Configuring "forever" Script to Run a Process Forever

This document describes how to use the "forever" command to keep a script running forever. The "forever" command is poorly designed, and has a million pitfalls. Because of its poor design, "forever" is the worst of these sorts of programs, except for all the others. We'll show you how to work around all the pitfalls.

I have code that should run forever. In this case it's a service that connects to Firebase and implements a search. It should run forever, so that the search is fast: it keeps a local copy of the list of search targets, and then stores search results into Firebase.

It's easy to write a Node.js script do do the search. And you can run the script from the command line.

./search.node &

The problem is that the service stops when I close the window.

You can use nohup:

nohup ./search.node &

The script will continue to run when the window is closed. But if the script should terminate, nothing will restart it.

That's what "forever" is for.

npm install forever

Then

node_modules/forever/bin/forever start search.js

That command might work, except, as mentioned above, there are a million stupid little issues with "forever".

The first issue is that the "node" command on linux (Ubuntu, at least) often doesn't exist. It's called nodejs by default. If you add a symlink to /bin, that might not solve the problem because the default Node.js is old. The best way to get a latest Node.js is

npm install n

The "n" program is created so you can run multiple versions of Node.js on your computer. By default, the latest version is installed in /usr/local/bin/node. Your PATH environment variable probably includes /usr/local/bin early on, so you can run node from the command line.

Now, you'll get the latest version of Node.js, so things like Firebase will run correctly.

Now

node_modules/forever/bin/forever start search.js

will probably start your script.

Ok, but what happens if the server reboots, perhaps to get the latest security patches? You can configure your crontab to start "forever" at boot.

crontab -e

then add

@reboot (cd work/search && node_modules/forever/bin/forever start search.js)

and it probably won't work. Cron runs with a very limted path that doesn't include /usr/local/bin.

So, create the following script, called "run".

PATH=/usr/local/bin:$PATH
export FOREVER_ROOT=.forever
node node_modules/forever/bin/forever stopall
node node_modules/forever/bin/forever --minUptime 1000 --spinSleepTime 1000 start server.js

This script adds /usr/local/bin to the path. It then sets the root for all files created by forever to be .forever, in the current directory, rather than your home directory. It then stops any previously running forever scripts. It then starts forever and your script.

Now update your crontab.

@reboot (cd work/search && ./run)

Then reboot your server to test.

Others will tell you to use the package forever_service. It solves a lot of the same problems in an easier way. However, it violates one of my prime rules, that you should be able to configure your app without needing root access. The reasoning behind that rule is for another post.

Tax Bill New Tax Brackets

The new tax bill offers across the board tax rate reductions, as long as you take the standard deduction. If you pay state and local taxes, though, the benefit of the new bill is not so clear.

This graph compares the old and new tax brackets for filing single, and earning less than $1,000,000. See how much money you will save by finding your income along the horizontal axis, then looking at the blue and red lines directly above. The blue line is what you would pay under the old rules, while the red line is what you will pay if the new tax bill passes.

When reading these graphs, LOWER IS BETTER. When the red line is below the blue line, it means that income level gets a tax cut from the new bill.

The graphs assume you take the standard deduction, which increases substantially in the new tax bill.

New Tax Bill

Federal Taxes by Income

filing single

If you earn under $169,500, you will save roughly 4%, which is due to the lower rate as well as the increased standard deduction.

At around $200,000, you will save roughly 2.5%.

At around $430,000 income, the curves almost touch. You save only 0.2% compared to the old rates.

After that, the savings rate climbs rapidly to 2.6%.

If you earn more than $1,000,000, you pay 37% instead of 39.6% on everything above $1,000,000. For each additional million dollars: you will save $26,000.

Of course, these tax savings only work for you if you take the standard deduction. If you are used to deducting more than $10,000 in combined state taxes and property taxes, the loss of deductions on the federal return quickly eliminates any tax cut or even turns your tax savings into tax increases.deducting

This graph shows the same blue and red lines as before. It adds two additional lines that show your total taxes: federal + state, and account for the changes in treatment of state and local deductions. This graph is for filing single in California and assumes you do not pay mortgage interest nor property tax (i.e., you do not own a home).

California, filing single

No Home (no property tax nor mortgage)

Notice that the purple line is above the black line for incomes under $200,000. This new tax bill is a tax cut for people who make under $200,000. Above $200,000 the new bill is a substantial tax increase.

The next chart shows the total taxes for a California home owner filing single. It assumes a mortgage of $417,000 at 3%/year and $16,500/year property tax.

State tax is computed based on income and deductions for both property tax and mortgage interest.

Federal tax is computed based on income and deductions for state + property tax (capped at $10,000) and mortgage interest.

California, filing single

$16,500 property tax

$417,000 mortgage at 3%

The purple line shows what your taxes would be using 2017 rates. The black line shows what your taxes would be if the new bill passes. Note that, if you own a home in California, at best, you get no tax cut. If you earn more than $200,000, your total taxes will increase substantially.

The graph looks pretty much the same for Kansas, assuming filing single, paying property tax of $3,300/year and a $217,000 mortgage at 4%. In this case, you do get a tax cut if you earn less than $200,000. That's because you don't max out the $10,000 state tax deduction. As your income increases, though, the loss of state tax deduction turns this bill a into tax increase.

Kansas, filing single

$3,300 property tax

$217,000 mortgage at 4%

That's not the whole story though. If you make enough money, and you live in a low tax state, then the curves cross again, and the new bill becomes a tax cut. Here is the same Kansas graph, but showing up to $3,000,000 income. Notice that the black line is (finally) below the purple line.

Kansas, filing single, out to $3,000,000 income

Even in Kansas, you get a tax hike if you make roughly $200,000 to $600,000.

Here is why this so called tax cut bill is a tax increase for most people: You pay federal taxes on the state taxes you already paid. Suppose your income is over 500000, so your tax bracket is 37%. Then you pay 37% federal tax on the money you used to pay your state taxes above $10,000. If the state tax is 4.6% (as in Kansas), then roughly $250,000 income gives you a state tax bill of $10,000. Therefore, 100% of your income above $500,000 is taxed twice. That's 4.6% of your income that wasn't taxed under the old law is now taxed (again) at 37%. That's 37% * 4.6% = 1.7%. That 1.7% is less than the 2.6% tax cut for the highest tax bracket (from 39.6% to 37%), so if you make enough money, you'll keep 0.9% more of it.

Of course, in high tax states, it's a different story. In California, the top tax bracket is 12.3%. Paying federal tax on money you already paid to state tax means you are double paying 37% of 12.3%, or 4.55%. 4.55% is bigger than the 2.6% tax cut, so the tax bill effectively increases the top bracket by 1.95%.

Here's that graph.

California, filing single out to $3,000,000 income

The new tax bill raises taxes on almost everybody. There are two cases where your taxes are lowered, both only if you live in a low tax states.

If you live in a low tax state and make less than $200,000, then you get a very small tax cut. If  you live in a low tax state and make many millions, then you get a substantial tax cut.

Here are the highest taxed states. They will see tax increases of 2.8% to 4.9%.

• California 13.3%
• Oregon 9.9%
• Minnesota 9.85%
• Iowa 8.98%
• New Jersey 8.97%
• Vermont 8.95%
• District of Columbia 8.95%
• New York 8.82%
• Hawaii 8.25%
• Wisconsin 7.65%

It's no surprise that these high tax states, who receive the biggest tax increase, are blue states.

Amazon Dash Buttons

Amazon makes lots of interesting hardware gadgets. This one, the Amazon Dash Button, is of questionable value. Amazon has a variety of dash buttons, branded with Tide, Glad, Charmin, etc. The consumer buys the Button for $5, peels off the sticky tape, and fastens the Button in some convenient location. Then, when he runs out of plastic bags or dog food, he pushes the appropriate button, and a corresponding order is placed automatically. Your first order is discounted by $5, so the Button is effectively free.

The product works fine, and is a nice piece of engineering. The problem is, people like to push buttons. I put a Tide button near my laundry machine, and people pushed it. Do you know how much laundry detergent I have?

I would prefer to have the dash button add the corresponding item to my shopping list, which would act as a reminder to order more.

I can think of lots of great uses for such "Internet Buttons." I'd like to have one that turns on my recirculating hot water, one that orders a pizza, etc. Fortunately, Amazon offers a generic AWS IoT Button that can be custom programmed. It's not free, of course, but costs $20.

With my interests in all things IoT, I'm a big fan of the Dash button, of course. In fact, I invented my own. This Amazon Dash button is for purchasing Amazon Dash buttons. When you run low on Amazon Dash buttons, simply press the button, and it will order another one for you.

A configuration option allows you to order two Buttons with a single press.

Trump Impeachment Day: Jan 21, 2018

From U.S. Constitution, Amendment XXII, Section 1:

no person who has held the office of President, or acted as President, for more than two years of a term to which some other person was elected President shall be elected to the office of President more than once

The Republican Senate will wait until Jan 21, 2018 to convict President Trump of impeachment. The House can impeach the president before then, but it's important that V.P. Pence become president less than two years before the end of the term. This way, Pence is still eligible for an additional eight years. Imagine that! Ten years of President Pence after two years of President Trump!

Here's a thought. It takes 2/3 of the Senate to convict, and the Republicans have only 52 seats. What if the Democrats decide to leave Trump in office? From their perspective, in many ways, Pence is worse. Also, the Democrats will have lots of leverage to make a deal with Trump before allowing him to stay in office. Trump will be happy to oblige because his thin skin will feel horribly slighted by the Republicans.

Net, net: No chance Trump will be impeached.

Sprint Overcharges for Overages

The new Sprint Framily plans are very inexpensive. It's a trade off. Sprint's service is mediocre and declining. The new LTE roll out was a disaster, with large areas (e.g., the entire downtown Sunnyvale) having no service for extended periods. The service seems to have stabilized now, and the very fast LTE is available in many areas.

The LTE service seems to cover much smaller regions than their 3G. If you happen to be in an LTE area, you will get terrific, very fast service. Outside of those smallish areas, you still have 3G, which performs quite fine. The problem is when you find yourself at the boundary of the LTE area, still inside the 3G. The LTE signal is too weak to provide service but strong enough to prevent the phone from switching to 3G. Thus, in the middle of a perfectly useful 3G signal, I often get no service.

I might blame the device, which should determine that LTE isn't strong enough and automatically switch to 3G. My device is a Samsung Galaxy 4S, the most popular phone in the world. It would surprise me if this most popular device were at fault. Doesn't matter, it's a problem.

To get 3G coverage when I am at the edge of LTE, I change the phone settings. (Settings / More Networks... / Mobile networks / Network mode / CDMA.) This change is inconvenient, as the phone has to reboot. Switching back is another reboot, so I find myself leaving the phone in 3G most of the time. I can stream Netflix in 3G no problem, so maybe LTE isn't that important.

Why not switch to AT&T, Verizon, or T-Mobile? Sprint has been less expensive than the others for a long time. And the new Family plans are priced even much lower. I now pay $25/mo/line, which includes unlimited voice, unlimited text, and 1 GB data. For 2 lines, I pay an extra $10 or $20, for 3 GB data or unlimited data.

Sprint bill is much lower than it would be with Ting, which is known as a low cost leader. If they were less expensive, I'd switch to Ting. Problem is, they haven't lowered their prices since they launched.

Most lines have only 1 GB of data. Not enough, you might think, given that people use a lot of data these days. Surprisingly, Sprint is very reasonable about overages these days. If you use more than the 1 GB but not enough to justify paying the extra $10/mo to get 3 GB, they charge for the overages at a reasonable amount.

How much do they charge? $15/GB. If you use 1.5 GB, then you are charged an overage of about $7.50. My overages this month was $13.06.

I have two points to make about the $15/GB overage charge. First, that amount is mentioned in the contract, but only in the context of the tablet plan and the 3 GB plan. There is no overage amount mentioned for the 1 GB default.

Second, whenever you are approaching your 1 GB limit, Sprint sends text messages:

These messages state that the overage amount is $0.01/MB (= $10/GB). According to these text messages, Sprint overcharges for the overages.

I called them to ask for my refund. They are sure the overage price is $0.015/MB. We refer to the contract. They are surprised that the overage amount isn't mentioned (for the 1 GB plan). I then quote the messages they send me.

Casey A.: I do believe you and we are seeing it here on our end. It's just that these notifications can only have two decimal digits that's why it's only $0.01.

You: not my problem

You: I make decisions based on the information you provide me

You: You can't expect any reasonable person to agree that a company with the technical sophistication to deploy a cell phone system is technically incompetent, as you describe.

Casey A.: I have been approved for an adjustment for $4.14 on the account. We will create a report for this one to have these notifications corrected. We do apologize for the inconvenience.

They then provide the refund. (My refund this month was $4.14. Not worth my time, I know, but it's the principle that motivates me.)

One final note. Sprint defines a MB as 210 bytes, so when requesting a refund, remember to divide the KB by 1024, not 1000, and you'll get a few more pennies back.

WolframAlpha Boils Wrong

Wolfram Alpha is very good at knowing or computing the answer to a wide range of questions, especially math and science related. I love Wolfram Alpha and use it all the time. Siri does too, and uses it to answer many arbitrary questions.

We learn in high school chemistry class that the boiling point of water at sea level is 100°C. At elevations above sea level, where pressure is reduced, the boiling point drops. For example, at 10,000 feet above sea level, water boils at 90°C.

I live in Los Altos, CA, at 157 feet above sea level. I expect water to boil at very close to 100°C, but I am calibrating a thermometer, so I want to be accurate. To my surprise, Wolfram Alpha gets it wrong:

For water to boil at 100.1°C, Los Altos would have to be below sea level, which we know is not true, and so does Wolfram Alpha:

Wolfram Alpha, I am disappointed in you!

While we are at it, why is Wolfram Alpha so slow?

Fast Alphanum Algorithm

The Alphanum algorithm sorts strings "the right way". By default, most sorting is done lexicographically, which means that you sort two strings based on the left-most character in each string that do not match. Ted comes before Tim because e comes before i. Also, A100 comes before A2 because 1 comes before 2. You've probably seen files sorted this way before:

    A1
    A10
    A11
    A2
    A3
    A4
    A5
    A6
    A7
    A8
    A9

You want to see it sorted this way:

    A1
    A2
    A3
    A4
    A5
    A6
    A7
    A8
    A9

    A10
    A11

That's where the Alphanum algorithm comes in. It compares two strings in a smarter way, so that you get the second result above, not the first.

Now ask yourself how should these strings be sorted?

    x3
    xy
    x
    x3
    x2
    x23
    000
    01
    009
    01
    2x
    2
    #2x
    #20
    #2
    40003
    4000000000002
    0000000000003
    00003
    000034
    xyz40003q
    xyz4000000000002q
    xyz0000000000003q
    xyz00003q
    xyz000034q
    00000
    00000000000000000000000
    0
    (empty string)

Which comes first, 0, 00000, or 00000000000000000000000 ? Let's say that numbers representing the same value should be sorted shortest first.

I wrote my alphanum algorithm a long, long time ago. It was slow, so I rewrote it, and now it's really fast. I published it, in the sense that it is used in various websites, but I never wrote it up. Why write it up now?

I just came across various implementations of alphanum collected by Dave Koelle at http://www.davekoelle.com/alphanum.html. So it's out there. And his versions seem to be pretty popular. There are a great many discussions about his collection.

Unfortunately, the code is pretty slow. The implementations use a "chunkify" approach, which converts each string into separate alpha and numeric parts. That's a whole lot of computation and memory management, which is slow.

My first implementation of alphanum did something similar, but I eventually moved past it. I found it is possible to write alphanum to do a single pass over the data, comparing the two strings in place, which is much faster. (In fact, this version is O(N), the same as lexicographic string comparison, so this version could potentially run the same speed as lexicographic string comparison.) Comparing to Dave's JavaScript version, mine runs about 9 times faster. (More on that later.) The code is also simpler, at least until the final optimized version. Here it is.

    function alphanum(a, b) {

        function isdigit(ch) {
            return '0' <= ch && ch <= '9';
        }

        var min = Math.min(a.length, b.length);

        for (var i = 0; i < min; i++) {
            if (a.charCodeAt(i) !== b.charCodeAt(i)) break;
        }

        if (i == min) return a.length - b.length;
        var cmp = a.charCodeAt(i) - b.charCodeAt(i);

        if (isdigit(a[i - 1]) || isdigit(a[i]) && isdigit(b[i])) {
            for (;; i++) {
                var ai = isdigit(a[i]);
                var bi = isdigit(b[i]);
                if (ai !== bi) return ai - bi;
                if (!ai || !bi) break;
            }
        }

        return cmp;
    }

Try it here: http://jsfiddle.net/SpP65/. The example uses the data set from Dave's page.

One problem: there is an important case that my code does not handle. See if you can figure it out. (It's tricky.)

==== SPOILER ====

I found Dave's page and saw that the algorithm was slow. However, my version didn't handle a significant case properly: numbers with leading zeros. How should frame2, frame3, and frame002 sort? It's obvious that frame002 belongs between the other two, but the code above doesn't do that.

For years I have been pondering how to fix my code while keeping it simple and fast. Today I figured it out. Unfortunately, I didn't manage to keep it simple. But the change to support leading zeros made the code even faster!

The approach: given all the edge conditions, I decided to simply blow out the code into a bunch of special cases, to deal with each edge condition separately. This way, you get lots of code, but only one code path runs for each case, so it's very fast. Here it is.

    function alphanum(a, b) {
        
        function isdigit(ch) {
            return '0' <= ch && ch <= '9';
        }
        
        var min = Math.min(a.length, b.length);
        
        for (var i = 0; i < min; i++) {
            if (a.charCodeAt(i) !== b.charCodeAt(i)) break;
        }
        
        if (i == min) return a.length - b.length;
        
        if (isdigit(a[i])) {
            if (isdigit(b[i])) {
                if (a[i] === '0' || b[i] === '0') {
                    for (var z = i - 1; z > 0; z--) {
                        if (a[z] !== '0') break;
                    }
                    if (!isdigit(a[z])) z++;
                    if (a[z] === '0' || b[z] === '0') {
                        for (var j = i; ; j++) {
                            if (b[j] !== '0') break;
                        }
                        for (;; i++) {
                            if (a[i] !== '0') break;
                        }
                        if (!isdigit(a[i])) {
                            if (isdigit(b[j])) {
                                return -1;
                            } else {
                                return i - j;
                            }
                        } else if (!isdigit(b[j])) {
                            return 1;
                        } else {
                            var cmp = a.charCodeAt(i) - b.charCodeAt(j);
                            for (i++, j++;; i++, j++) {
                                if (!isdigit(a[i])) {
                                    if (isdigit(b[j])) {
                                        return -1;
                                    } else {
                                        if (cmp) return cmp;
                                        return i - j;
                                    }
                                } else if (!isdigit(b[j])) {
                                    return 1;
                                }
                            }
                        }
                    }
                }
                
                var cmp = a.charCodeAt(i) - b.charCodeAt(i);
                for (i++;; i++) {
                    if (!isdigit(a[i])) {
                        if (isdigit(b[i])) {
                            return -1;
                        } else {
                            return cmp;
                        }
                    } else if (!isdigit(b[i])) {
                        return 1;
                    }
                }
            } else if (isdigit(a[i - 1])) {
                return 1;
            } else {
                return a.charCodeAt(i) - b.charCodeAt(i);
            }
        } else if (isdigit(b[i])) {
            if (isdigit(b[i - 1])) {
                return -1;
            } else {
                return a.charCodeAt(i) - b.charCodeAt(i);
            }
        } else {
            return a.charCodeAt(i) - b.charCodeAt(i);
        }
    }
    
    $('#a').text([
        '1000X Radonius Maximus',
        '10X Radonius',
        '200X Radonius',
        '20X Radonius',
        '20X Radonius Prime',
        '30X Radonius',
        '40X Radonius',
        'Allegia 50 Clasteron',
        'Allegia 500 Clasteron',
        'Allegia 50B Clasteron',
        'Allegia 51 Clasteron',
        'Allegia 6R Clasteron',
        'Alpha 100',
        'Alpha 2',
        'Alpha 200',
        'Alpha 2A',
        'Alpha 2A-8000',
        'Alpha 2A-900',
        'Callisto Morphamax',
        'Callisto Morphamax 500',
        'Callisto Morphamax 5000',
        'Callisto Morphamax 600',
        'Callisto Morphamax 6000 SE',
        'Callisto Morphamax 6000 SE2',
        'Callisto Morphamax 700',
        'Callisto Morphamax 7000',
        'Xiph Xlater 10000',
        'Xiph Xlater 2000',
        'Xiph Xlater 300',
        'Xiph Xlater 40',
        'Xiph Xlater 5',
        'Xiph Xlater 50',
        'Xiph Xlater 500',
        'Xiph Xlater 5000',
        'Xiph Xlater 58'
    ].sort(alphanum).join("\n"));
    
    $('#b').text([
        'x3', 'xy', 'x', 'x3', 'x2', 'x23',
        '000', '01', '009', '01',
        '2x', '2', '#2x', '#20', '#2',
        '40003', '4000000000002', '0000000000003', '00003', '000034',
        'xyz40003q', 'xyz4000000000002q', 'xyz0000000000003q', 'xyz00003q', 'xyz000034q',
        '00000', '00000000000000000000000', '0', ''
    ].sort(alphanum).join("\n"));

Try the new code: http://jsfiddle.net/nz6uU/3/

I find it fascinating how complicated the code got. There is nothing surprising in the code, just a lot of it. To understand it, pick various cases and follow the code through.

And how fast is it? See the benchmark: http://jsperf.com/alphanum/4. The first is Dave's alphanum. The second, named "timkay" is the broken (but simple) version, and the last, named "timkay 2" is the optimized version.

The version that handles leading zeros correctly also runs over 10x faster than the chunkify version. Sweet!