Live data from Hacker News

Forecasts need to have error bars

andrewpwheeler.com

161–165 of 165 posts

Re: Forecasts need to have error bars

#161
post #124

Earlier quoted context omitted.

> Why do they persist with models that don't represent reality despite knowing it? Why do physicists ignore friction whenever possible? In general, for any task, you take the simplest model that represents the aspects of reality that you care about. But you stay aware of the limits. That's true in physics or engineering just as much as in economics. That's why NASA uses Newtonian mechanics for all their rocket scienc…

Many of the policies you mentioned sound great in a world of spherical cows but break down in the real world. For example you say a basic policy is a tax on guns instead of a ban. First of all I dispute that is even orthodox economics. Second there is some strong evidence that gun bans reduce violence. Free migration is another one. It is an insanely complicated issue in the real world. No country has 100% free migra…

> For example you say a basic policy is a tax on guns instead of a ban. First of all I dispute that is even orthodox economics. Second there is some strong evidence that gun bans reduce violence.

First, I apologize for using guns as an example. That's a needlessly divisive topic. The general principle of 'taxes instead of bans' is rather orthodox. You see that more often applied to the example of drugs or emissions.

Second, what evidence do you have for gun bans reducing violence? And reduce violence compared to what baseline?

I am very willing to believe that if you compare a free-for-all with a ban on guns, that the latter will see less violence. (I haven't looked into the evidence. Results might differ depending on details and on when and where you do that, and who gets exceptions to the bans. Eg police and military presumably are still allowed guns? Hunters probably as well? Etc. It's not so important.)

My point is that in terms of violent crime avoided, a situation where each gun and each bullet comes with a million dollar tax would be statistically indistinguishable from a ban.

And in practice, a less severe tax would probably be enough to achieve those goals whilst still preserving access to guns for those who prefer it that way.

> No country has 100% free migration or they wouldn’t be a country.

What kind of definition of 'country' are you using here that breaks down in this way? (And what do you mean by '100%'? How nitpicky do you want to be?)

A history lesson from Wikipedia https://en.wikipedia.org/wiki/Passport

> A rapid expansion of railway infrastructure and wealth in Europe beginning in the mid-nineteenth century led to large increases in the volume of international travel and a consequent unique dilution of the passport system for approximately thirty years prior to World War I. The speed of trains, as well as the number of passengers that crossed multiple borders, made enforcement of passport laws difficult. The general reaction was the relaxation of passport requirements.[18] In the later part of the nineteenth century and up to World War I, passports were not required, on the whole, for travel within Europe, and crossing a border was a relatively straightforward procedure. Consequently, comparatively few people held passports.

> During World War I, European governments introduced border passport requirements for security reasons, and to control the emigration of people with useful skills. These controls remained in place after the war, becoming a standard, though controversial, procedure. British tourists of the 1920s complained, especially about attached photographs and physical descriptions, which they considered led to a "nasty dehumanisation".[19] The British Nationality and Status of Aliens Act was passed in 1914, clearly defining the notions of citizenship and creating a booklet form of the passport.

Btw, Switzerland as a country does not restrict immigration. That's left to the Kantone (which are sort-of the equivalent to American states). Yet, you'd be hard pressed to argue that Switzerland is not a country. If memory serves right, the US used to have similar arrangements in their past?

Re: Forecasts need to have error bars

#162
post #102

Earlier quoted context omitted.

How do economists have skin in the game? Many of them eg work in universities and some even have tenure. There's not much skin in the game between any forecasts they might make and their academic prospects. Economists working for companies often have to help them understand micro and macro-economics. Eg (some of) Google's economists help them design the ad auctions. It's relatively easy to figure out for Google how w…

You aren’t going to get hired by the Chicago crowd if you start espousing Kensyian ideas let alone Marxist ones. You aren’t getting hired by Exxon if you start talking about the negative externalities of climate change.

That might or might not be true, but it's not what 'skin in the game' means.

Re: Forecasts need to have error bars

#163
post #114

Earlier quoted context omitted.

> 95% is 95% regardless of the distribution. Standard deviation away from the mean don't correspond to the same percentiles for all distributions, or do they? If you want to be (almost) independent of distribution, you need Chebyshev's inequality. But that one is far weaker. > Its practical usage is similar to the 68–95–99.7 rule, which applies only to normal distributions. Chebyshev's inequality is more general, sta…

> Standard deviation away from the mean don't correspond to the same percentiles for all distributions, or do they? If they had said a two standard deviation interval then you would have needed to know the distribution, but they said 95% which gives you all the information you need to make the bet.

I was replying to this addendum, which only really works for the normal distribution:

> Edit for OP's explicit question: One standard-deviation errorbars are 68% confidence intervals. Two standard deviations are 95% confidence intervals. (assuming you're a frequentist, of course)

You are right about the first part of the original comment.

Re: Forecasts need to have error bars

#164

Earlier quoted context omitted.

Any anecdotes you can share? Also, I meant "make sure" in a negative way. As in you "make sure things are statistically significant" by e.g. p-hacking. Not that this isn't done in science but I think you're more in danger of being embarrassed during peer review than by the C-suite reading your executive summary...

Most companies that care will run everything though an AB test, the number of AB tests is physically limited by traffic volume and the team in charge of measuring the results of the AB tests is not the team that created the experiment. That makes it much harder to p-hack since you cannot re-run experiments infinitely on a laptop, and the measurement team is judged on the accuracy of their forecasts versus the revenue…

The worse things can and often do generate more revenue. So what are the goals of the AB test?

Re: Forecasts need to have error bars

#165

Earlier quoted context omitted.

Most companies that care will run everything though an AB test, the number of AB tests is physically limited by traffic volume and the team in charge of measuring the results of the AB tests is not the team that created the experiment. That makes it much harder to p-hack since you cannot re-run experiments infinitely on a laptop, and the measurement team is judged on the accuracy of their forecasts versus the revenue…

The worse things can and often do generate more revenue. So what are the goals of the AB test?

>The worse things can and often do generate more revenue.

The goal of a business is usually to generate revenue so I'm confused about your question.

Post reply on HN