Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overfitting and the strong version of Goodhart’s law

sohl-dickstein.github.io

Post reply on HN
Print view View original

101–110 of 111 posts

Re: Overfitting and the strong version of Goodhart’s law

#101

Post by pksebben »

agalunar wrote: post #54
a_c wrote: post #43

Maybe the ultimate proxy measurement is the pursuit of human economic productivity at the expense of longevity of our planet. Contrived example, human need food to survive. We need food, efficiently, since there are so many of us. So we need efficient farming. Efficient means smallest amount of land to produce most biomass (plant, animal). To that, we need single purpose farmland and artificial feeds, fertilizer and…

Ironically, we're not very efficient at all with our land usage today. Nearly 60% of global agricultural land is used for beef (either for pasture or for growing feed) and yet it accounts for only 2% of our calories. And about 50% of all food is wasted – simply thrown away.

that's an optimization error of another stripe - thrown away or not, the beef gets sold and FarmCo makes money

Re: Overfitting and the strong version of Goodhart’s law

#102

Post by gweinberg »

pixl97 wrote: post #57
club_tropical wrote: post #45

I have mentioned before - I hate Goodhart's law. It makes no sense. There is no example that is a good measure but a bad target. The canonical example I have heard is hospital emergency rooms that started to be measured by wait times, so they refused to admit patients until staff was ready to receive them, literally having ambulances circling around the block. This was supposed to be a "good measure turned into bad t…

So is measuring a programmers produced LOC a good or bad measure?

Bad, obviously. All else being equal, the better the coder, the less code he needs to write to achieve the same effect.

Re: Overfitting and the strong version of Goodhart’s law

#103

Post by epgui »

visarga wrote: post #94
epgui wrote: post #86

Earlier quoted context omitted.

Not in the same sense whatsoever. Training a neural net, backpropagation or not, doesn't affect the data. It's basically just some variation of / remix of a linear regression.

Yes, for that you need RL. An environment beats a fixed, even large, training set.

I think we're probably using different words to make the same distinction, and in any case the underlying mechanism is very different.

Re: Overfitting and the strong version of Goodhart’s law

#104

Post by AnthonyMouse »

mannykannot wrote: post #100
AnthonyMouse wrote: post #96

Earlier quoted context omitted.

The moral hazard is the difference between the insurance payout and the amount your future rates change as a result of the claim. If this amount is zero, there is no insurance. If it isn't, there is that much less incentive to avoid the harm.

Much less? Can you explain this immediate threshold as soon as the moral hazard is non-zero?

"That much less" meaning equal to the difference between the amount of the harm and what they'd expect to pay in increased premiums.

If spending $X would avoid a small chance of a million dollars in damage, the X you're inclined to spend is a lot larger if the potential loss is the full million dollars than if it would increase your insurance premiums by a net present cost of $1000.

Re: Overfitting and the strong version of Goodhart’s law

#105

Post by AnthonyMouse »

User23 wrote: post #98
AnthonyMouse wrote: post #80

Earlier quoted context omitted.

But that only moves the problem somewhere else, because insurance is a moral hazard. Someone who would otherwise be cautious to prevent harm has less incentive to do it because when the harm comes the insurance pays.

Not necessarily. In the case of accidents, liability insurance presents no moral hazard. The ordinary person is not going to leave a broken railing in his house that could cause a guest to fall and injure himself just because he has homeowner's insurance. Similarly, the ordinary person doesn't seek out automobile accidents on account of having mandatory automobile insurance. Perhaps most strikingly, the moral hazard…

> In the case of accidents, liability insurance presents no moral hazard. The ordinary person is not going to leave a broken railing in his house that could cause a guest to fall and injure himself just because he has homeowner's insurance.

Maybe not if repairing the railing costs $15, but what if the safety repairs would cost $15,000? When not doing it could cause someone who gets hurt to render you bankrupt and homeless, you find the money. When you're insured, you may have other priorities.

> Similarly, the ordinary person doesn't seek out automobile accidents on account of having mandatory automobile insurance.

It's not about seeking them out. You don't want an accident, but you do want to read that text you just got, and you're more likely to wait until you're stationary if an at fault accident could ruin you instead of just raising your insurance premiums.

> Perhaps most strikingly, the moral hazard theory would suggest that life insurance policy holders are more likely to commit suicide, but in reality they are less likely to!

Isn't suicide an exception to nearly all life insurance polices, among other reasons to remove that very incentive?

Re: Overfitting and the strong version of Goodhart’s law

#106

Post by User23 »

AnthonyMouse wrote: post #105
User23 wrote: post #98

Earlier quoted context omitted.

Not necessarily. In the case of accidents, liability insurance presents no moral hazard. The ordinary person is not going to leave a broken railing in his house that could cause a guest to fall and injure himself just because he has homeowner's insurance. Similarly, the ordinary person doesn't seek out automobile accidents on account of having mandatory automobile insurance. Perhaps most strikingly, the moral hazard…

> In the case of accidents, liability insurance presents no moral hazard. The ordinary person is not going to leave a broken railing in his house that could cause a guest to fall and injure himself just because he has homeowner's insurance. Maybe not if repairing the railing costs $15, but what if the safety repairs would cost $15,000? When not doing it could cause someone who gets hurt to render you bankrupt and hom…

> You don't want an accident, but you do want to read that text you just got, and you're more likely to wait until you're stationary if an at fault accident could ruin you instead of just raising your insurance premiums.

Literally no one ever has thought “self, I’m going to look at this text while I drive because I’m insured!” In the real world they’re doing it because they’re addicted and not because of some rational calculus. That might make a good scenario for a comedic skit though.

I don’t feel like looking it up on my phone, but I’d bet at worse than even odds that drunk drivers are in fact less likely to be insured, when the moral hazard theory would predict they’re more likely to be.

> Isn't suicide an exception to nearly all life insurance polices, among other reasons to remove that very incentive?

The answer is either not really or even an outright no. Individual policies usually have a 1-2 year no suicides clause and after that they pay. Group policies like employer offered ones usually have no wait period and will just pay out.

Re: Overfitting and the strong version of Goodhart’s law

#107

Post by mannykannot »

AnthonyMouse wrote: post #104
mannykannot wrote: post #100

Earlier quoted context omitted.

Much less? Can you explain this immediate threshold as soon as the moral hazard is non-zero?

"That much less" meaning equal to the difference between the amount of the harm and what they'd expect to pay in increased premiums. If spending $X would avoid a small chance of a million dollars in damage, the X you're inclined to spend is a lot larger if the potential loss is the full million dollars than if it would increase your insurance premiums by a net present cost of $1000.

The model you present here is semi-quantitative, in that it has an example value of $1000 for insurance premiums, another of $X for liability, but "small chance" is not introduced as a variable, and neither are "lot larger" and the change in premiums as a function of change in safety spending. I suspect that if this model were completed in accordance with your premise of equality, it would imply there is no rational case for insurance.

This seems moot, however, as this is not shaping up to be a plausible model for how things actually went since Winterbottom's unfortunate accident. Road transport vehicles have become a great deal safer since then, even as the potential for them to do harm has increased enormously. For your argument to be pertinent, it would have to be likely that, in the alternative reality where Rolf's ruling remained the law and liability insurance did not come about, they would be safer than they are now.

Re: Overfitting and the strong version of Goodhart’s law

#109

Post by AnthonyMouse »

mannykannot wrote: post #107
AnthonyMouse wrote: post #104

Earlier quoted context omitted.

"That much less" meaning equal to the difference between the amount of the harm and what they'd expect to pay in increased premiums. If spending $X would avoid a small chance of a million dollars in damage, the X you're inclined to spend is a lot larger if the potential loss is the full million dollars than if it would increase your insurance premiums by a net present cost of $1000.

The model you present here is semi-quantitative, in that it has an example value of $1000 for insurance premiums, another of $X for liability, but "small chance" is not introduced as a variable, and neither are "lot larger" and the change in premiums as a function of change in safety spending. I suspect that if this model were completed in accordance with your premise of equality, it would imply there is no rational…

> The model you present here is semi-quantitative, in that it has an example value of $1000 for insurance premiums, another of $X for liability, but "small chance" is not introduced as a variable, and neither are "lot larger" and the change in premiums as a function of change in safety spending.

All of the numbers are obviously made up examples because in practice they depend on what the risky behavior is and the value of the dollar etc. But we can make up more of the numbers, if you like example numbers.

A 1% chance of a million dollar liability has an expected value of -$10,000. A 1% chance of a premium increase with a net present cost of $1000 has an expected value of -$10. Therefore, the resources the party would rationally expend to prevent the harm is $10,000 in the first case and $10 in the second case. That difference is a lot.

> I suspect that if this model were completed in accordance with your premise of equality, it would imply there is no rational case for insurance.

Insurance is a net loss to the average insured, even before the moral hazard, because the sum of the premiums is necessarily more than the sum of the claims since premiums also have to cover the insurance company's overhead (or the insurance company becomes insolvent).

Its only purpose is to pool risk. Many people prefer a 100% chance of a $1050 loss to a 1% chance of a $100,000 loss. But pooling risk introduces moral hazard -- that's one of the reasons people like to be insured. "Peace of mind" = don't have to worry because the insurance will cover it.

> For your argument to be pertinent, it would have to be likely that, in the alternative reality where Rolf's ruling remained the law and liability insurance did not come about, they would be safer than they are now.

The market wants cars that are safer for their occupants because insurance can't bring you back from the dead.

The market doesn't care if cars are less safe for pedestrians or other motorists, because that cost is on the other party or the insurance company. And so we see cars getting heavier over time, as expected from that set of incentives.

Re: Overfitting and the strong version of Goodhart’s law

#110

Post by AnthonyMouse »

User23 wrote: post #106
AnthonyMouse wrote: post #105

Earlier quoted context omitted.

> In the case of accidents, liability insurance presents no moral hazard. The ordinary person is not going to leave a broken railing in his house that could cause a guest to fall and injure himself just because he has homeowner's insurance. Maybe not if repairing the railing costs $15, but what if the safety repairs would cost $15,000? When not doing it could cause someone who gets hurt to render you bankrupt and hom…

> You don't want an accident, but you do want to read that text you just got, and you're more likely to wait until you're stationary if an at fault accident could ruin you instead of just raising your insurance premiums. Literally no one ever has thought “self, I’m going to look at this text while I drive because I’m insured!” In the real world they’re doing it because they’re addicted and not because of some rationa…

> Literally no one ever has thought “self, I’m going to look at this text while I drive because I’m insured!”

It works the other way. If you have no insurance, you think, "self, I'm not going to look at this text while I drive because I'm not insured, and if I hit someone it could cause me to lose my house."

Same reason undocumented immigrants follow the speed limit.

> I don’t feel like looking it up on my phone, but I’d bet at worse than even odds that drunk drivers are in fact less likely to be insured, when the moral hazard theory would predict they’re more likely to be.

There are obvious reasons for this to be the case independently. People with a DUI record are more likely to drive drunk, but people with a DUI record may not be able to get or afford insurance. Drunk driving and not having insurance might both be correlated with poverty. Things like that. Is it your argument that not having insurance causes you to be less likely to drive drunk, all else equal?

> The answer is either not really or even an outright no. Individual policies usually have a 1-2 year no suicides clause and after that they pay. Group policies like employer offered ones usually have no wait period and will just pay out.

But the 1-2 year clause is there specifically because of the moral hazard. Otherwise not only would anyone planning to commit suicide have the incentive to take out life insurance first, anyone who needed a quick big payout for their loved ones would have the incentive to take out a policy and then commit suicide.

And the general trend in the opposite direction is caused by both the removal of that incentive, and the same kind of confounders as in the DUI case. People with stable employer-provided insurance coverage or with the financial stability to afford premiums for >2 years are the sort of people less likely to take their own lives.

Post reply on HN