Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overfitting and the strong version of Goodhart’s law

sohl-dickstein.github.io

Post reply on HN
Print view View original

51–60 of 111 posts

Re: Overfitting and the strong version of Goodhart’s law

#51

Post by kwhitefoot »

lqet wrote: post #33

I recommend reading "The Collapse of Complex Societies" by Joseph Tainter [0]. Complex societies tend to address problems by adding more and more rules and regulations, simply because they have always done so and it has been successful in the past. More importantly, though, it is typically the only tool they have. Essentially these societies are increasingly overfitting their legislature to narrow special cases until…

This is not new, merely ignored:

"What's the origin of the phrase 'Hard cases make bad law'?

... 'Hard cases make bad law' isn't so much a universal proverb as a legal adage. It came to light in a comment made by Judge Robert Rolf in the case of Winterbottom v Wright in 1842:

    This is one of those unfortunate cases...in which, it is, no doubt, a hardship upon the plaintiff to be without a remedy but by that consideration we ought not to be influenced. Hard cases, it has frequently been observed, are apt to introduce bad law.
The case required a judgment on whether third parties are able to sue for injury. The unusual nature of the case caused the judge to realise that, in the true sense of the expression, exceptions prove the rule and that, unfair as it might have appeared in some circumstances, the law was better drafted under the influence of the average case rather than the exceptional one.

The point was made explicitly in 1903 by V. S. Lean, in Collectanea:

    Hard cases make bad law. that is, lead to legislation for exceptions."
https://www.phrases.org.uk/meanings/hard-cases-make-bad-law....

Re: Overfitting and the strong version of Goodhart’s law

#52

Post by agalunar »

paulluuk wrote: post #48
lqet wrote: post #33

I recommend reading "The Collapse of Complex Societies" by Joseph Tainter [0]. Complex societies tend to address problems by adding more and more rules and regulations, simply because they have always done so and it has been successful in the past. More importantly, though, it is typically the only tool they have. Essentially these societies are increasingly overfitting their legislature to narrow special cases until…

Interesting idea! Seeing as my "should read" list is already hundreds of books long, can you give us a spoiler as to what he suggests (if anything) to fix this? Or is it simply inevitable and therefore not actionable?

Here's what I can recall:

The big idea is diminishing marginal returns – increasing societal complexity pays off enormously at first, but eventually the returns level off while the overhead remains high or even continues to increase. Collapse is then seen as a natural (and not necessarily cataclysmic) response to this. It's a process of simplification.

Tainter also points out that collapse may not occur (1) when there are strong, neighboring powers, which simply absorb the foundering society, or (2) when there are strong cultural reasons, such as national identity, for which the members of the society may be willing to put up with the bureaucratic overhead indefinitely (in order to prevent being absorbed); he gives Europe as an example, I believe.

Re: Overfitting and the strong version of Goodhart’s law

#53

Post by kwhitefoot »

bigfudge wrote: post #44

A lot of the suggested mitigations made a lot of sense to me. For example reducing time to prepare grant applications, or to add 1s jitter to stock trades. Perhaps there are flaws to these specific ideas, but the thing that struck me most was that we have almost no way of implementing ideas like this in Anglo political countries right now. Entrenched interests, and the technical detail involved makes it hard to imagi…

> reducing time to prepare grant applications,

Reminds me of a story about, I think, the take over of the Avis car rental company when it was failing. The guy who took charge said that the single thing that he did to turn the company around was to institute a rule saying that any decision, however large or small was not to take more than three weeks. If at the end of that period a decision was not forthcoming then a coin should be tossed to decide between the competing solutions.

Apologies if I have misremembered any, or all, of it but I think the idea works. Even if the wrong decision is made one gets to know early and can change course.

Re: Overfitting and the strong version of Goodhart’s law

#54

Post by agalunar »

a_c wrote: post #43

Maybe the ultimate proxy measurement is the pursuit of human economic productivity at the expense of longevity of our planet. Contrived example, human need food to survive. We need food, efficiently, since there are so many of us. So we need efficient farming. Efficient means smallest amount of land to produce most biomass (plant, animal). To that, we need single purpose farmland and artificial feeds, fertilizer and…

Ironically, we're not very efficient at all with our land usage today. Nearly 60% of global agricultural land is used for beef (either for pasture or for growing feed) and yet it accounts for only 2% of our calories. And about 50% of all food is wasted – simply thrown away.

Re: Overfitting and the strong version of Goodhart’s law

#55

Post by pixl97 »

ToJans wrote: post #27

This is exactly what Taleb wrote about [1]. Systems start out very fragile. Over time they get more and more robust, as edge cases and events result in more rules and rigidity. This works until they become so robust that they can no longer cope with the change when something unexpected -i.e. a black swan event - happens. The only way to counter this, to make a system that improves in coping with disorder/chaos as it…

Also reminds me of min/maxing in a multi-player game with an economy.

Some times in a balanced game a new min/max strategy is found in a game like this that it quickly becomes the dominant form of wealth generation in the economy. Asset inflation typically explodes in this scenario driving prices for everything way up.

The game itself typically becomes less fun at this point as min/max is now the only valid economic activity, hence intervention by the gods in the form of a patch. But while this fixes the exploit it does not fix the economic ruin. Players that made massive amounts of wealth in the event can have strangle holds on the economy and cause stagflation as material prices only drop slowly. New/poor players are hobbled with poor purchasing power.

It would be an interesting study on how many of the effects in this article could be studied by gamifying these situations.

Re: Overfitting and the strong version of Goodhart’s law

#56

Post by seri4l »

brookst wrote: post #4

This keeps getting rediscovered in new domains. JIT was the savior of manufacturing, until people learned that a single traffic jam that delayed a single delivery could create costs far in excess of the inventory savings. Optimizations are critical, everywhere. And measuring optimizations is important because it is is easier and cheaper and earlier than measuring end results. Measuring days of inventory, or dollars i…

The issue with JIT is not so much about proxies as it's about optimizing for the general case at the cost of worst case performance. Things that suppose an inefficiency 99.99% of the time can be an indispensable redundancy or buffer in the face of events that are difficult or impossible to account for.

Re: Overfitting and the strong version of Goodhart’s law

#57

Post by pixl97 »

club_tropical wrote: post #45

I have mentioned before - I hate Goodhart's law. It makes no sense. There is no example that is a good measure but a bad target. The canonical example I have heard is hospital emergency rooms that started to be measured by wait times, so they refused to admit patients until staff was ready to receive them, literally having ambulances circling around the block. This was supposed to be a "good measure turned into bad t…

So is measuring a programmers produced LOC a good or bad measure?

Re: Overfitting and the strong version of Goodhart’s law

#58

Post by Filligree »

agalunar wrote: post #54
a_c wrote: post #43

Maybe the ultimate proxy measurement is the pursuit of human economic productivity at the expense of longevity of our planet. Contrived example, human need food to survive. We need food, efficiently, since there are so many of us. So we need efficient farming. Efficient means smallest amount of land to produce most biomass (plant, animal). To that, we need single purpose farmland and artificial feeds, fertilizer and…

Ironically, we're not very efficient at all with our land usage today. Nearly 60% of global agricultural land is used for beef (either for pasture or for growing feed) and yet it accounts for only 2% of our calories. And about 50% of all food is wasted – simply thrown away.

I doubt it affects your central claim, but do note that a lot of pasture land is unusable for anything else.

Re: Overfitting and the strong version of Goodhart’s law

#59

Post by Filligree »

smartmic wrote: post #42
visarga wrote: post #30

Earlier quoted context omitted.

> as you increase the number of parameters in the model, your error bars become larger In large neural nets the effect is reversed. The larger the model, the better it generalises, even from the same training data.

> The larger the model, the better it generalises, even from the same training data Do you have some references for this claim? For me, it seems counterintuitive.

Can’t provide a reference, but I can confirm that this is common knowledge. It’s why e.g. GPT-3 outperforms GPT-2.

Though as stable diffusion shows, network architecture still matters a lot!

Note that the article points out you’ll get more overfitting as your number or parameters approaches that of the training set, which is what I suspect you’ve seen. The trend does reverse later on, but only once the parameter count is orders of magnitude beyond that point, and I don’t know if that ever happens outside of ML. It’s a lot of parameters.

Re: Overfitting and the strong version of Goodhart’s law

#60

Post by amadeuspagel »

> Goal: An informed, thoughtful, and involved populace

> Proxy: The ease with which people can share and find ideas

Are people today not more informed, thoughtful and involved? Maybe you just don't like the information, the thougths and the involvement. Is someone who shares and finds ideas on the internet not more involved then someone who passively watches TV? I harp on the term "involved" here, because that's the most neutral one. The capital rioters were quite involved.

Post reply on HN