Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overfitting and the strong version of Goodhart’s law

sohl-dickstein.github.io

Post reply on HN
Print view View original

61–70 of 111 posts

Re: Overfitting and the strong version of Goodhart’s law

#61

Post by brookst »

kqr wrote: post #9
brookst wrote: post #4

This keeps getting rediscovered in new domains. JIT was the savior of manufacturing, until people learned that a single traffic jam that delayed a single delivery could create costs far in excess of the inventory savings. Optimizations are critical, everywhere. And measuring optimizations is important because it is is easier and cheaper and earlier than measuring end results. Measuring days of inventory, or dollars i…

I feel like i have to repeat this very often: if a single traffic jam or other predictable common-cause variation results in your JIT implementation costing "far in excess of inventory savings" all you're telling me is that you have a really shitty JIT implementation. JIT means having the buffer on hand to handle at the very least common-cause variation. But it also should come with the flexibility to handle variatio…

Yes? My point was that people get carried away with maximizing optimizations because they start measuring the degree of optimization rather than the ultimate goal.

I’m not saying JIT always goes wrong, any more than neural net training always goes wrong. Just that when it does, it is often because an overzealousness that leads to a disconnect between the local and global goals.

Re: Overfitting and the strong version of Goodhart’s law

#62

Post by bombcar »

kwhitefoot wrote: post #51
lqet wrote: post #33

I recommend reading "The Collapse of Complex Societies" by Joseph Tainter [0]. Complex societies tend to address problems by adding more and more rules and regulations, simply because they have always done so and it has been successful in the past. More importantly, though, it is typically the only tool they have. Essentially these societies are increasingly overfitting their legislature to narrow special cases until…

This is not new, merely ignored: "What's the origin of the phrase 'Hard cases make bad law'? ... 'Hard cases make bad law' isn't so much a universal proverb as a legal adage. It came to light in a comment made by Judge Robert Rolf in the case of Winterbottom v Wright in 1842: This is one of those unfortunate cases...in which, it is, no doubt, a hardship upon the plaintiff to be without a remedy but by that considerat…

Something like this could (should?) perhaps be handled by governmental "remedy" service, a small insurance/tax against everyone that the judges can use to award remedy without forcing the defendant to pay.

Re: Overfitting and the strong version of Goodhart’s law

#63

Post by shanusmagnus »

astrobe_ wrote: post #22
brookst wrote: post #4

This keeps getting rediscovered in new domains. JIT was the savior of manufacturing, until people learned that a single traffic jam that delayed a single delivery could create costs far in excess of the inventory savings. Optimizations are critical, everywhere. And measuring optimizations is important because it is is easier and cheaper and earlier than measuring end results. Measuring days of inventory, or dollars i…

> This keeps getting rediscovered in new domains. I'm not sure there's a need for a "strong version" of Goodhart's law, or I fail to understand the distinction the author is trying to make. Goodhart is "so true" precisely because it warns about the fact that the measure is often not the goal itself, just like "the map is not the territory", and is indeed an ap proxi mation of the goal. One finds a similar problem wit…

See footnote 4. I would quote it except it's pretty long and full of links, and an edit, which HN "formatting" would make confusing.

Re: Overfitting and the strong version of Goodhart’s law

#64

Post by mannykannot »

bombcar wrote: post #62
kwhitefoot wrote: post #51

Earlier quoted context omitted.

This is not new, merely ignored: "What's the origin of the phrase 'Hard cases make bad law'? ... 'Hard cases make bad law' isn't so much a universal proverb as a legal adage. It came to light in a comment made by Judge Robert Rolf in the case of Winterbottom v Wright in 1842: This is one of those unfortunate cases...in which, it is, no doubt, a hardship upon the plaintiff to be without a remedy but by that considerat…

Something like this could (should?) perhaps be handled by governmental "remedy" service, a small insurance/tax against everyone that the judges can use to award remedy without forcing the defendant to pay.

We have already moved on, and a case like this would no longer be regarded as unusual, let alone difficult. In the process, something like what you propose arose, in the form of liability insurance. The system works reasonably well, in that there are a lot fewer hard cases without creating a crippling burden in the average case.

If anything, this case, together with the way things have changed since then, demonstrates that there is often a good deal of subjectivity in what constitutes a 'hard' case.

Re: Overfitting and the strong version of Goodhart’s law

#65

Post by ramshanker »

lqet wrote: post #33

I recommend reading "The Collapse of Complex Societies" by Joseph Tainter [0]. Complex societies tend to address problems by adding more and more rules and regulations, simply because they have always done so and it has been successful in the past. More importantly, though, it is typically the only tool they have. Essentially these societies are increasingly overfitting their legislature to narrow special cases until…

Meanwhile current Indian government repealed 100s of Laws / Acts.

https://www.indiatoday.in/mail-today/story/narendra-modi-law...

Re: Overfitting and the strong version of Goodhart’s law

#66

Post by marcosdumay »

brookst wrote: post #4

This keeps getting rediscovered in new domains. JIT was the savior of manufacturing, until people learned that a single traffic jam that delayed a single delivery could create costs far in excess of the inventory savings. Optimizations are critical, everywhere. And measuring optimizations is important because it is is easier and cheaper and earlier than measuring end results. Measuring days of inventory, or dollars i…

I highly recommend playing the beer game with different inventory sizes, and looking at the results.

Inventory management is not a simple task, and can not be generalized like this. JIT was adopted because it reduced the number of supply chain disasters, not despite increasing it like you claim. But, of course, that reduction wasn't homogeneous and not every single place saw a gain.

Re: Overfitting and the strong version of Goodhart’s law

#67

Post by marcosdumay »

jeffreygoesto wrote: post #31
kqr wrote: post #29

Earlier quoted context omitted.

Yes, but that it literally what JIT means, though. It's a mindset of solving assignable-cause variation at its cause rather than papering over it with more inventory.

Literally, "just" in time would mean "not more (material, goods,...) than needed" in time, wouldn't it? A real world analogy to Kanban.

JIT is mostly a name for inter-companies Kanban (the factory version of Kanban, not the software one).

And because of that, yes, the amount of inventory is pretty much arbitrary.

Re: Overfitting and the strong version of Goodhart’s law

#68

Post by jaschasd »

c7b wrote: post #36

Not sure I agree. The problem with overfitting is fitting too closely to the data points at hand, but you might still be measuring the right thing, as discussed in other posts here. The problem with Goodhart's law is, as I've always taken it, closer to the Lucas critique in economics than to the bias-variance trade-off in machine learning. Namely, when it comes to human behavior, structural relations that are very re…

A note that the datapoints you train on are part of the training objective. If you are using different data at test time than you use at training time, then you are measuring the wrong thing during training, the same as if you used a different loss function at training time.

Also -- as you say, feedback loops and non-stationarity make everything more complex, and are ubiquitous in the real world! But in machine learning we also see overfitting phenomena in systems with feedback loops -- e.g. in reinforcement learning or robotics, where the system changes depending on the agent's behavior.

(blog author here)

Re: Overfitting and the strong version of Goodhart’s law

#69

Post by marcosdumay »

smartmic wrote: post #42
visarga wrote: post #30

Earlier quoted context omitted.

> as you increase the number of parameters in the model, your error bars become larger In large neural nets the effect is reversed. The larger the model, the better it generalises, even from the same training data.

> The larger the model, the better it generalises, even from the same training data Do you have some references for this claim? For me, it seems counterintuitive.

It it very counterintuitive. It is also a very common observation that has taken everybody by surprise for almost 2 decades by now. At the beginning, people were very resistant to the idea, even when every experiment confirmed it.

The catch is that you need a huge amount of data to train those.

It also seems to have limits. There has been a few well documented cases where our current huge and very well trained kind of networks got errors there were lower than the rate of mislabeling of the data.

Re: Overfitting and the strong version of Goodhart’s law

#70

Post by wrp »

brookst wrote: post #4

This keeps getting rediscovered in new domains. JIT was the savior of manufacturing, until people learned that a single traffic jam that delayed a single delivery could create costs far in excess of the inventory savings. Optimizations are critical, everywhere. And measuring optimizations is important because it is is easier and cheaper and earlier than measuring end results. Measuring days of inventory, or dollars i…

A major consideration people seem to miss when discussing JIT is that it originated in a country with extremely reliable transportation. I had some chance to observe highly resilient and flexible logistical operations in Japan in the 1990s, and have since wondered just how well JIT in the USA follows that model.
Post reply on HN