The Economics of Recursive Self-Improvement [pdf] • Hacker News Board

Re: The Economics of Recursive Self-Improvement [pdf]

#101

Post by podocarp » Thu, Jul 16, 2026, 2:09 AM UTC

dinfinity wrote: post #70

jamesrcole wrote: post #61

Earlier quoted context omitted.

> As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. This isn't some foregone conclusion. It completely depends on the rate at which the intelligence and abilities of the AI increases. If that rate was high enough, then the harder and harder problems would become easier and easier for it.

Actually, evolution seems to show the opposite : The rate of advancement has only sped up, with billions of years between significant changes going to millions, to thousands, to tens and arguably to mere years now. Having said that, we're probably looking at an S-curve with the physical limits of reality getting in the way in the end.

The more you evolve I guess there's more surface area to evolve on. Similarly cavemen could only do so much. But today you can take a degree in physics, art, history, etc. and it's probably impossible for one person to know all that ever was and is. But it's possible for a caveman shaman to know everything the tribe knew up to that point. Probably. So while it's harder to make progress there's also a lot more places you can progress.

Re: The Economics of Recursive Self-Improvement [pdf]

#102

Post by red75prime » Thu, Jul 16, 2026, 7:38 AM UTC

ben_w wrote: post #96

godelski wrote: post #88

Earlier quoted context omitted.

> solving multiple previously unsolved Erdős problems. So what? Machines have solved tons of unsolved problems in mathematics. That's not a proof of intelligence. If you brute force a solution, we congratulate you on your effort. If you stumble into a solution, we congratulate you for being lucky (if we can distinguish) If you find a unique solution that no one else imagined, we congratulate you on your intelligence.…

> If you find a unique solution that no one else imagined, we congratulate you on your intelligence. If that's your definition, bad luck: that is exactly what the AI did. There are other definitions where AI fail, my example of which would be "how many examples did it take to learn the basics?", ML is as thick as plankton by this definition. > When we're talking about intelligence you can't distill it to "getting the…

> ML is as thick as plankton by this definition

...during autoregressive pretraining ([2]). A model pretrained on texts with 1930 data cutoff can solve a few programming problems when given a few examples. Its success rate is understandably much worse.

[1] https://talkie-lm.com/introducing-talkie

[2] That is when a model starts from a blank state that can be described in a few kilobytes.

Re: The Economics of Recursive Self-Improvement [pdf]

#103

Post by ben_w » Thu, Jul 16, 2026, 9:50 AM UTC

red75prime wrote: post #102

ben_w wrote: post #96

Earlier quoted context omitted.

> If you find a unique solution that no one else imagined, we congratulate you on your intelligence. If that's your definition, bad luck: that is exactly what the AI did. There are other definitions where AI fail, my example of which would be "how many examples did it take to learn the basics?", ML is as thick as plankton by this definition. > When we're talking about intelligence you can't distill it to "getting the…

> ML is as thick as plankton by this definition ...during autoregressive pretraining ([2]). A model pretrained on texts with 1930 data cutoff can solve a few programming problems when given a few examples. Its success rate is understandably much worse. [1] https://talkie-lm.com/introducing-talkie [2] That is when a model starts from a blank state that can be described in a few kilobytes.

Careful, looks like you're shifting definitions.

Remember, I was saying "by this definition", where "this" isn't about capabilities, but about effort needed to get there, which in the case of your link is "260B tokens of historical pre-1931 English text".

A human who read 260e9 tokens will take something like 1900 years of 24/7 reading to get that far, while a more realistic human (though one who still reads a lot) would take 260e9/((50000/0.75)*365) ~= 10,685 years, and that still only gets you something "interesting" rather than "competent".

Don't get me wrong, even merely HumanEval pass@100 ~= 0.04 (eyeballing that chart) shows the model has clearly learned something and isn't just randomly throwing things at the wall. All I'm saying is this took a huge effort to get even that far (and, implicitly, that this is a reasonable argument to use if you want to say they're "not intelligent").