Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Failing to Understand the Exponential, Again

julian.ac

Post reply on HN
Print view View original

161–170 of 266 posts

Re: Failing to Understand the Exponential, Again

#161

Post by adammarples »

Inufu wrote: post #111
hnlmorg wrote: post #30

Just because something exhibits an exponential growth at one point in time, that doesn’t mean that a particular subject is capable of sustaining exponential growth. Their Covid example is a great counter argument to their point in that covid isn’t still growing exponentially. Where the AI skeptics (or even just pragmatists, like myself) chime in is saying “yeah AI will improve. But LLMs are a limited technology that…

Author here. The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that: - given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continu…

The most common part of the S-curve by far is the flat bit before and the flat bit after. We just don't graph it because it's boring. Besides which there is no reason at all to assume that this process will follow that shape. Seems like guesswork backed up by hand waving.

Re: Failing to Understand the Exponential, Again

#162

Post by jijijijij »

Nevermark wrote: post #58
crazygringo wrote: post #7

> Given consistent trends of exponential performance improvements over many years and across many industries, it would be extremely surprising if these improvements suddenly stopped. I'm sure people were saying that about commercial airline speeds in the 1970's too. But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors. With LLM's at the moment, the limiting fac…

Progress in information systems cannot be compared to progress in physical systems. For starters, physical systems compete for limited resources and labor. For another, progress in software vastly reduces the cost of improved designs. Whereas progress in physical systems can enable but still increase the cost of improved designs. Finally, the underlying substrate of software is digital hardware, which has been improv…

That's fallacious reasoning, you are extrapolating from survivorship bias. A lot of technologies, genes, or species have failed along the way. You are also subjectively attributing progression as improvements, which is problematic as well, if you speak about general trends. Evolution selects for adaptation not innovation. We use the theory of evolution to explain the emergence of complexity, but that's not the sole direction and there are many examples where species evolved towards simplicity (again).

Resource expense alone could be the end of AI. You may look up historic island populations, where technological demands (e.g. timber) usually led to extinction by resource exhaustion and consequent ecosystem collapse (e.g. deforestation leading to soil erosion).

Re: Failing to Understand the Exponential, Again

#163

Post by oofbey »

danlitt wrote: post #142

I am constantly astonished that articles like this even pass the smell test. It is not rational to predict exponential growth just because you've seen exponential growth before! Incidentally, that is not what people did during COVID, they predicted exponential growth for reasons . Specific, articulable reasons, that consisted of more than just "look, like go up. line go up more?". Incidentally, the benchmarks quoted…

"It’s Difficult to Make Predictions, Especially About the Future" - Yogi Berra. It's funny because it's true.

So if you want to try to do this difficult task, because say there's billions of dollars and millions of people's livelihoods on the line, how do you do it? Gather a bunch of data, and see if there's some trend? Then maybe it makes sense to extrapolate. Seems pretty reasonable to me. Definitely passes the sniff test. Not sure why you think "line go up more" is such a stupid concept.

Re: Failing to Understand the Exponential, Again

#164

Post by Nevermark »

breuleux wrote: post #113
Nevermark wrote: post #58

Earlier quoted context omitted.

Progress in information systems cannot be compared to progress in physical systems. For starters, physical systems compete for limited resources and labor. For another, progress in software vastly reduces the cost of improved designs. Whereas progress in physical systems can enable but still increase the cost of improved designs. Finally, the underlying substrate of software is digital hardware, which has been improv…

> Looking at information systems as far back as the first coordination of differentiating cells to human civilization is one of exponential improvement. Under what metric? Most of the things you mention don't have numerical values to plot on a curve. It's a vibe exponential, at best. Life and humans have become better and better at extracting available resources and energy, but there's a clear limit to that (100%) an…

> It's a vibe exponential, at best.

I am a little stunned you think so.

Life has been on Earth about 3.5-3.8 billion years.

Break that into 0.5-0.8, 1 billion, 1 billion, 1 billion "quarters", and you will find exponential increases in evolutions rate of change and production of diversity across them by many many objective measures.

Now break up the last 1 billion into 100 million year segments. Again exponential.

Then break up the last 100 million into segments. Again.

Then the last 10 million years into segments, and watch humans progress.

The last million, in 100k year segments, watch modern humans appear.

the last 10k years into segments, watch agriculture, civilizations, technology, writing ...

The last 1000 years, incredible aggregation of technology, math, and the appearance of formal science

last 100 years, gets crazy. Information systems appear in labs, then become ubiquitous.

last 10 years, major changes, AI starts having mainstream impact

last 1 year - even the basic improvements to AI models in the last 12 months are an unprecedented level of change, per time, looking back.

I am not sure how any of could appear "vibe", given any historical and situational awareness.

This progression is universally recognized. Aside from creationists and similar contingents.

Re: Failing to Understand the Exponential, Again

#165

Post by xenobeb »

senordevnyc wrote: post #119
credit_guy wrote: post #62

I think the author of this blog is not a heavy user of AI in real life. If you are, you know there things AI is very good at, and thing AI is bad at. AI may see exponential improvements in some aspects, but not in other aspects. In the end, those "laggard" aspects of AI will put a ceiling on its real-world performance. I use AI in my coding for many hours each day. AI is great. But AI will not replace me in 2026 or i…

How much better is AI-assisted coding than it was in September 2023?

This all depends on how you define "better".

I am an amateur programmer and tried to port a python 2.7 library to python 3 with GPT5 a few weeks ago.

After a few tries, I realized both myself and the model missed that a large part of the library is based on another library that was never ported to 3 either.

That doesn't stop GPT5 from trying to write the code as best it can with a library that doesn't exist for python 3.

That is the part we have made absolutely no progress on.

Of course, it can do a much better react crud app than in Sept 2023.

In one sense, LLMs are so amazing and impressive and quite fugazi in another sense.

Re: Failing to Understand the Exponential, Again

#166

Post by travisjungroth »

Nevermark wrote: post #153
travisjungroth wrote: post #74

Earlier quoted context omitted.

> Progress in information systems cannot be compared to progress in physical systems. > For starters, physical systems compete for limited resources and labor. > Finally, the underlying substrate of software is digital hardware… See how these are related?

By physical systems, I meant systems whose purpose is to do physical work. Mechanical things. Gears. Struts. Computer hardware is an information system. You are correct that it is has a physical component. But its power comes from its organization (information) not its mass, weight, etc. Transistors get more powerful, not less, when made from less matter. Information systems move from substrate to more efficient subs…

They still depend on physical resources and labor. They’re made by people and machines. There’s never been more resources going into information systems than right now, and AI accelerated that greatly. Think of all the server farms being built next to power plants.

Re: Failing to Understand the Exponential, Again

#167

Post by Nevermark »

jijijijij wrote: post #162
Nevermark wrote: post #58

Earlier quoted context omitted.

Progress in information systems cannot be compared to progress in physical systems. For starters, physical systems compete for limited resources and labor. For another, progress in software vastly reduces the cost of improved designs. Whereas progress in physical systems can enable but still increase the cost of improved designs. Finally, the underlying substrate of software is digital hardware, which has been improv…

That's fallacious reasoning, you are extrapolating from survivorship bias. A lot of technologies, genes, or species have failed along the way. You are also subjectively attributing progression as improvements, which is problematic as well, if you speak about general trends. Evolution selects for adaptation not innovation. We use the theory of evolution to explain the emergence of complexity, but that's not the sole d…

See replies to sibling comments.

Re: Failing to Understand the Exponential, Again

#168

Post by xenobeb »

slaucon wrote: post #77

I feel like there should be some take away from the fact that we have to come up with new and interesting metrics like “Length of a Task That Can Be Automated” in order to point out that exponential growth is still happening. Fwiw, it does seem like a good metric, but it also feels like you can often find some metric that’s improving exponentially even when the base function is leveling out.

From Nassim Taleb

"Unless you have confidence in the ruler’s reliability, if you use a ruler to measure a table you may also be using the table to measure the ruler."

Seems like that is exactly what we are doing.

Re: Failing to Understand the Exponential, Again

#169

Post by bwfan123 »

joshdavham wrote: post #144

> - Models will be able to autonomously work for full days (8 working hours) by mid-2026. > - At least one model will match the performance of human experts across many industries before the end of 2026. > - By the end of 2027, models will frequently outperform experts on many tasks. I’ve seen a lot of people make predictions like this and it will be interesting to see how this turns out. But my question is, what sho…

> Should the person lose credibility for future predictions and we no longer take them seriously

If this were the case, almost every sell-side analyst should have been blacklisted by now. Its more about entertainment than facts - sort of like astrology.

Re: Failing to Understand the Exponential, Again

#170

Post by Nevermark »

travisjungroth wrote: post #166
Nevermark wrote: post #153

Earlier quoted context omitted.

By physical systems, I meant systems whose purpose is to do physical work. Mechanical things. Gears. Struts. Computer hardware is an information system. You are correct that it is has a physical component. But its power comes from its organization (information) not its mass, weight, etc. Transistors get more powerful, not less, when made from less matter. Information systems move from substrate to more efficient subs…

They still depend on physical resources and labor. They’re made by people and machines. There’s never been more resources going into information systems than right now, and AI accelerated that greatly. Think of all the server farms being built next to power plants.

Yes. Of course.

All information has a substrate at any given time.

But the amount of computation per resource drops because computation is not something tied to any unit matter. Nor any particular substrate.

It is not the same as a steam engine, which can only be made so efficient.

The amount of both matter and labor per quantity of computing power is dropping exponentially. Right?

See a sibling reply on the physical limits of computation. We are several singularities away from any hard limit.

Evidence: History of industrialization vs. history of computing. Fundamental physics.

Post reply on HN