Measuring "how long" an AI can work for seems bizarre to me.
Its a computer program. What does it even mean that soon it "will be able to work 8 hour days"?
Failing to Understand the Exponential, Again
131–140 of 266 posts
Re: Failing to Understand the Exponential, Again
#132“Guy who personally benefits from AI hype says we aren’t in a bubble” - don’t we have enough of these already ??
(I'm the one who posted the url, not author post)
Julian Schrittwieser (author of this post) has been in AI for a long time, he was in the core team who worked on AlphaGo, AlphaZero and MuZero at DeepMind, you can see him in the AlphaGo movie. While it doesn't make his opinion automatically true, I think it makes it worth considering, especially since he's a technical person, not a CEO trying to raise money
"extrapolating an exponential" seems dubious, but I think the point is more that there is no clear sign of slowing down in models capabilities from the benchmarks, so we can still expect improvements
Re: Failing to Understand the Exponential, Again
#133I'm less concerned about "parity with industry expert" and more concerned about "Error/hallucination rate compared to industry expert".
Without some guarantee of correctness, just posting the # of wins seems vacuous.
Re: Failing to Understand the Exponential, Again
#134Just because something exhibits an exponential growth at one point in time, that doesn’t mean that a particular subject is capable of sustaining exponential growth. Their Covid example is a great counter argument to their point in that covid isn’t still growing exponentially. Where the AI skeptics (or even just pragmatists, like myself) chime in is saying “yeah AI will improve. But LLMs are a limited technology that…
Author here. The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that: - given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continu…
That feels like you're moving the goal posts a bit.
Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound.
E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.
Re: Failing to Understand the Exponential, Again
#135Earlier quoted context omitted.
> Where are they going to get more data? This is a great question, but note that folks were freaking out about this a year or so ago and we seem to be doing fine. We seem to be making progress with some combination of synthetic training datasets on coding/math tasks, textbooks authored by paid experts, and new tokens (plus preference signals) generated by users of the LLM systems. It wouldn’t surprise me if coding/ma…
I am extremely confident that AGI, if it is achievable at all (which is a different argument and one I'm not getting into right now), requires a world model / fact model / whatever terminology you prefer, and is therefore not achievable by models that simply chain words together without having any kind of understanding baked into the model. In other words, LLMs cannot lead to AGI.
Agreed, it surely does require a world-model.
I disagree that generic LLMs plus CoT/reasoning/tool calling (ie the current stack) cannot in principle implement a world model.
I believe LLMs are doing some sort of world modeling and likely are mostly lacking a medium-/long-term memory system in which to store it.
(I wouldn’t be surprised if one or two more architectural overhauls end up occurring before AGI, I also wouldn’t be surprised if these occurred seamlessly with our current trajectory of progress)
Re: Failing to Understand the Exponential, Again
#136Exponential curves don't last for long fortunately, or the universe would have turned into a quark soup. The example of COVID is especially ironic, considering it stopped being a real concern within 3 years of its advent despite the exponential growth in the early years. Those who understand exponentials should also try to understand stock and flow.
Exponentials exist in their environment. Didn't Covid stop because we ran out of people to infect. Of course it can't keep going exponential, because there aren't exponential people to infect. What is this limit on AI? It is technology, energy, something. All these things can be over-come, to keep the exponential going. And of course, systems also break at the exponential. Maybe AI is stopped by the world economy col…
> What is this limit on AI? It is technology, energy, something. All these things can be over-come, to keep the exponential going.
That's kind of begging the question. Obviously if all the limitations on AI can be overcome growth would be exponential. Even the biggest ai skeptic would agree. The question is, will it?
Re: Failing to Understand the Exponential, Again
#137Just because something exhibits an exponential growth at one point in time, that doesn’t mean that a particular subject is capable of sustaining exponential growth. Their Covid example is a great counter argument to their point in that covid isn’t still growing exponentially. Where the AI skeptics (or even just pragmatists, like myself) chime in is saying “yeah AI will improve. But LLMs are a limited technology that…
Author here. The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that: - given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continu…
My point is that the limits of LLMs will be hit long before we they start to take on human capabilities.
The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI.
The human like capabilities are really just smoke and mirrors.
It’s like when people anthropomorphisise their car; “she’s being temperamental today”. Except we know the car is not intelligence and it’s just a mechanical problem. Whereas it’s in the AI tech firms best interest to upsell the human-like characteristics of LLMs because that’s how they get VC money. And as we know, building and running models isn’t cheap.
Re: Failing to Understand the Exponential, Again
#138Earlier quoted context omitted.
Barring fully reversible processes as the basis for technology, you still quickly run into energy and cooling constraints. Even with that, you'd have time or energy density constraints. Unlimited exponentials are clearly unphysical.
Yes, this is an accurate description, and also completely irrelevant to the issue at hand. At the stage of development we are today, no one cares how fast it takes for the exponent to go from eating our galaxy to eating the whole universe, or whether it'll break some energy density constraint before it and leave a gaping zero-point energy hole where our local cluster used to be. It'll stop eventually . What we care a…
Sure, but it reminds us that we are dealing with an S-curve, so we need to ask where the inflection point is. i.e. what are the relevant constraints, and can they reasonably sustain exponential growth for a while still? At least as an outsider, it's not obvious to me whether we won't e.g. run into bandwidth or efficiency constraints that make scaling to larger models infeasible without reimagining the sorts of processors we're using. Perhaps we'll need to shift to analog computers or something to break through cooling problems, and if the machine cannot find the designs for the new paradigm it needs, it can't make those exponential self-improvements (until it matches its current performance within the new paradigm, it gets no benefit from design improvements it makes).
My experience is that "AI can write programs" is only true for the smallest tasks, and anything slightly nontrivial will leave it incapable of even getting started. It doesn't "often makes mistakes or goes in a wrong direction". I've never seen it go anywhere near the right direction for a nontrivial task.
That doesn't mean it won't have a large impact; as an autocomplete these things can be quite useful today. But when we have a more honest look at what it can do now, it's less obvious that we'll hit some kind of singularity before hitting a constraint.
Re: Failing to Understand the Exponential, Again
#139Earlier quoted context omitted.
> Why predict that the (absolute) growth rate is going to keep accelerating past exactly now? By following this logic you should have predicted Moore’s law would halt every year for the last five decades. I hope you see why this is a flawed argument. You prove too much. But I will answer your “why”: plenty of exponential curves exist in reality, and empirically, they can last for a long time. This is just how technol…
> By following this logic you should have predicted Moore’s law would halt every year for the last five decades. I hope you see why this is a flawed argument. You prove too much. I do think it's continually amazing that Moore's law has continued in some capacity for decades. But before trumpeting the age of exponential growth, I'd love to see plenty of examples that aren't named "Moore's law": as it stands, one easy…
Solar panel cost per watt has been dropping exponentially for decades as well...
Partly these are matters of economies of scale - reduction in production costs at scale - and partly it's a matter of increasing human attention leading to steady improvements as the technology itself becomes more ubiquitous.
Re: Failing to Understand the Exponential, Again
#140Earlier quoted context omitted.
Author here. The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that: - given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continu…
That feels like you're moving the goal posts a bit. Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound. E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.
Well, growth has been on this exponential already for 5+ years (for the METR eval), and we are at the point where models are very close to matching human expert capabilities in many domains - only one or two more years of growth would put us well beyond that point.
Personally I think we'll see way more growth than that, but to see profound impacts on our economy you only need to believe the much more conservative assumption of a little extra growth along the same trend.