Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

591–600 of 1001 posts

Re: GPT-5 is behind schedule

#591

Post by rwyinuse »

concerndc1tizen wrote: post #571
ciconia wrote: post #551

Earlier quoted context omitted.

When you think about it it's astounding how much energy this technology consumes versus a human brain which runs at ~20W [1]. [1] https://hypertextbook.com/facts/2001/JacquelineLing.shtml

20w for 20 years to answer questions slowly and error-prone at the level of a 30B model. An additional 10 years with highly trained supervision and the brain might start contributing original work.

And yet that 20w brain can make me a sandwich and bring it to me, while state of the art AI models will fail that task.

Until we get major advances in robotics and models designed to control them, true AGI will be nowhere near.

Re: GPT-5 is behind schedule

#592

Post by palata »

foolfoolz wrote: post #294
missedthecue wrote: post #271

Earlier quoted context omitted.

Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.

AGI will arrive like self driving cars. it’s not that you will wake up one day and we have it. cars gained auto-braking, parallel parking, cruise control assist. and over a long time you get to something like waymo, which still is location dependent. i think AGI will take decades but sooner will be some special cases that are effectively the same

What makes you believe that AGI will happen, as opposed to all the beliefs that other people have had in history? Tons of people have "predicted" the next evolution of technology, and most of the time it ends up not happening, right?

Re: GPT-5 is behind schedule

#593

Post by wokwokwok »

NitpickLawyer wrote: post #487
wokwokwok wrote: post #460

Earlier quoted context omitted.

We already have agentic systems; they're not particularly impressive (1). There's no specific reason to expect them to get better. Things that will shift the status quo are: MCST-LLMs (like with ARC-AGI) and Much Bigger LLMs (like GPT-5, if they ever turn up) or some completely novel architecture. [1] - It's provable; if just chaining LLMs are a particular size into agentic systems could scale indefinitely, then you…

> It's provable; if just chaining LLMs are a particular size into agentic systems could scale indefinitely, then you could use a 1-param LLM and get AGI. You can't. QED. Perhaps I missunderstand your reply, but that has not been my experience at all. There are 3 types of "agentic" behaviour that has worked for a while for me, and I don't know how else it would work without "agents": 1. Task decomposition - this was m…

I didn’t say they don’t work, I said there is an upper bound on the function they provide.

If a discrete system can be composed of multiple LLMs the upper bound on the function they provide is by the function of the LLM, not the number of agents.

Ie. We have agentic systems.

Saying “wait till you see those agentic systems!” is like saying “wait til you see those c++ programs!”

Yes. I see them. Mmm. Ok. I don’t think I’m going to be surprised by seeing them doing exactly the same things in a year.

The impressive part in a year will the non agentic part of things.

Ie. Explicitly; if the underlying LLMs dont get any better, there is no reason to expect the system built out of them to get any better.

If that was untrue, you would expect to be able to build agentic systems out of much smaller LLMs, but that overwhelmingly doesn’t work.

Re: GPT-5 is behind schedule

#594

Post by palata »

> Altman told students in a talk at Stanford University that OpenAI could say with “a high degree of scientific certainty” that GPT-5 would be much smarter than the current model.

I love that quote. "I can tell you with a high degree of scientific certainty that my product will be badass... for some definition of badass, please don't ask".

I, too, could say with a high degree of scientific certainty that while putting those words together sounds good, it makes no sense.

Re: GPT-5 is behind schedule

#595

Post by badgersnake »

SamPatt wrote: post #415

I'm sure the debate over the definition of AGI is important and will continue for a while, but... I can't care about it anymore. Between Perplexity searching and summarizing, Claude explaining, and qwen (and other tools) coding, I'm already as happy as can be with whatever you want to call this level of intelligence. Just today I used a completely local AI research tool, based on Ollama. It worked great. Maybe it won…

They’re garbage, they will always be garbage. Changing a 4 to a 5 will not make it not garbage.

The whole sector is a hype bubble artificially inflating stock prices.

Re: GPT-5 is behind schedule

#596

Post by fsloth »

sollewitt wrote: post #493
atonse wrote: post #457

Earlier quoted context omitted.

Same here. The ability to “talk to an expert” about any topic I’m curious about and ask very specific questions has been invaluable to me. It reminds me of being a kid and asking my grandpa a million questions, like how light bulbs worked, or what was inside his radio, or how do we have day and night. And before anyone talks about accuracy or hallucinations, these conversations usually are treated as starting off poi…

LLMs suffer from the "Igon Value Problem" https://rationalwiki.org/wiki/Igon_Value_Problem Similar to reading a pop sci book, you're getting an entertainment from a thing with no actual understanding of the source material rather than an education.

Yes! But now you get a specific pop sci book _in any subject you want to learn about_ and _you can ask the book about comparisons_ (e.g. how were Roman and Parthian legal systems similar?). This at leas gives you a bunch of keywords to go silly in wikipedia and publications (sci-hub! Cough! Sci-hub!)

Re: GPT-5 is behind schedule

#597

Post by globular-toast »

phist_mcgee wrote: post #122
globular-toast wrote: post #45

Earlier quoted context omitted.

The ones who use it extensively are the same that used to hit up stackoverflow as the first port of call for every trivial problem that came their way. They're not really engineers, they just want to get stuff done.

No ad hominem please.

Hmm... calling people "not engineers" is considered an attack now? I'm afraid this is actually revealing your own bias towards engineers. I never said engineers were superior or that we'd be better off with a whole world full of them.

Re: GPT-5 is behind schedule

#598

Post by Y_Y »

sebzim4500 wrote: post #234
Y_Y wrote: post #222

Earlier quoted context omitted.

> You can't hill climb tokens, but you can climb manifolds. Could you explain this a bit please?

I imagine he means that when you reason in latent space the final answer is a smooth function of the parameters, which means you can use gradient descent to directly optimize the model to produce a desired final output without knowing the correct reasoning steps to get there. When you reason in token space (like everyone is doing now) you are executing nonlinear functions when you sample after each token, so you have…

I think there's a subtlety here about what makes (e.g. English) tokens different to points in latent space. Everything is still differentiable (at least in the ML sense) until you do random sampling. Even then you can exclude the sampling when calculating the gradient (or is this equivalent to the "manifold"?).

I don't see a priori why it would be better or worse to reason with the "superposition" of arguments in the pre-sampling phase rather than concrete realizations of those arguments found only after choosing the token. It may well be a contingent rather than necessary fact.

Re: GPT-5 is behind schedule

#599

Post by soheil »

ciconia wrote: post #551
t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

When you think about it it's astounding how much energy this technology consumes versus a human brain which runs at ~20W [1]. [1] https://hypertextbook.com/facts/2001/JacquelineLing.shtml

A human brain has been in continuous training for hundreds of thousands of years consuming slightly more than 20 watts.

Re: GPT-5 is behind schedule

#600

Post by soheil »

t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

It's like saying bacteria reproduction is way faster than humans so that's where we should be looking for the next breakthroughs.
Post reply on HN