Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

351–360 of 1001 posts

Re: GPT-5 is behind schedule

#351

Post by missedthecue »

foolfoolz wrote: post #294
missedthecue wrote: post #271

Earlier quoted context omitted.

Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.

AGI will arrive like self driving cars. it’s not that you will wake up one day and we have it. cars gained auto-braking, parallel parking, cruise control assist. and over a long time you get to something like waymo, which still is location dependent. i think AGI will take decades but sooner will be some special cases that are effectively the same

But maybe thses LLMs are like building bigger and bigger engines. It's not getting you closer to the self driving car.

Re: GPT-5 is behind schedule

#352

Post by phillipharris »

Xcelerate wrote: post #58
A_D_E_P_T wrote: post #2

Counterpoint: o1-Pro is insanely good -- subjectively, it's as far above GPT4 as GPT4 was above 3. It's almost too good. Use it properly for an extended period of time, and one begins to worry about the future of one's children and the utility of their schooling. o3, by all accounts, is better still. Seems to me that things are progressing quickly enough.

I had a 30 min argument with o1-pro where it was convinced it had solved the halting problem. Tried to gaslight me into thinking I just didn’t understand the subtlety of the argument. But it’s susceptible to appeal to authority and when I started quoting snippets of textbooks and mathoverflow it finally relented and claimed there had been a “misunderstanding”. It really does argue like a human though now...

This sounds fun to read, can you share the transcript?

Re: GPT-5 is behind schedule

#353

Post by icpmacdo »

arthurcolle wrote: post #337
arthurcolle wrote: post #329

Earlier quoted context omitted.

LLMs are a key piece of understanding that token sequences can trigger actions in the real world. AGI is here. You can trivially spin up a computer using agent to self improve itself to being a competent office worker

Tokens don't need to be text either, you can move to higher level "take_action" semantics where "stream back 1 character to session#117" as every single function call. Training cheap models that can do things in the real world is going to change a huge amount of present capabilities over the next 10 years

can you share learning resources on this topic

Re: GPT-5 is behind schedule

#354

Post by gerdesj »

bloodyplonker22 wrote: post #334
t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

I am working at an AI company that is not OpenAI. We have found ways to modularize training so we can test on narrower sets before training is "completely done". That said, I am sure there are plenty of ways others are innovating to solve the long training time problem.

Perhaps the real issue is that learning takes time and that there may not be a shortcut. I'll grant you that argument's analogue was complete wank when comparing say the horse and cart to a modern car.

However, we are not comparing cars to horses but computers to a human.

I do want "AI" to work. I am not a luddite. The current efforts that I've tried are not very good. On the surface they offer a lot but very quickly the lustre comes off very quickly.

(1) How often do you find yourself arguing with someone about a "fact"? Your fact may be fiction for someone else.

(2) LLMs cannot reason

A next token guesser does not think. I wish you all the best. Rome was not burned down within a day!

I can sit down with you and discuss ideas about what constitutes truth and cobblers (rubbish/false). I have indicated via parenthesis (brackets in en_GB) another way to describe something and you will probably get that but I doubt that your programme will.

Re: GPT-5 is behind schedule

#356

Post by anticorporate »

dgfitz wrote: post #284
anothernewdude wrote: post #263

Earlier quoted context omitted.

[flagged]

I wonder how Russian and North Korean citizens would feel about a capitalist, representative democracy?

I think they'd have thing or two to say about living under the rule of wealthy elites. We'd do well to listen to them.

Re: GPT-5 is behind schedule

#357

Post by icpmacdo »

bloodyplonker22 wrote: post #334
t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

I am working at an AI company that is not OpenAI. We have found ways to modularize training so we can test on narrower sets before training is "completely done". That said, I am sure there are plenty of ways others are innovating to solve the long training time problem.

This is literally just the scaling laws, "Scaling laws predict the loss of a target machine learning model by extrapolating from easier-to-train models with fewer parameters or smaller training sets. This provides an efficient way for practitioners and researchers alike to compare pretraining decisions involving optimizers, datasets, and model architectures"

https://arxiv.org/html/2410.11840v1#:~:text=Scaling%20laws%2....

Re: GPT-5 is behind schedule

#358

Post by icpmacdo »

overgard wrote: post #301
simonw wrote: post #113

"OpenAI’s is called GPT-4, the fourth LLM the company has developed since its 2015 founding." - that sentence doesn't fill me with confidence in the quality of the rest of the article, sadly.

The article definitely has issues, but to me what's relevant is where it's published. The smart money and experts without a vested interest have been well aware LLMs are an expensive dead for over a year and have been saying as much (Gary Marcus for instance). That this is starting to enter mainstream consciousness is what's newsworthy.

Gary Marcus is continuously lambasted and not taken seriously

Re: GPT-5 is behind schedule

#359

Post by Workaccount2 »

wruza wrote: post #293
dkobia wrote: post #232

Earlier quoted context omitted.

AGI is the Sisyphean task of our age. We’ll push this boulder up the mountain because we have to, even if it kills us.

Says who? And more importantly, is this the boulder? All I (and many others here) see is that people engage others to sponsor pushing some boulder, screaming promises which aren’t even that consistent with intermediate results that come out. This particular boulder may be on a wrong mountain, and likely is. It all feels like doubling down on astrology because good telescopes aren’t there yet. I’m pretty sure that whe…

The buzzkill when you fire up the latest most powerful model only for it to tell you that peanut is not typically found in peanut butter and jelly sandwiches.

Re: GPT-5 is behind schedule

#360

Post by mrbungie »

famouswaffles wrote: post #306
mrbungie wrote: post #236

Earlier quoted context omitted.

It is still not economical: in Arc at least 20 usd for task vs ~3 usd for a human (avg mturker) for the same perf.

Not necessarily. And this is the problem with ARC that people seem to forget. - It's just a suite of visual puzzles. It's not like say GSM8K where proficiency in it gives some indication on Math proficiency in general. - It's specifically a suite of puzzles that LLMs have shown particular difficulty in. Basically how much compute it takes to handle a task in this benchmark does not correlate with how much it will tak…

If the benchmark is not representative of normal usage* then the benchmark and the plot being shown are not useful at all from a user/business perspective and the focus on the breakthrough scores of o3-low and o3-high in ARC-AGI would be highly misleading. And also the "representative" point is really moot from the discussion perspective (i.e. saying o3 stomps benchmarks, but the benchmarks aren't representative).

*I don't think that is the case as you can at least make relative conclusions (i.e. o3 vs o1 series, o3-low is 4x to 20x the cost for ~3x the perf). Even if it is pure marketing they expect people to draw conclusions using the perf/cost plot from Arc.

PS: I know there are more benchmarks like SWE-Bench and Frontier Math, but this is the only one showing data about o3-low/high costs without considering the CodeForces plot that includes o3-mini (that one does look interesting, though right now is vaporware) but does not separate between compute scale modes.

Post reply on HN