Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: GPT-5 is behind schedule

#271

Post by missedthecue »

dkobia wrote: post #232
t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

AGI is the Sisyphean task of our age. We’ll push this boulder up the mountain because we have to, even if it kills us.

Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.

Re: GPT-5 is behind schedule

#272

Post by [deleted] »

fny wrote: post #247
t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

O3 is not a smaller model. It's an iterative GPT of sorts with the magic dust of reinforcement learning.

[deleted]

Re: GPT-5 is behind schedule

#274

Post by threeseed »

aantix wrote: post #268
threeseed wrote: post #233

Earlier quoted context omitted.

a) There is evidence e.g. private data deals that we are starting to hit the limitations of what data is available. b) There is no evidence that LLMs are the roadmap to AGI. c) Continued investment hinges on their being a large enough cohort of startups that can leverage LLMs to generate outsized returns. There is no evidence yet this is the case.

Have we really hit the wall? Do they use GPS based data? Feels like there’s data all around us. Sure they’ve hit the wall with obvious conversations and blog articles that humans produced, but data is a by product of our environment. Surely there’s more. Tons more.

We also could just measure the background noise of the universe and produce unlimited data.

But just like GPS data it isn't suited for LLMs given that you know it has no relevance what so ever to language.

Re: GPT-5 is behind schedule

#275

Post by thrwthsnw »

threeseed wrote: post #233
ramesh31 wrote: post #220

Earlier quoted context omitted.

But if the scaling law holds true, more dollars should at some point translate into AGI, which is priceless. We haven't reached the limits yet of that hypothesis.

a) There is evidence e.g. private data deals that we are starting to hit the limitations of what data is available. b) There is no evidence that LLMs are the roadmap to AGI. c) Continued investment hinges on their being a large enough cohort of startups that can leverage LLMs to generate outsized returns. There is no evidence yet this is the case.

Private data is 90% garbage too

Re: GPT-5 is behind schedule

#276

Post by maxrmk »

simonw wrote: post #113

"OpenAI’s is called GPT-4, the fourth LLM the company has developed since its 2015 founding." - that sentence doesn't fill me with confidence in the quality of the rest of the article, sadly.

I was wondering about this one too...

> At best, they say, Orion performs better than OpenAI’s current offerings, but hasn’t advanced enough to justify the enormous cost of keeping the new model running.

wdym "keep it running"?

Re: GPT-5 is behind schedule

#277

Post by jprete »

anothernewdude wrote: post #263
idiotsecant wrote: post #257

Earlier quoted context omitted.

And when we get it there, it kills us.

[flagged]

I think you're both right and wrong. You're right that capitalism has become a paperclip machine, but capitalism also wants AI so it can cheaply and at scale replace the human components of the machine with something that has more work capacity for fewer demands.

Re: GPT-5 is behind schedule

#278

Post by n144q »

ComplexSystems wrote: post #252
threeseed wrote: post #233

Earlier quoted context omitted.

a) There is evidence e.g. private data deals that we are starting to hit the limitations of what data is available. b) There is no evidence that LLMs are the roadmap to AGI. c) Continued investment hinges on their being a large enough cohort of startups that can leverage LLMs to generate outsized returns. There is no evidence yet this is the case.

"There is no evidence that LLMs are the roadmap to AGI." - There's plenty of evidence. What do you think the last few years have been all about? Hell, GPT-4 would already have qualified as AGI about a decade ago.

> GPT-4 would already have qualified as AGI about a decade ago.

Did you just make that up?

Re: GPT-5 is behind schedule

#279

Post by thrwthsnw »

coldtea wrote: post #267
ComplexSystems wrote: post #252

Earlier quoted context omitted.

"There is no evidence that LLMs are the roadmap to AGI." - There's plenty of evidence. What do you think the last few years have been all about? Hell, GPT-4 would already have qualified as AGI about a decade ago.

> What do you think the last few years have been all about? Next token language-based predictors with no more intelligence than brute force GIGO which parrot existing human intelligence captured as text/audio and fed in the form of input data. 4o agrees: "What you are describing is a language model or next-token predictor that operates solely as a computational system without inherent intelligence or understanding. T…

Everything you said is parroting data you’ve trained on, two thirds of it is actual copy paste

Re: GPT-5 is behind schedule

#280

Post by User23 »

xanderlewis wrote: post #250
plufz wrote: post #242

Earlier quoted context omitted.

Technically…? Does anyone here believe that the EU and Europe is the same thing? Would you find it weird if someone said that a Norwegian company was in Europe?

Many people certainly seem to! And it annoys me. I wasn’t talking about the EU, though. I was just commenting on the fact that in the UK, ‘Europe’ generally means ‘continental Europe’. > Would you find it weird if someone said that a Norwegian company was in Europe? I’d find it weird if a European did. But from Americans it’s to be expected.

If Norway isn’t in Europe where is it? Asia?
Post reply on HN