Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

421–430 of 1001 posts

Re: GPT-5 is behind schedule

#421

Post by BenFranklin100 »

afavour wrote: post #409
foolfoolz wrote: post #294

Earlier quoted context omitted.

AGI will arrive like self driving cars. it’s not that you will wake up one day and we have it. cars gained auto-braking, parallel parking, cruise control assist. and over a long time you get to something like waymo, which still is location dependent. i think AGI will take decades but sooner will be some special cases that are effectively the same

Your position assumes an answer to OPs question: that yes, LLMs are the path to AGI. But the question still remains, what if they’re not? We can be reasonably confident that the components we’re adding to cars today are progress toward full self driving. But AGI is a conceptual leap beyond an LLM.

To buttress your point, reason and human language are not the same thing. This fact is not fully and widely appreciated as it deserves to be.

Re: GPT-5 is behind schedule

#422

Post by eru »

EGreg wrote: post #340
apsec112 wrote: post #324

Earlier quoted context omitted.

What do you think, if we saw it, would constitute hard evidence of logical thinking or a sense of truth?

We have it, it’s called Cyc But it is far behind the breadth of LLMs

Alas, Cyc is pretty much a useless pipe dream.

Re: GPT-5 is behind schedule

#423

Post by eru »

NBJack wrote: post #321
missedthecue wrote: post #271

Earlier quoted context omitted.

Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.

No. But it won't stop the industry from trying. LLMs have no real sense of truth or hard evidence of logical thinking. Even the latest models still trip up on very basic tasks. I think they can be very entertaining, sure, but not practical for many applications.

> LLMs have no real sense of truth or hard evidence of logical thinking.

Most humans don't have that either, most of the time.

Re: GPT-5 is behind schedule

#424

Post by sooheon »

whoisthemachine wrote: post #138
Animats wrote: post #123

"Orion’s problems signaled to some at OpenAI that the more-is-more strategy, which had driven much of its earlier success, was running out of steam." So LLMs finally hit the wall. For a long time, more data, bigger models, and more compute to drive them worked. But that's apparently not enough any more. Now someone has to have a new idea. There's plenty of money available if someone has one. The current level of LLM…

Unfortunately, the best they can do is "This is my confidence on what someone would say given the prior context".

What someone from the past would have said.

Re: GPT-5 is behind schedule

#425

Post by eru »

arthurcolle wrote: post #391
jazzyjackson wrote: post #376

Earlier quoted context omitted.

If agents can self improve why hasn't gpt4 improved itself into gpt5 yet

Agents can trivially self improve. I'd be happy to show you - contact me at arthur@distributed.systems Why wouldn't you hand me 35 million dollars right now if I can clearly illustrate to you that I have technology you haven't seen? Edge. Maybe you know something I don't, or maybe you just haven't seen it. While loops go hard ;) They don't need to release their internal developments to you to show that they can scale…

Perhaps you should pitch that to a VC?

Re: GPT-5 is behind schedule

#426

Post by wat10000 »

n144q wrote: post #278
ComplexSystems wrote: post #252

Earlier quoted context omitted.

"There is no evidence that LLMs are the roadmap to AGI." - There's plenty of evidence. What do you think the last few years have been all about? Hell, GPT-4 would already have qualified as AGI about a decade ago.

> GPT-4 would already have qualified as AGI about a decade ago. Did you just make that up?

A lot of people held that passing the Turing Test would indicate human-level intelligence. GPT-4 passes.

Re: GPT-5 is behind schedule

#427

Post by eru »

threeseed wrote: post #233
ramesh31 wrote: post #220

Earlier quoted context omitted.

But if the scaling law holds true, more dollars should at some point translate into AGI, which is priceless. We haven't reached the limits yet of that hypothesis.

a) There is evidence e.g. private data deals that we are starting to hit the limitations of what data is available. b) There is no evidence that LLMs are the roadmap to AGI. c) Continued investment hinges on their being a large enough cohort of startups that can leverage LLMs to generate outsized returns. There is no evidence yet this is the case.

> c) Continued investment hinges on their being a large enough cohort of startups that can leverage LLMs to generate outsized returns. There is no evidence yet this is the case.

Why does it have to be startups? And why does it have to be LLMs?

Btw, we might be running out of text data. But there's lots and lots more data you can have (and generate), if you are willing to consider other modalities.

You can also get a bit further with text data by using it for multiple epochs, like we used to do in the past. (But that only really gives you at best an order of magnitude. I read some paper that the returns diminish drastically after four epochs.)

Re: GPT-5 is behind schedule

#428

Post by eru »

aantix wrote: post #288
threeseed wrote: post #274

Earlier quoted context omitted.

We also could just measure the background noise of the universe and produce unlimited data. But just like GPS data it isn't suited for LLMs given that you know it has no relevance what so ever to language.

You’re thinking of language in the strictest of sense. GPS data as it relates to location names, people, cultures, path finding.

What does culture and names and people have to do with the Global Position System?

You are right that we can have lots more data, if you are willing to consider other modalities. But that's not 'GPS'. Unless you are using an idiosyncratic definition of GPS?

Re: GPT-5 is behind schedule

#429

Post by eru »

threeseed wrote: post #274
aantix wrote: post #268

Earlier quoted context omitted.

Have we really hit the wall? Do they use GPS based data? Feels like there’s data all around us. Sure they’ve hit the wall with obvious conversations and blog articles that humans produced, but data is a by product of our environment. Surely there’s more. Tons more.

We also could just measure the background noise of the universe and produce unlimited data. But just like GPS data it isn't suited for LLMs given that you know it has no relevance what so ever to language.

Ignoring the confusion about 'GPS' for a moment: there's lots and lots of other data that could be used for training AI systems.

But, you need to go multi-modal for that; and you need to find data that's somewhat useful, not just random fluctuations like the CMB. So eg you could use YouTube videos, or even just point webcams at the real world. That might be able to give your AI a grounding in everyday physics?

There's also lots of program code you can train your AI on. Not so much the code itself, because compared to the world's total text (that we are running out of), the world's total human written code is relatively small.

But you can generate new code and make it useful for training, by also having the AI predict what happens when you (compile and) run the code. A bit like self-playing for improving AlphaGo.

Re: GPT-5 is behind schedule

#430

Post by ferminaut »

SamPatt wrote: post #415

I'm sure the debate over the definition of AGI is important and will continue for a while, but... I can't care about it anymore. Between Perplexity searching and summarizing, Claude explaining, and qwen (and other tools) coding, I'm already as happy as can be with whatever you want to call this level of intelligence. Just today I used a completely local AI research tool, based on Ollama. It worked great. Maybe it won…

vscode + cline extension + gemini2.0 is pretty awesome. Highly recommend checking out cline. it quickly became one of my favorite coding tools.
Post reply on HN