GPT-5 is behind schedule • Hacker News Board

Re: GPT-5 is behind schedule

#411

Post by Workaccount2 » Mon, Dec 23, 2024, 3:31 AM UTC

Yizahi wrote: post #165

GPT-5 is not behind schedule. GPT-5 is called GPT-4o and it has been already released half a year ago. It was not revolutionary enough to be called 5, and prophet saint Altman was probably afraid to release new gen not exponentially improving, so it was rebranded in the last moment. It's speculation of course, but it is kinda obvious speculation.

Not really, 4o was purpose built to be a light weight 4. Remember that 4o was also when GPT-4 became available to everyone. Before that ou had to be premium to use GPT-4, and got limited inquiries.

4o was all about compute optimization.

Re: GPT-5 is behind schedule

#412

Post by [deleted] » Mon, Dec 23, 2024, 3:33 AM UTC

razodactyl wrote: post #410

Two thoughts: 1. Even if LLM architecture doesn't work out, it's wise to remember that it's the quality of training data which is a deciding factor. A pivot is easily doable since this is a constant and disconnected from the technology itself. 2. This is clearly a moonshot project but it still feels wasteful especially with context of previous iterations of models and their shortcomings-it feels like the tremendous a…

[deleted]

Re: GPT-5 is behind schedule

#413

Post by eru » Mon, Dec 23, 2024, 3:36 AM UTC

marcus_holmes wrote: post #333

danpalmer wrote: post #320

Earlier quoted context omitted.

Interesting idea. The concept of The Singularity would seem to go against this, but I do feel that seems unlikely and that a gradual transition is more likely. However, is that AGI, or is it just ubiquitous AI? I’d agree that, like self driving cars, we’re going to experience a decade or so transition into AI being everywhere. But is it AGI when we get there? I think it’ll be many different systems each providing an…

The Singularity is caused by AI being able to design better AI. There's probably some AI startup trying to work on this at the moment, but I don't think any of the big boys are working on how to get an LLM to design a better LLM. I still like the analogy of this being a really smart lawn mower, and we're expecting it to suddenly be able to do the laundry because it gets so smart at mowing the lawn. I think LLMs are g…

> The Singularity is caused by AI being able to design better AI.

That's perhaps necessary, but not sufficient.

Suppose you have such a self-improving AI system, but the new and better AIs still need exponentially more and more resources (data, memory, compute) for training and inference for incremental gains. Then you still don't get a singularity. If the increase in resource usage is steep enough, even the new AIs helping with designing better computers isn't gonna unleash a singularity.

I don't know if that's the world we live in, or whether we are living in one where resources requirements don't balloon as sharply.

Re: GPT-5 is behind schedule

#414

Post by mkl » Mon, Dec 23, 2024, 3:36 AM UTC

arthurcolle wrote: post #329

missedthecue wrote: post #271

Earlier quoted context omitted.

Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.

LLMs are a key piece of understanding that token sequences can trigger actions in the real world. AGI is here. You can trivially spin up a computer using agent to self improve itself to being a competent office worker

> You can trivially spin up a computer using agent to self improve itself to being a competent office worker

If that was true, office workers would be being replaced at large scale and we'd know about it.

Re: GPT-5 is behind schedule

#415

Post by SamPatt » Mon, Dec 23, 2024, 3:41 AM UTC

I'm sure the debate over the definition of AGI is important and will continue for a while, but... I can't care about it anymore.

Between Perplexity searching and summarizing, Claude explaining, and qwen (and other tools) coding, I'm already as happy as can be with whatever you want to call this level of intelligence.

Just today I used a completely local AI research tool, based on Ollama. It worked great.

Maybe it won't get much better? Or maybe it'll take decades instead of years? Ok. I remember not having these tools. I never want to go back.

Re: GPT-5 is behind schedule

#416

Post by lobochrome » Mon, Dec 23, 2024, 3:42 AM UTC

xanderlewis wrote: post #206

simonw wrote: post #146

Earlier quoted context omitted.

No, I'm complaining that just because GPT-4 is called GPT-4 doesn't mean it's the fourth LLM from OpenAI. Off the top of my head: GPT-2, Codex, GPT-3 in three different flavors (babbage, curie, davinci), GPT-3.5. Suggesting that GPT-4 was "fourth" simply isn't credible. Just the other day they announced a jump from o1 to o3, skipping o2 purely because it's already the name of a major telecommunications brand in Europ…

It’s somehow funny to hear a British company being described as ‘in Europe’, but I suppose you’re technically correct…

Well - it’s Spanish now no? Telefonica bought them.

Re: GPT-5 is behind schedule

#417

Post by eru » Mon, Dec 23, 2024, 3:42 AM UTC

worik wrote: post #387

dartos wrote: post #385

Earlier quoted context omitted.

I don’t think that’s true for AGI. AGI is the holy grail of technology. A technology so advanced that not only does it subsume all other technology, but it is able to improve itself. Truly general intelligence like that will either exist or not. And the instant it becomes public, the world will have changed overnight (maybe the span of a year) Note: I don’t think statistical models like these will get us there.

If that is what AGI looks like. There may well be an upper limit on cognition (we are not really sure what cognition is - even as we do it) and it may be that human minds are close to it.

Yes, we can imagine that there's an upper limit to how smart a single system can be. Even suppose that this limit is pretty close to what humans can achieve.

But: you can still run more of these systems in parallel, and you can still try to increase processing speeds.

Signals in the human brain travel, at best, roughly at the speed of sound. Electronic signals in computers play in the same league as the speed of light.

Human IO is optimised for surviving in the wild. We are really bad at taking in symbolic information (compared to a computer) and our memory is also really bad for that. A computer system that's only as smart as a human but has instant access to all the information of the Internet and to a calculator and to writing and running code, can already be effectively act much smarter than a human.

Re: GPT-5 is behind schedule

#418

Post by sooheon » Mon, Dec 23, 2024, 3:42 AM UTC

exhaze wrote: post #286

ynniv wrote: post #177

Earlier quoted context omitted.

Is it "eerie"? LeCun has been talking about it for some time, and may also be OpenAI's rumored q-star, mentioned shortly after Noam Brown (diplomacybot) joining OpenAI. You can't hill climb tokens, but you can climb manifolds.

I wasn’t aware of others attempting manifolds for this before - just something I stumbled upon independently. To me the “eerie” part is the thought of an LLM no longer using human language to reason - it’s like something out of a sci fi movie where humans encounter an alien species that thinks in a way that humans cannot even comprehend due to biological limitations. I am hopeful that progress in mechanistic interpre…

I remember (apocryphal?) Microsoft's chatbot developing pidgin to communicate to other chatbots. Every layer of the NN except the first and last already "think" in latent space, is this surprising?

Re: GPT-5 is behind schedule

#419

Post by keenmaster » Mon, Dec 23, 2024, 3:43 AM UTC

wongarsu wrote: post #342

danpalmer wrote: post #320

Earlier quoted context omitted.

Interesting idea. The concept of The Singularity would seem to go against this, but I do feel that seems unlikely and that a gradual transition is more likely. However, is that AGI, or is it just ubiquitous AI? I’d agree that, like self driving cars, we’re going to experience a decade or so transition into AI being everywhere. But is it AGI when we get there? I think it’ll be many different systems each providing an…

The idea of the singularity presumes that running the AGI is either free or trivially cheap compared to what it can do, so we are fine expending compute to let the AGI improve itself. That may eventually be true, but it's unlikely to be true for the first generation of AGI. The first AGI will be a research project that's completely uneconomical to run for actual tasks because humans will just be orders of magnitude c…

If the first AGI is a very uneconomical system with human intelligence but knowledge of literally everything and the capability to work 24/7, then it is not human equivalent.

It will have human intelligence, superhuman knowledge, superhuman stamina, and complete devotion to the task at hand.

We really need to start building those nuclear power plants. Many of them.

Re: GPT-5 is behind schedule

#420

Post by NBJack » Mon, Dec 23, 2024, 3:43 AM UTC

apsec112 wrote: post #324

NBJack wrote: post #321

Earlier quoted context omitted.

No. But it won't stop the industry from trying. LLMs have no real sense of truth or hard evidence of logical thinking. Even the latest models still trip up on very basic tasks. I think they can be very entertaining, sure, but not practical for many applications.

What do you think, if we saw it, would constitute hard evidence of logical thinking or a sense of truth?

Consistent, algorithmic performance on basic tasks.

A great example is the simple 'count how many letters' problem. If I prompt it with a word or phrase, and it gets it wrong, me pointing out the error should translate into a consistent course correction for the entire session.

If I ask it to tell me how long President Lincoln will be in power after the 2024 election, it should have a consistent ground truth to correct me (or at least ask for clarification of which country I'm referring to). If facts change, and I can cite credible sources, it should be able to assimilate that knowledge on the fly.