Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

561–570 of 1001 posts

Re: GPT-5 is behind schedule

#561

Post by coffeebeqn »

throwaway314155 wrote: post #270
emeg wrote: post #181

Earlier quoted context omitted.

What makes you say that?

They hyped them like crazy and haven't discussed them once since then. I agree that the inability to change the model is pretty absurd when the whole point was to "supercharge" specific tasks. There was even talk of some sort of profit sharing with creators which clearly never happened. I just think the premise is too confusing for many and can still be served by using a custom system prompt via the API.

Was it hyped? I tried a few of them and they seemed absolutely useless. Like I could install a “custom GPT” that just appends something to my prompt? How great..

Re: GPT-5 is behind schedule

#562

Post by jstummbillig »

Fargren wrote: post #555
jstummbillig wrote: post #552

Earlier quoted context omitted.

You just state this as if it was obviously true, but I don't see how. Why is using LLM like reading a pop sci book and not like reading a history book? Or even less like either, because you have to continually ask questions to get anything?

A history book is written by someone who knows the topic, and then reviewed by more people who also know the topic, and then it's out there where people can read it and criticize it if it's wrong about the topic. A question asked to an AI is not reviewed by anyone, and it's ephemeral. The AI can answer "yes" today, and "no" tomorrow, so it's not possible to build a consensus on whether it answers specific questions c…

A pop sci fi book can be written by someone who knows the topic and reviewed by people who know the topic — and a history book can also not.

LLM generated answers are more comparable to ad-hoc human expert's answers and not to written books. But it's much simpler to statistically evaluate and correct them. That is how we can know that, on average, LLMs are improving and are outperforming human experts on an increasing number of tasks and topics.

Re: GPT-5 is behind schedule

#563

Post by NBJack »

eru wrote: post #423
NBJack wrote: post #321

Earlier quoted context omitted.

No. But it won't stop the industry from trying. LLMs have no real sense of truth or hard evidence of logical thinking. Even the latest models still trip up on very basic tasks. I think they can be very entertaining, sure, but not practical for many applications.

> LLMs have no real sense of truth or hard evidence of logical thinking. Most humans don't have that either, most of the time.

Then we already have access to a cheaper, scalable, abundant, and (in most cases) renewable resource, at least compared to how much a few H100s cost. Take good care of them, and they'll probably outlast most a GPU's average lifespans (~10 years).

We're also biodegradable.

Re: GPT-5 is behind schedule

#564

Post by leesec »

croes wrote: post #367
leesec wrote: post #229

Tech journalism is so cooked man lol. They just rocked everyones world with o3 and they still gotta drop this post.

I doubt they rocked they world of 10% of the people. Time to get out of the tech bubble.

This here is a technology forum bucko. Also it's a figure of speech. Also I've done more manual labor than you'll ever do in your life. Time to get out of whatever bubble youre in where you be pedantic and annoying

Re: GPT-5 is behind schedule

#565

Post by silisili »

josu wrote: post #558
bloppe wrote: post #485

Earlier quoted context omitted.

Ya I think they probably will, but "the chatgpt company" is not worth 157B. It might not even be worth 1B.

It has replaced ~50% of my Google searches.

Yes but it also hasn't been attacked by ads yet. Google doesn't suck for lack of search results, it sucks because of ads.

Imagine asking chatgpt to tell you about slopes in Colorado, and the first five answers are about how awesome North Face is and how you can order from them. You probably wouldn't use it as much.

Re: GPT-5 is behind schedule

#566

Post by vixen99 »

beefnugs wrote: post #394
missedthecue wrote: post #271

Earlier quoted context omitted.

Do we know LLMs are the path to AGI? If they're not, we'll just end up with some neat but eye wateringly expensive LLMs.

LLMs will end up being the good human-machine interface that lets us talk to whatever AGI really looks like (whoops expensive... will be hard pushes to make all further layers even more expensive though, capitalism will crash before this happens)

And then what?

Re: GPT-5 is behind schedule

#567

Post by az226 »

Animats wrote: post #123

"Orion’s problems signaled to some at OpenAI that the more-is-more strategy, which had driven much of its earlier success, was running out of steam." So LLMs finally hit the wall. For a long time, more data, bigger models, and more compute to drive them worked. But that's apparently not enough any more. Now someone has to have a new idea. There's plenty of money available if someone has one. The current level of LLM…

The problem is data.

GPT-3 was trained on 4:1 ratio of data to parameters. And for GPT-4 the ratio was 10:1. So to scale this out, GPT-5 should be 25:1. The parameter count jumped from 175B to 1.3T, which means GPT-5 should be 10T parameters and 250T training tokens. There is zero chance OpenAI has a training set of high quality data that is 250T tokens.

If I had to guess, they trained a model that was maybe 3-4T in size and used 30-50T high quality tokens and maybe 10-30 medium and low quality ones.

There is only one company in the world that stores the data that could get us past the wall.

The training cost of the above scaled GPT-5 is 150x GPT-4, which was 25k A100 for 90 days, which poor MFU.

Let’s assume they double MFU, it would mean 1M H100s. But let’s say they made algorithmic improvements, so maybe it’s only 250-500k H100s.

While the training cluster size was 100k and then grew to 150k, this cluster is suggestive of a smaller model and less data.

But ultimately data is the bottleneck.

Re: GPT-5 is behind schedule

#568

Post by isoprophlex »

divan wrote: post #445
SamPatt wrote: post #415

I'm sure the debate over the definition of AGI is important and will continue for a while, but... I can't care about it anymore. Between Perplexity searching and summarizing, Claude explaining, and qwen (and other tools) coding, I'm already as happy as can be with whatever you want to call this level of intelligence. Just today I used a completely local AI research tool, based on Ollama. It worked great. Maybe it won…

Let's revisit this comment in one year – after the explosion of agentic systems. (:

You mean, the explosion of human centipede LLM prompts shitting into eachother?

Yes that will be a sight to behold.

Re: GPT-5 is behind schedule

#569

Post by tugu77 »

SamPatt wrote: post #415

I'm sure the debate over the definition of AGI is important and will continue for a while, but... I can't care about it anymore. Between Perplexity searching and summarizing, Claude explaining, and qwen (and other tools) coding, I'm already as happy as can be with whatever you want to call this level of intelligence. Just today I used a completely local AI research tool, based on Ollama. It worked great. Maybe it won…

Thats all fine, but I think you are missing the bigger picture. It's not about whether what we already got out of this is good. Of course it is. But this is about where it's going.

Until about 120 years ago, people were happy with horses and horse carriages. Such a great help! Travel long distances, pull weights, I never want to go back! But then the automobile was invented and within a few years little travel was done by horses anymore.

More recently, everybody had a landline phone at home. Such great tech! Talk to grandma hundreds of miles away! I never want to go back! Then suddenly the mobile phone and just shortly after the smart phone came along and now nobody has a landline anymore but everybody can record tiktoks anywhere anytime and share them with the world within seconds.

Now imagine "AI". Sure, we have some new tools right now. Sure we don't want to go back. But imagine the transformative effects that could come if the train didn't stop here. Question is just: will it?

Re: GPT-5 is behind schedule

#570

Post by SheinhardtWigCo »

Fargren wrote: post #555
jstummbillig wrote: post #552

Earlier quoted context omitted.

You just state this as if it was obviously true, but I don't see how. Why is using LLM like reading a pop sci book and not like reading a history book? Or even less like either, because you have to continually ask questions to get anything?

A history book is written by someone who knows the topic, and then reviewed by more people who also know the topic, and then it's out there where people can read it and criticize it if it's wrong about the topic. A question asked to an AI is not reviewed by anyone, and it's ephemeral. The AI can answer "yes" today, and "no" tomorrow, so it's not possible to build a consensus on whether it answers specific questions c…

The same is true of Google, no?
Post reply on HN