Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

971–980 of 1001 posts

Re: GPT-5 is behind schedule

#971

Post by int_19h »

az226 wrote: post #567
Animats wrote: post #123

"Orion’s problems signaled to some at OpenAI that the more-is-more strategy, which had driven much of its earlier success, was running out of steam." So LLMs finally hit the wall. For a long time, more data, bigger models, and more compute to drive them worked. But that's apparently not enough any more. Now someone has to have a new idea. There's plenty of money available if someone has one. The current level of LLM…

The problem is data. GPT-3 was trained on 4:1 ratio of data to parameters. And for GPT-4 the ratio was 10:1. So to scale this out, GPT-5 should be 25:1. The parameter count jumped from 175B to 1.3T, which means GPT-5 should be 10T parameters and 250T training tokens. There is zero chance OpenAI has a training set of high quality data that is 250T tokens. If I had to guess, they trained a model that was maybe 3-4T in…

We're also increasingly using synthetic data to train them on, though, and the race now is in coming up with better ways to generate it.

Re: GPT-5 is behind schedule

#972

Post by overgard »

icpmacdo wrote: post #965
overgard wrote: post #540

Earlier quoted context omitted.

By whom? He seems highly credible to me, and his credentials check out, especially compared to hype men like Sam Altman. All youre doing is spreading FUD by an unnamed "they"

He only criticizes ai capabilities, without creating anything himself. Credentials are effectively meaningless. With every new release, he clamors for attention to prove how right he was—and always will be. That’s precisely why he lacks credibility.

He started and then sold a machine learning startup to Uber. He's also written multiple books about the construction of the human mind and he has a PhD from MIT. I would hardly call that creating nothing. He's not clamoring for attention, he's asking that AI be regulated and pointing out a lot of the glaring issues with the field.

Re: GPT-5 is behind schedule

#973

Post by xboxnolifes »

solardev wrote: post #732
LASR wrote: post #610

So the team I lead does a lot of research around all the “plumbing” around LLMs. Both technical and from a product-market perspectives. What I’ve learned is that for the most part that AI revolution is not going to be because of PHD-level LLMs. It will be because people are better equipped to use the high-schooler level LLMs to do their work more efficiently. We have some knowledge graph experiments where LLMs contin…

> a revolution in knowledge-worker productivity. That's a nice euphemism for "imminent mass layoffs and a race to the bottom"...

Productivity has always meant the ability to do more with less. Or it can mean doing even more with more.

Were we at a peak 1000+ years ago and have only gone downhill since at every technological breakthrough?

Re: GPT-5 is behind schedule

#974

Post by patcon »

jval43 wrote: post #532
patcon wrote: post #512

Earlier quoted context omitted.

There's something here that I feel is pretty deep, though offensive for some minds: What is the actual consequence of being wrong? Of not getting right the base reality of a situation? Usually, stasis is the enemy that is much great than false information. If people with 90% truth can take a step forward in the world, even if they mistakenly think they have 100% truth, what does it matter? They're learning more and a…

Incorrect information by itself is at best useless. Incorrect information that is thought to be correct is outright dangerous. Objective truth is crucial to science and progress. We've come too far since the age of enlightenment to just give it all up.

The hundred year functioning of democracy begs to differ. It literally works nothing like how anyone tells themselves it does, not just laypeople, but arguably even political scientists. It's quite possible that no echelon of society has had the correct story so far, and yet... (again, see "Democracy for Realists")

Also, the vision heuristics that brains use to help us monitor motion as another obvious example. They lie. They work. They won.

https://x.com/foone/status/1014267515696922624?s=46

> Objective truth is crucial to science

Agreed. We define science and science is truth about base reality.

> Objective truth is crucial to [...] progress.

More contentious imho. Depends if progress is some abstract human ideal that we pursue, or simply "survival". If it's the former, maybe objective truth is required. If it's the latter, I find the simulation evidence to be that over-adherence to objective truth (at least information-theoretically) is in fact detrimental to our survival.

Re: GPT-5 is behind schedule

#975

Post by lobsterthief »

josu wrote: post #558
bloppe wrote: post #485

Earlier quoted context omitted.

Ya I think they probably will, but "the chatgpt company" is not worth 157B. It might not even be worth 1B.

It has replaced ~50% of my Google searches.

Yes but remember we’re in a tech bubble and the average person still doesn’t know what ChatGPT is.

Re: GPT-5 is behind schedule

#976

Post by drawfloat »

leesec wrote: post #229

Tech journalism is so cooked man lol. They just rocked everyones world with o3 and they still gotta drop this post.

o3 isn't even available to more than a few researchers. Either everyone you know is a researcher, or they're believing hype about something they've not tried.

Re: GPT-5 is behind schedule

#977

Post by atoav »

kridsdale1 wrote: post #946
atoav wrote: post #738

Earlier quoted context omitted.

These productivity gains won't be shared with the employees. I think some people underestimate what a violent populus can do to them if they squeeze out even more Yacht money from the people.

Every one of those employees is capable of either using the new tools to start their own companies and solve unaddressed customer problems, or negotiate for comp that has equity. This has always been true. The losers here are algorithm junkies who refuse to learn new skills and want to solve yesterday’s problems.

So you surely wouldn't be against a harsh inheritance tax so every generation can get a fair shot at the same issues?

Re: GPT-5 is behind schedule

#978

Post by Eupolemos »

mort96 wrote: post #648
LASR wrote: post #610

So the team I lead does a lot of research around all the “plumbing” around LLMs. Both technical and from a product-market perspectives. What I’ve learned is that for the most part that AI revolution is not going to be because of PHD-level LLMs. It will be because people are better equipped to use the high-schooler level LLMs to do their work more efficiently. We have some knowledge graph experiments where LLMs contin…

Honestly I wish you people would stop forcing this "AI revolution" on us. It's not good. It's not useful. It's not creating value. It's not "another team member"; other team members have their own minds with their own ideas and their own opinions. Your autocomplete takes my attention away from what I want to write and replaces it with what you want me to write. We don't want it.

I find inline AIs like Github Copilot to be annoying, but browser based AIs like Mistral og ChatGPT a really good and welcome help.

Re: GPT-5 is behind schedule

#979

Post by gizmo »

FrustratedMonky wrote: post #877
gizmo wrote: post #838

Earlier quoted context omitted.

Many terrific books have been published in the past 500 years. The median book is not worth your time, however, and neither is the top 10%. You cannot possibly read everything so you have to be very selective or you will read only dreck. This is the opposite of being anti-science or anti-education.

But compared to the content on the internet? So Top 10% of Books. Ok 90 % of Books. marginal, lot of bad. Internet. Just millions of pages of junk. - Books still take some effort. So why not start there. It isn't either/or, binary, a lot of books are bad, so guess I'll learn my medical degree from browsing the web because I don't trust those 'experts'.

The median book about medicine is over 100 years old, written in a language you don't speak, and filled to the brim with quackery. Worse than useless. Maybe you don't realize that bookstores and libraries only carry a minuscule fraction of all published works? You will get better information from reddit than from a book written before the discovery of penicillin.

I'll get you started with one of the great works from the 1600s:

https://www.gutenberg.org/cache/epub/49513/pg49513-images.ht...

Re: GPT-5 is behind schedule

#980

Post by ulbu »

parineum wrote: post #843
gizmo wrote: post #686

Earlier quoted context omitted.

People who are experts (PhD and 20 years of experience) often have very dumb opinions in their field of expertise. Experts make amateur mistakes too. Look at the books written by expert economists, expert psychologists, expert historians, expert philosophers, expert software engineers. Most books are not worth the paper they're written on, despite the authors being experts with decades of experience in their respecti…

> People who are experts (PhD and 20 years of experience) often have very dumb opinions in their field of expertise. And the training data contains all those dumb opinions.

and rehashes it unthinkingly, without an idea of what it means to consider and disagree with it.
Post reply on HN