Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

171–180 of 1001 posts

Re: GPT-5 is behind schedule

#171

Post by asadalt »

exhaze wrote: post #160
simonw wrote: post #140

Earlier quoted context omitted.

The new idea is inference-time scaling, as seen in o1 (and o3 and Qwen's QwQ and DeepSeek's DeepSeek-R1-Lite-Preview and Google's gemini-2.0-flash-thinking-exp). I suggest reading these two pieces about that: - https://www.aisnakeoil.com/p/is-ai-progress-slowing-down - best explanation I've seen of inference scaling anywhere - https://arcprize.org/blog/oai-o3-pub-breakthrough - François Chollet's deep dive into o3 I'…

I think the wildest thing is actually Meta’s latest paper where they show a method for LLMs reasoning not in English, but in latent space https://arxiv.org/pdf/2412.06769 I’ve done research myself adjacent to this (mapping parts of a latent space onto a manifold), but this is a bit eerie, even to me.

kinda how we do it. language is just an io interface(but also neural obv) on top of our reasoning engine.

Re: GPT-5 is behind schedule

#172

Post by Yizahi »

Animats wrote: post #123

"Orion’s problems signaled to some at OpenAI that the more-is-more strategy, which had driven much of its earlier success, was running out of steam." So LLMs finally hit the wall. For a long time, more data, bigger models, and more compute to drive them worked. But that's apparently not enough any more. Now someone has to have a new idea. There's plenty of money available if someone has one. The current level of LLM…

To output "don't know" a system needs to "know" too. Random token generator can't know. It can guess better and better, maybe it can even guess 99.99% of time, but it can't know, it can't decide or reason (not even o1 can "reason").

Re: GPT-5 is behind schedule

#173

Post by vasco »

nialv7 wrote: post #97
ericskiff wrote: post #25

What we can reasonably assume from statements made by insiders: They want a 10x improvement from scaling and a 10x improvement from data and algorithmic changes The sources of public data are essentially tapped Algorithmic changes will be an unknown to us until they release, but from published research this remains a steady source of improvement Scaling seems to stall if data is limited So with all of that taken toge…

> OpenAI’s next moat I don't think oai has any moat at all. If you look around, QwQ from Alibaba is already pushing o1-preview performances. I think oai is only ahead by 3~6 months at most.

If their AGI dreams would come true it might be more than enough to have 3 months head start. They probably won't, but it's interesting to ponder what the next few hours, days, weeks would be for someone that would wield AGI.

Like let's say you have a few datacenters of compute at your disposal and the ability to instantiate millions of AGI agents - what do you have them do?

I wonder if the USA already has a secret program for this under national defense. But it is interesting that once you do control an actual AGI you'd want to speed-run a bunch of things. In opposition to that, how do you detect an adversary already has / is using it and what to do in that case.

Re: GPT-5 is behind schedule

#174

Post by im3w1l »

viraptor wrote: post #133
noman-land wrote: post #69

Earlier quoted context omitted.

I completely don't understand the use for synthetic data. What good it's it to train a model basically on itself?

This is a good read for some examples https://arxiv.org/abs/2203.14465 > This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat. We show…

One issue with that is that the model may learn to smuggle data. You as a human think that the plain reading of the words is what is doing the reasoning, but (part of) the processing is done by the exact comma placement and synonym choice etc.

Data smuggling is a known phenomenon in similar tasks.

Re: GPT-5 is behind schedule

#175

Post by layer8 »

kelnos wrote: post #125
bobxmax wrote: post #121

Earlier quoted context omitted.

We are close. Language models and large vision models have transformed robotics. It just takes some time to get hardware up and running.

I think it would be many decades before I'd trust a robot like that around small children or pets. Robots with that kind of movement capability, as well as the ability it pick up and move things around, will be heavy enough that a small mistake could easily kill a small child or pet.

Even for adults, a robot that would likely have to be close to as massive as a human being, in order to do laundry and the like, would spook me out, moving freely through my place.

Re: GPT-5 is behind schedule

#176

Post by achierius »

bobxmax wrote: post #132
leonheld wrote: post #128

Earlier quoted context omitted.

> have transformed robotics Did they? Where? Seriously, I genuinely want to know who is employing these techniques.

All frontier labs are now employing LVMs or LLMs. But that's my point is you won't see the fruits of it this early.

That's the point being made. It's transformed robotics research, yes, but it both remains to see whether it will have a truly transformative effect on the field as experienced by people outside academia (I think this is quite probable) and more pointedly when.

Re: GPT-5 is behind schedule

#177

Post by ynniv »

exhaze wrote: post #160
simonw wrote: post #140

Earlier quoted context omitted.

The new idea is inference-time scaling, as seen in o1 (and o3 and Qwen's QwQ and DeepSeek's DeepSeek-R1-Lite-Preview and Google's gemini-2.0-flash-thinking-exp). I suggest reading these two pieces about that: - https://www.aisnakeoil.com/p/is-ai-progress-slowing-down - best explanation I've seen of inference scaling anywhere - https://arcprize.org/blog/oai-o3-pub-breakthrough - François Chollet's deep dive into o3 I'…

I think the wildest thing is actually Meta’s latest paper where they show a method for LLMs reasoning not in English, but in latent space https://arxiv.org/pdf/2412.06769 I’ve done research myself adjacent to this (mapping parts of a latent space onto a manifold), but this is a bit eerie, even to me.

Is it "eerie"? LeCun has been talking about it for some time, and may also be OpenAI's rumored q-star, mentioned shortly after Noam Brown (diplomacybot) joining OpenAI. You can't hill climb tokens, but you can climb manifolds.

Re: GPT-5 is behind schedule

#178

Post by oceanplexian »

rubymamis wrote: post #84
anonzzzies wrote: post #20

Earlier quoted context omitted.

We do the same (all requests go to o1, sonnet and gemini and we store the results for later to compare) automatically for our research: Claude always wins. Even with specific prompting on both platforms. Especially frontend it seems o1 really is terrible.

Every time I try Gemini, it's really subpar. I found that qwen2.5-coder-32b-instruct can be better. Also, for me 50% 50% for Sonnet and o1, but although I'm not 100% sure about it, I think o1 is better with longer and more complicated (C++) code and debugging. At least from my brief testing. Also, OpenAI models seem to be more verbose - sometimes it's better - where I'd like additional explanation on chosen fields in…

Very anecdotal but I’ve found that for things that are well spec’d out with a good prompt Sonnet 3.5 is far better. For problems where I might have introduced a subtle logical error o1 seems to catch it extremely well. So better reasoning might be occurring but reasoning is only a small part of what we would consider intelligence.

Re: GPT-5 is behind schedule

#179

Post by jonas21 »

mathieuh wrote: post #26
phito wrote: post #5

Earlier quoted context omitted.

I keep reading this on HN so I believe it has to be true in some ways, but I don't really feel like there is any difference in my limited use (programming questions or explaining some concepts). If anything I feel like it's all been worse compared to the first release of ChatGPT, but I might be wearing rose colored glasses.

It’s the same for me. I genuinely don’t understand how I can be having such a completely different experience from the people who rave about ChatGPT. Every time I’ve tried it’s been useless. How can some people think it’s amazing and has completely changed how they work, while for me it makes mistakes that a static analyser would catch? It’s not like I’m doing anything remarkable, for the past couple of months I’ve b…

I think the difference comes down to interacting with it like IDE autocomplete vs. interacting with it like a colleague.

It sounds like you're doing the former -- and yeah, it can make mistakes that autocomplete wouldn't or generate code that's wrong or overly complex.

On the other hand, I've found that if you treat it more like a colleague, it works wonderfully. Ask it to do something, then read the code and ask follow-up questions. If you see something that's wrong or just seems off, tell it, and ask it to fix it. If you don't understand something, ask for an explanation. I've found that this process generates great code that I often understand better than if I had written it from scratch, and in a fraction of the time.

It also sounds like you're asking it to do basic tasks that you already know how to do. I find that it's most useful in tackling things that I don't know how to do. It'll already have read all of the documentation and know the right way to call whatever APIs, etc, and -- this is key -- you can have a conversation with it to clear up anything that's confusing.

This takes a big shift in mindset if you've been using IDEs all your life and have expectations of LLMs being a fancy autocomplete. And you really have to unlearn a lot of stuff to get the most out of them.

Re: GPT-5 is behind schedule

#180

Post by glenstein »

Yizahi wrote: post #165

GPT-5 is not behind schedule. GPT-5 is called GPT-4o and it has been already released half a year ago. It was not revolutionary enough to be called 5, and prophet saint Altman was probably afraid to release new gen not exponentially improving, so it was rebranded in the last moment. It's speculation of course, but it is kinda obvious speculation.

>GPT-5 is called GPT-4o

This is the first I have heard of this in particular. Do you know of any article or source for more on the efforts to train GPT 5 and the decision to call it GPT 4o?

Post reply on HN