Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

71–80 of 1001 posts

Re: GPT-5 is behind schedule

#71

Post by DougN7 »

Stevvo wrote: post #31
ericskiff wrote: post #25

What we can reasonably assume from statements made by insiders: They want a 10x improvement from scaling and a 10x improvement from data and algorithmic changes The sources of public data are essentially tapped Algorithmic changes will be an unknown to us until they release, but from published research this remains a steady source of improvement Scaling seems to stall if data is limited So with all of that taken toge…

"With o3 now public knowledge, imagine how long it’s been churning out new thinking at expert level across every field." I highly doubt that. o3 is many orders of magnitude more expensive than paying subject matter experts to create new data. It just doesn't make sense to pay six figures in compute to get o3 to make data a human could make for a few hundred dollars.

Someone needs to dress up Mechanical Turk and repackage it as an AI company…..

Re: GPT-5 is behind schedule

#72

Post by jncfhnb »

ADeerAppeared wrote: post #47
beacon294 wrote: post #19

Earlier quoted context omitted.

There's someone with this comment in every thread. Meanwhile, no one answers this because they are getting value. Please take the time to learn, it will give you value.

> Meanwhile, no one answers this because they are getting value. You're literally doing the same thing you're accusing of. Every HN thread is full of AI boosters claiming AI to be the future with no backing evidence. Riddle me this. If all these people are "getting value", why are all these companies losing horrendous amounts of money? Why has nobody figured out how to be profitable? > Please take the time to learn,…

> Riddle me this. If all these people are "getting value", why are all these companies losing horrendous amounts of money? Why has nobody figured out how to be profitable?

While I agree that LLMs are not currently working great for most envisioned use cases; this premise here is not a good argument. Large LLM providers are not trying to be profitable at the moment. They’re trying to grow and that’s pretty sensible.

Uber was the poster child of this, and for all its mockery, Uber is now an unqualified profitable company.

Re: GPT-5 is behind schedule

#73

Post by psb217 »

noman-land wrote: post #69
ericskiff wrote: post #25

What we can reasonably assume from statements made by insiders: They want a 10x improvement from scaling and a 10x improvement from data and algorithmic changes The sources of public data are essentially tapped Algorithmic changes will be an unknown to us until they release, but from published research this remains a steady source of improvement Scaling seems to stall if data is limited So with all of that taken toge…

I completely don't understand the use for synthetic data. What good it's it to train a model basically on itself?

The value of synthetic data relies on having non-zero signal about which generated data is "better" or "worse". In a sense, this what reinforcement learning is about. Ie, generate some data, have that data scored by some evaluator, and then feed the data back into the model with higher weight on the better stuff and lower weight on the worse stuff.

The basic loop is: (i) generate synthetic data, (ii) rate synthetic data, (iii) update model to put more probability on better data and less probability on worse data, then go back to (i).

Re: GPT-5 is behind schedule

#74

Post by orbital-decay »

h_tbob wrote: post #35
anonzzzies wrote: post #9

Earlier quoted context omitted.

Not sure what you are using it for, but it is terrible for me for coding; claude beats it always and hands down. o1 just thinks forever to come up with stuff it already tried the previous time. People say that's just prompting without pointing to real million line+ repositories or realistic apps to show how that can be improved. So I say they are making todo and hello world apps and yes, there it works really well. C…

That so weird, it’s seems like everybody here prefers Claude. I’ve been using Claude and openai in copilot and I find even 4o seems to understand the problem better. O1 definitely seems to get it right more for me.

Different languages maybe? I find Sonnet v2 to be lacking in Rust knowledge compared to 4o 11-20, but excelling at Python and JS/TS. O1's strong side seems to be complex or quirky puzzle-like coding problems that can be answered in a short manner, it's meh at everything else, especially considering the price. Which is understandable given its purpose and training, but I have no use for it as that's exactly the sort of problem I wouldn't trust an LLM to solve.

Sonnet v2 in particular seems to be a bit broken with its reasoning (?) feature. The one where it detects it might be hallucinating (what's even the condition?) and reviews the reply, reflecting on it. It can make it stop halfway into the reply and decide it wrote enough, or invent some ridiculous excuse to output a worse answer. Annoying, although it doesn't trigger too often.

Re: GPT-5 is behind schedule

#75

Post by shadowerm »

beacon294 wrote: post #19
jsheard wrote: post #13

Earlier quoted context omitted.

[flagged]

There's someone with this comment in every thread. Meanwhile, no one answers this because they are getting value. Please take the time to learn, it will give you value.

It gives me value but I am not even sure it is $20 a month of value at this point.

It was in 2023 but I picked all the low hanging fruit.

More importantly though, where is all the great output from the people who are getting so much value out of the models?

It is all privately held? How can that be with millions of people using these models?

Re: GPT-5 is behind schedule

#76

Post by ynniv »

dartos wrote: post #63
ynniv wrote: post #60

Earlier quoted context omitted.

Everyone's obsessed with new training tokens... It doesn't need to be more knowledgeable, it just needs to practice more. Ask any student: practice is synthetic data.

That leads to overfitting in ML land, which hurts overall performance. We know that unique data improves performance. These LLM systems are not students… Also, which students graduate and are immediately experts in their fields? Almost none. It takes years of practice in unique, often one-off, situations after graduation for most people to develop the intuition needed for a given field.

It's overfitting when you train too large a model on too many details. Rote memorization isn't rewarding.

The more concepts the model manages to grok, the more nonlinear its capabilities will be: we don't have a data problem, we have an educational one.

Claude 3.5 was safety trained by Claude 3.0, and it's more coherent for it. https://www.anthropic.com/news/claudes-constitution

Re: GPT-5 is behind schedule

#77

Post by a1j9o94 »

onemoresoop wrote: post #43
h_tbob wrote: post #36

It seems google has a massive advantage here since they can tap all of YouTube to train. I wonder what openai is using for its video data source.

Train for what? For making videos? Train from people’s comments? There’s a lot of garbage on AI slop on youtube, how would this be sifted out? I think there’s more value here on HN in terms of training, but even that, to what avail?

YouTube is such a great multimodal dataset—videos, auto-generated captions, and real engagement data all in one place. That’s a strong starting point for training, even before you filter for quality. Microsoft’s Phi-series models already show how focusing on smaller, high-quality datasets, like textbooks, can produce great results. You could totally imagine doing the same thing with YouTube by filtering for high-quality educational videos.

Down the line, I think models will start using video generation as part of how they “think.” Picture a version of GPT that works frame by frame—ask it to solve a geometry problem, and it generates a sequence of images to visualize the solution before responding. YouTube’s massive library of visual content could make something like that possible.

Re: GPT-5 is behind schedule

#78

Post by williamcotton »

mathieuh wrote: post #26
phito wrote: post #5

Earlier quoted context omitted.

I keep reading this on HN so I believe it has to be true in some ways, but I don't really feel like there is any difference in my limited use (programming questions or explaining some concepts). If anything I feel like it's all been worse compared to the first release of ChatGPT, but I might be wearing rose colored glasses.

It’s the same for me. I genuinely don’t understand how I can be having such a completely different experience from the people who rave about ChatGPT. Every time I’ve tried it’s been useless. How can some people think it’s amazing and has completely changed how they work, while for me it makes mistakes that a static analyser would catch? It’s not like I’m doing anything remarkable, for the past couple of months I’ve b…

I found it very useful for writing a lexer and parser for a search DSL and React component recently:

https://github.com/williamcotton/search-input-query

Re: GPT-5 is behind schedule

#79

Post by omega3 »

taneq wrote: post #51
omega3 wrote: post #23

Earlier quoted context omitted.

People who say „it’s bullshit” are the ones that push the technological advance forward.

No, people who say "it's bullshit" and then do something to fix the bullshit are the ones that push technology forward. Most people who say "it's bullshit" instantly when something isn't perfect for exactly what they want right now are just whingers and will never contribute anything except unconstructive criticism.

Sounds like "yes but" rather than "no" otherwise you're responding to self created straw man.

Re: GPT-5 is behind schedule

#80

Post by david-gpu »

jillesvangurp wrote: post #16
david-gpu wrote: post #8

What I find odd is that o1 doesn't support attaching text documents to chats the way 4o does. For a model that specializes in reasoning, reading long documents seems like a natural feature to have.

You can use the new project feature for that. That's a way of grouping conversations, adding files, etc. Should work with o1 pro as well apparently.

"When using custom instructions or files, only GPT-4o is available". Straight out of the ChatGPT web interface when you try to select which model you want to use.
Post reply on HN