Live data from Hacker News

OpenAI researcher announced GPT-5 math breakthrough that never happened

the-decoder.com

81–90 of 258 posts

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#81

Yann LeCun's "Hoisted by their own GPTards" is fantastic.

While Yann is clearly brilliant, and has a deeper understanding of the roots of the filed than many of us mortals, I think he's been on a debbie downer trend lately, and more importantly, some of his public stances have been proven wrong in mere months / years after he made them.

I remember a public talk, where he was on the stage with some young researcher from MS. (I think it was one of the authors of the "sparks of brilliance in gpt4" paper, but not sure).

Anyway, throughout that talk he kept talking above the guy, and didn't seem to listen, even though he obviously didn't try the "raw", "unaligned" model that the folks at MS were talking about.

And he made 2 big claims:

1) LLMs can't do math. He went on to "argue" that LLMs trick you with poetry that sounds good, but is highly subjective, and when tested on hard verifiable problems like math, they fail.

2) LLMs can't plan.

Well, merely one year later, here we are. AIME is saturated (with tool use), gold at IMO, and current agentic uses clearly can plan (and follow up with the plan, re-write parts, finish tasks, etc etc).

So, yeah, I'd take everything any one singular person says with a huge grain of salt. No matter how brilliant said individual is.

Edit: oh, and I forgot another important argument that Yann made at that time:

3) because of the nature of LLMs, errors compound. So the longer you go in a session, the more errors accumulate so they devolve in nonsense.

Again, mere months later the o series of models came out, and basically proved this point moot. Turns out RL + long context mitigate this fairly well. And a year later, we have all SotA models being able to "solve" problems 100k+ tokens deep.

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#82
post #55

The original tweet was clearly misunderstood... https://x.com/SebastienBubeck/status/1977181716457701775 : > gpt5-pro is superhuman at literature search: > it just solved Erdos Problem #339 (listed as open in the official database https://erdosproblems.com/forum/thread/339 ) by realizing that it had actually been solved 20 years ago https://x.com/MarkSellke/status/1979226538059931886 : > Update: Mehtaab and I pushed…

If holding the CTO of OpenAI accountable for his wildly inaccurate statement constitutes "dunking on OpenAI", then I'd say dunk away.

He, more than anyone else, should be able to for one parse the original statements correctly and for another maybe realize that if one of their models had accomplished what he seemed to think GPT-5 had, that may require some more scrutiny and research before posting it. That would have, after all, been a clear and incredibly massive development for the space, something the CTO of OpenAI should recognize instantly.

The amount of people that told me this is clear and indisputable proof that AGI/ASI/whatever is either around the corner or already here is far more than zero and arguing against their misunderstanding was made all the more challenging because "the CTO of OpenAI knows more than you" is quite a solid appeal to authority.

I'd recommend maybe a waiting period of 48h before any authority in any field can send a tweet, that might resolve some of the inaccuracies and the incredibly annoying need to just jump on wild bandwagons...

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#83
I make mistakes all the time. This seems like a genuine mistake, not malice.

Imagine if you were talking about your own work online, you make an honest mistake, then the whole industry roasts you for it.

I’m so tired of hearing everyone take stabs at people at OpenAI just because they don’t personally like sama or something.

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#84

Humans hallucinating about AI.

"OpenAI Researcher Hallucinates GPT-5 Math Breakthrough" could be a headline from The Onion.

"OpenAI Researcher Hallucinates GPT-5 Math Breakthrough" could be a headline from The Onion.

Off topic, but I saw The Onion on sale in the magazine rack of Barnes and Noble last month.

For those who miss when it was a free rag in sidewalk newsstands, and don't want to pony up for a full subscription, this is an option.

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#85
post #55

The original tweet was clearly misunderstood... https://x.com/SebastienBubeck/status/1977181716457701775 : > gpt5-pro is superhuman at literature search: > it just solved Erdos Problem #339 (listed as open in the official database https://erdosproblems.com/forum/thread/339 ) by realizing that it had actually been solved 20 years ago https://x.com/MarkSellke/status/1979226538059931886 : > Update: Mehtaab and I pushed…

"you are totally right—I actually misunderstood" ...like, seriously? Did an AI come up with this retraction, or are humans actually talking like robots now?

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#86
> GPT-5 is proving useful as a literature review assistant

No, it does not. It only produces a highly convincing counterfeit. I am honestly happy for people who are satisfied with its output: life is way easier for them than for me. Obviously, the machine discriminates me personally. When I spend hours in the library looking for some engineering-related math made in the 70s-80s, as a last resort measure, I can try to play this gambling with chat, hoping for any tiny clue to answer my question. And then for the following hours, I am trying to understand what is wrong with the chat output. Most often, I experience the "it simply can't be" feeling, and I know I am not the only one having it.

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#87

Earlier quoted context omitted.

Hanlon's Razor

Lying is a stupid way of selling something and making money

Lying is a stupid way of selling something and making money

Works for Elon.

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#88

Earlier quoted context omitted.

Best case: Hallucination Worst case (more probable): Lying

Hanlon's Razor

They are expanding into the adult market because they are running out of ideas. I think common sense is enough to decide what is what here.

Re: OpenAI researcher announced GPT-5 math breakthrough that never happened

#90
post #9

This honestly doesn’t surprise me. We have reached a point where it’s becoming clearer and clearer that AGI is nowhere to be seen, whereas advances in LLM ability to ‘reason’ have slowed down to (almost?) a halt.

In my book, chat-based AGI has been reached years ago, when I couldn't reliably distinguish computer from human. Solving problems that humanity couldn't solve is super-AGI or something like that. It's not there indeed.

We're not even solving problems that humanity can solve. There's been several times where I've posed to models a geometry problem that was novel but possible for me to solve on my own, but LLMs have fallen flat on executing them every time. I'm no mathematician, these are not complex problems, but they're well beyond any AI, even when guided. Instead, they're left to me, my trusty whiteboard, and a non-negligible amount of manual brute force shuffling of terms until it comes out right.

They're good at the Turing test. But that only marks them as indistinguishable from humans in casual conversation. They are fantastic at that. And a few other things, to be clear. Quick comprehension of an entire codebase for fast queries is horribly useful. But they are a long way from human-level general intelligence.

Post reply on HN