Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.2 derives a new result in theoretical physics

openai.com

Post reply on HN
Print view View original

321–330 of 430 posts

Re: GPT-5.2 derives a new result in theoretical physics

#321

Post by virgildotcodes »

utopiah wrote: post #304
btown wrote: post #118

Earlier quoted context omitted.

It bears repeating that modern LLMs are incredibly capable, and relentless, at solving problems that have a verification test suite. It seems like this problem did (at least for some finite subset of n)! This result, by itself, does not generalize to open-ended problems, though, whether in business or in research in general. Discovering the specification to build is often the majority of the battle. LLMs aren't bad a…

> modern LLMs are incredibly capable, and relentless, at solving problems that have a verification test suite. Feel like it's a bit what I tried to expressed few weeks ago https://news.ycombinator.com/item?id=46791642 namely that we are just pouring computational resources at verifiable problems then claim that astonishingly sometimes it works. Sure LLMs even have a slight bias, namely they do rely on statistics so i…

> throw stuff at the wall, see what sticks, once something finally does report it as grandiose and claim to be "intelligent".

What do we think humans are doing? I think it’s not unfair to say our minds are constantly trying to assemble the pieces available to them in various ways. Whether we’re actively thinking about a problem or in the background as we go about our day.

Every once in a while the pieces fit together in an interesting way and it feels like inspiration.

The techniques we’ve learned likely influence the strategies we attempt, but beyond all this what else could there be but brute force when it comes to “novel” insights?

If it’s just a matter of following a predefined formula, it’s not intelligence.

If it’s a matter of assembling these formulas and strategies in an interesting way, again what else do we have but brute force?

Re: GPT-5.2 derives a new result in theoretical physics

#322

Post by srcreigh »

kilroy123 wrote: post #232
Toutouxc wrote: post #217

Earlier quoted context omitted.

I am constantly seeing this thing do most of my work (which is good actually, I don't enjoy typing code), but requiring my constant supervision and frequent intervention and always trying to sneak in subtle bugs or weird architectural decisions that, I feel with every bone in my body, would bite me in the ass later. I see JS developers with little experience and zero CS or SWE education rave about how LLMs are so muc…

I agree with you on all of it. But _what if_ they work out all of that in the next 2 years and it stops needing constant supervision and intervention? Then what?

It’s literally not possible. It has nothing to do with intelligence. A perfectly intelligent AI still can’t read minds. 1000 people give the same prompt and want 1000 different things. Of course it will need supervision and intervention.

We can synthesize answers to questions more easily, yes. We can make better use of extensive test suites, yes. We cannot give 1000 different correct answers to the same prompt. We cannot read minds.

Re: GPT-5.2 derives a new result in theoretical physics

#323

Post by D-Machine »

btown wrote: post #118
CGMthrowaway wrote: post #24

Earlier quoted context omitted.

This is the critical bit (paraphrasing): Humans have worked out the amplitudes for integer n up to n = 6 by hand, obtaining very complicated expressions, which correspond to a “Feynman diagram expansion” whose complexity grows superexponentially in n. But no one has been able to greatly reduce the complexity of these expressions, providing much simpler forms. And from these base cases, no one was then able to spot a…

It bears repeating that modern LLMs are incredibly capable, and relentless, at solving problems that have a verification test suite. It seems like this problem did (at least for some finite subset of n)! This result, by itself, does not generalize to open-ended problems, though, whether in business or in research in general. Discovering the specification to build is often the majority of the battle. LLMs aren't bad a…

Even more generally than verification, just being tied to a loss function that represent something we actually care about. E.g. compiler and test errors, LEAN verification in Aristotle, basic physics energy configs in AlphaFold, or win conditions in e.g. RL, such as in AlphaGo.

RLHF is an attempt to push LLMs pre-trained with a dopey reconstruction loss toward something we actually care about: imagine if we could find a pre-training criterion that actually cared about truth and/or plausibility in the first place!

Re: GPT-5.2 derives a new result in theoretical physics

#324

Post by famouswaffles »

D-Machine wrote: post #317
famouswaffles wrote: post #314

Earlier quoted context omitted.

OP's original comment is something that is actually happening in a bunch of comments on this very thread, and yours...not even remotely. You certainly tried to paint it as disingenuous but it really just fell flat. I'm not surprised you failed to understand that though. >Given our previous interactions ( https://news.ycombinator.com/item?id=46938446 ), it is clear you don't understand much things about AI and/or LLMs…

> It's interesting to me that whenever a new breakthrough in AI use comes up, there's always a flood of people who come in to handwave away why this isn't actually a win for LLMs. >> OP's original comment is something that is actually happening in a bunch of comments on this very thread OPs original comment was obviously a general claim not tied to responses to this thread. As usual, you fail to understand even the b…

His comment was a general claim, but he made it here specifically because this thread was already full of examples proving his point. Shouldn't that be Obvious?

Re: GPT-5.2 derives a new result in theoretical physics

#325

Post by utopiah »

virgildotcodes wrote: post #321
utopiah wrote: post #304

Earlier quoted context omitted.

> modern LLMs are incredibly capable, and relentless, at solving problems that have a verification test suite. Feel like it's a bit what I tried to expressed few weeks ago https://news.ycombinator.com/item?id=46791642 namely that we are just pouring computational resources at verifiable problems then claim that astonishingly sometimes it works. Sure LLMs even have a slight bias, namely they do rely on statistics so i…

> throw stuff at the wall, see what sticks, once something finally does report it as grandiose and claim to be "intelligent". What do we think humans are doing? I think it’s not unfair to say our minds are constantly trying to assemble the pieces available to them in various ways. Whether we’re actively thinking about a problem or in the background as we go about our day. Every once in a while the pieces fit together…

See what I replied just earlier https://news.ycombinator.com/item?id=47011884 namely the different regimes, within paradigm versus challenging it by going back to first principles. The ability to notice something is off beyond "just" assembling existing pieces, to backtrack within the process when failures get too many and actually understand the relationship is precisely different.

Re: GPT-5.2 derives a new result in theoretical physics

#326

Post by D-Machine »

MatejKafka wrote: post #262
square_usual wrote: post #64

It's interesting to me that whenever a new breakthrough in AI use comes up, there's always a flood of people who come in to handwave away why this isn't actually a win for LLMs. Like with the novel solutions GPT 5.2 has been able to find for erdos problems - many users here (even in this very thread!) think they know more about this than Fields medalist Terence Tao, who maintains this list showing that, yes, LLMs hav…

I don't think it's about trying to handwave away the achievement. The problem is that many AI proponents, and especially companies producing the LLM tools constantly overstate the wins while downplaying the issues, and that leads to a (not always rational) counter-reaction from the other side.

It is especially glaring in this case because, when queried, it is clear that far too many of the most zealous proponents don't even understand the simplest basics of how these models actually work (e.g. tokenization, positional or other encoding schemes, linear algebra, pre-training, basic input/output shaping/dimensions, recursive application, training data sources, etc).

There are simple limitations that follow from these basic facts (or which follow with e.g. extreme but not 100% certainty), such that many experts openly state that e.g. LLMs have serious limitations, but, still, despite all this, you get some very extreme claims about capabilities, from supporters, that are extremely hard to reconcile with these basic and indisputable facts.

That, and the massive investment and financial incentives means that the counter-reaction is really quite rational (but still potentially unwarranted, in some/many practical cases).

Re: GPT-5.2 derives a new result in theoretical physics

#327

Post by Kuxe »

tjr wrote: post #104
randomtoast wrote: post #53

Earlier quoted context omitted.

> but I haven’t been to get them to do something totally out of distribution yet from first principles Can humans actually do that? Sometimes it appears as if we have made a completely new discovery. However, if you look more closely, you will find that many events and developments led up to this breakthrough, and that it is actually an improvement on something that already existed. We are always building on the shou…

Go enough shoulders down, and someone had to have been the first giant.

A discovery by a giant is in some sense a new base vector in the space of discoveries. The interesting question is if a statistical machine can only perform a linear combination in the space of discoveries, or if a statistical machine can discover a new base vector in the space of discoveries.. whatever that is.

Re: GPT-5.2 derives a new result in theoretical physics

#328

Post by matwood »

supern0va wrote: post #165
cpard wrote: post #126

AI can be an amazing productivity multiplier for people who know what they're doing. This result reminded me of the C compiler case that Anthropic posted recently. Sure, agents wrote the code for hours but there was a human there giving them directions, scoping the problem, finding the test suites needed for the agentic loops to actually work etc etc. In general making sure the output actually works and that it's a s…

>AI can be an amazing productivity multiplier for people who know what they're doing. >[...] >The "AI replaces humans in X" narrative is primarily a tool for driving attention and funding. You're sort of acting like it's all or nothing. What about the the humans that used to be that "force multiplier" on a team with the person guiding the research? If a piece of software required a team of ten to people, and instead…

> If a piece of software required a team of ten to people, and instead it's built with one engineer overseeing an AI, that's still 90% job loss.

Yes, but this assumes a finite amount of software that people and businesses need and want. Will AI be the first productivity increase where humanity says ‘now we have enough’? I’m skeptical.

Re: GPT-5.2 derives a new result in theoretical physics

#329

Post by ece »

cpard wrote: post #126

AI can be an amazing productivity multiplier for people who know what they're doing. This result reminded me of the C compiler case that Anthropic posted recently. Sure, agents wrote the code for hours but there was a human there giving them directions, scoping the problem, finding the test suites needed for the agentic loops to actually work etc etc. In general making sure the output actually works and that it's a s…

Everytime I see a RL startup, a data startup or even a startup focused on a specific vertical, I think this exact same thing about LLMs.

Re: GPT-5.2 derives a new result in theoretical physics

#330

Post by hananova »

cpard wrote: post #126

AI can be an amazing productivity multiplier for people who know what they're doing. This result reminded me of the C compiler case that Anthropic posted recently. Sure, agents wrote the code for hours but there was a human there giving them directions, scoping the problem, finding the test suites needed for the agentic loops to actually work etc etc. In general making sure the output actually works and that it's a s…

AI is indeed an amazing productivity multiplier! Sadly that multiplier is in the range [0; 1).
Post reply on HN