Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 used a prompt to close a 30-year gap in convex optimization

old.reddit.com

Post reply on HN
Print view View original

151–160 of 414 posts

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#151

Post by tripleee »

slashdave wrote: post #144
sashank_1509 wrote: post #116

This is all a depressing and bleak future that I don’t look forward to. One solution is to ban LLM’s, to artificially create a demand for human thought, that just feels like living in an artificially constructed zoo. Another solution is humans don’t do anything that AI can do better , / doesn’t need the human touch. So I suppose we will all become artists, sportsmen or politicians, the only jobs that will remain exce…

There is more to human life than programming and math proofs.

unfortunately many won't get to experience it much because we'll be stressed and struggling to make a living

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#152

Post by tripleee »

qsera wrote: post #4
baal80spam wrote: post #2

Waiting for comments saying that LLMs can't produce anything new and general goalpost moving.

From the post lol >So I wouldn't really say that this result is using or creating some fundamentally new techniques in convex geometry or optimization theory. What this means from my perspective is that if a result is attainable with existing techniques, modern AI methods will be able to solve those problems. I don't think researchers in math/TCS will be made obsolete, but I think it will instead no longer make sense…

this is a fairly bleak outlook even when you're trying to make it sound the opposite. Only the cream of the crop talent will have value going on?

Most of us aren't Terence Tao

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#153

Post by pydry »

p-e-w wrote: post #139
Quothling wrote: post #88

Earlier quoted context omitted.

Around here AI isn't really more of a threat to juniors than it is to seniors. It's a threat to the people who have been taught "recipies" rather than applied computer science. You can have excellent seniors who can do TDD, DRY, SOLID and so on, who also happen to have no idea what a L1 cache miss is. The current AI models know all of those things, but they struggle applying them correctly without someone piloting th…

Unless you’re claiming that AIs will suddenly (and very soon) stop improving, they are obviously a threat to everyone’s job. Calling notable conjectures that have been open for decades “low-hanging fruit” is an act of desperation. Most professional mathematicians couldn’t have proved those conjectures if their lives depended on it.

>Unless you’re claiming that AIs will suddenly (and very soon) stop improving

Most technologies level off sharply after bouts of boundless improvements.

In 1968 they thought we'd be flying to the moon by now but instead we're flying across the ocean in planes not that different from the 747 that existed back then.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#154

Post by [deleted] »

hodgehog11 wrote: post #92
LPisGood wrote: post #86

Earlier quoted context omitted.

It should be noted that optimization of a convex bounded lipschitz function is exactly what most modern statistical learning (AI) models are based on.

Very confused by this comment. The older (poorer) parts of the ML literature focus on models with convex and (gradient-)Lipschitz objectives, but that's not representative of reality, not even close. Modern objectives for AI models are famously nonconvex (catastrophically, from the point of view of classical optimisation theory), and that's where the interesting research is.

[deleted]

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#155

Post by nicf »

vatsachak wrote: post #91
rakel_rakel wrote: post #29

> I don't think researchers in math/TCS will be made obsolete, but I think it will instead no longer make sense to work on any low-hanging, or even medium-hanging (you know what I mean) fruit. We'll be needed for problems where actual novel approaches are needed. I wonder how this compares to what we see happening with "juniors" in software development? In math research, do you also get the training for the professio…

Math is way more automatable than programming. In math, a proof is a proof. We don't know if we can get there and so getting there is the hard part. In software, we always know that we can solve the problem. So HOW to solve the problem is the hard part. Because the type of solution involves maintainability, which involves planning, LLMs suck at it. This leads to "LLM slop code" whereby the LLM creates ad-hoc convolut…

I've spent some time working both as a math researcher and as a software engineer, and I think this comment actually underrates the similarity between the two fields as they're actually practiced.

Some math research does involve grabbing a single, fully specified conjecture off the shelf and hunting for a proof of it, and it's true that if you manage to solve a long-standing open problem, other mathematicians will be interested no matter how you did it.

But this isn't all of what they do, probably not even most of what they do. Like in software engineering, it's not always obvious which question would be the most useful one to ask. A lot of mathematical work also goes into what we call "theory-building", where you could say that primary work goes into coming up with definitions rather than theorems. Mathematicians also care a great deal about how something is proved; a lot of them are some of the most aesthetically picky people I've ever met. Words like "ugly", "beautiful", "creative", and "boring" are used to describe both definitions and proofs all the time.

From the outside, it can look like all they're doing is pumping out proofs at any cost. But I promise you that when I talk to mathematicians who don't have any experience building software, they have a similarly narrow view of that field as well! Both fields, from the inside, look a lot more human than you might expect.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#156

Post by qsera »

peddling-brink wrote: post #13
WA wrote: post #6

Earlier quoted context omitted.

If knowledge is a Swiss cheese, LLMs can help fill the holes, but not make the cheese bigger.

Today maybe. I disagree in the long term. While they’ll never have the same subjective experience as humans, what stops an LLM from applying similar lines of thought* in a manner that results in a novel conjecture? They are prediction machines, and so are we in a way. We can give them nearly limitless resources to scale their predictive capabilities. We have billions of years of training baked in. They distill direct…

>what stops an LLM from applying similar lines of thought* in a manner that results in a novel conjecture?

One thing is that an LLM can never assume, or find out, an inconsistency in its training data. Novel ideas often require correction of existing assumptions. As far as I understand, it is impossible, by design, for LLMs to contradict what is in its training data.

For example, an LLM trained on the data from an internet comprised of people who believe in the earth centric hypothesis can never say "Hey, that cannot be correct", or come up with the heliocentric alternative

But maybe it is not applicable to pure Math...

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#157

Post by ACCount37 »

fidotron wrote: post #136
ACCount37 wrote: post #125

Earlier quoted context omitted.

"Lack" isn't the right word. "Lacking" is more like it. If there was a deep fundamental inability, we wouldn't see things like newer generations of LLMs consistently improving on ARC-AGI series (heavy spatial reasoning loading) and SimpleBench (a lot of commonsense + spatial reasoning components). In a way, it's a surprise that LLMs, notoriously lacking any sort of embodied experience, can even get this close to huma…

> "Lack" isn't the right word. "Lacking" is more like it. Yeah, that's fair. > My takeaway is that text is a far richer modality than anyone has expected - and that high end LLMs are often sharp and flexible enough to recognize their weak points and substitute their strengths. I.e. all the LLMs implementing A* to optimally solve pathfinding in ARC-AGI-3 tasks, often unprompted. I agree and disagree with this. I think…

Grown? LLMs were always "large enough for other reasoning modes to conceivably be hiding in the parameter space".

Basic LLMs don't reason in text, and never did. They use it as an interface - for input, output and some of the intermediate products. Heavy use of those "pseudo-recurrence" intermediates in "reasoning models" is a relatively late post-training adaptation. But the process that happens between those endpoints is not at all text-based. What happens in the hidden dimension is part "output logit domain", tied to probability distributions over possible output tokens, and part "incomprehensible concept-space madness".

The latter being where things like latent world models live. LLMs develop partial world models, right in pre-training, despite not being explicitly forced to - because it brings them closer to heaven of accurate next token prediction.

And yes, larger models like Fable seem to be better at spatial reasoning. Maybe because their large size increases the sample efficiency and improves generalization, allowing them to absorb the sparse signal of "spatial reasoning" in the training text better. Maybe because this extra size means more layers, allowing for deeper latent space reasoning in lieu of true recurrence. Maybe because the default "next token prediction" reward underrates rare spatial reasoning challenges, and the model only starts to "get good" at them once the other sources of loss reduction are heavily depleted. Maybe because no true recurrence is suboptimal for spatial reasoning architecturally. But it is what it is. Spatial reasoning gains in LLMs are extractable, but extracting them is nontrivial.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#158

Post by pfdietz »

skybrian wrote: post #148
p-e-w wrote: post #139

Earlier quoted context omitted.

Unless you’re claiming that AIs will suddenly (and very soon) stop improving, they are obviously a threat to everyone’s job. Calling notable conjectures that have been open for decades “low-hanging fruit” is an act of desperation. Most professional mathematicians couldn’t have proved those conjectures if their lives depended on it.

I wouldn’t call it “low hanging fruit” but it’s easy to think of problems that seem harder. Apparently solving notable math conjectures is easier than building a practical robot to deliver a package to someone’s porch? So, yes, AI is a big deal and we don’t know what it’s going to affect, but the goal of replacing everyone’s job is extremely ambitious and there’s a long way to go. This has to be assessed separately f…

Moravec's Paradox strikes again!

Moravec must be at some level gratified things are arriving close to his predicted timeline.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#159

Post by piloto_ciego »

sashank_1509 wrote: post #116

This is all a depressing and bleak future that I don’t look forward to. One solution is to ban LLM’s, to artificially create a demand for human thought, that just feels like living in an artificially constructed zoo. Another solution is humans don’t do anything that AI can do better , / doesn’t need the human touch. So I suppose we will all become artists, sportsmen or politicians, the only jobs that will remain exce…

This is utopian friend!

Literally anything you wanted to make is no plausible to make if not now then in the next couple years.

The thing you’re worried about is capitalism and the connection with working to having the right to keep living. If you can throw off that mental shackle you can start to see how this can be amazing, but you have to drop the idea that everyone has to work at a job for someone else to provide some service in order to do it. It’s hard, I know, but change your mindset some and dream for a better world and we can make it.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#160

Post by pfdietz »

pydry wrote: post #153
p-e-w wrote: post #139

Earlier quoted context omitted.

Unless you’re claiming that AIs will suddenly (and very soon) stop improving, they are obviously a threat to everyone’s job. Calling notable conjectures that have been open for decades “low-hanging fruit” is an act of desperation. Most professional mathematicians couldn’t have proved those conjectures if their lives depended on it.

>Unless you’re claiming that AIs will suddenly (and very soon) stop improving Most technologies level off sharply after bouts of boundless improvements. In 1968 they thought we'd be flying to the moon by now but instead we're flying across the ocean in planes not that different from the 747 that existed back then.

They sometimes start improving again. In the context of your comment, look how the cost/kg to LEO has suddenly dropped radically. This was mostly due to institutional change that allowed previous non-technological barriers to improvement to be bypassed.
Post reply on HN