Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 used a prompt to close a 30-year gap in convex optimization

old.reddit.com

Post reply on HN
Print view View original

121–130 of 414 posts

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#121

Post by pessimizer »

semiquaver wrote: post #101
cromka wrote: post #19

Earlier quoted context omitted.

That doesn't make any sense; you can't have one LLM to read your mind to prompt another LLM.

> you can't have one LLM to read your mind to prompt another LLM I’m excited to inform you that we as a species have developed a particularly useful facility known as Language which these LLM tools are evidently rather handy at wielding. This facility is particularly useful in this context when it takes the form of “dialog” or “questioning”, which can be used to propagate abstract ideas by means of mutually-feedback-…

This is a lot of words to say that a human can prompt an LLM to tell it what they want.

edit: it reminds me of all that I have to wade through after I've asked an LLM a straightforward question and the answer should have been "yes, you're right."

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#122

Post by wanderlust123 »

vatsachak wrote: post #105
sashank_1509 wrote: post #103

Earlier quoted context omitted.

> So I'll say it again, AI will win a fields medal for before managing a McDonald's simply because there are enough big problems within arms reach than their current capacity to plan over time AI can manage a McDonald’s already. If manage means directing humans to do something to ensure the store is running. If manage means running robots, then yes maybe that is 5 years away but just directing humans to run a store,…

No it can't. Show me a business which uses in context learning to manage a McDonald's

Well that’s a problem of incentives. Why would a manager outsource their own job to an AI?

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#123

Post by vatsachak »

fsmv wrote: post #119
vatsachak wrote: post #91

Earlier quoted context omitted.

Math is way more automatable than programming. In math, a proof is a proof. We don't know if we can get there and so getting there is the hard part. In software, we always know that we can solve the problem. So HOW to solve the problem is the hard part. Because the type of solution involves maintainability, which involves planning, LLMs suck at it. This leads to "LLM slop code" whereby the LLM creates ad-hoc convolut…

I think the difference is in math the problem is fully specified and easily verifiable and in programming it's not. I don't agree that we always know we can solve the problem.

Not always, sure but 90% of the time yes.

For example, create a DFA for a regex, not too bad just use Thompson's algorithm and then NFA->DFA. But now we have to care about efficiency, user API, maintainability of definitions etc.

Coding is more of a human problem than math

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#124

Post by vatsachak »

wanderlust123 wrote: post #122
vatsachak wrote: post #105

Earlier quoted context omitted.

No it can't. Show me a business which uses in context learning to manage a McDonald's

Well that’s a problem of incentives. Why would a manager outsource their own job to an AI?

It's not a problem of incentives. Every executive wants to inject LLMs everywhere these days. If they haven't somewhere it means that it does not work.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#125

Post by ACCount37 »

fidotron wrote: post #32
mw67 wrote: post #14

Crazy how intelligence is cheap, efficient and commonplace now. We humans better refocusing our energy on our core values/principles, given most of our skills are becoming irrelevant

It's still clear that LLMs lack spatial reasoning, either in the concrete or abstract, and while that sort of reasoning has been downplayed by academia for at least a century it is fundamental to technology and industry. (And many would say for science and mathematics too). They will, however, get there as well either directly or as interfaces to models that do, and your core point stands.

"Lack" isn't the right word. "Lacking" is more like it.

If there was a deep fundamental inability, we wouldn't see things like newer generations of LLMs consistently improving on ARC-AGI series (heavy spatial reasoning loading) and SimpleBench (a lot of commonsense + spatial reasoning components).

In a way, it's a surprise that LLMs, notoriously lacking any sort of embodied experience, can even get this close to human baselines on tasks like this.

My takeaway is that text is a far richer modality than anyone has expected - and that high end LLMs are often sharp and flexible enough to recognize their weak points and substitute their strengths. I.e. all the LLMs implementing A* to optimally solve pathfinding in ARC-AGI-3 tasks, often unprompted.

There might still be unrealized gains there from true depth-unbounded recurrence, or maybe from finding better ways to integrate modalities in training. But clearly, a "fundamental limit" it ain't.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#126

Post by pessimizer »

threethirtytwo wrote: post #108

Genuine question: If you still or did think LLMs are just stochastic parrots that just summarize everything and have no form of creativity, what do you think after seeing results like this? I'm very curious how people reconcile their fear/hatred of AI with actual objective reality. This is actually what interests me most about the whole AI thing. How we tell ourselves what we tell ourselves.

I'm very curious why people conflate thinking LLMs are stochastic parrots with "fear/hatred" of AI. It seems like you're arguing with people who agree that it works and it helps, but you're trying to insist that this implies that they should kneel down and pray to it.

Is "stochastic parrot" too disrespectful for you? Do you think it is a slur?

edit: and this is a genuine question, also. How do you do stochastic parrot = "just summarize everything" = "no form of creativity" = "fear/hatred" so quickly?

Are summaries not creative? Are Maxwell's equations not summaries? Do people hate and fear parrots?

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#127

Post by skeledrew »

qarl2 wrote: post #45
peddling-brink wrote: post #13

Earlier quoted context omitted.

Today maybe. I disagree in the long term. While they’ll never have the same subjective experience as humans, what stops an LLM from applying similar lines of thought* in a manner that results in a novel conjecture? They are prediction machines, and so are we in a way. We can give them nearly limitless resources to scale their predictive capabilities. We have billions of years of training baked in. They distill direct…

> While they’ll never have the same subjective experience as humans You state this as a fact - are you aware the question is unresolved? EDIT: I'd love to know why you're downvoting me for stating a known fact.

Fear spreads.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#128

Post by artninja1988 »

William_BB wrote: post #85
mw67 wrote: post #14

Crazy how intelligence is cheap, efficient and commonplace now. We humans better refocusing our energy on our core values/principles, given most of our skills are becoming irrelevant

Ever heard of the infinite monkey theorem? This is basically what LLMs do on really hard tasks. Prompt it a million times on a really hard problem and it might output the correct answer once.

>Ever heard of the infinite monkey theorem?

Even if every atom in the universe were a supercomputer generating a trillion trillion random characters every second since the Big Bang, the chance of producing Hamlet would still be essentially zero.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#129

Post by skeledrew »

semiquaver wrote: post #101
cromka wrote: post #19

Earlier quoted context omitted.

That doesn't make any sense; you can't have one LLM to read your mind to prompt another LLM.

> you can't have one LLM to read your mind to prompt another LLM I’m excited to inform you that we as a species have developed a particularly useful facility known as Language which these LLM tools are evidently rather handy at wielding. This facility is particularly useful in this context when it takes the form of “dialog” or “questioning”, which can be used to propagate abstract ideas by means of mutually-feedback-…

Made my day XD

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#130

Post by skeledrew »

jdw64 wrote: post #24
semiquaver wrote: post #15

Earlier quoted context omitted.

You’re at least 18 months out of date claiming that prompting will be the new hot skill. Turns out LLMs are also good at prompting other LLMs.

Rather than prompt engineering, I think it should be called overall harness engineering. Anyway, that's how I feel these days

I think harness engineering is more broad, including not only the - system - prompt but also tools and skills made available to the LLM.
Post reply on HN