Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6 used a prompt to close a 30-year gap in convex optimization

old.reddit.com

Post reply on HN
Print view View original

391–400 of 414 posts

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#391

Post by My_Name »

phillip_kerger wrote: post #352
YeGoblynQueenne wrote: post #326

So if you dig down a bit it turns out the author had been trying to solve that problem for a year with GPT 5.4 and 5.5 and he fed all that information to the prompt he gave to Sol Pro which may or may not had direct access to the author's chat history. So the claimed "148 minutes" was really "a year plus 148 minutes". Moreover, it seems the prompt included the technique used to solve the problem: https://old.reddit.c…

I (author of the original post and paper) can add a few things here: 1. My previous approaches with GPT 5.5 were really not very sophisticated in terms of my input. I threw the problem at it, and just kept encouraging it to go iterate through ideas without any success. 2. The approaches that are in the prompt, though they will seem cryptic to someone not in the field, are relatively natural ideas. In fact, the constr…

There is a firm opinion in some circles that an AI can never do anything but recombine already known things despite a rising number of cases where that very much appears to be an inaccurate view.

Any slim possibility that vital information was given to the AI to make the task just one of recombination, rather than coming up with anything original, is grabbed with both hands no matter how tenuous.

I can understand why, nobody likes the idea of a machine capable of doing the job of a human, and jobs requiring the mind are especially sacrosanct. But like farm workers in the early part of the C20th, not liking the incoming technology won't change the effect it will have when companies get the idea they can make more money and hire less people.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#392

Post by _alternator_ »

hodgehog11 wrote: post #362
_alternator_ wrote: post #115

Earlier quoted context omitted.

I'd push back on this. Most of the core optimization techniques (eg, ADAM, stochastic gradient descent) are straight out of the convex optimization literature. Generally you need to use optimizers that work well on convex objectives because near minimizers, functions tend to be convex. (Proof by contradiction: a non-convex point has a strict descent direction.) The fact that neural networks are highly nonconvex has e…

No, I have to push back as well, sorry. It takes a very long time to get to the "near-minimizer" stage when training a neural network, and in practice, you never get there (see neural scaling law regimes). What you are saying is the viewpoint from 6-7 years ago. Things have changed. The reasons why optimizers work well for neural networks in their highly nonconvex landscapes has absolutely nothing to do with their pe…

It seems that we have two different stories here: in one, the new optimization theory represents a stark departure from the prior art, a sort of revolutionary new view of the understanding of optimization as applied to neural networks.

In the other story, the current understanding of optimization is a natural evolution of past work, where a new generation of researchers respond to social and technological changes, adapting and building on the work of the past, taking what's useful, downplaying the importance of some ideas, and inventing new language to describe concepts that seem most relevant to the current situation.

Both stories tell some of the truth. A revolution or evolution? Looking at the literature (eg the sibling comment here) shows that even today, convexity is used as an intuition pump for modern optimization techniques. But there are also new ideas that apply to the specific exigencies of neural nets, and downplayed ideas (eg convergence rates) that seem less relevant.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#394

Post by My_Name »

sscaryterry wrote: post #46
fidotron wrote: post #36

Earlier quoted context omitted.

Do we employ mathematicians in customer service roles?

Thats a silly and obtuse comment.

It was a sensible, valid and pertinent reply to the parent comment though...

As with most comments, context is quite important.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#395

Post by smaudet »

neerajsi wrote: post #317
smaudet wrote: post #291

Earlier quoted context omitted.

> It's not a zero sum game. Resources are, though. The planet cannot support a race of digital super-people, and us, and an continually growing economy. It's the height of folly to think that, as things are going, we are going anywhere "good".

I was arguing with a friend about this point today. I'd posit that our economy has been growing in a way where goods that require more human time and natural resources to produce are giving way to goods that are intangible. Once we've met our basic material needs, we're tending to consume things that are replicable with low marginal costs, and which do not interfere with the production of other goods. So maybe we can…

> Once we've met our basic material needs

The concern is not goods produced, those are not "resources", except perhaps in the very limited sense that food is a "produce".

Energy, materials, those are resources.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#396

Post by My_Name »

tripleee wrote: post #230
kevincox wrote: post #223

Earlier quoted context omitted.

The problem isn't that we are gaining more knowledge and better technology. The problem is that we are allowing the rich to use the technology to subjugate the majority. If AI improves human productivity so much that millions of people no longer need to work that should be an incredible thing. But the flawed structure of our society punishes those people rather than freeing them to persue endeavors that interest then…

Only solution I can think of is higher taxes for the rich used to provide universal income. I'm not sure how the realities of that would play out, though. Anything based on a free market just ends up in.. this.

I fully agree. Tax wealth, not work. Then it is far easier to lift yourself from poverty, but harder to hold on to vast amounts of wealth.

After WW2, when tax on the wealthy was far greater, sometimes 95% of their income (in the UK at least), it was the period when the largest number of ordinary people were raised from poverty to middle class status, yet the number of rich people also increased greatly. When that stopped, wages stagnated, and now we have a cost of living crisis. Connect the dots, lol...

Many people think that the market is either 100% free, or 100% controlled, but a free market that operates within defined limits that will stop the greatest excesses of the rich ruining things for everyone else is the sweet spot we should be aiming for.

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#397

Post by My_Name »

kevincox wrote: post #240
tripleee wrote: post #230

Earlier quoted context omitted.

Only solution I can think of is higher taxes for the rich used to provide universal income. I'm not sure how the realities of that would play out, though. Anything based on a free market just ends up in.. this.

I'm strongly in favour of this. I think income tax should approach 100%. So your second billion you still make more. But not twice as much. The exact decay function is up to date. But I really don't think your post-tax take home should scale linearly with your income once you hit the max bracket. We can also remove all of the loopholes that result in the biggest companies paying basically no taxes. I also think it ma…

As I say above, here in the UK we used to have the top rate of tax at 95% of your earnings, and it was the period of fastest growth for poor and middle class people (Boomers at the time)

Unfortunately, the rich are the ones most capable of buying democracy, and the current set of politicians seem to very much welcome the money...

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#398

Post by My_Name »

pfdietz wrote: post #356
sashank_1509 wrote: post #116

This is all a depressing and bleak future that I don’t look forward to. One solution is to ban LLM’s, to artificially create a demand for human thought, that just feels like living in an artificially constructed zoo. Another solution is humans don’t do anything that AI can do better , / doesn’t need the human touch. So I suppose we will all become artists, sportsmen or politicians, the only jobs that will remain exce…

> This is all a depressing and bleak future that I don’t look forward to. Or, for the vast majority of humanity who aren't professional mathematicians, it's a liberating and empowering advancement. I'm sorry if the automated digital theorem looms are eliminating the market for your handwoven proofs.

Indeed. Where do you throw your wooden shoes these days? (Not that it was an effective tactic then anyway)

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#399

Post by smaudet »

astrange wrote: post #358
smaudet wrote: post #291

Earlier quoted context omitted.

> It's not a zero sum game. Resources are, though. The planet cannot support a race of digital super-people, and us, and an continually growing economy. It's the height of folly to think that, as things are going, we are going anywhere "good".

Economic growth doesn't mean using up resources faster, it means trading things faster.

You are correct about growth, but speed is also a resource - kinetic energy is still energy, which comes from somewhere, and unless you have 100% perfect energy usage, there is thermodynamic loss, contributing to entropy, which itself is another resource. Temperature is a physical resource all lifeforms and machine processes share in a gravity well, and all processes breakdown after a melting point (unable to maintain their own required entropy levels)

Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization

#400

Post by MinimalAction »

weregiraffe wrote: post #376
MinimalAction wrote: post #66

Earlier quoted context omitted.

That view is incredibly reductionist. It really is an efficient encoding of how nature behaves. It might be a human construct, but given how best it allows to understand nature (through principles of physics), it is uncanny to be any different from the language of nature. Reminds me of Wigner's Unreasonable effectiveness of mathematics in natural sciences [0]. [0]: https://en.wikipedia.org/wiki/The_Unreasonable_Effec…

Nature has no language. Language is a human tool to imperfectly describe nature. Putting language before nature is magical thinking (also known as Platonism).

Yes, of course mathematics is a tool to model nature as accurately as we could. That still doesn't mean mathematics is as trivial or frivolous as manipulating symbols.
Post reply on HN