nah, scientist with the name Shakey Onail found this and all creds are given to LLMs is crazy
Do you have a link to Shakey's publication?
GPT-5.6 used a prompt to close a 30-year gap in convex optimization
261–270 of 414 posts
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#262Earlier quoted context omitted.
It genuinely scares me that some people's first reaction to this news is banning LLM.
Maybe we should also burn books and lobotomize ourselves to make science more fun and challenging. The ego here sickens me. I don't care about your ego. I want accelerated material science, medical science, energy, better outcomes for people across the world.
> I want accelerated material science, medical science, energy, better outcomes for people across the world.
There will be people benefiting from advances in these fields.
Neither you nor I will be one of them.
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#263I know a bit about this field. This conjecture reads as somewhat more niche than the cyclic double cover conjecture recently proved by OpenAI, but nevertheless represents a real contribution. You want to know how long it takes to solve an optimization problem, in this case over convex, lipschitz functions. (The restriction to a spherical domain is not really a restriction, you can just change variables for any bounde…
[flagged]
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#264Earlier quoted context omitted.
Around here AI isn't really more of a threat to juniors than it is to seniors. It's a threat to the people who have been taught "recipies" rather than applied computer science. You can have excellent seniors who can do TDD, DRY, SOLID and so on, who also happen to have no idea what a L1 cache miss is. The current AI models know all of those things, but they struggle applying them correctly without someone piloting th…
Interesting, thanks. I don't know where "around here" is, but the signals I've seen in a lot of articles is that the demand for junior software people has taken a dive since a year or two back, with student programs etc getting cancelled. One googler said they were getting a junior to their team and that was kind of a big deal because it hadn't happened in that whole department for a long time. In relation to that, I…
There is definitely the effect of less or de-growth in the industry, which started before the current AI hype. And now there's the additional effect of companies hoping AI will replace their need for (junior) devs. Nobody knows if or to what degree this will work out (yes, we all have opinions, but no crystal balls), but they are holding back the hiring until they know how all this pans out.
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#265Earlier quoted context omitted.
Math is way more automatable than programming. In math, a proof is a proof. We don't know if we can get there and so getting there is the hard part. In software, we always know that we can solve the problem. So HOW to solve the problem is the hard part. Because the type of solution involves maintainability, which involves planning, LLMs suck at it. This leads to "LLM slop code" whereby the LLM creates ad-hoc convolut…
I've spent some time working both as a math researcher and as a software engineer, and I think this comment actually underrates the similarity between the two fields as they're actually practiced. Some math research does involve grabbing a single, fully specified conjecture off the shelf and hunting for a proof of it, and it's true that if you manage to solve a long-standing open problem, other mathematicians will be…
The types who do ugly proofs, when they write code, produce spaghetti code. It's the same thought process going into how to approach something.
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#266Earlier quoted context omitted.
The author also used GPT-5.6 to write the prompt. This did involve giving GPT-5.6 access to his previous work and a back and forth process (so definitely still used the author's expertise to some degree), but the prompt itself is also largely AI generated.
What's the difference between using GPT to write the prompt to GPT, and "thinking"? The LLM uses the first tokens to predict more tokens, and then uses those tokens to predict even more tokens.
The ability to correct the plan/prompt.
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#267Earlier quoted context omitted.
ADAM does not work on simple convex problems [1]. [1] https://parameterfree.com/2020/12/06/neural-network-maybe-evolved-to-make-adam-the-best-optimizer/ [2] https://arxiv.org/pdf/1905.09997 [1] refers to [2], which shows that ADAM is not as efficient as gradient descent with line search on some problems, including neural networks.
I'll point out that "does not work" is not the same as "not as efficient" :) But it does seem the Adam paper had an error. I think that Nesterov's first order method is the most efficient general first order algorithm on convex problems, so anything else is in some sense worse. (Edit: removed incorrect ADAM comment.)
Yours' "not as efficient" in [2] means that, sometimes, ADAM "does not work." Look at figure 2, ADAM literally does not work in the case of "true model."
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#268Earlier quoted context omitted.
You ever play a video game with god mode cheats enabled, so you can unlock all the unlocks, get all the best gear, and be an unstoppable force with unlimited money? Yeah, it's fun for 30 minutes.
Exactly, a lot of people keep attacking me as anxiety for losing my livelihood. It’s not that. It’s just depression that another avenue for human craftsmanship has been taken over by the machines. Humans are not just infinite consumers. If AI solves all of humanity problems, it’s like living in a zoo, not a life worth living imo.
This has always been the case though that "another avenue of human craftsmanship has been taken over by the machines."
It's good if we can have robots building things instead of having humans slumped over a workbench in a sweatshop piecing things together. It's good if we can have LLMs spitting out code rather than CS grads working 15 hour days at fintech startups or whatever. The conditions were never (ever) good before.
And, it won't be like a zoo - you'll be able to go wherever you want, do more or less whatever you want. Think about living in The Culture, or the Star Trek universe or whatever. There are options beyond "I'm a pet to the machines." Think big, dream big, then help make it a reality!
Like, out in space you're still going to need a human to make decisions because you can't wait 30min for the tight-beam signal to get back to earth. Also, we're pretty good at soaking up rads and still being "useful" - at least so far I don't see that being a major advantage to the robots. Maybe our place is to be the deep space mechanics that keep the robots alive? I don't know, regardless, you should dream big. What kind of world do you want to live in? Ok, how do we make that world happen?
My big (somewhat unspoken and somewhat immature dream) is that advances in regenerative health tech fix my optic nerves and I can get into the cockpit again one day, then maybe later I can fly some space vehicle like I wanted to since I was a 12 year old. Immature I know, but I miss flying still.
Is that world possible without AI? Probably/maybe? But it's a lot more plausible in a world where we have folded every protein, we have robot surgeons doing robotic procedures, AI generated research, etc.
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#269Earlier quoted context omitted.
I was thinking this past week I have gotten so lazy w my prompting via CLIs. Back in the before I had put such discipline into my prompting and supporting context. Now I’m like, “look here and here and here are some tools, and /skill /skill okay go.” Or “restate this request in your own words and enrich it as appropriate handling any gaps. Okay go”
We're also at the point where you can roll out context to your entire organisation. I created an app for our m365 Cowork and deployed it to everyone who develops software. It does a couple of things, but it main knows our compliance policies and can guide developers through writing the documentation needed for NIS2 compliance. It also guardrails against non-approved packages, and helps developers find alternatives, o…
Yes. I worked on two large monorepos, about one quarter per project. Maybe 80+ devs on first, maybe 40+ on second. Both were not sure how to describe this but ultra-high-velocity agentic driven development efforts.
On the first, there were ~no shared skills. There were some requirements set up but they were not minded properly and became stale / ate context for little gain. The hardest hit was in E2E tests which would flake and create long running, too-often failing CI. People would disable them, because they were not reliable and velocity was so high, no one was happy w them.
I maintained my own set of skills and CLIs to back them. I'd share them if they came up but it was like the old days of manage your own stuff. Not much credit for building and sharing devex tooling to the team.
But then on the second one we were in better shape--we had vendoring set up to distro skills automatically.
Before the project was well underway, I put time into understanding how all of our tests aught to be written. Finding the forbidden things, etc, getting review from our best test folks and ultimately landed on a `/test` that routed across all possible test types.
Like night and day. Instead of finding out while trying to get a release out the door that some corner of the project had a handful of flakes, tests were written the right way from the start.
Like, it was beautiful. And I don't think devs noted difference while building. Only that there was an absence of BS in CI.
Hard to quantify the lack of pain, but it was big!
Re: GPT-5.6 used a prompt to close a 30-year gap in convex optimization
#270Crazy how intelligence is cheap, efficient and commonplace now. We humans better refocusing our energy on our core values/principles, given most of our skills are becoming irrelevant
Intelligence on its own is not very useful though. We put it on a pedestal because it creates huge potential when paired with other things, wisdom, discipline, empathy, but on its own?
Llm's do have discipline and persistence, though.