Earlier quoted context omitted.
People used to complain about compilers as well.
Compilers are deterministic.
So are AI.
Unsolved Problem by Fields Medalist Breached by Two High School Students
51–58 of 58 posts
Re: Unsolved Problem by Fields Medalist Breached by Two High School Students
#52Earlier quoted context omitted.
In my experience - a robust testing harness will get you the safety you need. And most software you describe has such testing. I think the Bun port is a great example where testing enabled a very successful implementation. (Both the original tests themselves and runtime comparisons to the previous implementation.)
Testing is an extremely inadequate measure of reliability and robustness.
You understand that robust testing can, should, and often does test directly for reliability and robustness, right?
Re: Unsolved Problem by Fields Medalist Breached by Two High School Students
#53Earlier quoted context omitted.
But isn't it a horrible thing that what you just described (being forced by the prisoner's dilemma), is what defines progress these days?
There is nothing I can do about that. Investors & shareholders believe that AI is the future so that's where all the money is going.
I know. I just thought it was kind of a sad state of affairs....
Re: Unsolved Problem by Fields Medalist Breached by Two High School Students
#54Real title includes: > Unsolved Problem by Fields Medalist Breached by Two High School Students with AI My title recommendation: Fields Medalist Problem Solved With AI They used AI for “ computation, proof idea generation , and editing assistance”. It’s a bit odd how they list the AIs used - “Claude Opus 5, Anthropic and ChatGPT Sol5.6 were used for calculations, proof ideas, and editorial assistance.”
note HEAVILY > The students heavily utilized AI assistants, specifically Claude Opus 5 and GPT-5.6 Sol, for computational exploration, proof idea generation, and editing. this is a bit like Enhanced Olympics( https://www.enhanced.com ), except that you have someone else compete for you. My issue is that it makes it hard to distinguish real insight/work etc. from effectively null one. An old instance of the same issue…
For short problems the LLM are just too good now, but for long problems they still get in trouble.
It's like bicycle or F1 race. It goes faster, but you still have to steer the boat to reach somewhere. (Or probably something in between, like a motorcycle race.) Also, the "kids" were guide by a postdoc, not completely on their own.
Re: Unsolved Problem by Fields Medalist Breached by Two High School Students
#55Real title includes: > Unsolved Problem by Fields Medalist Breached by Two High School Students with AI My title recommendation: Fields Medalist Problem Solved With AI They used AI for “ computation, proof idea generation , and editing assistance”. It’s a bit odd how they list the AIs used - “Claude Opus 5, Anthropic and ChatGPT Sol5.6 were used for calculations, proof ideas, and editorial assistance.”
note HEAVILY > The students heavily utilized AI assistants, specifically Claude Opus 5 and GPT-5.6 Sol, for computational exploration, proof idea generation, and editing. this is a bit like Enhanced Olympics( https://www.enhanced.com ), except that you have someone else compete for you. My issue is that it makes it hard to distinguish real insight/work etc. from effectively null one. An old instance of the same issue…
Difference being anyone can be a "script kiddie". I don't think anyone could direct an AI to proofs like this one.
Re: Unsolved Problem by Fields Medalist Breached by Two High School Students
#56Earlier quoted context omitted.
Testing is an extremely inadequate measure of reliability and robustness.
You understand that robust testing can, should, and often does test directly for reliability and robustness, right?
Yes of course, but you also understand that as Dijkstra said "tests cannot show the absence of bugs."
Tests can show you problems, if you can find them, but they cannot show that there are no problems. Property based testing or fuzzing gets your more coverage, and is a good step, but it is still nothing compared to proving things or understanding how something is built and that it is solid. Testing works towards checking for reliability and robustness, but often it's only 10% (?) of the job.