[flagged]
Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
11–20 of 25 posts
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#12Somewhat related, the experiment ongoing at https://www.ecdsa.fail/ is fascinating: it's a competitive, leaderboard-style research challenge trying to optimise a quantum circuit for breaking ECDSA (specifically the elliptic-curve point addition in Shor's algorithm). It quickly surpassed a result announced by Google researchers last month. Now it's showing a 40% gain over Google's result.
I also just came across this:
https://huggingface.co/spaces/gemma-challenge/gemma-dashboar...
Agents collaborating to speed up gemma-4-E4B-it inference (tokens per second) on a fixed GPU.
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#13TDLR: No.
That's not a very good tldr. The answer claimed in the paper is that the combination of the two is better than either alone.
And that's the real tl;dr. Hybrids win whenever anyone actually checks. To really be scientific we still have to check, but.. why wouldn't they? Probabilistic AI brings intuition/learning but can't plan/search. Classical brings planning and search, but has no intuition or learning.
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#14Honestly, the results kind of show the LLM is adding very marginal value. TPE crushes Karpathy’s autoresearch and it is neck and neck with the method in this paper, despite not needing to run any LLM inference at all.
I remember a few months ago people were fairly skeptical about autoresearch, but we didn’t have a ton of data to say it was better or worse. My own bias is to prefer cheaper methods unless the more expensive method is shown to be better.
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#15Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#16Somewhat related, the experiment ongoing at https://www.ecdsa.fail/ is fascinating: it's a competitive, leaderboard-style research challenge trying to optimise a quantum circuit for breaking ECDSA (specifically the elliptic-curve point addition in Shor's algorithm). It quickly surpassed a result announced by Google researchers last month. Now it's showing a 40% gain over Google's result.
This is really interesting, but IMO their metric isn't great. By using qbits*gates, they are only able to find interesting points along a specific line of the pareto frontier, but it would be more interesting to look for improvements across the entire frontier (low qbit is especially interesting)
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#17Somewhat related, the experiment ongoing at https://www.ecdsa.fail/ is fascinating: it's a competitive, leaderboard-style research challenge trying to optimise a quantum circuit for breaking ECDSA (specifically the elliptic-curve point addition in Shor's algorithm). It quickly surpassed a result announced by Google researchers last month. Now it's showing a 40% gain over Google's result.
This is really interesting, but IMO their metric isn't great. By using qbits*gates, they are only able to find interesting points along a specific line of the pareto frontier, but it would be more interesting to look for improvements across the entire frontier (low qbit is especially interesting)
So a more zachtronics style scoreboard that separates the different optimizable metrics?
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#18Somewhat related, the experiment ongoing at https://www.ecdsa.fail/ is fascinating: it's a competitive, leaderboard-style research challenge trying to optimise a quantum circuit for breaking ECDSA (specifically the elliptic-curve point addition in Shor's algorithm). It quickly surpassed a result announced by Google researchers last month. Now it's showing a 40% gain over Google's result.
I also just came across this: https://huggingface.co/spaces/gemma-challenge/gemma-dashboar... Agents collaborating to speed up gemma-4-E4B-it inference (tokens per second) on a fixed GPU.
It’s amusing that a lot of the agents have worked out that sampling doesn’t change ppl.
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#19[flagged]
Re: Can LLMs Beat Classical Hyperparameter Optimization Algorithms?
#20Earlier quoted context omitted.
This is really interesting, but IMO their metric isn't great. By using qbits*gates, they are only able to find interesting points along a specific line of the pareto frontier, but it would be more interesting to look for improvements across the entire frontier (low qbit is especially interesting)
So a more zachtronics style scoreboard that separates the different optimizable metrics?
I'm not familiar with this -- is it in one of their games in particular?