231–240 of 298 posts
I wonder if evolvable hardware [0] is the next step. In 1996, they optimized an FPGA using a genetic algorithm. It evolved gates disconnected from the circuit, but were required. The circuit exploited the minuscule magnetic fields from the disconnected gates rather than the logical connections. [0] https://en.wikipedia.org/wiki/Evolvable_hardware
This looks like something that can (and should) be reimplemented open-source. It doesn't look like a particularly daunting project.
I've been working on something very similar as a tool for my own AI research -- though I don't have the success they claim. Mine often plateaus on the optimization metric. I think there's secret sauce in the meta-prompting and meta-heuristic comments from the paper that are quite vague, but it makes sense -- it changes the dynamics of the search space and helps the LLM get out of ruts. I'm now going to try to integra…
Show the training set, and PROVE that the tasks and answers aren't in there. I don't understand why this is not a default first step for proving that this is creating new knowledge.
Are you claiming that for the open problems they give record-breaking solutions for, there were just answers on the web waiting to be found?
Show the training set, and PROVE that the tasks and answers aren't in there. I don't understand why this is not a default first step for proving that this is creating new knowledge.
It's Google. Assume the training set contains, as a subset, the entirety of all public digitized information. How would you like to them to share it?
Show the training set, and PROVE that the tasks and answers aren't in there. I don't understand why this is not a default first step for proving that this is creating new knowledge.
How can you actually verify it, even if they provide something?
Maybe this one can stop writing a fucking essay in code comments. I'm now no longer surprised just how consistently all the gemini models overcomplicate coding challenges or just plain get them wrong. Claude is just consistently spot on. A few salient comments for tricky code instead of incessantly telling me what it's changed and what I might want to do, incorrect assumptions when it has the code or is something we'…
Just ask it to only add comments on complex parts (or not at all). Prompt engineering.
Maybe this one can stop writing a fucking essay in code comments. I'm now no longer surprised just how consistently all the gemini models overcomplicate coding challenges or just plain get them wrong. Claude is just consistently spot on. A few salient comments for tricky code instead of incessantly telling me what it's changed and what I might want to do, incorrect assumptions when it has the code or is something we'…
The model likely is doing it more for itself than for you. You can take the code and give it to another LLM instance and ask it to strip all comments.
Earlier quoted context omitted.
What it essentially does is a debugging/optimization loop where you change one thing, eval, repeat it again and compare results. Previously we needed to have a human in the loop to do the change. Of course we have automated hyperparameter tuning (and similar things), but that only works only in a rigidly defined search space. Will we see LLMs generating new improved LLM architectures, now fully incomprehensible to hu…
The “fully incomprehensible to humans” aspect of this potential future state interests me as a software person. The last 50 years of software evolution have been driven by a need to scale human comprehension for larger and more integrated codebases. If we decreasingly need/rely on humans to understand our code, source code’s forward-progress flywheel is going to slow down and will bring us closer to (as you suggest)…
Earlier quoted context omitted.
They address this in the AlphaEvolve paper: "While AI Co-Scientist represents scientific hypotheses and their evaluation criteria in natural language, AlphaEvolve focuses on evolving code, and directs evolution using programmatic evaluation functions. This choice enables us to substantially sidestep LLM hallucinations, which allows AlphaEvolve to carry on the evolution process for a large number of time steps."
It is interesting how google turned the tide on GenAI race, and seems to be leading the pack, with not only great fundamental research, but also interesting model and products. To what extent these remain a niche/nice to have or become a sensation remains to be seen, but I hope if they don't reach hype status, they might be released to the open weights world.
For example, for self driving, it makes much more sense to treat it like a game, where the model learns the evolution of the surrounding environment, and learns how its own actions affect it, and can MCTS its way into correct behavior - specifically because once it learns the environment dynamics, it can internally simulate crashes and retrain itself.
If this process is refined (namely the functions that control direction of training) , you can pretty much start training a model on the dataset of real world (sights, sounds, physical interactions, as well as digital ones), and as it learns the environment, it can be further and further refined, and then we get to the point where it can self evolve its decision making to be truly considered "intelligent".
From the paper, "Notably, for multiplying two 4 × 4 matrices, applying the algorithm of Strassen recursively results in an algorithm with 49 multiplications, which works over any field...AlphaEvolve is the first method to find an algorithm to multiply two 4 × 4 complex-valued matrices using 48 multiplications." If you do naive matrix multiplication, you get a sense that you're doing similar work multiple times, but i…