Show the training set, and PROVE that the tasks and answers aren't in there. I don't understand why this is not a default first step for proving that this is creating new knowledge.
How can you actually verify it, even if they provide something?
AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
191–200 of 298 posts
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#192Maybe the actual solution to the interpretability/blackbox problem is to not ask the llm to execute a given task, but rather to write deterministic programs that can execute the task.
That is what I think is most interesting about it. You get repeatable efficiency gains rather than burning GPU time in data centres.
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#193sheeesh
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#194Software engineering will be completely solved. Even systems like v0 are astounding in their ability to generate code, and are very primitive to whats coming. I get downvoted on HN for this opinion, but its truly going to happen. Any system that can produce code, test the code, and iterate if needed will eventually outperform humans. Add in the reinforcement learning, where they can run the code, and train the model…
lol
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#195Earlier quoted context omitted.
That right and In fact it’s the core purpose of the tool. This is complex automation which by definition compresses the solution into a computable process that works more efficiently than the non-automated process That, in fact, is the revolutionary part - you’re changing how energy is used to solve the problem.
Faster, yes; more efficiently...I guess that's why they're funding nuclear plants then?
Yes. Electrical power generation, transmission and transformation into work is more efficient, per labor unit, than using a human.
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#196From the paper, "Notably, for multiplying two 4 × 4 matrices, applying the algorithm of Strassen recursively results in an algorithm with 49 multiplications, which works over any field...AlphaEvolve is the first method to find an algorithm to multiply two 4 × 4 complex-valued matrices using 48 multiplications." If you do naive matrix multiplication, you get a sense that you're doing similar work multiple times, but i…
Are you sure the saving needs complex values? I think their algorithm works over any char 0 field. Probably needs to just divide by some divisor of 4!=24 if I had to guess.
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#197Show the training set, and PROVE that the tasks and answers aren't in there. I don't understand why this is not a default first step for proving that this is creating new knowledge.
Well that's harder than maybe solving well-known open problems (whose soln's are presumably not in training set lol) but it seems that their examples are not clearly breaking sota, especially on matmul
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#198It's hard to stake out a defensible position on bold claims like these because, if they were as presented, it's hard to see how you haven't simply completed runaway AI. Philosophically, let's say you talk an old LLM through a new discovery. Thanks to your instruction, the LLM now has access to "new" information not in its training data. It is certainly capable of this. The problem in is that this is just laundered hu…
runaway AI is a process not a moment.
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#199[dead]
Re: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
#200Earlier quoted context omitted.
100%. LLMs are extremely useful for doing obvious but repetitive optimizations that a human might miss.
What it essentially does is a debugging/optimization loop where you change one thing, eval, repeat it again and compare results. Previously we needed to have a human in the loop to do the change. Of course we have automated hyperparameter tuning (and similar things), but that only works only in a rigidly defined search space. Will we see LLMs generating new improved LLM architectures, now fully incomprehensible to hu…
The “fully incomprehensible to humans” aspect of this potential future state interests me as a software person.
The last 50 years of software evolution have been driven by a need to scale human comprehension for larger and more integrated codebases. If we decreasingly need/rely on humans to understand our code, source code’s forward-progress flywheel is going to slow down and will bring us closer to (as you suggest) incomprehensibility.
Not only did we scale the breadth of codebases - the flywheel built layers and layers of abstraction over time (have you seen the code sample in this article??), fostering a growing market of professional developers and their career progressions; if most code becomes incomprehensible, itll be the code closer to “the bottom”, a thin wrapper of API on top of an expanding mass of throwaway whatever-language AlphaAlgo creates.
If we don’t wrangle this, it will destroy a profession and leave us with trillions of LoC that only people with GPUs can understand. Which may be another profession I suppose.