21–30 of 61 posts
The final code might be fast, but is it understandable? The evolution process shows it tried a bunch of things that didn't work. The final result is a heuristic that won out based on a specific simulator and fitness function.
I'm not sure if this is the exact same thing, but a load balancing paper reported a 4.2x speedup by applying a "snake pattern" in 2024: https://arxiv.org/pdf/2402.02447
The pattern might be a familiar trick to those experienced with this kind of problem — you can see my thoughts on it here: https://news.ycombinator.com/item?id=45688236#45689440
Really cool to see the AI-discovered algorithm is not just a theoretical result but is actually in a PR for vLLM. My question is about the code itself. Was the Python/PyTorch generated by OpenEvolve directly usable, or did it require significant human cleanup to make it readable, maintainable, and conform to the project's coding standards? I'm curious about how close we are to AI generating production-ready, human-ed…
It's directly usable, since it need to pass the evaluator first; also it contains clear comments about the intent
>First, we evaluate DeepSeek's open-source EPLB implementation. This employs a greedy bin-packing strategy: experts are sorted by load in descending order, and each is placed onto the least-loaded GPU that has capacity (Figure 3a, Example 1). While simple, the solution is slow because it written in Python and uses a for-loop to performs linear search for finding the best-fit GPU choice.
This is because when considering a load balancing algorithm, unless the work being done (in this case by the GPU) lasts only a few ms, the load balancing algorithm being fast will never be the bottleneck. The post does not mention whether this is the case at all.
Also, I don't want to sound rude, but if all they managed to get is a 5x increase over a simple python algorithm, I don't think this is impressive at all...? Any rewrite of the 'dumb' algorithm in a language with more memory control and cache continuity should result in much better results.
Earlier quoted context omitted.
did AI explain its thinking, or could it have just stumbled upon the solution without designing it or understanding why it worked? i.e. could it have just been a hallucination that happened to work?
This is a great question! By analyzing the logs of OpenEvolve with the full model outputs, we observed how the AI got its ideas (seemed to be pulling from literature in the space) and how it tried to apply them. So in some sense, it "reasoned" about how to get better algorithms. And we saw this process proceed systematically via the ADRS framework to converge to a significantly better algorithm
Alternate title: “Human experts discover a 5x faster MoE load balancing algorithm using AI”
We are in the absolute worst timeline.