Really cool to see the AI-discovered algorithm is not just a theoretical result but is actually in a PR for vLLM.
My question is about the code itself. Was the Python/PyTorch generated by OpenEvolve directly usable, or did it require significant human cleanup to make it readable, maintainable, and conform to the project's coding standards? I'm curious about how close we are to AI generating production-ready, human-editable code for complex algorithms.
Untitled topic
11–20 of 61 posts
Re: undefined
#12I'm not sure if this is the exact same thing, but a load balancing paper reported a 4.2x speedup by applying a "snake pattern" in 2024:
Re: undefined
#13this feels less like Copilot and more like AlphaGo for systems programming. it's not just finding patterns in existing code, but discovering novel and more efficient strategies in a given problem space. Very cool.
Re: undefined
#14Nice result, but the snake pattern is pretty obvious and intuitive even for a human who just glances over the problem. It kinda breaks if there is huge variance (if the top load expert is orders of magnitude higher than #2 it probably should just get its own GPU), but I'm not familiar enough with MoE to know if that's a realistic possibility.
Thanks! In realistic workloads, the differences won’t be orders of magnitude. I agree that this is a fairly simple problem. Experienced engineers—or anyone who has faced similar challenges—can quickly come up with such solutions. The key point, however, is that others might get stuck in their research simply because they don’t realize these quick solutions exist (“I don’t know what I don’t know”). AI helps bridge tha…
Except that "AI" steals and mostly does not do citations.
EDIT: The chutzpah of downvoting this is striking. The paper says "surpasses highly optimized algorithms engineered by human experts to achieve a 5.0x speedup" and https://news.ycombinator.com/item?id=45689663 links to a 2024 paper where humans discovered a 4.2x speedup using a snake pattern. The 2024 paper is not cited.
Re: undefined
#15I'm not sure if this is the exact same thing, but a load balancing paper reported a 4.2x speedup by applying a "snake pattern" in 2024: https://arxiv.org/pdf/2402.02447
Seems the same tbh
Re: undefined
#16Earlier quoted context omitted.
Thanks! In realistic workloads, the differences won’t be orders of magnitude. I agree that this is a fairly simple problem. Experienced engineers—or anyone who has faced similar challenges—can quickly come up with such solutions. The key point, however, is that others might get stuck in their research simply because they don’t realize these quick solutions exist (“I don’t know what I don’t know”). AI helps bridge tha…
Except that "AI" steals and mostly does not do citations. EDIT: The chutzpah of downvoting this is striking. The paper says "surpasses highly optimized algorithms engineered by human experts to achieve a 5.0x speedup" and https://news.ycombinator.com/item?id=45689663 links to a 2024 paper where humans discovered a 4.2x speedup using a snake pattern. The 2024 paper is not cited .
that's true for any application of AI :(
Re: undefined
#17Really cool to see the AI-discovered algorithm is not just a theoretical result but is actually in a PR for vLLM. My question is about the code itself. Was the Python/PyTorch generated by OpenEvolve directly usable, or did it require significant human cleanup to make it readable, maintainable, and conform to the project's coding standards? I'm curious about how close we are to AI generating production-ready, human-ed…
It's directly usable, since it need to pass the evaluator first; also it contains clear comments about the intent
Re: undefined
#18i wonder how hard it is to get the setup for AI to evolve on?
I spent 2~3 hours setting up, most of the time was spent on writing the evaluator
Actually I think the evaluator will be the most important part for the whole pipeline to work
Re: undefined
#19The final code might be fast, but is it understandable? The evolution process shows it tried a bunch of things that didn't work. The final result is a heuristic that won out based on a specific simulator and fitness function.
Re: undefined
#20i wonder how hard it is to get the setup for AI to evolve on?
I spent 2~3 hours setting up, most of the time was spent on writing the evaluator Actually I think the evaluator will be the most important part for the whole pipeline to work
Yes, getting the right workloads and ensuring correctness are crucial parts of the process