41–50 of 61 posts
As an author of the blog, I'll note that this was one of the easiest applications of ADRS. Bowen, who was leading this effort, got things running within a day or two and the initial runs were with free Google credits! It was exciting to see how quickly these kinds of frameworks could be applied to real-world engineering and algorithmic challenges.
Earlier quoted context omitted.
Theoretically if I were to type into an LLM " Write a novel compression algorithm for images that is at least 25% smaller at the same speed and quality as ___ " and it did, and I ran the code (which I didn't understand) and it worked, wouldn't that count? The odds of that working, though, are of course pretty near 0. But theoretically, it could happen.
You might find that if it did produce something, it might not be _novel_
I'm not sure if this is the exact same thing, but a load balancing paper reported a 4.2x speedup by applying a "snake pattern" in 2024: https://arxiv.org/pdf/2402.02447
Thanks for letting us know! While we’re tackling different problems, the core idea around load balancing is quite similar. The pattern might be a familiar trick to those experienced with this kind of problem — you can see my thoughts on it here: https://news.ycombinator.com/item?id=45688236#45689440
> On average, it takes about 540 ms to re-balance the experts and achieves a load balance factor of 0.66 (calculated as the ratio of average to maximum tokens generated per GPU).
> ...
> We also consider a non-public reference implementation from a frontier lab that we have access to. This implementation avoids explicit iteration and reduces the rebalancing algorithm runtime to 19.6 ms while achieving the same balance factor as the open-source algorithm.
> ...
> The resulting algorithm matches the load balance factor of the other baselines while reducing runtime to just 3.7 ms, yielding a 5.0x speedup over the internal reference implementation.
As an author of the blog, I'll note that this was one of the easiest applications of ADRS. Bowen, who was leading this effort, got things running within a day or two and the initial runs were with free Google credits! It was exciting to see how quickly these kinds of frameworks could be applied to real-world engineering and algorithmic challenges.
What does ADRS stand for?
From TFA: https://arxiv.org/pdf/2510.06189
> We term this approach as AI-Driven Research for Systems (ADRS), which iteratively generates, evaluates, and refines solutions.
> The central thesis of this paper is that a new class of AI-driven approaches, which we term AI-Driven Research for Systems (ADRS), is beginning to show promising results in automated algorithm discovery, and will ultimately prompt a re-evaluation of the traditional role of systems researchers.
So, if I got this right, this is just about re-implementing an existing load balancing algorithm faster...? If so, this is really dumb. As you guys checked out, yes most load balancing algorithms are slow/dumb: >First, we evaluate DeepSeek's open-source EPLB implementation. This employs a greedy bin-packing strategy: experts are sorted by load in descending order, and each is placed onto the least-loaded GPU that has…
Thanks for commenting! Actually in this case, "the work being done" can be really fast because it can be done asynchronously. For context, here’s how this translates in a real-world application. The original algorithm was provided by DeepSeek, and our optimized implementation achieves a 92× speedup over it. The 5x number is comparing with another baseline that is undisclosed yet. When integrating EPLB into vLLM, I di…
e: also comparison a fixed (nothing faster than 0!) and random policy might be informative if your intent is to publish this as improvement for the object problem, not just a demonstration of ARDS.
Earlier quoted context omitted.
Thanks! In realistic workloads, the differences won’t be orders of magnitude. I agree that this is a fairly simple problem. Experienced engineers—or anyone who has faced similar challenges—can quickly come up with such solutions. The key point, however, is that others might get stuck in their research simply because they don’t realize these quick solutions exist (“I don’t know what I don’t know”). AI helps bridge tha…
Except that "AI" steals and mostly does not do citations. EDIT: The chutzpah of downvoting this is striking. The paper says "surpasses highly optimized algorithms engineered by human experts to achieve a 5.0x speedup" and https://news.ycombinator.com/item?id=45689663 links to a 2024 paper where humans discovered a 4.2x speedup using a snake pattern. The 2024 paper is not cited .
What "AI" is best at is enabling theft without crediting the true creators
Earlier quoted context omitted.
Thanks! In realistic workloads, the differences won’t be orders of magnitude. I agree that this is a fairly simple problem. Experienced engineers—or anyone who has faced similar challenges—can quickly come up with such solutions. The key point, however, is that others might get stuck in their research simply because they don’t realize these quick solutions exist (“I don’t know what I don’t know”). AI helps bridge tha…
Except that "AI" steals and mostly does not do citations. EDIT: The chutzpah of downvoting this is striking. The paper says "surpasses highly optimized algorithms engineered by human experts to achieve a 5.0x speedup" and https://news.ycombinator.com/item?id=45689663 links to a 2024 paper where humans discovered a 4.2x speedup using a snake pattern. The 2024 paper is not cited .
Earlier quoted context omitted.
Theoretically if I were to type into an LLM " Write a novel compression algorithm for images that is at least 25% smaller at the same speed and quality as ___ " and it did, and I ran the code (which I didn't understand) and it worked, wouldn't that count? The odds of that working, though, are of course pretty near 0. But theoretically, it could happen.
As you say, the odds of this happening are very close to zero. But suppose for a minute that this was possible. Did you learn anything? Do you really have a discovery? Was this done using a novel method or applying something that already exists? If you give this to somebody else, should they believe it works? Is this result even understandable by human beings? You'd need to answer so many questions that in the end ev…
A sufficiently advanced discovery in, say, mathematics can only be understood by other mathematicians. Does that make it less of a discovery? So what's wrong if a machine discovers something that can only be analysed and proved by other machines?