[flagged]
Auto-research with codex: How I achieved a 232x Faster Kernel
91–100 of 104 posts
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#92One thing worth to note in the competition is that 8 out of the 10 top solutions, which all happened to be optimized this way completely broke at any other input than the competition ones. The only solutions that did not break when tested with OOD shapes were made by experts who know a lot about GPU programming and that did not create 25k lines of CUDA but followed and adjusted their solution in reasonable bounds. Th…
This is one of the dilemmas that I am trying to wrap my head around. I love optimizing software pipelines, which often boils down to figuring out the operational constraints that the compiler and the generic libraries can’t assume. Then I exploit these to squeeze out performance. But in a world I can start from scratch and code a domain specific solution from line zero in a matter of hours/days, I do not need general…
You already answered your own question. Testing and trust have always been the most important part.
This has nothing to do with LLMs. There have always been plenty of solutions that are more capable, but untrusted.
Your real question seems to be whether you can prioritize better. What are your project goals? If you have no say in or insight into those goals, you have an even bigger problem. What are you even working on?
For sure, you don't want to be maintaining your dependencies. LLMs make it trivial to rack up insane amounts of technical debt. Why is that appealing to anyone? How is it meaningfully different from the idiots wanting to fork everything on github and copypasta their way to startup success over a decade ago?
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#93Damn! If a solo engineer can do this, it makes the most around OAI/Anthropic start to look pretty weak.
Yeah, but here's a dirty little secret that very few people are discussing: You can't use Claude for this sort of thing if the goal is to make better AI systems. Anthropic finetunes Claude to dissuade people and the agent from using research that actually works. Anything that they use internally in their own models is poisoned, to protect their moat. By proxy, that also means any openweights model that was distilled…
> Thankfully, I don't believe OpenAI does this - they are far more honest and seem to care about their reputation.
Bro. Sam Altman?
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#94In the last couple of days I wanted to try out the new definitive DeepSeek v4 releases. I gave it the repository of a semi-abandoned video compression codec and I told it to perform the usual benchmark -> profile -> verify -> research -> improve loop. I specifically chose this codec because the authors include a verifier for the bitstream to make sure you don't break stuff if you want to try your own implementation.…
I've had a lot of success decompiling old video game ROMs in exactly this way. Like you say - give it a way of verifying correctness - put it in a loop - and they are quite surprising. https://github.com/qarl/arcade-js
I wonder if this will result in discovery of edge cases in the logic that could be exploited to get higher scores (glitches/bugs and such!)
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#95Earlier quoted context omitted.
Yeah, but here's a dirty little secret that very few people are discussing: You can't use Claude for this sort of thing if the goal is to make better AI systems. Anthropic finetunes Claude to dissuade people and the agent from using research that actually works. Anything that they use internally in their own models is poisoned, to protect their moat. By proxy, that also means any openweights model that was distilled…
> Thankfully, I don't believe OpenAI does this - they are far more honest and seem to care about their reputation. Bro. Sam Altman?
OpenAI is run by its employees and their culture is far healthier. Unlike Anthropic, they don't have a CEO that actively encourages their employees to be dishonest.
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#96[flagged]
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#97Earlier quoted context omitted.
I did something similar recently with Google's C# protobuf library. I had spotted I was getting CPU bound rather than memory bandwidth bound when doing streaming of uint32 buffers in dotnet gRPC. I then asked claude to compare the C#/.NET implementation in the library with the C++ version, and it quickly identified that the C# library was missing a couple of fairly cheap optimisations that were present in the C++ ver…
You presented another thing LLMs excel at: integrating something from a project that is not present in another one. I think they work so well at this because both the starting and ending points have an already existing structure, so the LLM can guide itself effectively. In your case it's even more egregious because we are talking about the same exact algorithm/functionality implemented in two different, but rather si…
Yeah. LLMs are also great at reading and understanding thousands of lines of code quickly. It can find things like this - or compare disparate implementations.
I’ve been using fable recently not to write code, but to write briefing notes on projects. “Read and compare these three libraries. What are the unique differences in their implementations? Why were those ideas chosen? What are the pros and cons of those designs? What ideas would you copy into a new project? Write a report on your findings.”
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#98Earlier quoted context omitted.
I tried kernel autoreasearch using DeepSeek-V4-Flash as well. It spent about 1-2 hours to complete the FlashAttention optimization job ( https://github.com/fengwang/FA5090/tree/main/v7 ) and cost me only $0.2. I believe we are ready to offload a lot of this kind well-defined constrained optimization problems to AI Agent autoresearch.
> I tried kernel autoreasearch using DeepSeek-V4-Flash as well. It spent about 1-2 hours to complete the FlashAttention optimization job Doing the same, re-implementing a lot of LLM/diffusion models in Rust+CUDA for my own usage, usually the initial implementation takes 1-2 days (of 100% autonomous work) then I put an agent to optimize the implementation which tends to get close to SOTA performance within another day…
If you have access to SOTA implementations, LLMs can often find, understand and copy their best optimisation ideas.
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#99Earlier quoted context omitted.
I've had a lot of success decompiling old video game ROMs in exactly this way. Like you say - give it a way of verifying correctness - put it in a loop - and they are quite surprising. https://github.com/qarl/arcade-js
I wonder if this will result in discovery of edge cases in the logic that could be exploited to get higher scores (glitches/bugs and such!)
Possibly. Do you have agents? Pull my repo and ask them to see if they can see any bugs in Donkey Kong.
Re: Auto-research with codex: How I achieved a 232x Faster Kernel
#100Earlier quoted context omitted.
> Thankfully, I don't believe OpenAI does this - they are far more honest and seem to care about their reputation. Bro. Sam Altman?
OpenAI is run by its employees and their culture is far healthier. Unlike Anthropic, they don't have a CEO that actively encourages their employees to be dishonest.
They have lost 12 executives in the past year.