Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Auto-research with codex: How I achieved a 232x Faster Kernel

sankalp.bearblog.dev

Post reply on HN
Print view View original

61–70 of 104 posts

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#61

Post by josu »

augment_me wrote: post #57

One thing worth to note in the competition is that 8 out of the 10 top solutions, which all happened to be optimized this way completely broke at any other input than the competition ones. The only solutions that did not break when tested with OOD shapes were made by experts who know a lot about GPU programming and that did not create 25k lines of CUDA but followed and adjusted their solution in reasonable bounds. Th…

Overfitting to the input is part of the meta in this type of challenges.

The goal is not to create good, general or maintainable code. The only goal is to produce the fastest code.

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#63

Post by dejavucoder »

augment_me wrote: post #57

One thing worth to note in the competition is that 8 out of the 10 top solutions, which all happened to be optimized this way completely broke at any other input than the competition ones. The only solutions that did not break when tested with OOD shapes were made by experts who know a lot about GPU programming and that did not create 25k lines of CUDA but followed and adjusted their solution in reasonable bounds. Th…

this is true. in one of the later problems (cholesky decomposition), the organizer ran the submissions on a tiny training run to validate... and also provided code for same for our reference. most of the top solutions hit 4/8 or so. not very numerically stable.

i found out that as i learnt more domain wise, i was (obviously) able to steer better. doing a re-write can also remove lots of slop and context rot (and subsequently make it easier for both human and LLM to make solution more numerically stable, less reward hackish)

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#64

Post by dejavucoder »

josu wrote: post #61
augment_me wrote: post #57

One thing worth to note in the competition is that 8 out of the 10 top solutions, which all happened to be optimized this way completely broke at any other input than the competition ones. The only solutions that did not break when tested with OOD shapes were made by experts who know a lot about GPU programming and that did not create 25k lines of CUDA but followed and adjusted their solution in reasonable bounds. Th…

Overfitting to the input is part of the meta in this type of challenges. The goal is not to create good, general or maintainable code. The only goal is to produce the fastest code.

fair argument

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#65

Post by dejavucoder »

dejavucoder wrote: post #53
dzbarsky wrote: post #12

Earlier quoted context omitted.

This was nowhere near the top submission. But even if a solo engineer could get a top kernel, you don't think that having thousands of engineers, infinite tokens, and stronger models than are available to the public would give the labs a significant edge?

hello author here. yes, it gives labs edge and leads to self-recursive improvement loops. also i was myself able to finish 7th in a later competition with 2-3 other approaches which are variants of the method discussed in this blog. in general, having a harness as thin as possible with some problem specific instructions while controlling for context rot is the key. point i am trying to make is there are a lot of opti…

you may notice Kimi, GLM have also started telling how their model is able to optimise it's own inference pipeline

https://www.kimi.com/blog/kimi-k3

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#66

Post by revetkn »

qarl2 wrote: post #18
Almondsetat wrote: post #2

In the last couple of days I wanted to try out the new definitive DeepSeek v4 releases. I gave it the repository of a semi-abandoned video compression codec and I told it to perform the usual benchmark -> profile -> verify -> research -> improve loop. I specifically chose this codec because the authors include a verifier for the bitstream to make sure you don't break stuff if you want to try your own implementation.…

I've had a lot of success decompiling old video game ROMs in exactly this way. Like you say - give it a way of verifying correctness - put it in a loop - and they are quite surprising. https://github.com/qarl/arcade-js

Very cool, me too! I've been working on Final Fantasy Legend (Game Boy and WonderSwan Color) and King's Bounty (PC - DOS). It's great for reversing. Really interesting to see the guts of the games, including bugs.

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#68

Post by dejavucoder »

msdz wrote: post #59
dsign wrote: post #23

Earlier quoted context omitted.

Anecdotally, I saw Opus 5 come up with a complicated loop unrolling technique when I asked it to implement a simple biquad in SIMD, missing a simpler solution. Maybe it was a downgraded session, who knows. That SIMD instruction set, the one for the ESP32-P4, is not very popular and all the documentation it has is a couple of blog posts. So I'm pretty sure it has at most seen some code for a predecessor during its tra…

> Maybe it was a downgraded session, who knows. Off-topic, but imagine us collectively being okay with (or powerless to do anything about) this sentiment about any other software service provided like two years ago. How the times have changed…

we live in exciting and scary times...

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#69

Post by bla3 »

Mirrors my experience: LLMs are really good at optimizing, better than most humans. But also, they tend to not reach absolute peak performance where people made an effort to optimize something.

Since most problems see fairly little optimization, that's still a big win most of the time.

Re: Auto-research with codex: How I achieved a 232x Faster Kernel

#70

Post by qarl2 »

kenerwin88 wrote: post #60
qarl2 wrote: post #18

Earlier quoted context omitted.

I've had a lot of success decompiling old video game ROMs in exactly this way. Like you say - give it a way of verifying correctness - put it in a loop - and they are quite surprising. https://github.com/qarl/arcade-js

Oh wow, this is almost exactly what I’ve been doing with Zelda LTTP. I have it in rust now, but just finished the “first pass” you reference. Mine is still not really readable, second step is the modernizing the actual code. I’ve really struggled with needing to handhold it though, I’ll see if I can plagiarize from you!

I'm working on getting the "handholding" down to zero. Frogger is almost done and I haven't had to intervene once.
Post reply on HN