I'm genuinely curious how much of this is LLM generated?
Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
11–20 of 30 posts
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#12Very weird coding style, did you run astyle --style=python on C code? Also, your LLM left a comment in the cuda source that it is untested, does the cuda stuff work?
Not sure, but the code is quite dense and lacking in comments. `nanoeuler` & `nanoeuler_check` is itself the binary checked straight into git with the `.log` file? All of the commit messages are "Add files via upload" and happened in quick succession. I suspect this is LLM generated, which is cool, but shouldn't then have the claim "forward and backward passes are written and verified by hand" unless it is true. Rega…
I haven't tested NanoEuler yet but Gutenberg is awesome. Maybe a matter of taste but I like it much better than modern English.
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#13I'm genuinely curious how much of this is LLM generated?
Most part of trasformer and sft!
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#14Mentioning neural ODE doesn't make sense here, as this is unrelated. Basically any implementation of transformer uses residuals, but you're not really training a neural ODE here. Also consider getting rid of the em-dashes. I don't know if you mostly vibe-coded this or not, but the README is pretty clearly AI generated.
Hi, thanks for the comment. Nanoeuler is starting as a study and research project that will obviously improve over time. I'll do my best to make the readme and other things more readable. Thank you very much.
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#15[dead]
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#16this is super interesting. Looking forward to trying this out!
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#17this is super interesting. Looking forward to trying this out!
Really thanks If you need any help or have any questions I'm here.
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#18Wow that's really cool i'll definitely check it out!
have played around with machine learning algorithms built from scratch in c / cuda too, but once i hit the cuda part of it i kinda just left it to the side.
i'm curious how did you use CUDA to optimize the matrix multiplications?
how optimized is training, does it take much longer then using pytorch?
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#19Wow that's really cool i'll definitely check it out! have played around with machine learning algorithms built from scratch in c / cuda too, but once i hit the cuda part of it i kinda just left it to the side. i'm curious how did you use CUDA to optimize the matrix multiplications? how optimized is training, does it take much longer then using pytorch?
Hi, in nanoeuler I use cuBLAS (NVIDIA's super-optimized library) for all matrix multiplications, with the tensor cores in TF32 mode. It's the same thing PyTorch uses underneath, so it's very fast.
What I've optimized (and will improve even more) and written by hand are the kernels for the other parts (like FlashAttention, which gave a nice 3x speedup), while I've delegated the large matrices to cuBLAS.
Training the 116M model on a 4070 runs well and in reasonable times. Compared to PyTorch, it's a bit slower (probably 1.5-2.5x), but nothing dramatic, especially considering it's all done from scratch without a framework and there are no other optimizations that would make it faster. I'm working on it.
Re: Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
#20Wow that's really cool i'll definitely check it out! have played around with machine learning algorithms built from scratch in c / cuda too, but once i hit the cuda part of it i kinda just left it to the side. i'm curious how did you use CUDA to optimize the matrix multiplications? how optimized is training, does it take much longer then using pytorch?
Hi, in nanoeuler I use cuBLAS (NVIDIA's super-optimized library) for all matrix multiplications, with the tensor cores in TF32 mode. It's the same thing PyTorch uses underneath, so it's very fast. What I've optimized (and will improve even more) and written by hand are the kernels for the other parts (like FlashAttention, which gave a nice 3x speedup), while I've delegated the large matrices to cuBLAS. Training the 1…
Do you have a guess why your code is so much slower than torch? I didn't look, but there must be no reason to have 2x slower code esp. for a simple grid of FMAs.