Live data from Hacker News

RustGPT: A pure-Rust transformer LLM built from scratch

github.com

31–40 of 186 posts

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#31
post #20

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

lowkey ppl who praise cargo seem to have no idea of the tradeoffs involved in dependency management the difficulty of including a dependency should be proportional to the risk you're taking on, meaning it shouldn't be as difficult as it in, say, C where every other library is continually reinventing the same 5 utilities, but also not as easy as it is with npm or cargo, because you get insane dependency clutter, and a…

Security is another problem, and should be tackled systematically. Artificially making dependency inclusion hard is not it and is detrimental to the more casual use cases.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#32

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me.

[1] https://github.com/astral-sh/uv

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#35
Congrats - there is a very small problem with the LLM - its reusing transformer blocks and you want to use different instances of them.

Its a very cool excercise, I did the same with Zig and MLX a while back, so I can get a nice foundation, but since then as I got hooked and kept adding stuff to it, switched to Pytorch/Transformers.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#36

Earlier quoted context omitted.

I know because this is how an AI generated project looks. Clearly AI generated README, "clean" code, the way files are named, etc.

To me it looks like LLM generated README, but not necessarily the source (or at least not all of it). Or there's been a cleaning pass done over it.

I think pretty clearly the source is also at least partially generated. None the less, just a README like that already sends a strong signal to stop looking and not trust anything written there.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#37

Earlier quoted context omitted.

How do you know? The over-commenting?

I know because this is how an AI generated project looks. Clearly AI generated README, "clean" code, the way files are named, etc.

Not sure myself. Commit messages look pretty human. But the emojis in readme and comments like "// Re-export key structs for easier access", "# Add any test-specific dependencies here if needed" are sus indeed.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#38
post #5
post #3

Absolutely love how readable the entire project is

Never knew Rust could be that readable. Makes me think other Rust engineers are stuck in a masochistic ego driven contest, which would explain everything else I've encountered about the Rust community and recruiting on that side.

Most Rust code looks like this - only generic library code goes crazy with all the generics and lifetimes, due to the need to avoid unnecessary mallocs and also provide a flexible API to users.

But most people aren't writing libraries.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#39
post #6

I’m curious where you got your training data? I will look myself, but saw this and thought I’d ask. I have a CPU-first, no-backprop architecture that works very well on classification datasets. It can do single‑example incremental updates which might be useful for continuous learning. I made a toy demo to train on tiny.txt and it can predict next characters, but I’ve never tried to make an LLM before. I think my arch…

https://huggingface.co/datasets/NousResearch/Hermes-3-Datase...

To my untrained eye, this looks more like an instruct dataset.

For just plain text, I really like this one - https://huggingface.co/datasets/roneneldan/TinyStories

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#40

Congrats - there is a very small problem with the LLM - its reusing transformer blocks and you want to use different instances of them. Its a very cool excercise, I did the same with Zig and MLX a while back, so I can get a nice foundation, but since then as I got hooked and kept adding stuff to it, switched to Pytorch/Transformers.

correction: It's a cool exercise if you write it yourself and not use GPT
Post reply on HN