Live data from Hacker News

RustGPT: A pure-Rust transformer LLM built from scratch

github.com

101–110 of 186 posts

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#101

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

uv has been amazing for me. It just works, and it works fast.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#102
post #5

Earlier quoted context omitted.

Never knew Rust could be that readable. Makes me think other Rust engineers are stuck in a masochistic ego driven contest, which would explain everything else I've encountered about the Rust community and recruiting on that side.

Most Rust code looks like this - only generic library code goes crazy with all the generics and lifetimes, due to the need to avoid unnecessary mallocs and also provide a flexible API to users. But most people aren't writing libraries.

Don't underestimate what some programmers trying to prove their cleverness (or just trying to have fun) can do if left unchecked. I think most Rust code does indeed look like this but I've seen plenty of projects that go crazy with lifetimes and generics juggling where they don't have to.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#103

Cool stuff! I can see some GPT comments that can be removed // Increased for better learning this doesn't tell me anything // Use the constants from lib.rs const MAX_SEQ_LEN: usize = 80; const EMBEDDING_DIM: usize = 128; const HIDDEN_DIM: usize = 256; these are already defined in lib.rs, why not use them (as the comment suggests)

Do you think vibe coded rust will rot the quality of language code generally?

These things will be corrected over time.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#105

Earlier quoted context omitted.

I'm sure it's true and all. But I've been hearing the same claim about all those tools uv is intended to replace, for years now. And every time I try to run any of those, as someone who's not really a python coder, but can shit out scripts in it if needed and sometimes tries to run python software from github, it's been a complete clusterfuck. So I guess what I'm wondering is, are you a python guy, or are you more li…

Isn’t UV essentially cargo for python?

Somewhat literally so. It is written in Rust and makes use of the cargo-util crate for some overlapping functionality.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#106
post #76

Earlier quoted context omitted.

Go performance is terrible for numeric stuff though, no SIMD support.

That's not really true, but we're talking about a Python replacement for scripting tasks, not core compute tasks, anyway. It is not like Python is the paragon of SIMD support. Any real Python workloads end up being written in C for good reason, using Python only as the glue. Go can also interface with C code, and despite all the flack it gets for its C call overhead it is still significantly faster at calling C code…

For the record of people reading this, I wrote a multithreaded SIMD-heavy compute task in Go, and it suffered only 5% slowdown vs the original hand-optimized C++ version.

The low level SIMD stuff was called out to over the c FFI bridge; golang was used for the rest of the program.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#107

Earlier quoted context omitted.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

I'm sure it's true and all. But I've been hearing the same claim about all those tools uv is intended to replace, for years now. And every time I try to run any of those, as someone who's not really a python coder, but can shit out scripts in it if needed and sometimes tries to run python software from github, it's been a complete clusterfuck. So I guess what I'm wondering is, are you a python guy, or are you more li…

I'm (reluctantly) a python guy, and uv really is a much different experience for me than all the other tools. I've otherwise had much the same experience as you describe here. Maybe it's because `uv` is built in rust? ¯\_ (ツ)_/¯

But I'd also hesitate to say it "solves all my problems". There's plenty of python problems outside of the core focus of `uv`. For example, I think building a python package for distribution is still awkward and docs are not straightforward (for example, pointing to non-python files which I want to include was fairly annoying to figure out).

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#108
post #73

Earlier quoted context omitted.

Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based. The sole exclusion I've seen to my social circle is those working at companies…

> Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. I wish this were broadly true. But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling. I think ML will fade away from Python eventually but right now it's still everywhere.

A lot of what I see in ML is all focused around Triton, which is why I mentioned it.

If someone wrote a Triton impl that is all Rust instead, that would do a _lot_ of the heavy lifting on switching... most of their hard code is in Triton DSL, not in Python, the Python is all boring code that calls Triton funcs. That changes the argument on cost for a lot of people, but sadly not all.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#109
post #71
post #64

Earlier quoted context omitted.

i'm saying that ease of dependency inclusion should not be a main criterion for evaluating how good a build system is, not that it isn't the main criterion for many people... like the entire point of my comment is that people have misguided criteria for evaluating build systems, and your comment seems to just affirm this?

> dependency inclusion _should not_ be a main criterion for evaluating how good a build system is That's just like, your opinion, man.

> That's just like, your opinion, man.

I would love to know how many younger readers recognize this classic movie reference.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#110
post #82

Earlier quoted context omitted.

Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based. The sole exclusion I've seen to my social circle is those working at companies…

Okay. Humor me. I want to write a transformer-based classifier for a project. I am accustomed to the pytorch and tensorflow libraries. What is the equivalent using C?

You do know that tensorflow was written in C++ and the Python API bolted on top?
Post reply on HN