Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RustGPT: A pure-Rust transformer LLM built from scratch

github.com

Post reply on HN
Print view View original

51–60 of 186 posts

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#51

Post by trklausss »

DiabloD3 wrote: post #49
ricardobeat wrote: post #32

Earlier quoted context omitted.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative.

Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language".

If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency management on development, I still have to test it for deployment :)

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#52

Post by Galanwe »

untrimmed wrote: post #16

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

> spent days wrestling with Python dependency hell

I mean I would understand that comment in 2010, but in 2025 it's grossly ridiculous.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#54

Post by DiabloD3 »

airza wrote: post #50
DiabloD3 wrote: post #49

Earlier quoted context omitted.

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

There's not really another game in town if you want to do fast ML development :/

Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum.

Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based.

The sole exclusion I've seen to my social circle is those working at companies that don't directly do ML, but provide drivers/hardware/supporting software to ML people in academia, and have to try to fix their cursed shit for them.

Also, fwiw, there is no reason why Triton is Python. I dislike Triton for a lot of reasons, but its just a matmul kernel DSL, there is nothing inherent in it that has to be, or benefits from, being Python.... it takes DSL in, outputs shader text out, then has the vendor's API run it (ie, CUDA, ROCm, etc). It, too, would benefit from becoming Rust.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#55

Post by Exuma »

DiabloD3 wrote: post #49
ricardobeat wrote: post #32

Earlier quoted context omitted.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

i hate python, but the idea of replacing python with rust is absurd

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#56

Post by imtringued »

kachapopopow wrote: post #4
techsystems wrote: post #2

> ndarray = "0.16.1" rand = "0.9.0" rand_distr = "0.5.0" Looking good!

I was slightly curious: cargo tree llm v0.1.0 (RustGPT) ├── ndarray v0.16.1 │ ├── matrixmultiply v0.3.9 │ │ └── rawpointer v0.2.1 │ │ [build-dependencies] │ │ └── autocfg v1.4.0 │ ├── num-complex v0.4.6 │ │ └── num-traits v0.2.19 │ │ └── libm v0.2.15 │ │ [build-dependencies] │ │ └── autocfg v1.4.0 │ ├── num-integer v0.1.46 │ │ └── num-traits v0.2.19 ( ) │ ├── num-traits v0.2.19 ( ) │ └── rawpointer v0.2.1 ├── rand v0…

    cargo tree llm v0.1.0 (RustGPT)
    ├── ndarray v0.16.1
    │   ├── matrixmultiply v0.3.9
    │   │   └── rawpointer v0.2.1
    │   │       [build-dependencies]
    │   │       └── autocfg v1.4.
    │   ├── num-complex v0.4.6
    │   │   └── num-traits v0.2.19
    │   │       └── libm v0.2.15
    │   │           [build-dependencies]
    │   │           └── autocfg v1.4.0
    │   ├── num-integer v0.1.46
    │   │   └── num-traits v0.2.19 ()
    │   ├── num-traits v0.2.19 ()
    │   └── rawpointer v0.2.1
    ├── rand v0.9.0
    │   ├── rand_chacha v0.9.0
    │   │   ├── ppv-lite86 v0.2.20
    │   │   │   └── zerocopy v0.7.35
    │   │   │       ├── byteorder v1.5.0
    │   │   │       └── zerocopy-derive v0.7.35 (proc-macro)
    │   │   │           ├── proc-macro2 v1.0.94
    │   │   │           │   └── unicode-ident v1.0.18
    │   │   │           ├── quote v1.0.39
    │   │   │           │   └── proc-macro2 v1.0.94 ()
    │   │   │           └── syn v2.0.99
    │   │   │               ├── proc-macro2 v1.0.94 ()
    │   │   │               ├── quote v1.0.39 ()
    │   │   │               └── unicode-ident v1.0.18
    │   │   └── rand_core v0.9.3
    │   │       └── getrandom v0.3.1
    │   │           ├── cfg-if v1.0.0
    │   │           └── libc v0.2.170
    │   ├── rand_core v0.9.3 ()
    │   └── zerocopy v0.8.23
    └── rand_distr v0.5.1
        ├── num-traits v0.2.19 ()
        └── rand v0.9.0 ()

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#57

Post by aeve890 »

trklausss wrote: post #51
DiabloD3 wrote: post #49

Earlier quoted context omitted.

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language". If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency man…

>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative.

I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#58

Post by sloppytoppy »

ramon156 wrote: post #27

Cool stuff! I can see some GPT comments that can be removed // Increased for better learning this doesn't tell me anything // Use the constants from lib.rs const MAX_SEQ_LEN: usize = 80; const EMBEDDING_DIM: usize = 128; const HIDDEN_DIM: usize = 256; these are already defined in lib.rs, why not use them (as the comment suggests)

[flagged]

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#60

Post by TheAceOfHearts »

ricardobeat wrote: post #32
untrimmed wrote: post #16

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

Switching to uv made my python experience drastically better.

If something doesn't work or I'm still encountering any kind of error with uv, LLMs have gotten good enough that I can just copy / paste the error and I'm very likely to zero-in on a working solution after a few iterations.

Sometimes it's a bit confusing figuring out how to run open source AI-related python projects, but the combination of uv and iterating on any errors with an LLM has so far been able to resolve all the issues I've experienced.

Post reply on HN