Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RustGPT: A pure-Rust transformer LLM built from scratch

github.com

Post reply on HN
Print view View original

71–80 of 186 posts

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#71

Post by Sl1mb0 »

taminka wrote: post #64
dev_l1x_be wrote: post #46

Earlier quoted context omitted.

> lowkey ppl who praise cargo seem to have no idea Way to go on insulting people on HN. Cargo is literally the reason why people coming to Rust from languages like C++ where the lack of standardized tooling is giant glaring bomb crater that poses burden on people every single time they need to do some basic things (like for example version upgrades). Example: https://github.com/facebook/folly/blob/main/build.sh

i'm saying that ease of dependency inclusion should not be a main criterion for evaluating how good a build system is, not that it isn't the main criterion for many people... like the entire point of my comment is that people have misguided criteria for evaluating build systems, and your comment seems to just affirm this?

> dependency inclusion _should not_ be a main criterion for evaluating how good a build system is

That's just like, your opinion, man.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#72

Post by pclmulqdq »

physicsguy wrote: post #67
aeve890 wrote: post #57

Earlier quoted context omitted.

>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.

Go performance is terrible for numeric stuff though, no SIMD support.

There are Go SIMD libraries now, and there's also easy use of C libraries via Cgo.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#73

Post by nkozyra »

DiabloD3 wrote: post #54
airza wrote: post #50

Earlier quoted context omitted.

There's not really another game in town if you want to do fast ML development :/

Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based. The sole exclusion I've seen to my social circle is those working at companies…

> Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum.

I wish this were broadly true.

But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling.

I think ML will fade away from Python eventually but right now it's still everywhere.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#74

Post by DiabloD3 »

physicsguy wrote: post #67
aeve890 wrote: post #57

Earlier quoted context omitted.

>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.

Go performance is terrible for numeric stuff though, no SIMD support.

(given the context of LLMs) Unless you're doing CPU-side inference for corner cases where GPU inference is worse, lack of SIMD isn't a huge issue.

There are libraries to write SIMD in Go now, but I think the better fix is being able to autovectorize during the LLVM IR optimization stage, so its available with multiple languages.

I think LLVM has it now, its just not super great yet.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#75

Post by taminka »

Sl1mb0 wrote: post #71
taminka wrote: post #64

Earlier quoted context omitted.

i'm saying that ease of dependency inclusion should not be a main criterion for evaluating how good a build system is, not that it isn't the main criterion for many people... like the entire point of my comment is that people have misguided criteria for evaluating build systems, and your comment seems to just affirm this?

> dependency inclusion _should not_ be a main criterion for evaluating how good a build system is That's just like, your opinion, man.

i mean, unless you have some absolute divine truths, that's kind of the best i have :shrug

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#76

Post by 9rx »

physicsguy wrote: post #67
aeve890 wrote: post #57

Earlier quoted context omitted.

>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.

Go performance is terrible for numeric stuff though, no SIMD support.

That's not really true, but we're talking about a Python replacement for scripting tasks, not core compute tasks, anyway. It is not like Python is the paragon of SIMD support. Any real Python workloads end up being written in C for good reason, using Python only as the glue. Go can also interface with C code, and despite all the flack it gets for its C call overhead it is still significantly faster at calling C code than Python is.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#77

Post by tialaramex »

ramon156 wrote: post #27

Cool stuff! I can see some GPT comments that can be removed // Increased for better learning this doesn't tell me anything // Use the constants from lib.rs const MAX_SEQ_LEN: usize = 80; const EMBEDDING_DIM: usize = 128; const HIDDEN_DIM: usize = 256; these are already defined in lib.rs, why not use them (as the comment suggests)

For the constants is it possible the author didn't know how? I remember in my first week of Rust I didn't understand how to name things properly, basically I was overthinking it.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#78

Post by mtlmtlmtlmtl »

ricardobeat wrote: post #32
untrimmed wrote: post #16

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

I'm sure it's true and all. But I've been hearing the same claim about all those tools uv is intended to replace, for years now. And every time I try to run any of those, as someone who's not really a python coder, but can shit out scripts in it if needed and sometimes tries to run python software from github, it's been a complete clusterfuck.

So I guess what I'm wondering is, are you a python guy, or are you more like me? because for basically any of these tools, python people tell me "tool X solved all my problems" and people from my own cohort tell me "it doesn't really solve anything, it's still a mess".

If you are one of us, then I'm really listening.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#80

Post by mountainriver »

DiabloD3 wrote: post #54
airza wrote: post #50

Earlier quoted context omitted.

There's not really another game in town if you want to do fast ML development :/

Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based. The sole exclusion I've seen to my social circle is those working at companies…

I love Rust and C, I write quite a bit of both. I am an ML engineer by trade.

To say most ML people are using Rust and C couldn’t be further from the truth

Post reply on HN