Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RustGPT: A pure-Rust transformer LLM built from scratch

github.com

Post reply on HN
Print view View original

131–140 of 186 posts

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#131

Post by eximius »

Diggsey wrote: post #119
cmrdporcupine wrote: post #23

Earlier quoted context omitted.

linking both rand-core 0.9.0 and rand-core 0.9.3 which the project could maybe avoid by just specifying 0.9 for its own dep on it

It doesn't link two versions of `rand-core`. That's not even possible with rust (you can only link two semver-incompatible versions of the same crate). And dependency specifications in Rust don't work like that - unless you explicitly override it, all dependencies are semver constraints, so "0.9.0" will happily match "0.9.3".

This doesn't sound right. If A depends on B and C - B and C can each bring their own versions of D, I thought?

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#132

Post by leoh »

ericdotlee wrote: post #92

This is incredibly cool, but I wonder when more of the AI ecosystem will move past python tooling into something more... performant? Very interesting to already see rust based inference frameworks as well.

"Python" is perfectly performant for AI and this demonstrates a deep lack of understanding. Virtually every library in python used for AI delegates to lower-level code written in C++.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#133

Post by pjmlp »

physicsguy wrote: post #67
aeve890 wrote: post #57

Earlier quoted context omitted.

>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.

Go performance is terrible for numeric stuff though, no SIMD support.

Go itself no, but luckily like in any compiler toolchain, there is an Assembler available.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#134

Post by wavemode »

DiabloD3 wrote: post #49
ricardobeat wrote: post #32

Earlier quoted context omitted.

Have you tried uv [1]? It has removed 90% of the pain of running python projects for me. [1] https://github.com/astral-sh/uv

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

Rust is not a viable replacement for Python except in a few domains.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#135

Post by pjmlp »

airza wrote: post #50
DiabloD3 wrote: post #49

Earlier quoted context omitted.

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

There's not really another game in town if you want to do fast ML development :/

PyTorch also supports C++ and Java, Tensorflow also does C++ and Java, Apple AI is exposing ML libraries via Swift, Microsoft is exposing their AI stuff via .NET and Java as well, then there is Julia and Mojo is coming along.

It is happening.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#136

Post by tripplyons »

yobbo wrote: post #128

Very nice! Next thing to add would be numerical gradient testing.

Is that where you approximate a partial derivative as a difference in loss over a small difference in a single parameter's value?

Seems like a great way to verify results, but it has the same downsides as forward mode automatic differentiation since it works in a pretty similar fashion.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#137

Post by pjmlp »

wolvesechoes wrote: post #120
adastra22 wrote: post #110

Earlier quoted context omitted.

You do know that tensorflow was written in C++ and the Python API bolted on top?

It could be written in mix of Cobol and APL. No one cares. People saying "oh those Python libraries are just C/C++ libraries with Python API, every language can have them" have one problem - no other language has them (with such extensive documentation, tutorials etc.)

PyTorch and Tensorflow also support C++ (naturally) and Java.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#138

Post by 0xffff2 »

Diggsey wrote: post #119
cmrdporcupine wrote: post #23

Earlier quoted context omitted.

linking both rand-core 0.9.0 and rand-core 0.9.3 which the project could maybe avoid by just specifying 0.9 for its own dep on it

It doesn't link two versions of `rand-core`. That's not even possible with rust (you can only link two semver-incompatible versions of the same crate). And dependency specifications in Rust don't work like that - unless you explicitly override it, all dependencies are semver constraints, so "0.9.0" will happily match "0.9.3".

So there's no difference at all between "0", "0.9" and "0.9.3" in cargo.toml (Since semver says only major version numbers are breaking)? As a decently experienced Rust developer, that's deeply surprising to me.

What if devs don't do a good job of versioning and there is a real incompatibility between 0.9.3 and 0.9.4? Surely there's some way to actually require an exact version?

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#139

Post by hadlock »

ndai wrote: post #6

I’m curious where you got your training data? I will look myself, but saw this and thought I’d ask. I have a CPU-first, no-backprop architecture that works very well on classification datasets. It can do single‑example incremental updates which might be useful for continuous learning. I made a toy demo to train on tiny.txt and it can predict next characters, but I’ve never tried to make an LLM before. I think my arch…

The training data is contained inside main.rs; it looks like about 50 statements about general facts. Probably to keep training time down. If you go off-script things fall apart pretty quickly:

Enter prompt: hello Model output: What are eclipses ? Assistant : Eclipses occur when one celestial body moves into the shadow of another

Enter prompt: what are facts Model output: eclipses Assistant : Eclipses Eclipses What What was when What through you ? through you meet through using of What was What tall ? of What thank help you explain using , and do you a patterns you you a patterns through air in tall you help metal metal

Enter prompt: how are mountains formed? Model output: I ? ' I ' : Of : in happy Hi wind in yeast altering it it

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#140

Post by [deleted] »

emporas wrote: post #14
Charon77 wrote: post #3

Absolutely love how readable the entire project is

It is very procedural/object oriented. This is not considered good Rust practice. Iterators make it more functional, which is better, more succinct that is, and enums more algebraic. But it's totally fine for a thought experiment.

[deleted]
Post reply on HN