Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RustGPT: A pure-Rust transformer LLM built from scratch

github.com

Post reply on HN
Print view View original

171–180 of 186 posts

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#171

Post by Diggsey »

eximius wrote: post #131
Diggsey wrote: post #119

Earlier quoted context omitted.

It doesn't link two versions of `rand-core`. That's not even possible with rust (you can only link two semver-incompatible versions of the same crate). And dependency specifications in Rust don't work like that - unless you explicitly override it, all dependencies are semver constraints, so "0.9.0" will happily match "0.9.3".

This doesn't sound right. If A depends on B and C - B and C can each bring their own versions of D, I thought?

Within a crate graph, for any given major version of a crate (eg. D v1) only a single minor version can exist. So if B depends on D v1.x, and C depends on D v2.x, then two versions of D will exist. If B depends on Dv1.2 and C depends on Dv1.3, then only Dv1.3 will exist.

I'm over-simplifying a few things here:

1. Semver has special treatment of 0.x versions. For these crates the minor version depends like the major version and the patch version behaves like the minor version. So technically you could have v0.1 and v0.2 of a crate in the same crate graph.

2. I'm assuming all dependencies are specified "the default way", ie. as just a number. When a dependency looks like "1.3", cargo actually treats this as "^1.3", ie. the version must be at least 1.3, but can be any semver compatible version (eg. 1.4). When you specify an exact dependency like "=1.3" instead, the rules above still apply (you still can't have 1.3 and 1.4 in the same crate graph) but cargo will error if no version can be found that satisfies all constraints, instead of just picking a version that's compatible with all dependents.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#172

Post by Diggsey »

0xffff2 wrote: post #138
Diggsey wrote: post #119

Earlier quoted context omitted.

It doesn't link two versions of `rand-core`. That's not even possible with rust (you can only link two semver-incompatible versions of the same crate). And dependency specifications in Rust don't work like that - unless you explicitly override it, all dependencies are semver constraints, so "0.9.0" will happily match "0.9.3".

So there's no difference at all between "0", "0.9" and "0.9.3" in cargo.toml (Since semver says only major version numbers are breaking)? As a decently experienced Rust developer, that's deeply surprising to me. What if devs don't do a good job of versioning and there is a real incompatibility between 0.9.3 and 0.9.4? Surely there's some way to actually require an exact version?

They are different:

    "0": ">=0.0.0, =0.9.0, =0.9.3,
Notice how the the minimum bound changes while the upper bound is the same for all of them.

The reason for this is that unless otherwise specified, the ^ operator is used, so "0.9" is actually "^0.9", which then gets translated into the kind of range specifier I showed above.

There are other operators you can use, these are the common ones:

    (default) ^ Semver compatible, as described above
    >= Inclusive lower bound only
Note that while an exact bound will force that exact version to be used, it still doesn't allow two semver compatible versions of a crate to exist together. For example. If cargo can't find a single version that satisfies all constraints, it will just error.

For this reason, if you are writing a library, you should in almost all cases stick to regular semver-compatible dependency specifications.

For binaries, it is more common to want exact control over versions and you don't have downstream consumers for whom your exact constraints would be a nightmare.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#173

Post by pjmlp »

famouswaffles wrote: post #159
pjmlp wrote: post #153

Earlier quoted context omitted.

The existing C++ API is done according to that "beautiful" Google guidelines, thus it could be much better. However people are definitely using it, as Android doesn't do Python, neither does ChromeOS.

>However people are definitely using it, as Android doesn't do Python, neither does ChromeOS. That's not really a reason to think people are using it for that when things like onnxruntime and executorch exist. In fact, they are very likely not using it for that, if only because the torch runtime is too heavy for distribution on the edge anyway (plus android can run python). Regardless, that's just inference of existi…

Well, onnxruntime is also having polyglot bindings, and yet another way to avoid Python.

Yes, you can package Python alongside your APK, if you feel like having fun making it compiled with NDK, and running stuff even more slowly in phone ARM chipsets over Dalvik JNI than it already is on desktops.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#175

Post by wolvesechoes »

adastra22 wrote: post #163
wolvesechoes wrote: post #155

Earlier quoted context omitted.

I hoped it was quite obvious that by "other languages" I meant "other than Python and C/C++ in which they are written". At least sibling actually mentioned Java.

Scroll up this thread and the other poster was asking if you can use pytorch and tensorflow from C. Both are C++ libraries, so accessing them from C/C++ is pretty trivial and has first-class support.

You should read more carefully before responding.

I said "beside Python, and C/C++ in which they are written"

You: "you can see people are using it from C".

What a surprise that library usable from Python through wrapped C API has C API!

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#176

Post by farhanhubble »

untrimmed wrote: post #16

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

I have heard of similar experiences on HN a few times. Haven't seen any such conflicts on real projects in the last five years or so, since I started using Poetry and then UV. I deal with data science code and the people writing it have a tendency to create dependency spaghetti, for example including the Scikit package in a mainly Pytorch code, just because they need a tried-and-tested accuracy() function.

I do remember banging my head against failed dependency resolution in my Early days of Python, circa 2014, with Pip and Conda, etc.

The dependency issues I have faced were mostly due to data science folks pinning exact package versions for the sake of replicability in requirements.txt for example

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#177

Post by farhanhubble »

untrimmed wrote: post #16

As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.

My biggest gripes with Python are:

- exports being broken if code is executed from a different directory

- packaging being more complicated than it should be

and I don't even have too much experience in the area of packaging, besides occasionally publishing to a private repo.

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#178

Post by ramon156 »

ericdotlee wrote: post #93
ramon156 wrote: post #27

Cool stuff! I can see some GPT comments that can be removed // Increased for better learning this doesn't tell me anything // Use the constants from lib.rs const MAX_SEQ_LEN: usize = 80; const EMBEDDING_DIM: usize = 128; const HIDDEN_DIM: usize = 256; these are already defined in lib.rs, why not use them (as the comment suggests)

Do you think vibe coded rust will rot the quality of language code generally?

Vibe coded is fine, but keep the comments useful. GPT's are so quick with putting a comment on everything that it kind of enriches your codebase with slop. I wouldn't call it rotting, but definitely redundant

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#179

Post by re »

beacon294 wrote: post #166
mtlmtlmtlmtl wrote: post #78

Earlier quoted context omitted.

I'm sure it's true and all. But I've been hearing the same claim about all those tools uv is intended to replace, for years now. And every time I try to run any of those, as someone who's not really a python coder, but can shit out scripts in it if needed and sometimes tries to run python software from github, it's been a complete clusterfuck. So I guess what I'm wondering is, are you a python guy, or are you more li…

It doesn't handle python version management, it only handles pip. It doesn't solve bundling Python.

It does handle python version management: https://docs.astral.sh/uv/concepts/python-versions/

Re: RustGPT: A pure-Rust transformer LLM built from scratch

#180

Post by virtualritz »

Galanwe wrote: post #157
virtualritz wrote: post #143

Earlier quoted context omitted.

So in 2025, in Python, if I depend on two packages. A and B, and they both depend on different, API-incompatible or behavior-incompatible (or both) versions of C, that won't be an issue? That's not my experience and e.g. uv hasn't helped me with that. I believe this is an issue with Python itself? If parent was saying something "grossly ridiculous" I must be doing something wrong too. And I'm happy to hear what as th…

Well, first, this a purposefully contrived example, that pretty much does not happen in real life scenarios. So you're pretty much acknowledging that there is no real problem by having to resort to such length. Second, what exactly would you like to happen in that instance? You want to have, in a single project, the same library but at different and conflicting versions. The only way to solve that is to disambiguate,…

> Well, first, this a purposefully contrived example [...]

So you are saying that (a) I made this up and (b) intentionally so.

How so? I am always flabbergasted when people make such statements.

You know nothing of my use of Python. I work in a specific field (computer graphics) and within that an even more specific sub field, visual effects.

I have to use Python maybe every three months. And there is some dependency related pain every single time. Python's dependency management "is straight up terrible" (quoted from elsewhere in this thread), I concur.

And thusly, in my world, this example is not "contrived" and given the aforementioned circumstances -- that were unknown to you -- even less so "purposefully".

> Second, what exactly would you like to happen in that instance?

I would expect Python to namespace-wrap (on-the-fly) conflicting versions.

See Rust for some similar solution.

> [...] a wasted effort for not-a-problem.

If this was "not-a-problem" why would Rust/cargo go out of its way to solve it? And why would people regularly point out for this to be one of the reasons dependencies are indeed a "not-a-problem" in Rust and how great that is compared to whatever else they battled with before?

Indeed you and I do live in different worlds.

Post reply on HN