Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting Rust

josephg.com

Post reply on HN
Print view View original

141–150 of 410 posts

Re: Rewriting Rust

#141

Post by cmrdporcupine »

lifthrasiir wrote: post #79
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

I consciously remove and rewrite various dependencies at work, but I feel it's only a half of the whole story because either 1K or 4M lines of code seem to be equally inaccurate estimates for the appropriate number of LoC for this project. It seems that most dependencies of cargo-watch are pulled from three direct requirements: clap, cargo_metadata and watchexec. Clap would pull lots of CLI things that would be natur…

It's frustrating because the grand-daddy of build systems with automatic transitive dependency management -- Maven -- already had tools from day one to handle this kind of thing through excluded dependencies (a blunt instrument, but sometimes necessary). In my experience, [patch] doesn't cut it or compare.

That, and the maven repository is moderated. Unlike crates.io.

Crates.io is a real problem. No namespaces, basically unmoderated, tons of abandoned stuff. Version hell like you're talking about.

I have a hard time taking it at all seriously as a professional tool. And it's only going to get worse.

If I were starting a Rust project from scratch inside a commercial company at this point, I'd use Bazel or Buck or GN/Ninja and vendored dependencies. No Cargo, no crates.io.

Re: Rewriting Rust

#142

Post by olalonde »

dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Why do you care how many lines of code the dependencies are? Compile time? Lack of disk space?

Re: Rewriting Rust

#143

Post by lifthrasiir »

wiseowise wrote: post #55
willvarfar wrote: post #46

Earlier quoted context omitted.

How does Go compare? I'm curious as I don't know Go but it often gets mentioned here on HN as very lightweight. (A quick googling finds https://pkg.go.dev/search?q=watch which makes me think that it's not any different?)

https://pkg.go.dev/std They’re much better.

I recall that I was very surprised to hear that Go standard library has extensive cryptographic stuffs. Generally that would be very unwise because they will become much harder to change or remove in spite of security issues. Turns out that this particular portion would be maintained by other maintainers who are actually trained in cryptography and security---something almost any other languages wouldn't be able to do with their resources.

Re: Rewriting Rust

#144

Post by cmrdporcupine »

olalonde wrote: post #142
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Why do you care how many lines of code the dependencies are? Compile time? Lack of disk space?

Some of us like to understand what's happening in the software we work on, and don't appreciate unnecessary complexity or unknown paths in the codebase that come through third party transitive dependencies.

Some of us have licensing restrictions we have to adhere to.

Some of us are very concerned about security and the potential problems of unaudited or unmoderated code that comes in through a long dependency chain.

Hard learned lessons through years of dealing with this kind of thing: good software projects try to minimize the size of their impact crater.

Re: Rewriting Rust

#145

Post by aapoalas »

dietr1ch wrote: post #38

Marking fixed stack size (and maybe even with an actual bound) would be helpful to ensure the tail-call optimisation is being done. I don't think any language helps verifying that., and even in the ones that require it by spec, it's unclear if it's happening. Maybe you didn't really wrote a tail-recursive function because of a helper that you expected to be inlined. I guess it's easy to notice if you try to blow the…

Guaranteed tail-calls with the `become` keyword have been moving forward recently.

Re: Rewriting Rust

#146

Post by cmrdporcupine »

simonask wrote: post #61
armitron wrote: post #44

Earlier quoted context omitted.

This is the main reason we have banned Rust across my Org. Every third party library needs to be audited before being introduced as a vendored dependency which is not easy to do with the bloated dependency chains that Cargo promotes.

I'm sorry, but that feels like an incredibly poorly informed decision. One thing is to decide to vendor everything - that's your prerogative - but it's very likely that pulling everything in also pulls in tons of stuff that you aren't using, because recursively vendoring dependencies means you are also pulling in dev-dependencies, optional dependencies (including default-off features), and so on. For the things you d…

> One thing is to decide to vendor everything - that's your prerogative - but it's very likely that pulling everything in also pulls in tons of stuff that you aren't using, because recursively vendoring dependencies means you are also pulling in dev-dependencies, optional dependencies (including default-off features), and so on.

What you're describing is a problem with how Cargo does vendoring, and yes, it's awful. It should not be called vendoring, it is just "local mirroring", which is not the same thing.

But Rust can work just fine without Cargo or Crates.io.

Re: Rewriting Rust

#147

Post by the_gipsy »

alexvitkov wrote: post #109
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

That's what inevitably happens when you make transitive dependencies easy and you have a culture of "if there's a library for it you must use it!" C/C++ are the only widely used languages without a popular npm-style package manager, and as a result most libraries are self-contained or have minimal, and often optional dependencies. efsw [1] is a 7000 lines (wc -l on the src directory) C++ FS watcher without dependenci…

> a library can provide generic compress/decompress callbacks, that the user can implement with whatever.

This only works for extremely simple cases. Beyond toy example, you have to glue together two whole blown APIs with a bunch of stuff not aligning at all.

Re: Rewriting Rust

#148

Post by louismerlin »

If I were to rewrite Rust, I'd probably go the route of less features, not more.

Make it 70% of Rust in 10% of the code, similarly to what QBE[0] is doing with LLVM.

You'd probably be able to achieve that if you remove macros and some of the rarely-used features.

[0]: https://c9x.me/compile/

Re: Rewriting Rust

#149

Post by wiseowise »

Denvercoder9 wrote: post #126
wiseowise wrote: post #45

Earlier quoted context omitted.

Yes, unironically they’re now. Node has improved greatly in last two years. They always had native JSON support. Now have native test runner, watch, fetch, working on permission system à la deno, added WebSockets and working on native SQLite driver. All of this makes it a really attractive platform for prototyping which scales from hello world without any dependencies to production. Good luck experimenting with Rust…

> without any dependencies Nah, you still have those dependencies, they're just integrated in your interpreter. That has advantages (you're now only trusting a single source) and disadvantages (you always get all the goodies and the associated risks with that, even if you don't need them).

You’re being pedantic for the sake of being pedantic.

Re: Rewriting Rust

#150

Post by lostmsu »

joatmon-snoo wrote: post #78
dist1ll wrote: post #42

Earlier quoted context omitted.

That's the usual response I get when I bring this issue up. "file watching is actually very complicated" or "if you avoided deps, you'd just reimplement millions of loc yourself. Forgive me if I'm making a very bold claim, but I think cross-platform file watching should not require this much code. It's 32x larger than the Linux memory management subsystem.

Good file watching that provides flexible primitives absolutely requires: - ok, a single ext4 file inode changes, and its filename matches my hardcoded string - oh, you don’t want to match against just changes to “package.json” but you want to match against a regex? voila, now you need a regex engine - what about handling a directory rename? should that trigger matches on all files in the renamed directory? - should…

Regex one stands out as a negative example here. Why does it have to be built-in instead of exposing a str -> bool filter lambda?
Post reply on HN