Live data from Hacker News

Cargo: predictable dependency management

blog.rust-lang.org

81–90 of 146 posts

Re: Cargo: predictable dependency management

#81
post #45
post #27

Earlier quoted context omitted.

Happier than Rust 1.0 and the end of The Rustup?

End of The Rustup? AFAICS, it's still downloading a very specific servo-only rustc.

I really wish downvotes would require a comment so that I knew why the two posts in this thread branch went negative after being positive for a long time. I feel like people treat it like a personal spam filter although it's the same view for everybody. Odd.

Re: Cargo: predictable dependency management

#82
post #4

Cargo assumes you want to cache downloaded/built dependencies at the granularity of a unix user account. It's very easy to break things if you try to force current cargo cache dependencies at the per-project level and there's been no interest in caching things at a per-machine or shared-between-machines level. If duplicating compilation work is desirable to ensure some amount of noninterference, it should be supporte…

It sounds like there's two obvious work-arounds, and I'm not sure if I'd consider them all that bad: 1) use a caching http proxy for the downloads (in my experience cargo itself is pretty quick), or 2) use a shared user account for building ("sudo -s cargo-build; ... ").

I'm not sure concurrently writing to a group-writeable cache-folder is such a hot idea -- even things like apt use a write-lock when doing updates.

I suppose a third option is to mount (or probably symlink) ~/.cargo on a filesystem with deduplication -- but that wouldn't buy you caching for free (might work well coupled with a caching http proxy though).

Re: Cargo: predictable dependency management

#83
post #66
post #44

It's great that Raph's line-breaking code has been reused, but in contrast to Mozilla projects any contribution to it will now require assigning copyright to Google (like Ubuntu or Microsoft's CLA require as well). At least it's a tiny piece, so may not hurt too much in terms of prevented contributions.

Or just forking it, it's released under a open source license. This wouldn't be the first time Servo has forked a dependency, for example they maintain a fork of glutin (library that handles opening windows) because they want certain features that upstream doesn't. https://github.com/servo/glutin

That can work.

Curious what features servo/glutin has that are exclusive?

Re: Cargo: predictable dependency management

#84

Something that I'm surprised this page doesn't talk about, and which is very important considering the recent hubbub over left-pad, is that any dependency you get on Cargo can be relied upon to continue to exist forever (well, as long as the crates.io site still exists, but if that goes away so does the Cargo index). The reason is because you can't ever remove a published version of your crate from crates.io. You can…

But that's really a crates.io feature, not cargo's. You could easily deploy your own repository which serves everyone a random file when you pull "some_crate-1.2.3".

Re: Cargo: predictable dependency management

#85
post #72

Earlier quoted context omitted.

> Something that I'm surprised this page doesn't talk about, and which is very important considering the recent hubbub over left-pad, is that any dependency you get on Cargo can be relied upon to continue to exist forever Maybe the reason why it's hardly talked about is because it's common sense and pretty much all dependency managers support it? Except node of course, because they have no idea what they're doing.

> Except node of course, because they have no idea what they're doing. NPM and node are two different things.

..his point stands.

Re: Cargo: predictable dependency management

#86

I appreciate that predictability is a big priority with Cargo. I feel like Rust and Cargo are in a great position to deterministically (per compiler version, per set of build flags) build libraries. That would be an amazing step forward in security if it could be enforced. Does anyone know if this is planned? https://reproducible-builds.org

In general, we are very interested in reproducible builds, and have fixed bugs where accidental non-determinism has crept in. However, it can be easy to reintroduce with build.rs, which allows for executing arbitrary Rust code before a build. Syntax extensions are another problem here. But the scope is much reduced, it's true.

That's awesome to hear. Would it be possible to make an attribute like

#[deny(non_deterministic)]

which would error if `build.rs` or similar is present? Also, for usage of things like the `file!` macro, which might mess up determinism based on the build directory.

Theoretically, syntax extensions could be written to be deterministic as well (for a given version, of course).

Is there a tracking issue on this currently?

Re: Cargo: predictable dependency management

#87

    Libraries can be compiled as static libraries or dynamic libraries.
    even static libraries might want to do some dynamic linking (for 
    example, against the system version of openssl).
haha... I like cargo a lot, but I feel this is glossing over the biggest issue building rust code has: Building and linking to native libraries.

    Similarly, applications are often built for different architectures,
    operating systems, or even operating system version...

    Compiling winapi v0.2.6 
    Compiling libc v0.2.10
Oh? I'd love to see that compiling on a different operating system.

Linking against the system version of openssl? Are you sure it's the right version of openssl not to break your binding?

The C ecosystem is fundamentally problematic to get repeatable cross platform builds with; and trying to get repeatable cross platform builds with rust when it touches any c / c++ library is quite a pain.

Piston is a good example of how this is done... but people still struggle to compile it; the 'uses the compiler to explicitly build the dependency as a static library and links it' (eg. git binding afaik) is a much better solution, but it massively increases build times.

It's a rough edge point; and I still feel like there's been no real progress towards solving the issues.

Everyone just does something completely different in their native build (build.rs) step, and then your 'repeatable build' is largely, hit 'cargo build' and 'hope nothing goes wrong and all dependencies are installed'.

There are other more important priorities for rust at the moment (like recovering from panics...), but this is certainly an area I'd love to see improvement.

Re: Cargo: predictable dependency management

#88
So, dependencies in, say, cabal or go or CL are unpredictable?

The trend to put meaningless but catchy adjectives, like "safe" and "predictable", as if everything else is opposite, is misleading.

Designers of classic languages were not some arrogant punks (they were brilliant, like David Moon and other guys of his generation), and the most of findamental problems were well understood in times of Standard ML or Common Lisp.

So, in comparison to C or C++ it might be "predictable" and even more "safe", but in comparison with well-researched classic languages, these mere redundant adjectives.

Re: Cargo: predictable dependency management

#89
post #34

I like many things about Rust, but Cargo alone is what initially got me started with it. I was in the process of setting up a fresh C++ project with vendored dependencies, and it was just an absolute nightmare. In contrast, it took me about 5 minutes to get a Rust project set up with comparable dependency complexity.

I understand that old languages like C++ or Java have problems providing a single package manager. It seems that newer languages have roughly equivalent services. Are there any big differences between Cargo and Python/Go/D/Ruby/Javascript/etc?

I wouldn't say Java has this problem. Look at Maven for instance. Then look back at Crates, you will notice how many common solutions they share.

On the other hand I see that Crates solves the problem of dependency when 2 different version of the same library are used across the project. Maven doesn't solve that and it's painful. However this seems a bigger problem that just a package manager I suppose.

Re: Cargo: predictable dependency management

#90

Earlier quoted context omitted.

In general, we are very interested in reproducible builds, and have fixed bugs where accidental non-determinism has crept in. However, it can be easy to reintroduce with build.rs, which allows for executing arbitrary Rust code before a build. Syntax extensions are another problem here. But the scope is much reduced, it's true.

That's awesome to hear. Would it be possible to make an attribute like #[deny(non_deterministic)] which would error if `build.rs` or similar is present? Also, for usage of things like the `file!` macro, which might mess up determinism based on the build directory. Theoretically, syntax extensions could be written to be deterministic as well (for a given version, of course). Is there a tracking issue on this currently…

There isn't, it might be an interesting idea! Not allowing build.rs would eliminate a lot of crates, and not all build.rs' are nondeterministic... So that's tough.
Post reply on HN