Cargo: predictable dependency management • Hacker News Board

Re: Cargo: predictable dependency management

#131

Post by airless_bar » Fri, May 06, 2016, 8:23 PM UTC

kibwen wrote: post #100

eyko wrote: post #95

Earlier quoted context omitted.

I believe he means that packages aren't namespaced (owner/package-name), so if I publish a package called nike, Nike could come and want to take over the package name.

Global namespacing has nothing to do with that. Even with per-user namespacing, there's nothing to stop a user from making their username "nike" and publishing all their code under "nike/package-name", which would be subject to just as much potential legal action as a package named "nike" itself. Likewise there's also nothing to stop Bob from uploading "bob/nike".

If course global name-spacing has something to do with it!

What's the difference between a "username" resulting in nike/package-name and a package called nike-package-name?

Not much in practice. Name-spacing doesn't mean "add another made up name entry somewhere".

That's why people who thought about these issues out-sourced this concern completely: Prove your ownership of nike.com and you can publish as nike.com.

Suddenly 99.9% of the trademark issues are gone, handled by registrars.

I think this approach is pretty obvious and I find the lack of thought coming from rust devs deeply concerning.

Re: Cargo: predictable dependency management

#132

Post by f2f » Fri, May 06, 2016, 9:11 PM UTC

dschiptsov wrote: post #117

dbaupp wrote: post #115

Earlier quoted context omitted.

Oh, indeed. I must've got mixed up: it doesn't guarantee that builds will the same from machine to machine (unless you run go get at the exact same instant).

So, in Golang there is no concept of package version? For them different location (to import from) for an incompatible version is good-enough.

go get is just a mechanism, not the tool. there are other tools build on top of go get that ensure proper versioning.

Re: Cargo: predictable dependency management

#133

Post by weinzierl » Fri, May 06, 2016, 9:54 PM UTC

steveklabnik wrote: post #129

weinzierl wrote: post #128

Earlier quoted context omitted.

> "look at this index first, but if it's not here, go look at this other one" I agree that this would be very helpful for a lot of people, but it is kind of the opposite of what I was asking about. As far as I understand it the commercial repos try to solve two main concerns: 1. license compliance 2. security Better performance and reliability are just additional benefits. I only know the details for a certain Fortun…

If you just want to ignore the broader OSS ecosystem, and run your own version of Crates.io behind a firewall, that's 100% supported today. The only real issue is that how to do so isn't particularly well-documented.

Sorry for pestering again but I think this is kind of important and I haven't made myself entirely clear yet (English is not my first language).

In maven the repo URL is configurable in settings.xml. This URL can be different for different departments of even different projects.

From what I see in the cargo source the crates.io URL is hard coded. So the DNS is the only level of redirection we have. Using varying IP addresses for crates.io for different departments or even projects wouldn't fly, at least not in the world I live in.

>ignore the broader OSS ecosystem

It's not about that either because the commercial repos contain very much the same OSS packages as the standard repo but don't present all of them to everyone all the time. Take for example a car company: GPL3 for in-house projects that are just used by the employees are discouraged but somewhat tolerated. GPL3 for projects that run in the car are a big big no no. You want to be certain that no dev ever introduces GPL3 source into anything that is in the car. You want your build to fail if any of packages change their license to GPL3. You want your build to fail if any of you packages has a known vulnerability.

I know Cargo is not maven, but I believe this is a feature which is crucial for industry adoption. I think I will just add a feature request for this on GitHub.

Re: Cargo: predictable dependency management

#134

Post by steveklabnik » Fri, May 06, 2016, 10:23 PM UTC

weinzierl wrote: post #133

steveklabnik wrote: post #129

Earlier quoted context omitted.

If you just want to ignore the broader OSS ecosystem, and run your own version of Crates.io behind a firewall, that's 100% supported today. The only real issue is that how to do so isn't particularly well-documented.

Sorry for pestering again but I think this is kind of important and I haven't made myself entirely clear yet (English is not my first language). In maven the repo URL is configurable in settings.xml. This URL can be different for different departments of even different projects. From what I see in the cargo source the crates.io URL is hard coded. So the DNS is the only level of redirection we have. Using varying IP a…

  > Sorry for pestering again

No worries! This thread is a bit old but I'll try to pay attention to it.

  > From what I see in the cargo source the crates.io URL is hard coded.

It's not: http://doc.crates.io/config.html#configuration-keys

TL;DR:

  [registry]
  index = "URL_GOES_HERE"

and you're good.

  > It's not about that either because the commercial repos contain very
  > much the same OSS packages as the standard repo but don't present all 
  > of them to everyone all the time.

Ahh yeah. What I mean is, you'd have to set up the packages in that registry yourself. Which sounds like what they'd want to do, so seems fine.

Re: Cargo: predictable dependency management

#135

Post by tatterdemalion » Fri, May 06, 2016, 11:13 PM UTC

gnoway wrote: post #113

riffraff wrote: post #108

Earlier quoted context omitted.

the parent comment might have been worded better, but it has a point, i.e. removing published code generally has to be supported in some way to handle special situations, e.g. * people publish stuff inadvertently (i.e. private information/keys) * people publish stuff they are not allowed to (i.e. copyright and trademark violations) * people publish stuff you do not want to see published (i.e. stuff related to breakin…

I think the key is that crates.io (and rubygems.org, nuget.org, etc.) as repo owners have to own the removal operation themselves and not delegate that to the package owner/maintainer. The repo owner is better positioned to take package consumers' needs into account and make good decisions about when and how to remove a package. As far as satisfying support requests: obviously exceptional stuff like DMCA gets handled…

> I think the key is that crates.io (and rubygems.org, nuget.org, etc.) as repo owners have to own the removal operation themselves and not delegate that to the package owner/maintainer. The repo owner is better positioned to take package consumers' needs into account and make good decisions about when and how to remove a package.

What does this mean? Rubygems and npm both allow a full unpublish, not only a yank. crates.io does not. Rubygems and npm are the same in this regard.

Remember that we are not talking about npm unpublishing someone's library without their approval (that's a different issue), we are talking about npm allowing a user to unpublish a library, which crates.io does not allow a user to do. I will repeat, because people seem not to believe it: rubygems allows this as well, specifically because the maintainers of rubygems could not handle the support tickets that resulted from not providing this feature.

The idea that this is some amateurish aberration from npm is a myth. If Rust is lucky, someday crates.io will have to choose between paying someone to field these tickets or letting users unpublish code.

Re: Cargo: predictable dependency management

#136

Post by tatterdemalion » Fri, May 06, 2016, 11:17 PM UTC

riffraff wrote: post #108

tatterdemalion wrote: post #70

Earlier quoted context omitted.

This is incorrect and very rude. Rubygems has also implemented an automated delete feature. Rubygems and NPM have orders of magnitude more users than crates.io; once crates.io approaches that scale the team will have to allow users to delete crates or somehow find funding to field these kinds of support tickets.

the parent comment might have been worded better, but it has a point, i.e. removing published code generally has to be supported in some way to handle special situations, e.g. * people publish stuff inadvertently (i.e. private information/keys) * people publish stuff they are not allowed to (i.e. copyright and trademark violations) * people publish stuff you do not want to see published (i.e. stuff related to breakin…

How could my comment have been worded better? It is incorrect to claim that npm is alone in allowing automated unpublish, and rude to suggest the reason they allow it is because they do not know what they are doing, when in fact the reason they are doing it is because they have far more users than any other package manager. My comment was completely true and not at all aggressive.

Re: Cargo: predictable dependency management

#137

Post by yazaddaruvala » Sat, May 07, 2016, 12:22 AM UTC

curun1r wrote: post #126

yazaddaruvala wrote: post #103

My Cargo wish list! 1. Like every enterprise build system I've ever gotten to use, I wish Cargo would manage the compiler version as well. 2. The compiler forces me to mark `unsafe` functions/blocks to use unsafe code. Why does Cargo not force me to mark `unsafe` dependencies to use unsafe code?[0] [0] Only dependencies that explicitly use `unsafe` should need to be marked in my Cargo.toml. eg. Iron doesn't use `unsa…

> 1. Like every enterprise build system I've ever gotten to use, I wish Cargo would manage the compiler version as well. With rustup.rs becoming official, it's a much better way to handle compiler versions since it handles cargo versions as well and allows the two to be coupled together. This allows for tighter integration between cargo and rustc as well as making testing of new versions of cargo significantly easier…

To me, as a user (rather than, as a dev), I don't really care about the "Single Responsibility Principle", and the difference between Rustup and Cargo seems artificial.

They are both binaries I need to install that help me manage my Rust application's dependencies. The only arguable difference is that Rustup manages "build dependencies" where Cargo manages "code dependencies". However, the tools I've used, managed both those concepts, and did so seamlessly (I wish I could share more).

Meanwhile, I don't believe Rustup.toml currently exists (I'd love to be corrected). Since other people like the difference between Cargo and Rustup, I'm happy to change my Wishlist 1.

1. I wish Rustup had a Rustup.toml file, such that my git repo could manage the currently used version of rustc. Primarily for consistency across teams, and seamless transition when I `cd ../ && cargo build`.

Re: Cargo: predictable dependency management

#138

Post by pyre » Sat, May 07, 2016, 2:02 AM UTC

ecnahc515 wrote: post #68

pyre wrote: post #63

Earlier quoted context omitted.

What would be useful is a sort of "caching proxy" that could have various knobs to handle situations like: - crates.io is down - crates.io says this cached package doesn't exist. - etc...

This is already possible today with existing caching proxies. This is a great way to make your CI/builds more reliable and quicker.

I meant a caching proxy that functioned as a mini Crates.io in the absence of actual crates.io being up. Depending on the crates.io protocol, just caching HTTP requests might not be enough, but also acting as a (offline-able) middle man that knows the protocol gives rise to other knobs and such (e.g. a configurable blacklist of packages).

Re: Cargo: predictable dependency management

#139

Post by dbaupp » Sat, May 07, 2016, 11:10 AM UTC

cm3 wrote: post #124

dbaupp wrote: post #96

Earlier quoted context omitted.

Could you explain what you're referring to? It seems like a total non sequitur, since as far as I know, there has never been any pain (or even rumour of pain) in a Rust upgrade caused by what libc the compiler uses, but rather breakage in the unstable features that servo uses.

If the official rust+cargo was built statically with musl, then a single download would work on glibc and musl distros. Right now, we don't have a download that works on musl, and one has to bootstrap on a glibc system, targeting musl, which makes it impossible to bootstrap rust+cargo on a musl distro without glibc because bootstrapping requires a snapshot.

Ah, it is a total non sequitur, unrelated to what servo encounters when changing Rust versions?

Re: Cargo: predictable dependency management

#140

Post by steveklabnik » Sat, May 07, 2016, 3:49 PM UTC

yazaddaruvala wrote: post #137

curun1r wrote: post #126

Earlier quoted context omitted.

> 1. Like every enterprise build system I've ever gotten to use, I wish Cargo would manage the compiler version as well. With rustup.rs becoming official, it's a much better way to handle compiler versions since it handles cargo versions as well and allows the two to be coupled together. This allows for tighter integration between cargo and rustc as well as making testing of new versions of cargo significantly easier…

To me, as a user (rather than, as a dev), I don't really care about the "Single Responsibility Principle", and the difference between Rustup and Cargo seems artificial. They are both binaries I need to install that help me manage my Rust application's dependencies. The only arguable difference is that Rustup manages "build dependencies" where Cargo manages "code dependencies". However, the tools I've used, managed bo…

Rustup supports your wish with "rustup override".