Live data from Hacker News

Release engineering is exhausting so here's cargo-dist

blog.axo.dev

21–30 of 66 posts

Re: Release engineering is exhausting so here's cargo-dist

#21
Awesome thank you. I'm adding it to the cargo favorites list: https://github.com/sixarm/cargo-install-favorites

Tiny feedback:

- Can you consider changing "git add ." to be explicit e.g. "git add Cargo.toml .github/workflows/release.yml"?

- How about modifying Cargo.toml to add cargo-dist as a dev-dependency? I know it's not strictly necessary; it's just very helpful for typical collaboration.

Re: Release engineering is exhausting so here's cargo-dist

#22
post #11

> (Did you know the warning you get on Windows isn't about code signing, but is actually just a special flag Windows' builtin unzipping tool sets on all executables it extracts?) My jaw hit the floor here.

Correction on this someone else sent me: The check of interest is for a Mark Of The Web[0] flag that Windows includes in file system metadata. The builtin unzipping utility just faithfully propagates this flag to the files it unpacks. Other utilities like 7zip are unlikely to do this propagation (effectively clearing it). But yeah either way it has nothing to do with code signing! [0]: https://nolongerset.com/mark-of…

macOS has a similar feature with Gatekeeper, which bit me when preparing a Pyinstaller binary for Mac. The flag doesn't get added when you download a file with curl, but it does when you download it through a web browser, which can cause difficult to debug issues with binaries downloaded from GitHub releases.

You can remove this flag with the xattr command:

    xattr -d com.apple.quarantine the_quarantined_binary
I wrote up the details of this in a PR [0] where I last dealt with it.

[0] https://github.com/splitgraph/sgr/pull/656

Re: Release engineering is exhausting so here's cargo-dist

#23

I'm really not a fan of the "download the prebuilt binary from github releases" workflow that's been proliferating along with the popularity of Rust. It seems like a step backward in terms of secure package management, and it's surprising to me that Rust doesn't offer a more out-of-box experience for this, instead encouraging users to build from source. I understand the arguments for this, and I even find some of the…

Worth considering using nix with cargo. Of course it still involves a lot of "download from github" or "download from nix cache" but reproducibility + tight source hash pinning helps guarantee provenance.

Re: Release engineering is exhausting so here's cargo-dist

#24
post #21

Awesome thank you. I'm adding it to the cargo favorites list: https://github.com/sixarm/cargo-install-favorites Tiny feedback: - Can you consider changing "git add ." to be explicit e.g. "git add Cargo.toml .github/workflows/release.yml"? - How about modifying Cargo.toml to add cargo-dist as a dev-dependency? I know it's not strictly necessary; it's just very helpful for typical collaboration.

Do you mean this kind of dep? https://rust-lang.github.io/rfcs/3028-cargo-binary-dependenc...

That's an interesting thought, I'm not sure I've ever seen someone employ that as a pattern. Actually no wait, I thought cargo bin-deps specifically gave the developer no way to manually invoke it (i.e. there's no equivalent functionality to npm's npx)? Without that, what use would the dependency be?

Re: Release engineering is exhausting so here's cargo-dist

#25
post #24
post #21

Awesome thank you. I'm adding it to the cargo favorites list: https://github.com/sixarm/cargo-install-favorites Tiny feedback: - Can you consider changing "git add ." to be explicit e.g. "git add Cargo.toml .github/workflows/release.yml"? - How about modifying Cargo.toml to add cargo-dist as a dev-dependency? I know it's not strictly necessary; it's just very helpful for typical collaboration.

Do you mean this kind of dep? https://rust-lang.github.io/rfcs/3028-cargo-binary-dependenc... That's an interesting thought, I'm not sure I've ever seen someone employ that as a pattern. Actually no wait, I thought cargo bin-deps specifically gave the developer no way to manually invoke it (i.e. there's no equivalent functionality to npm's npx)? Without that, what use would the dependency be?

Yes, where that link describes "[dev-dependencies]" and "[build-dependencies]".

I use the dev-dependencies section often, and the build-dependencies rarely. And anyone here, please correct my understanding if there's a better way to do what I'm describing.

For me, these sections are an easy way to be explicit with collaborators that the project needs the deps installed in order to work on the project, and does not want/need the deps to be compiled into the release.

My #1 use case is to have a collaborator start with a git clone, then run "cargo build", and have cargo download and cache everything that's needed to work on the project and release it. My #2 use case is to deal with semver, so a project can be explicit that it needs a greater cargo-dist version in the future.

To your point about cargo bin deps akin to npx, yes, your understanding matches mine i.e. not available yet. I do advocate for that feature to be added because it's helpful for local environments. Cargo does offer "default-run" which shows there an awareness of a developer preferring specific local exectuables-- maybe Cargo can/will add more like that for deps?

Re: Release engineering is exhausting so here's cargo-dist

#26
post #11

Earlier quoted context omitted.

Correction on this someone else sent me: The check of interest is for a Mark Of The Web[0] flag that Windows includes in file system metadata. The builtin unzipping utility just faithfully propagates this flag to the files it unpacks. Other utilities like 7zip are unlikely to do this propagation (effectively clearing it). But yeah either way it has nothing to do with code signing! [0]: https://nolongerset.com/mark-of…

macOS has a similar feature with Gatekeeper, which bit me when preparing a Pyinstaller binary for Mac. The flag doesn't get added when you download a file with curl, but it does when you download it through a web browser, which can cause difficult to debug issues with binaries downloaded from GitHub releases. You can remove this flag with the xattr command: xattr -d com.apple.quarantine the_quarantined_binary I wrote…

This is actually pretty similar. The OS has an alternative data stream(An idea they stole from Mac), and they list what site a exe was downloaded on, or if it came from somewhere else. Others incorrectly called it a flag, when it works by having two different file data streams for a single file, one is the default one.

So for example, a single file can actually contain two different "files"(File data).

So, foo.exe, actually will effectively open the file foo.exe:DEFAULT. You could also add a piece of malware to the foo file in place of a datastream. So foo.exe is legit, but if you open foo.exe:MALWARE , it will open up the malware datastream.

So tldr, how Windows does this, it when you get a file from a third party source(Internet, USB Drive, etc), it adds a new datastream in the form of a textfile. And the textfile contains info about the source. Namely, a number for location it came from(3? for web), and then some more info.

Re: Release engineering is exhausting so here's cargo-dist

#27

I'm really not a fan of the "download the prebuilt binary from github releases" workflow that's been proliferating along with the popularity of Rust. It seems like a step backward in terms of secure package management, and it's surprising to me that Rust doesn't offer a more out-of-box experience for this, instead encouraging users to build from source. I understand the arguments for this, and I even find some of the…

we actually agree and are working on this! github releases are just an easy initial target, and makes our tool a drop-in replacement for the kinds of things people are already doing. longer-term we'd like to see something more robust, and cargo-dist is the first cog in that machine.

i have personally packaged and published many rust devtools on npm (cloudflare's wrangler, apollo's rover, wasm-pack) but that was largely because they were targeted at a javascript developer audience.

as a former npm registry engineer i'm curious what you find to be the particular value of publishing to npm? installing node is actually very unpleasant and then getting global installs to work is also... very unpleasant. i think it works well for people already in that ecosystem but i think we can build something better for a more agnostic audience that can give a similar out-of-box exp without requiring a centralized registry. would love to learn more about your perspective!

Re: Release engineering is exhausting so here's cargo-dist

#28

I'm really not a fan of the "download the prebuilt binary from github releases" workflow that's been proliferating along with the popularity of Rust. It seems like a step backward in terms of secure package management, and it's surprising to me that Rust doesn't offer a more out-of-box experience for this, instead encouraging users to build from source. I understand the arguments for this, and I even find some of the…

I've noticed people publishing binaries to PyPI too - you can run "pip install ziglang" to get Zig, for example.

I wrote a bit about that pattern here: https://simonwillison.net/2022/May/23/bundling-binary-tools-...

Post reply on HN