Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Release engineering is exhausting so here's cargo-dist

blog.axo.dev

Post reply on HN
Print view View original

11–20 of 66 posts

Re: Release engineering is exhausting so here's cargo-dist

#11

Post by Gankra »

Shared404 wrote: post #10

> (Did you know the warning you get on Windows isn't about code signing, but is actually just a special flag Windows' builtin unzipping tool sets on all executables it extracts?) My jaw hit the floor here.

Correction on this someone else sent me:

The check of interest is for a Mark Of The Web[0] flag that Windows includes in file system metadata. The builtin unzipping utility just faithfully propagates this flag to the files it unpacks. Other utilities like 7zip are unlikely to do this propagation (effectively clearing it).

But yeah either way it has nothing to do with code signing!

[0]: https://nolongerset.com/mark-of-the-web-details/

Re: Release engineering is exhausting so here's cargo-dist

#12

Post by capableweb »

IshKebab wrote: post #4

> Congrats kid you're A Release Engineer now and your life is hell. Enjoy debugging basic typos on a remote machine with 20 minute latency because you sure can't run those Github CI bash-scripts-in-yaml files locally! Yes! Why is this accepted?? Gitlab has a way of running CI locally (for Docker based builds anyway; who knows about Windows or Mac) but a) it doesn't support the same features at the "proper" one (even…

CircleCI solved this so many years ago, allow users to SSH into the environment the build happens.

Workflow is something like:

- Guess together a random CI or CD workflow, this is just to kick off the process. You can also start with a empty config.

- Something fails, get SSH host+port to connect to

- Enter environment and manually do everything you want to be able to automatically do

- Execute `history` and copy output, trim it into something nicer and put in your config

Re: Release engineering is exhausting so here's cargo-dist

#13

Post by brundolf »

IshKebab wrote: post #4

> Congrats kid you're A Release Engineer now and your life is hell. Enjoy debugging basic typos on a remote machine with 20 minute latency because you sure can't run those Github CI bash-scripts-in-yaml files locally! Yes! Why is this accepted?? Gitlab has a way of running CI locally (for Docker based builds anyway; who knows about Windows or Mac) but a) it doesn't support the same features at the "proper" one (even…

What we need is a standard CI language/config format across vendors (stuffing a custom DSL inside YAML doesn't count)

That would allow for a tooling ecosystem of static checkers, offline runners, IDE integrations, etc etc, and would also cut down on the learning barrier every time you switch to a new company that uses a different vendor

Re: Release engineering is exhausting so here's cargo-dist

#14

Post by jakub_g »

Gankra wrote: post #11
Shared404 wrote: post #10

> (Did you know the warning you get on Windows isn't about code signing, but is actually just a special flag Windows' builtin unzipping tool sets on all executables it extracts?) My jaw hit the floor here.

Correction on this someone else sent me: The check of interest is for a Mark Of The Web[0] flag that Windows includes in file system metadata. The builtin unzipping utility just faithfully propagates this flag to the files it unpacks. Other utilities like 7zip are unlikely to do this propagation (effectively clearing it). But yeah either way it has nothing to do with code signing! [0]: https://nolongerset.com/mark-of…

If someone's interested with more details about MoTW, EricLaw (long time MSFT engineer at IE and Edge teams) got you covered:

https://textslashplain.com/2016/04/04/downloads-and-the-mark...

https://textslashplain.com/2022/12/02/mark-of-the-web-additi...

Re: Release engineering is exhausting so here's cargo-dist

#15

Post by MuffinFlavored »

I think this would benefit from an example repo that shows just Cargo.toml for a simple src/main.rs with `fn main() { println!("Hello, world!"); }` project with the simplest needed .github/workflows/foo.yaml possible to actually use this.

If it was in the article and I missed it I apologize.

Re: Release engineering is exhausting so here's cargo-dist

#16

Post by WorldMaker »

wizzwizz4 wrote: post #9
IshKebab wrote: post #4

> Congrats kid you're A Release Engineer now and your life is hell. Enjoy debugging basic typos on a remote machine with 20 minute latency because you sure can't run those Github CI bash-scripts-in-yaml files locally! Yes! Why is this accepted?? Gitlab has a way of running CI locally (for Docker based builds anyway; who knows about Windows or Mac) but a) it doesn't support the same features at the "proper" one (even…

> But still, how is this not a core feature of all CI systems? Vendor lock-in, presumably.

In multiple directions, too. It's easier to build the CI system itself if you are only targeting one class of servers/one means of hosting servers/one specific "cloud".

Re: Release engineering is exhausting so here's cargo-dist

#17

Post by Gankra »

MuffinFlavored wrote: post #15

I think this would benefit from an example repo that shows just Cargo.toml for a simple src/main.rs with `fn main() { println!("Hello, world!"); }` project with the simplest needed .github/workflows/foo.yaml possible to actually use this. If it was in the article and I missed it I apologize.

The "way-too-quickstart" is the minimal example: https://github.com/axodotdev/cargo-dist#way-too-quick-start

A key feature of cargo-dist is that

cargo dist init --ci=github

should simply set everything up for an arbitrary* Rust workspace and you just check in the results.

* Not sufficiently tested for all the wild workspaces you can build, but a "boring" one with a bunch of libraries supporting one binary should work -- that's what cargo-dist's own workspace is, and it's self-hosting without any configuration. Any time I want to update the bootstrap dist I just install that version of cargo-dist on my machine and run `cargo dist generate-ci github --installer=...` to completely overwrite the ci with the latest impl.

Re: Release engineering is exhausting so here's cargo-dist

#18

Post by Gankra »

Gankra wrote: post #17
MuffinFlavored wrote: post #15

I think this would benefit from an example repo that shows just Cargo.toml for a simple src/main.rs with `fn main() { println!("Hello, world!"); }` project with the simplest needed .github/workflows/foo.yaml possible to actually use this. If it was in the article and I missed it I apologize.

The "way-too-quickstart" is the minimal example: https://github.com/axodotdev/cargo-dist#way-too-quick-start A key feature of cargo-dist is that cargo dist init --ci=github should simply set everything up for an arbitrary* Rust workspace and you just check in the results. * Not sufficiently tested for all the wild workspaces you can build, but a "boring" one with a bunch of libraries supporting one binary should work…

Just to elaborate on this a bit: as discussed in the Concepts section of the docs[0] the core of cargo-dist absolutely supports workspaces with multiple binaries, and will chunk them out into their own distinct logical applications and provide builds/metadata for all them.

However this isn't fully supported by the actual github CI integration yet[1], as I haven't implemented proper support for detecting that you're only trying to publish a new version of only one of the applications (or none of them!), and it doesn't properly merge the release notes if you're trying to publish multiple ones at once.

I never build workspaces like that so I'm waiting for someone who does to chime in with the behaviour they want (since there's lots of defensible choices and I only have so many waking hours to implement stuff).

[0]: https://github.com/axodotdev/cargo-dist/#concepts [1]: https://github.com/axodotdev/cargo-dist/issues/69

Re: Release engineering is exhausting so here's cargo-dist

#19

Post by MuffinFlavored »

Gankra wrote: post #17
MuffinFlavored wrote: post #15

I think this would benefit from an example repo that shows just Cargo.toml for a simple src/main.rs with `fn main() { println!("Hello, world!"); }` project with the simplest needed .github/workflows/foo.yaml possible to actually use this. If it was in the article and I missed it I apologize.

The "way-too-quickstart" is the minimal example: https://github.com/axodotdev/cargo-dist#way-too-quick-start A key feature of cargo-dist is that cargo dist init --ci=github should simply set everything up for an arbitrary* Rust workspace and you just check in the results. * Not sufficiently tested for all the wild workspaces you can build, but a "boring" one with a bunch of libraries supporting one binary should work…

Cool, thanks.

It creates a ~100 line release.yml file

1. create GitHub release

2. upload artifacts

3. upload manifest

4. publish GitHub release

I like it, it's opinionated, but I don't know how much it will catch on. If somebody needs to maintain a 100 line GitHub Actions release YAML file, in my experience you typically want to understand everything in it in case you need to adjust it for future needs/as they grow.

It's well done though. Curious to see how much adoption it picks up.

Re: Release engineering is exhausting so here's cargo-dist

#20

Post by chatmasta »

I'm really not a fan of the "download the prebuilt binary from github releases" workflow that's been proliferating along with the popularity of Rust. It seems like a step backward in terms of secure package management, and it's surprising to me that Rust doesn't offer a more out-of-box experience for this, instead encouraging users to build from source. I understand the arguments for this, and I even find some of them convincing - namely, that the inverse problem of opaque intermediate binaries and object files would be much worse as it would cause a cascade of brittle builds and barely any packages would work.

But the fact remains that end users want to download a binary, and the common approach to this is to simply publish them to GitHub actions. Surely Cargo could offer a better option than this, while retaining the ability to build all packages from source (maybe you can only publish binaries to Cargo _in addition_ to the source files... or maybe Cargo.toml could include a link to GitHub releases, and Cargo could include a command for downloading directly from there.)

In the meantime, I've been considering publishing Rust binaries to npm (in addition to GitHub releases). I got the idea from esbuild, which is written in Go but distributed via npm. What do people think of this approach? Here's a recent blog post [0] describing the setup.

[0] https://blog.orhun.dev/packaging-rust-for-npm/

Post reply on HN