Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zb: An Early-Stage Build System

zombiezen.com

Post reply on HN
Print view View original

71–80 of 132 posts

Re: Zb: An Early-Stage Build System

#71

Post by imiric »

kortex wrote: post #41
imiric wrote: post #13

Happy to see someone inspired by Nix, but wanting to carve their own path. Nix popularized some powerful ideas in the Linux world, but it has a steep learning curve and a very unfriendly UI, so there is plenty of room for improvement there. I'm not sure if Lua is the right choice, though. A declarative language seems like a better fit for reproducibility. The goal of supporting non-deterministic builds also seems to…

> The goal of supporting non-deterministic builds also seems to go against this. I think this is actually a great escape hatch. Supporting non-deterministic builds means more folks will be able to migrate their existing build to zb. Postel's law and all that.

Right, could be.

One of the insane things with Nix is that the suggested workflow is to manage _everything_ with it. This means that it wants to replace every package manager in existence, so you see Python, Emacs and other dependency trees entirely replicated in Nix. As well as every possible configuration format. It's craziness... Now I don't need to depend on just the upstream package, I also have to wait for these changes to propagate to Nix packages. And sometimes I just want to do things manually as a quick fix, instead of spending hours figuring out why the Nix implementation doesn't work.

So, yeah, having an escape hatch that allows easier integration with other ecosystems or doing things manually in some cases, would be nice to have.

Re: Zb: An Early-Stage Build System

#72

Post by theLiminator »

aseipp wrote: post #57

Whatever choices this project makes (I have some opinions, but I think they're not too important) I don't see it mentioning one of the most absolutely critical choices Nix made that was absolutely key to its insane success (at least, IMO, as a hardcore contributor and user for like 10+ years): the monorepo, containing all of the packages and all the libraries for use by everyone downstream, and all contributions tryi…

Buck2 looks very principled. Will definitely be interesting as it gets mature in the open source world.

I'm personally convinced monorepo is strictly superior (provided you have the right tooling to support it).

Re: Zb: An Early-Stage Build System

#73

Post by imiric »

pdimitar wrote: post #48
zombiezen wrote: post #34

Earlier quoted context omitted.

That's accurate (unless the config file attempts to read something from the build process, that will trigger a build). It's a good point about debugging build problems. This is an issue I've experienced in Nix and Bazel as well. I'm not convinced that I have a great solution yet, but at least for my own debugging while using the system, I've included a `zb derivation env` command which spits out a .env file that matc…

Surface-level feedback: get rid of the word "derivation". Surely there must be a better way to describe the underlying thing...

Agreed! It's such an alien term to describe something quite mundane. Language clarity is a big part of a friendly UI.

Re: Zb: An Early-Stage Build System

#74

Post by School-Cotton »

kstenerud wrote: post #67
School-Cotton wrote: post #54

Earlier quoted context omitted.

What’s wrong with it? It’s a term of art that means a specific thing in both nix and guix; it’d just be confusing if zb renamed it to something else.

I'm 80% finished moving all of my servers from NixOS to Debian. I used NixOS for 3 years (even wrote some custom flakes) before finally giving up (for the final year I was just too scared to touch it, and then said "I shouldn't be scared of my OS"). I should know what "derivation" means, but I can't for the life of me remember...

I don’t know Nix, but I’ll describe how Guix works, and hopefully it will be obvious what the corresponding Nix concepts are.

A “package” is a high-level description (written in scheme) of how to build something, like: “using the GNU build system with inputs a, b, c, and configure flags x, y, z, build the source available at https://github.com/foo/bar”

The actual builder daemon doesn’t know about the GNU build system, or how to fetch things from GitHub, or how to compute nested dependencies, etc.; it is very simple. All it knows is how to build derivations, which are low-level descriptions of how to build something: “create a container that can see paths a, b, and c (which are themselves other derivations or files stored in the store and addressed by their hash), then invoke the builder script x.”

So when you ask guix to build something, it reads the package definition, finds the source and stores it in the store, generates the builder script (which is by convention usually also written in scheme, though theoretically nothing stops you from defining a package whose builder was written in some other language), computes the input derivation paths, etc., and ultimately generates a derivation which it then asks the daemon to build.

I believe in Nix, rather than scheme, packages are written in nix lang and builder scripts can be written in any language but by convention are usually bash.

So basically long story short, the package is the higher-level representation on the guix side, and the derivation is the lower-level representation on the guix-daemon side.

Re: Zb: An Early-Stage Build System

#75

Post by danmur »

sweeter wrote: post #66
imiric wrote: post #13

Happy to see someone inspired by Nix, but wanting to carve their own path. Nix popularized some powerful ideas in the Linux world, but it has a steep learning curve and a very unfriendly UI, so there is plenty of room for improvement there. I'm not sure if Lua is the right choice, though. A declarative language seems like a better fit for reproducibility. The goal of supporting non-deterministic builds also seems to…

I thought of creating something similar and I was going to use a personal fork of the Go compiler with some mods, anko (which is a really cool go binding language) or righting my own DSL. It's quite the undertaking. I like Nix and NixOS a lot, its really cool, but it has some really odd management issues and the language IMO is horrendous. I used NixOS for around a year and I was changing my Nixpkgs version and I got…

The language has grown on me a bit. I initially hated it but a lot of my pain was not actually the language but the lack of good docs for the standard library.

Still struggle with the tracebacks though. It's painful when things go wrong.

Re: Zb: An Early-Stage Build System

#76

Post by nurettin »

pdimitar wrote: post #48
zombiezen wrote: post #34

Earlier quoted context omitted.

That's accurate (unless the config file attempts to read something from the build process, that will trigger a build). It's a good point about debugging build problems. This is an issue I've experienced in Nix and Bazel as well. I'm not convinced that I have a great solution yet, but at least for my own debugging while using the system, I've included a `zb derivation env` command which spits out a .env file that matc…

Surface-level feedback: get rid of the word "derivation". Surely there must be a better way to describe the underlying thing...

It is the name of a feature in Nix. This is as obfuscated as calling a rock a rock.

Re: Zb: An Early-Stage Build System

#77

Post by pdimitar »

nurettin wrote: post #76
pdimitar wrote: post #48

Earlier quoted context omitted.

Surface-level feedback: get rid of the word "derivation". Surely there must be a better way to describe the underlying thing...

It is the name of a feature in Nix. This is as obfuscated as calling a rock a rock.

Strange thing to say but you do you.

I tried to dabble in Nix several times and the term never stuck.

I suppose for you it's impossible to accept that the term is just bad and unintuitive. And other comments here say the same.

Re: Zb: An Early-Stage Build System

#78

Post by steeleduncan »

zombiezen wrote: post #30
steeleduncan wrote: post #8

Looks great, Nix-with-Lua that also supports Windows would be amazing. Two questions if I may - Does this sandbox builds the way flakes do? - What is MinGW used for on Windows? Does this rely on the MinGW userland, or is it just because it would be painful to write a full bootstrap for a windows compiler while also developing Zb? Also, its great to see the live-bootstrap in there. I love the purity of how Guix's pack…

Nix sandboxes derivation runs on Linux even without flakes, and I'm planning on implementing that, yes: https://github.com/256lights/zb/issues/29 and https://github.com/256lights/zb/issues/31 MinGW is used to build Lua using cgo. I'd like to remove that part, see https://github.com/256lights/zb/issues/28 I haven't started the userspace for Windows yet ( https://github.com/256lights/zb/issues/6 ), but I suspect that i…

Thanks for answering and I really hope it works out. A Nix alternative with less friction would be very welcome!

Re: Zb: An Early-Stage Build System

#80

Post by kaba0 »

aseipp wrote: post #60
zombiezen wrote: post #25

Earlier quoted context omitted.

(Hi! I recognize your name from Bazel mailing lists but I forget whether we've talked before.) I'm mostly contrasting from Nix, which has difficulty with poisoning cache when faced with non-deterministic build steps when using input-addressing (the default mode). If zb encounters a build target with multiple cached outputs for the same inputs, it rebuilds and then relies on content-addressing to obtain build outputs…

I mean, to be fair, Nix is nothing more than a big ass pile of genrule() calls, at the end of the day. Everything is really just genrule. Nix just makes it all work with the sandbox it puts all builds in. Bazel has an equivalent sandbox and I'm pretty sure you can sandbox genrule so it's in a nice, hermetic container. (Side note, but one of my biggest pet peeves is that Nix without the sandbox is actually fundamental…

You (plural) seem to know a great deal about build systems, so I figured I would ask - what’s your opinion about Mill? It’s a not so well known build tool written in scala, but I find its underlying primitives are absolutely on point.

For those who don’t know, its build descriptors are just Scala classes with functions. A function calling another function denotes a dependency, and that’s pretty much it. The build tool will automatically take care of parallelizing build steps and caching them.

How do you think it relates to Nix and alia on a technical level?

Post reply on HN