Live data from Hacker News

Bazel – Correct, reproducible, fast builds for everyone

bazel.io

51–60 of 185 posts

Re: Bazel – Correct, reproducible, fast builds for everyone

#51
post #4

Working at Google, Blaze is one of the technologies that amazes me most. Any engineer can build any Google product from source on any machine just by invoking a Blaze command. I may not want to build GMail from source (could take a while) but it's awesome to know that I can. I think this could be hugely useful to very large open source projects (like databases or operating systems) that may be intimidating for contri…

Wow, sounds like enterprise Gentoo, in a good way.

Re: Bazel – Correct, reproducible, fast builds for everyone

#52
post #27

Earlier quoted context omitted.

I didn't downvote him but it sounds like "he's driving angry". He does work for that "evil" company in Redmond. :-). Maybe he can help make F# a great cross-platform language and increase the goodwill? Microsoft did an incredible job with F# but it really only runs well on Windows.

If he works for MS he has zero right to even be mildly annoyed, since we have for decades been living in a world where "only runs on Windows" isn't even in the small print.

I've run into some (but def. not all) current and former MS folks who carry this ironic shoulder-chip. One such indignantly complained about a major open-source project's janky Windows support something like "well, that's just because they choose not to support the platform!"

I found this really interesting, as after a bit of conversation, the speaker was clearly unaware of how MS' technical and business models around Windows have impeded open source work. A for-pay operating system with a profit-center development toolchain presents a very large barrier to Unix-centric OSS projects. Not to mention the numerous technical impedance mismatches between the $unix and Windows worlds. (And these days we have tools like libuv to help with that, but still.)

Re: Bazel – Correct, reproducible, fast builds for everyone

#53
post #22

Earlier quoted context omitted.

I've started at Google 4 months ago, and it's one of the best things to discover. Now open-sourced :)

Though not really open source ;)

https://news.ycombinator.com/item?id=9257571

Re: Bazel – Correct, reproducible, fast builds for everyone

#54
post #35

I had a bit of a read but I didn't find where it explains (code or doc) how it achieves reproducible builds. It seems like a stricter, huge make-like harness (in fact it reminds me of the mozilla firefox python build system a bit). It's not bad by any means, but it seems like to me it doesn't "magically" fix the "be reproducible" problem at all (which is what it seem to claim) Am I missing something?

You are absolutely correct: Bazel by itself does not make your builds reproducible. If a tool calls rand() or bakes the current time into its output, reproducibility goes out of the window.

What Bazel does, however, is to make it possible to run build steps in a sandbox (although the current one is kinda leaky) so that your build is isolated from the environment and thus behaves in the same way on any computer. It also tracks dependencies correctly so that it knows when a specific action needs to be re-run.

This makes it possible to diagnose non-reproducible build steps easily. At Google, the hit rate of our distributed build cache usually floats around 99%, and this would be impossible without reproducible build steps.

Re: Bazel – Correct, reproducible, fast builds for everyone

#55
post #35

I had a bit of a read but I didn't find where it explains (code or doc) how it achieves reproducible builds. It seems like a stricter, huge make-like harness (in fact it reminds me of the mozilla firefox python build system a bit). It's not bad by any means, but it seems like to me it doesn't "magically" fix the "be reproducible" problem at all (which is what it seem to claim) Am I missing something?

You're right - it doesn't magically solve build reproducibility. Bazel pushes you towards a build configuration where you have to describe (in a terse way) the entire dependency graph of what is being built. It allows Bazel to be smart about where in the graph things are stale.

If you run a script that outputs intermediate files, Bazel needs to know about that scripts inputs and outputs. And it works better if it knows them ahead of time.

Re: Bazel – Correct, reproducible, fast builds for everyone

#56
post #27
post #21

Earlier quoted context omitted.

Not really sure why this is getting downvoted - it's kind of an important detail when choosing a build system if you have to do multiplatform deployment.

I didn't downvote him but it sounds like "he's driving angry". He does work for that "evil" company in Redmond. :-). Maybe he can help make F# a great cross-platform language and increase the goodwill? Microsoft did an incredible job with F# but it really only runs well on Windows.

But they're already working on doing that.

Re: Bazel – Correct, reproducible, fast builds for everyone

#57
post #35

I had a bit of a read but I didn't find where it explains (code or doc) how it achieves reproducible builds. It seems like a stricter, huge make-like harness (in fact it reminds me of the mozilla firefox python build system a bit). It's not bad by any means, but it seems like to me it doesn't "magically" fix the "be reproducible" problem at all (which is what it seem to claim) Am I missing something?

It's not magic; you have to work at it. (For example, make sure that zip doesn't put timestamps in the file.) But it's designed so that code generators should act as pure functions from input files to output files, and many generators actually are, especially the built in ones. If you do this then the build system will help you.

Writing generators to run this way is kind of a pain, actually, sort of like writing code to run in a sandbox. Also, the generators themselves must be checked in, and often built from source. But we consider the results worth it.

Re: Bazel – Correct, reproducible, fast builds for everyone

#60
post #37
post #23

A couple of questions: * If I have a Maven-based project with heavy reliance on pre-built jars from Maven Central, what's the recipe to port it to Bazel? * Related, if I have multiple github repos, say a couple open source libraries and a couple private repos, what's a good recipe in conjunction to Bazel?

Check out http://bazel.io/docs/build-encyclopedia.html#maven_jar . In the root of your build, specify the jars you want from maven and then add them as dependencies in your BUILD files. The first time you run "bazel build", they'll be downloaded and cached from then on. It's somewhat limited in functionality at the moment, but should work for basic "download and depend on a jar". For multiple Github repos, use http:/…

Thanks for the tips. I'm super-hyped that blaze was open sourced, it is one of the best systems I've ever had the pleasure to work with.

A couple more questions :)

* Any pointers for adding Scala (sbt?) support? I'd start here: http://bazel.io/docs/skylark/rules.html.

* Suppose I develop using multiple repos and http_archive. I'd like to make changes both to a library and to a project that depends on it simultaneously, without committing the library patches to master github repo just yet. Is there a way to configure the http_archive, let's say by saying "bazel --mode=local", and have it customize the remote archive http to use a different url (say, my github's fork instead of the master github) for that build?

Post reply on HN