Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel – Correct, reproducible, fast builds for everyone

bazel.io

Post reply on HN
Print view View original

141–150 of 185 posts

Re: Bazel – Correct, reproducible, fast builds for everyone

#141

Post by alblue »

spullara wrote: post #94
jibu wrote: post #88

Maven doesn't work so well when there are loads of small self contained 'micro-libraries' (yes, sub-projects, but they are so involved to set up they almost defeat the purpose). Was considering pants -- which doesnt seem like it has great adoption? -- but this seems like its substantially more fully featured. Presumably will also make opensourcing internal projects easier. That can't be a bad thing :)

WRT to Java support: Since it doesn't appear to generate poms or publish to maven repositories it doesn't seem very useful on the open source part of things. It seems explicitly for generating internal, proprietary software from a monolithic source tree. I would have much rather seen the incremental compiler and jar generator integrated to maven than replacing the entire build system.

Actually Maven 3.3 was released recently which has a smart builder for building separate parts in parallel, and using Takari plugins you can use the Eclipse complier which is parallelising in itself. See http://takari.io for more details.

Re: Bazel – Correct, reproducible, fast builds for everyone

#142

Post by asuffield »

astral303 wrote: post #50
setheron wrote: post #36

If i'm sticking to primarily Java; is there a benefit to using Bazel as opposed to Maven / Gradle / Sbt ?

At first impression, unless you have a single gigantic source code base, unlikely. From their FAQ: >> "Gradle: Bazel configuration files are much more structured than Gradle's, letting Bazel understand exactly what each action does. This allows for more parallelism and better reproducibility" The value of "more parallelism" depends on the complexity of your Java source code base. I can easily imagine why this extra s…

blaze is nothing remotely like the wall of cruft that maven forces you to climb for everything you do. I would describe it as "almost entirely unlike maven".

Re: Bazel – Correct, reproducible, fast builds for everyone

#143

Post by zak_mc_kracken »

ovidiup wrote: post #130
OneMoreGoogler wrote: post #104

Earlier quoted context omitted.

> Any engineer can build any Google product from source on any machine A little too optimistic :) You can't build Android, Chrome, ChromeOS, iOS apps, etc. via blaze.

When I worked at Google I built a Blaze extension to be able to build Android apps. It worked really well, though I'm not sure how well it was maintained after I left in 2010. Internally at Google, Blaze was extremely customizable, and I hope Bazel too, so one can easily add support for building iOS apps etc. EDIT #1: I see support for building Objective-C apps is already present in Bazel. EDIT #2: Bazel uses Skylark…

The Chromium tool chain is pretty insane. ninja, fetch, etc... There's really no excuse for this since Google has such a strong (and now open source) build system.

Re: Bazel – Correct, reproducible, fast builds for everyone

#147

Post by cromwellian »

w4tson wrote: post #140

It's another impressive feat from Google and reading the comments I've kind of established that 1. Binaries are checked in to source 2. It's more structured than Gradle 3. It's for very large code bases 5. It's nix only But... 1. We've already had the "chuck it in a lib directory" approach. The distributed approach maven/ivy etc seems to be working for the millions of developers out there who just have to get through…

I don't know about Bazel, but Blaze doesn't "check in binaries". Build artifacts are cached, but not "checked in".

The problem with maven and gradle is that their build actions/plugins can have have unobservable side effects.

This approach is more 'pure functional'. You have rules which take inputs, run actions, produce outputs and memoize them. If inputs don't change, then you use memoized outputs and don't run the action.

As long as your actions produce observable side effects in the outputs (and don't produce side effects which are not part of the outputs, but product state which depended upon in some manner), then you can do a lot of optimizations on this graph.

In my experience with maven and gradle, they are way way slower, and that's on relatively small projects

Re: Bazel – Correct, reproducible, fast builds for everyone

#148

Post by esrauch »

hansjorg wrote: post #87
hanwenn wrote: post #10

Earlier quoted context omitted.

See http://bazel.io/docs/FAQ.html , "Will Bazel make my builds reproducible automatically? For Java and C++ binaries, yes, assuming you do not change the toolchain. If you have build steps that involve custom recipes (eg. executing binaries through a shell script inside a rule), you will need to take some extra care: Do not use dependencies that were not declared. Sandboxed execution (–spawn_strategy=sandboxed, only…

When you say Java, does that include Android? I see that Android is supported, but couldn't find anything about reproducibility. Reproducible Android builds would be very interesting.

http://bazel.io/docs/roadmap.html

It looks like they only have iOS and not Android in this first release, but they are planning on adding Android support ~June of this year.

Re: Bazel – Correct, reproducible, fast builds for everyone

#149

Post by fizbin »

hanwenn wrote: post #10
middleclick wrote: post #7

Earlier quoted context omitted.

So the builds are reproducible automatically?

See http://bazel.io/docs/FAQ.html , "Will Bazel make my builds reproducible automatically? For Java and C++ binaries, yes, assuming you do not change the toolchain. If you have build steps that involve custom recipes (eg. executing binaries through a shell script inside a rule), you will need to take some extra care: Do not use dependencies that were not declared. Sandboxed execution (–spawn_strategy=sandboxed, only…

Specifically, people should note that many code generators are not carefully designed for strict reproducibility, and will stick time stamps in generated output.

Even if you undo that, code generation tools are liable to at some point traverse a dictionary without caring about whether the result is deterministic. I spent some time at Google fighting with antlr to try to get it to have deterministic output and I still think that I left some corner case uncovered.

Re: Bazel – Correct, reproducible, fast builds for everyone

#150

Post by nharmata-google »

georgehm wrote: post #138
lberki wrote: post #49

Earlier quoted context omitted.

Currently about 60% of our code (in terms of lines of Java code, excluding tests) is open sourced. The rest is glue logic to internal Google systems or build rules that we haven't open sourced. Some of these rules, we are planning to open source in the future, and some others are specific to Google, so they don't really make much sense in the open source tree.

What about skyframe? http://bazel.io/docs/skyframe.html looks like an overview without any examples. Couldn't find any references to it in the bazel code at github too.

https://github.com/google/bazel/tree/master/src/main/java/co... is the implementation of the skyframe engine (the general-purpose memoizing, incremental, functional evaluation framework) and https://github.com/google/bazel/tree/master/src/main/java/co... contains the implementation of bazel-on-top-of-skyframe.
Post reply on HN