Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel 1.0

opensource.googleblog.com

Post reply on HN
Print view View original

101–110 of 204 posts

Re: Bazel 1.0

#101

Post by sayrer »

lenkite wrote: post #42

Whats the advantage of using Bazel over CMake ? What does one do about package management ?

For packages, you download things by specifying an archive (zip file, git repo, etc), and a SHA256 hash. Think of it as extremely-precise version pinning. You can also refer to other Bazel projects and use their internal rules if permitted.

The biggest advantages of Bazel over something like CMake are:

1) It is very general, so you can easily express dependencies of this sort:

- concat three text files, then

- run them through a parser generator, then

- use the generated parser in a C library, then

- use that C library in a Go server library and a Rust client library, then

- package the Go server into a Docker container and build an iOS binary that uses the Rust client

This is a cross-language dependency chain that also runs a few Unix tools. In this example, if you edit the Rust client, only the Rust library and the iOS client will be rebuilt. If you edit one of the text files that compose the grammar, everything will get rebuilt.

The dependencies are also a DAG, so if your projects have a shared Go library and you change that, the Go server would get rebuilt, as would the Docker container.

While all this might seem distressingly general, it removes the need for things like "bootstrap.sh" files, "build.rs" files, and all of the unstated dependencies they entail.

2) If you download all of the toolchains (C++, Java, Rust, Go, etc), it's impossible to have "works on my machine" problems. Exceptions are use of "local_*" rules, and Apple binaries (you need machine-local SDKs).

3) This allows great caching. You can run the equivalent of ccache for all of this stuff, locally or shared on a LAN. RBE is even more advanced.

Re: Bazel 1.0

#102

Post by jxramos »

RcouF1uZ4gsC wrote: post #46
paulddraper wrote: post #35

Earlier quoted context omitted.

Bazel doesn't start a server for Java JIT reasons. It starts a server for (1) concurrency control (2) management of worker processes some languages use (3) caching the build graph (recall that Bazel works with very large code bases). These reasons are independent of implementation in C++, Go, Rust, Java AOT, etc. (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it cho…

> (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it chooses.) How much did the fact that Java is very slow starting up influence the decision to use a persistent process instead of some other solution?

Aren't the hot compilers kept around under this mechanism to speed up the next incremental build round? I think it's not just caching of intermediate build output but also not letting the tools start from a cold start.

Re: Bazel 1.0

#103

Post by sayrer »

bradfitz wrote: post #25
paulddraper wrote: post #19

Earlier quoted context omitted.

Parts of Bazel are written in C/C++. The command line client as well as some low level file utilities. But Bazel is distributed as a standalone package in several packaging systems; no need to install Java yourself. Though I doubt many people know this; there's virtually no reason you would be exposed to its language implementation choices. EDIT: Actually I believe Bazel is changing its .deb package to not be standal…

> there's virtually no reason you would be exposed to its language implementation choices Um, doesn't it still have a client that starts up a persistent server and you then have to wait for that server to start up (which, being Java, takes forever) and then deal with those server processes hanging around? That sounds like being exposed to its language implementation choices. As opposed to, say, having fast start-up t…

Does the initial startup time matter?

I think the server is kept around for a number of reasons that have nothing to do with Java startup time. Probably the biggest one is file-watching, using inotify, FSEvents, etc.

https://stackoverflow.com/questions/57982028/how-does-bazel-...

Re: Bazel 1.0

#104

Post by spuz »

pcj-github wrote: post #51

For me, the difference between other build systems and bazel was like the shift from CVS to git... Profoundly better internal data model, but definitely a learning curve.

That makes sense but there aren't many people out there using CVS right now. What were you using before Bazel? I.e. what are you comparing Bazel to?

Re: Bazel 1.0

#105

Post by amelius »

martius wrote: post #95
RcouF1uZ4gsC wrote: post #46

Earlier quoted context omitted.

> (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it chooses.) How much did the fact that Java is very slow starting up influence the decision to use a persistent process instead of some other solution?

My guess is that startup time is not the issue, but loading a large data structure (its cache) from disk can be. Especially if you can't or don't want to use memory mapping because it's hard to do well. But more importantly, I think that the server can monitor file changes ahead of time with things like inotify, saving the time of stat()-ing files when the user wants to perform a build.

> Especially if you can't or don't want to use memory mapping because it's hard to do well.

Why do new languages never address this issue?

Re: Bazel 1.0

#106

Post by arianvanp »

imrehg wrote: post #93
pgrulich wrote: post #84

Earlier quoted context omitted.

Is not the openjdk available for arm? What do you need from Oracle? https://adoptopenjdk.net/releases.html#aarch64_linux

This is aarch64 (that is 64-bit ARM), and NixOS seems more 32-bit arm looking at their wiki https://nixos.wiki/wiki/NixOS_on_ARM On the other hand, they do 64-bit too, so maybe there would be some way to do for one, then bootstrap the other? Other idea, distributions like Debian, seems to build OpenJDK as well, so maybe that would be a way to get the initial package (through an existing distro) https://buildd.debian.…

We only support aarch64 officially. 32 bit ARM is only community-supported.

Re: Bazel 1.0

#107

Post by pja »

halflings wrote: post #26
taneq wrote: post #6

Problem: There are N confusing build systems with arcane rules and you just want to compile your program. Solution: Design a simple, straightforward build tool. Problem: There are N+1 confusing build systems with arcane rules.

Bazel is almost a standard for build systems at this point. Did you ever use it, or at least see what a bazel build rule looks like [1] before calling it arcane? [1] https://docs.bazel.build/versions/1.0.0/tutorial/cpp.html#un...

Build rules for bazel are straightforward when you’re on the happy path & trying to do something that bazel already understands. It’s when you’re not on the happy path that things get awkward.

E.g. How easy is it to integrate a new C++ compiler into bazel? (and I don’t just mean what you get by changing CC to point somewhere else. I want the full integration with the bazel build system in order to get those hermetic builds that are the whole point of using bazel in the first place).

How easy is it to build things which are not in the set of blessed languages? What if I have a Haskell component? Can I easily integrate that into my bazel build? Last time I tried the answer was definitely not.

If you can use the pre-defined BUILD rules then bazel is pretty great. If you can’t and have to start wading into the weeds of bazel’s hinterland then it rapidly stops being so. If you’re inside Google, then you’re probably fine because a blaze/bazel engineer is ultimately only an email away. For the rest of us outside the Googleplex, extending bazel is an extended exercise in frustration in my experience.

Re: Bazel 1.0

#108

Post by fnord123 »

sz4kerto wrote: post #82
RcouF1uZ4gsC wrote: post #46

Earlier quoted context omitted.

> (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it chooses.) How much did the fact that Java is very slow starting up influence the decision to use a persistent process instead of some other solution?

JVM startup is sub-second. Still not great for e.g. small command line utilities, but completely fine otherwise.

JVM startup for `java -version` is around 150ms on my machine:

    $ time java -version
    openjdk version "11.0.1" 2018-10-16
    OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
    OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

    real 0m0.159s
    user 0m0.116s
    sys 0m0.052s
This is how long it takes for a program like git to do actual work. Once you start running an actual program like Maven, it blows up to 500ms for `mvn -version`:

    $ time  mvn -version > /dev/null

    real 0m0.570s
    user 0m0.431s
    sys 0m0.145s
Bazel takes 150ms to do nothing:

    $ time bazel --version > /dev/null

    real 0m0.144s
    user 0m0.028s
    sys 0m0.063s
As a baseline, make takes under 30ms to do nothing:

    $ time make -version > /dev/null

    real 0m0.025s
    user 0m0.002s
    sys 0m0.016s

Re: Bazel 1.0

#109

Post by _the_inflator »

klodolph wrote: post #10
taneq wrote: post #6

Problem: There are N confusing build systems with arcane rules and you just want to compile your program. Solution: Design a simple, straightforward build tool. Problem: There are N+1 confusing build systems with arcane rules.

I’m getting a little tired of this particular brand of lazy cynicism. I’m happy to see people inventing new build systems, programming languages, game engines, ORMs, or anything else where the conventional wisdom is to use the existing tools. It may be crazy optimistic but we need people to keep these skill sets alive.

I side with you. Why build Angular, React? There was jQuery...

After all, there are only tools. If you need a better tool, build it and ignore the naysayers. Everything is an experiment. If it inspires people, the better.

Re: Bazel 1.0

#110

Post by gpderetta »

boris wrote: post #49
RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

There are alternatives that satisfy this (valid, IMO) requirement. For example, there is build2 ( https://build2.org ; full disclousure: I am involved with the project) that tries to achieve the same objectives (correct & fast builds, uniform interface accross platforms, being language agnostic, etc) though not necessarily using the same mechanisms (build2, for example, tries to be fairly light-weight). Here is the i…

I checked as I was interested, but it seems that build2 doesn't do distributed builds.

edit: remote->distributed

Post reply on HN