Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel 1.0

opensource.googleblog.com

Post reply on HN
Print view View original

91–100 of 204 posts

Re: Bazel 1.0

#91

Post by emelski »

rienbdj wrote: post #88
loeg wrote: post #67

Earlier quoted context omitted.

> Having a proper dependency graph makes it possible to cache maximally and accurately, and that results in fast builds. Pretty much every other build system (e.g., make) is rooted in having a proper dependency graph. This is not a unique property of Bazel.

Make (to use your example) does not do any sandboxing to ensure that the declared dependency graph is actually correct. This is the big innovation in Bazel (and Co).

Electric Make has had sandboxing since 2002, no conversion from your familiar make-based builds to a new shiny build tool required, and it can make on-the-fly corrections to execution order if that sandboxing reveals that incomplete dependency specifications caused something to run in the wrong order (relative to a strictly serial build).

"Sandboxing" in a build tool cannot be claimed as Bazel's innovation.

Re: Bazel 1.0

#92

Post by meisterT »

ksec wrote: post #33
RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

>I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. Java can be compiled into native binaries using GraalVM. Not to mention you dont actually have to install Java as it comes in bundled.

native-image performance without PGO might be worse than with JIT, at least in some experiments that I did.

See also https://github.com/oracle/graal/issues/979

Re: Bazel 1.0

#93

Post by imrehg »

pgrulich wrote: post #84
arianvanp wrote: post #79

Earlier quoted context omitted.

We didn't have Bazel on ARM for NixOS because we can't bootstrap openJDK on ARM. There's no source code for a JVM that compiles on ARM afaik. Though in theory we could package binary blobs from oracle and bootstrap JDK and Bazel from there, it means now our trust path for your critical build tool has a random Oracle blob in its trust path there that is extremely hard to get rid off. Build systems should be easy to bu…

Is not the openjdk available for arm? What do you need from Oracle? https://adoptopenjdk.net/releases.html#aarch64_linux

This is aarch64 (that is 64-bit ARM), and NixOS seems more 32-bit arm looking at their wiki https://nixos.wiki/wiki/NixOS_on_ARM On the other hand, they do 64-bit too, so maybe there would be some way to do for one, then bootstrap the other?

Other idea, distributions like Debian, seems to build OpenJDK as well, so maybe that would be a way to get the initial package (through an existing distro) https://buildd.debian.org/status/logs.php?pkg=openjdk-11 - see the armel/armhf/arm64 architectures (arm64 equals aarch64).

Re: Bazel 1.0

#94

Post by 9nGQluzmnq3M »

choppaface wrote: post #89

One of the major time savers of bazel is Remote Build Execution (RBE), which allows you to build modules in parallel in the cloud. So if you have 1000 CPUs, you can really just have a client do `bazel build -j 1000 //...` and you can get a huge speed-up. Remote (and local) builds all happen in a sandbox, so you don't have to worry about e.g. preparing a docker image with the worker / build slave environment. (You do,…

> GCE is slow to work on supporting auto-scale, so you have to pay for unused workers.

Can you expand on what you mean here? GCE has had autoscaling for years and it's quite configurable, I don't see why you would need to keep unused workers around unless your build load is extremely spiky.

Re: Bazel 1.0

#95

Post by martius »

RcouF1uZ4gsC wrote: post #46
paulddraper wrote: post #35

Earlier quoted context omitted.

Bazel doesn't start a server for Java JIT reasons. It starts a server for (1) concurrency control (2) management of worker processes some languages use (3) caching the build graph (recall that Bazel works with very large code bases). These reasons are independent of implementation in C++, Go, Rust, Java AOT, etc. (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it cho…

> (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it chooses.) How much did the fact that Java is very slow starting up influence the decision to use a persistent process instead of some other solution?

My guess is that startup time is not the issue, but loading a large data structure (its cache) from disk can be.

Especially if you can't or don't want to use memory mapping because it's hard to do well.

But more importantly, I think that the server can monitor file changes ahead of time with things like inotify, saving the time of stat()-ing files when the user wants to perform a build.

Re: Bazel 1.0

#96

Post by pjmlp »

cromwellian wrote: post #75
RcouF1uZ4gsC wrote: post #46

Earlier quoted context omitted.

> (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it chooses.) How much did the fact that Java is very slow starting up influence the decision to use a persistent process instead of some other solution?

Java starts up in ~1s, something like Graal could make this faster. But I find this Java criticism more a symptom of Java-derangement syndrome, because python and node build systems have even worse startup times and no one says anything.

Indeed, but it isn't a cool toy among this crowd.

Re: Bazel 1.0

#97

Post by choppaface »

9nGQluzmnq3M wrote: post #94
choppaface wrote: post #89

One of the major time savers of bazel is Remote Build Execution (RBE), which allows you to build modules in parallel in the cloud. So if you have 1000 CPUs, you can really just have a client do `bazel build -j 1000 //...` and you can get a huge speed-up. Remote (and local) builds all happen in a sandbox, so you don't have to worry about e.g. preparing a docker image with the worker / build slave environment. (You do,…

> GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. Can you expand on what you mean here? GCE has had autoscaling for years and it's quite configurable, I don't see why you would need to keep unused workers around unless your build load is extremely spiky.

Not auto-scaling VM compute instances, auto-scaling the Google Cloud RBE service (I think it's called Cloud Build right now?). If you have a worker pool of size 1,000, you have to pay for that even if your dev team only uses it at full capacity 8-10hrs per day. Or maybe the Google Cloud sales people working with my company are misinformed? IDK, but my experience with TPUs, GPUs, AI Engine, Dataproc, BigQuery, etc... every GCloud feature I've used (besides vanilla Compute VMs) have gotchas and/or time sinks that are costly unless you have a load of free credits. For AI Engine, it took like 6 months to upgrade their internal Kubernetes cluster; in the meantime, crashed jobs would burn an extra 15 minutes of GPU dollars.

There's a lot of good stuff in Google Cloud, but be sure to get Google to pay you to wade through the construction with free credits.

Re: Bazel 1.0

#98

Post by kristiandupont »

RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

I am going to be trying out Bazel next week and I intend to run it in a container. I assume this is possible?

Re: Bazel 1.0

#99

Post by jacquesm »

arianvanp wrote: post #79
RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

We didn't have Bazel on ARM for NixOS because we can't bootstrap openJDK on ARM. There's no source code for a JVM that compiles on ARM afaik. Though in theory we could package binary blobs from oracle and bootstrap JDK and Bazel from there, it means now our trust path for your critical build tool has a random Oracle blob in its trust path there that is extremely hard to get rid off. Build systems should be easy to bu…

Isn't there a way to get the Android toolchain to give you what you need? After all Dalvik is just Java warmed over and has a working VM, and there are plenty of ARM based Android phones.

That might get you bootstrapped to the point where you can use that to re-compile the community edition of Java.

Re: Bazel 1.0

#100

Post by laurentlb »

willtim wrote: post #74
throwaway1777 wrote: post #54

Earlier quoted context omitted.

Bazel is hardly new, it’s been in use at Google for many many years as Blaze.

This is misleading. Bazel is full of brand new code and only parts of it have been in use at Google as part of Blaze.

No. I work on Blaze/Bazel. The vast majority of the code is shared between Blaze and Bazel. Examples of Bazel-specific code include support for Windows and support for external deps (needed for multirepos and interaction with package managers).

The core is exactly the same.

Post reply on HN