Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel 1.0

opensource.googleblog.com

Post reply on HN
Print view View original

111–120 of 204 posts

Re: Bazel 1.0

#111

Post by londons_explore »

amelius wrote: post #105
martius wrote: post #95

Earlier quoted context omitted.

My guess is that startup time is not the issue, but loading a large data structure (its cache) from disk can be. Especially if you can't or don't want to use memory mapping because it's hard to do well. But more importantly, I think that the server can monitor file changes ahead of time with things like inotify, saving the time of stat()-ing files when the user wants to perform a build.

> Especially if you can't or don't want to use memory mapping because it's hard to do well. Why do new languages never address this issue?

In-kernal block caching of files is pretty crude - since the kernel has no knowledge of your data structures or which bits of your files you're going to be reading next, you end up having hundreds of page faults requiring single-sector disk reads for most workloads.

Re: Bazel 1.0

#112

Post by merb »

choppaface wrote: post #97
9nGQluzmnq3M wrote: post #94

Earlier quoted context omitted.

> GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. Can you expand on what you mean here? GCE has had autoscaling for years and it's quite configurable, I don't see why you would need to keep unused workers around unless your build load is extremely spiky.

Not auto-scaling VM compute instances, auto-scaling the Google Cloud RBE service (I think it's called Cloud Build right now?). If you have a worker pool of size 1,000, you have to pay for that even if your dev team only uses it at full capacity 8-10hrs per day. Or maybe the Google Cloud sales people working with my company are misinformed? IDK, but my experience with TPUs, GPUs, AI Engine, Dataproc, BigQuery, etc...…

in Google Cloud Build you pay per minute, not per instance.

Re: Bazel 1.0

#113

Post by zaphirplane »

arianvanp wrote: post #79
RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

We didn't have Bazel on ARM for NixOS because we can't bootstrap openJDK on ARM. There's no source code for a JVM that compiles on ARM afaik. Though in theory we could package binary blobs from oracle and bootstrap JDK and Bazel from there, it means now our trust path for your critical build tool has a random Oracle blob in its trust path there that is extremely hard to get rid off. Build systems should be easy to bu…

You want to build java from source on arm ? Can’t you borrow what AdoptopenJdk Does

Re: Bazel 1.0

#114

Post by shusson »

choppaface wrote: post #97
9nGQluzmnq3M wrote: post #94

Earlier quoted context omitted.

> GCE is slow to work on supporting auto-scale, so you have to pay for unused workers. Can you expand on what you mean here? GCE has had autoscaling for years and it's quite configurable, I don't see why you would need to keep unused workers around unless your build load is extremely spiky.

Not auto-scaling VM compute instances, auto-scaling the Google Cloud RBE service (I think it's called Cloud Build right now?). If you have a worker pool of size 1,000, you have to pay for that even if your dev team only uses it at full capacity 8-10hrs per day. Or maybe the Google Cloud sales people working with my company are misinformed? IDK, but my experience with TPUs, GPUs, AI Engine, Dataproc, BigQuery, etc...…

> Cloud Build

If you're talking about [1], it is billed in minutes. You can have 10 concurrent workers. Custom Workers is in alpha.

[1] https://cloud.google.com/cloud-build/

Re: Bazel 1.0

#115

Post by rjsw »

arianvanp wrote: post #79
RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

We didn't have Bazel on ARM for NixOS because we can't bootstrap openJDK on ARM. There's no source code for a JVM that compiles on ARM afaik. Though in theory we could package binary blobs from oracle and bootstrap JDK and Bazel from there, it means now our trust path for your critical build tool has a random Oracle blob in its trust path there that is extremely hard to get rid off. Build systems should be easy to bu…

You could try cross-compiling OpenJDK from x86_64 to ARM. This is a lot easier in newer OpenJDK releases than it used to be.

Currently doing this for BSD using OpenJDK 11.

Re: Bazel 1.0

#116

Post by shusson »

jingwen wrote: post #69
slavik81 wrote: post #65

The lack of backwards compatibility between Bazel versions has been the deal-killer for me thus far. Old build instructions should work with newer releases of the same tools. It's important for the tools to remain compatible because you can't change the instructions contained in old releases. I hope the release of 1.0 is an indication that Bazel is taking interface stability more seriously.

Yes. 1.0 indicates the beginning of semantic versioning, which was not in place during the alpha and beta phases.

Exactly this. It's not fair to complain about breaking changes in a product pre 1.0.

Also, unless they had wildly different versioning pre 1.0, they could have just said "we are releasing 1.0". semver already states that 0.y.z should not be considered a stable API [1].

[1] https://github.com/semver/semver/blob/master/semver.md

Re: Bazel 1.0

#117

Post by henryaj »

RcouF1uZ4gsC wrote: post #8

I only wish Bazel had been written in a language that produced native binaries like C++ or Go instead of Java. It seems like a waste for me as a non-Java developer to install Java on my dev machine just for Bazel.

You should look at Please: https://please.build

Bazel-like in many ways; written in Go.

Re: Bazel 1.0

#118

Post by kajecounterhack »

mc3 wrote: post #61
jdkee wrote: post #45

What does Bazel do exactly?

Bazel is an open-source build and test tool similar to Make, Maven, and Gradle. It uses a human-readable, high-level build language. Bazel supports projects in multiple languages and builds outputs for multiple platforms. Bazel supports large codebases across multiple repositories, and large numbers of users.

It also builds code repeatably & hermetically (interpreted languages excepted; at google python is built super hermetically where a python binary + environment is bundled with python script(s) into an executable; some companies like dropbox have copied this). You run build, and you get binaries for N kinds of systems (e.g x86_64, ARM). It builds a dependency graph of your code that you can query and results in cache-able builds (only rebuild the part you need). Since building dependencies in a tree can be massively parallelized, it also allows you to parallelize your builds well across machines (with some effort).

You use the one tool to compile C++, Java, Python, Go, Rust, etc.

Re: Bazel 1.0

#119

Post by willtim »

laurentlb wrote: post #100
willtim wrote: post #74

Earlier quoted context omitted.

This is misleading. Bazel is full of brand new code and only parts of it have been in use at Google as part of Blaze.

No. I work on Blaze/Bazel. The vast majority of the code is shared between Blaze and Bazel. Examples of Bazel-specific code include support for Windows and support for external deps (needed for multirepos and interaction with package managers). The core is exactly the same.

My employer encountered lots of basic bugs, like file handle exhaustion, which together with the amount of recent activity in GitHub, suggested to me that there is a lot of new code in Bazel. If it's essentially the same as Blaze as you say, then why the new name?

Re: Bazel 1.0

#120

Post by jhasse »

cromwellian wrote: post #75
RcouF1uZ4gsC wrote: post #46

Earlier quoted context omitted.

> (And yes it doesn't have to use a persistent process to solve these problems. That is the solution it chooses.) How much did the fact that Java is very slow starting up influence the decision to use a persistent process instead of some other solution?

Java starts up in ~1s, something like Graal could make this faster. But I find this Java criticism more a symptom of Java-derangement syndrome, because python and node build systems have even worse startup times and no one says anything.

> python and node build systems have even worse startup times and no one says anything.

Noop build with Waf (build system written in Python) takes 0.13s on my system. Waf reports that the build actually takes 0.04s, so I guess 0.09s is Python's start-up time (and some other overhead).

Post reply on HN