Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel 2.0

blog.bazel.build

Post reply on HN
Print view View original

91–100 of 117 posts

Re: Bazel 2.0

#91

Post by [deleted] »

habitue wrote: post #64
zellyn wrote: post #56

Earlier quoted context omitted.

If I understand it correctly (unlikely), Nix has the degree of purely-functional rigor necessary to do this correctly, right? Sounds like it would eventually be awesome for Bazel usecases.

Nix isn't great as a build system, because it throws a lot out and rebuilds everything when something changes. It's intended to get a correct, isolated package installed, not to maximize sharing. Bazel goes through great pains to only rebuild the minimum necessary for correctness. It's able to do that because bazel build files get a lot more information about the source level dependencies than a Nix file does

[deleted]

Re: Bazel 2.0

#92

Post by m0zg »

joshuamorton wrote: post #83
m0zg wrote: post #81

Earlier quoted context omitted.

Well, don't have version skew then? Require that anything merged to master doesn't break any tests? Require that tests exist in the first place? Google makes it work at a dramatically larger scale. Everything at tip-of-tree is always ready to go. EDIT: Looks like I've misread the parent's argument as one against monorepo. It was in fact an argument in favor, and one I agree with.

Yeah but Google does that by being a monorepo.

Looks like I've misread the parent's argument as one against monorepo. My error.

Re: Bazel 2.0

#93

Post by dehrmann »

erik_seaberg wrote: post #88
ecnahc515 wrote: post #74

Earlier quoted context omitted.

Deployment ordering at large scale is avoided and usually done by not making breaking changes. 4 phase migrations, always. Roll out new API, update existing software to use new API, wait for everything to stop using old API + backfill, remove old API.

I agree that gradual adoption of new APIs is the way to go, but once you're doing that you no longer need an atomic commit across all projects.

You actually never want an atomic commit for that class of changes across projects because HEAD should always be deployable to all services. It's obviously messier at FAANG-scale, but with even 25 devs, not properly staging API-breaking changes leads to a lot of "only deploy commits before xxxx to service foo."

Re: Bazel 2.0

#94

Post by karlding »

Boulth wrote: post #87
dub wrote: post #77

Earlier quoted context omitted.

Bazel is part of the story of how Google manages provenance for build artifacts ( https://cloud.google.com/security/binary-authorization-for-b... )

This is not entirely correct. It's not Bazel but "build system very similar to Bazel" (from your source) and that's I guess their internal Blaze tool. I wonder what's the real usage of Bazel (not Blaze) in Google.

According to this comment [0] by laurentlb (one of the people working on Bazel who also commented in this post) from a year ago, Blaze is just Bazel but with integrations to Google-internal tools.

[0] https://news.ycombinator.com/item?id=18823546

Re: Bazel 2.0

#95

Post by vlovich123 »

habitue wrote: post #85
vlovich123 wrote: post #72

Earlier quoted context omitted.

So I always wonder how Google does this. Somehow they're able to determine which individual unit test are impacted by a change. Maybe this is only for Java but I'm pretty sure I recall claims that they can change 1 line & know that only 1 other test case is impacted (i.e. not even the other test cases within the same unit). Now Google's monorepo does imply also that if you are changing a line that is foundationally p…

It depends on the granularity you write your BUILD files with. You can certainly write a target for each individual test, and track dependencies independently. In practice, you might glob together all files in a directory and put up with a few extra test runs in exchange for less bookkeeping

What I was saying is that Google somehow manages to track test dependencies at the source level so globbing wouldn't matter.

Re: Bazel 2.0

#96

Post by klodolph »

malkia wrote: post #70

One issue we hit with our CI, and mix of build systems is this - given a changelist, find out which targets needs to be built, and which one needs to be tested on pre-submit, and which on post-submit. With that, we end up paying so much extra time building everything over and over without need, and then not building things that we ought to. So that's one reason to switch, but at the same time lots of people simply do…

> Then it hit me... And I'm sure the same is for buck, pants, please.build, gn and other similar systems.

There’s an exercise you can do where you design a build system on the basis that it shouldn't do unnecessary work (which can be very slow and frustrating in practice).

My personal experience is that you can really quickly get to the point where just reading the entire graph into memory gets expensive. People talk about how Google is huge… but long before you get to that scale, you can end up with a build graph that just takes forever to parse and evaluate. (At Google's scale, it doesn't even fit in memory any more.)

So you decide that, as a hard design requirement, you should be able to only load the portion of the repository that you are building. And then you want to make this cacheable, so you can change the repository and know what’s changed in some quick / reasonable way.

If you go down this path, you end up rediscovering some of the big design decisions behind Bazel, Buck, Pants, Please, and GN.

Re: Bazel 2.0

#97

Post by joshuamorton »

vlovich123 wrote: post #95
habitue wrote: post #85

Earlier quoted context omitted.

It depends on the granularity you write your BUILD files with. You can certainly write a target for each individual test, and track dependencies independently. In practice, you might glob together all files in a directory and put up with a few extra test runs in exchange for less bookkeeping

What I was saying is that Google somehow manages to track test dependencies at the source level so globbing wouldn't matter.

Frankly you heard wrong. There isn't anything like this.

Re: Bazel 2.0

#98

Post by joshuamorton »

weberc2 wrote: post #80
shrewduser wrote: post #33

Earlier quoted context omitted.

I'll never understand the fascination with mono repo's.

There are tradeoffs both ways. With multirepos you likely have a dependency hell problem and you often have to submit and release several PRs for otherwise small updates. With monorepos, (if you want reasonable build times) you have to be able to determine what has changed and what needs to build (including tests, etc) as a result. This is technically true of multirepos as well, but the problem is pushed into git and…

rules_python has supported py3 for a while.

The next obvious question is, what would you do to make it simpler? Tons of people have tried (you listed 5), and they all rebuilt the same thing. What features do you drop?

Re: Bazel 2.0

#99

Post by steeve »

jacquesm wrote: post #59
steeve wrote: post #17

Earlier quoted context omitted.

To each their own I guess. I’ve been using it for the last 2 years, and I am not doing any project again without it. Bazel is not that complex if you start a project with it. Migrating to it and learning it at the same time will be hard though since you’re likely to uncover a lot of skeletons.

This comment is not internally consistent.

How so?

Re: Bazel 2.0

#100

Post by jacquesm »

steeve wrote: post #99
jacquesm wrote: post #59

Earlier quoted context omitted.

This comment is not internally consistent.

How so?

> Bazel is not that complex if you start a project with it.

Bazel is quite complex and when you start a project you do not yet need it, rarely will an organization start with something like Bazel, they use it because - you hope - they need it.

vs:

> Migrating to it and learning it at the same time will be hard though since you’re likely to uncover a lot of skeletons.

So the bulk of Bazel use cases will revolve around migrating an existing build system to use Bazel instead, and that is hard, because Bazel is difficult and has a very steep learning curve, and requires a lot of work to keep it running.

Tooling should adapt to use cases, if you need to adapt your use cases to the tooling then that's a fault of the tool. If that limits use of the tool to those projects that are started with it then you have already lost the vast majority of your potential audience. So yes, if you start using Bazel right from day #1 then that might be the way to go. But I suspect - and so far have not seen any evidence - that that is the way it is actually used.

Post reply on HN