Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel 2.0

blog.bazel.build

Post reply on HN
Print view View original

81–90 of 117 posts

Re: Bazel 2.0

#81

Post by m0zg »

kylecordes wrote: post #61
shrewduser wrote: post #33

Earlier quoted context omitted.

I'll never understand the fascination with mono repo's.

At a certain size, monorepo becomes the worst way to do it except for all the others. Essentially: version skew across numerous artifacts in a large organization starts to look like the version skew across an industry or ecosystem. The aggregate cost of dealing with it project by project is probably higher, at least that is what most of the biggest tech companies have concluded, than dealing with it at the source lev…

Well, don't have version skew then? Require that anything merged to master doesn't break any tests? Require that tests exist in the first place? Google makes it work at a dramatically larger scale. Everything at tip-of-tree is always ready to go.

EDIT: Looks like I've misread the parent's argument as one against monorepo. It was in fact an argument in favor, and one I agree with.

Re: Bazel 2.0

#82

Post by weberc2 »

dehrmann wrote: post #45
serverholic wrote: post #40

Earlier quoted context omitted.

Well for one you can commit to multiple projects in a single PR. Makes coordinating changes across projects much easier.

It gives you that illusion; it doesn't solve versioning and deployment orders, and I'd argue that that's the harder part of changes across projects. Polyrepos make messy things...messy.

It pretty much does solve the versioning issue. “Latest, always”. The downside is the abysmal state of monorepo build tools. With multirepos, who updates the downstream repos’ dependency files (e.g., requirements.txt) when an upstream project releases a change? And is the policy “latest, always” or do you support N versions of every package? I would argue that the latter is insane at any scale, and the former leaves you dealing with dependencies manually (someone is updating the downstream repos’ dependency files when an upstream change is released) or you build automation that does it and you’re well on your way to implementing your own monorepo-like build tool.

Everything is hard, unfortunately.

Re: Bazel 2.0

#83

Post by joshuamorton »

m0zg wrote: post #81
kylecordes wrote: post #61

Earlier quoted context omitted.

At a certain size, monorepo becomes the worst way to do it except for all the others. Essentially: version skew across numerous artifacts in a large organization starts to look like the version skew across an industry or ecosystem. The aggregate cost of dealing with it project by project is probably higher, at least that is what most of the biggest tech companies have concluded, than dealing with it at the source lev…

Well, don't have version skew then? Require that anything merged to master doesn't break any tests? Require that tests exist in the first place? Google makes it work at a dramatically larger scale. Everything at tip-of-tree is always ready to go. EDIT: Looks like I've misread the parent's argument as one against monorepo. It was in fact an argument in favor, and one I agree with.

Yeah but Google does that by being a monorepo.

Re: Bazel 2.0

#84

Post by IshKebab »

habitue wrote: post #62

Good heuristic for whether it's worth considering moving to bazel for your build system: - Do you have 200+ developers working on a monorepo? - Are you willing to vendor all of your dependencies and maintain their builds yourself? If so, consider it. The productivity you're losing to unnecessary rebuilding and re-running unchanged unit tests will probably be paid back if you can contort your development process to th…

I don't think you need 200 developers to make it worth considering.

Re: Bazel 2.0

#85

Post by habitue »

vlovich123 wrote: post #72
habitue wrote: post #62

Good heuristic for whether it's worth considering moving to bazel for your build system: - Do you have 200+ developers working on a monorepo? - Are you willing to vendor all of your dependencies and maintain their builds yourself? If so, consider it. The productivity you're losing to unnecessary rebuilding and re-running unchanged unit tests will probably be paid back if you can contort your development process to th…

So I always wonder how Google does this. Somehow they're able to determine which individual unit test are impacted by a change. Maybe this is only for Java but I'm pretty sure I recall claims that they can change 1 line & know that only 1 other test case is impacted (i.e. not even the other test cases within the same unit). Now Google's monorepo does imply also that if you are changing a line that is foundationally p…

It depends on the granularity you write your BUILD files with. You can certainly write a target for each individual test, and track dependencies independently. In practice, you might glob together all files in a directory and put up with a few extra test runs in exchange for less bookkeeping

Re: Bazel 2.0

#86

Post by wereHamster »

habitue wrote: post #62

Good heuristic for whether it's worth considering moving to bazel for your build system: - Do you have 200+ developers working on a monorepo? - Are you willing to vendor all of your dependencies and maintain their builds yourself? If so, consider it. The productivity you're losing to unnecessary rebuilding and re-running unchanged unit tests will probably be paid back if you can contort your development process to th…

Another factor: are your languages supported by bazel? If you use the same languages that google uses (C++, Python, go), it's fair to say that those are well supported. For all other languages, even if they are widely used outside of Google (JavaScript, nodejs), you may be out of luck.

Re: Bazel 2.0

#87

Post by Boulth »

dub wrote: post #77
breatheoften wrote: post #67

Does bazel use the word “provenance“ at all? Provenance is a word I first saw advertised in a platform called dotscience.io — that I find fundamentally interesting. And it seems quite relevant to hermetic builds. Provenance is about giving any state derived from an arbitrary computation an identity that is derived from the content hash of the inputs needed to re-compute that state ... in dotscience they achieve this…

Bazel is part of the story of how Google manages provenance for build artifacts ( https://cloud.google.com/security/binary-authorization-for-b... )

This is not entirely correct. It's not Bazel but "build system very similar to Bazel" (from your source) and that's I guess their internal Blaze tool.

I wonder what's the real usage of Bazel (not Blaze) in Google.

Re: Bazel 2.0

#88

Post by erik_seaberg »

ecnahc515 wrote: post #74
dehrmann wrote: post #45

Earlier quoted context omitted.

It gives you that illusion; it doesn't solve versioning and deployment orders, and I'd argue that that's the harder part of changes across projects. Polyrepos make messy things...messy.

Deployment ordering at large scale is avoided and usually done by not making breaking changes. 4 phase migrations, always. Roll out new API, update existing software to use new API, wait for everything to stop using old API + backfill, remove old API.

I agree that gradual adoption of new APIs is the way to go, but once you're doing that you no longer need an atomic commit across all projects.

Re: Bazel 2.0

#89

Post by jefftk »

wereHamster wrote: post #86
habitue wrote: post #62

Good heuristic for whether it's worth considering moving to bazel for your build system: - Do you have 200+ developers working on a monorepo? - Are you willing to vendor all of your dependencies and maintain their builds yourself? If so, consider it. The productivity you're losing to unnecessary rebuilding and re-running unchanged unit tests will probably be paid back if you can contort your development process to th…

Another factor: are your languages supported by bazel? If you use the same languages that google uses (C++, Python, go), it's fair to say that those are well supported. For all other languages, even if they are widely used outside of Google (JavaScript, nodejs), you may be out of luck.

I don't know much about bazel, but JavaScript is very widely used inside Google. It's the main language I work in.

Re: Bazel 2.0

#90

Post by zapita »

wereHamster wrote: post #86
habitue wrote: post #62

Good heuristic for whether it's worth considering moving to bazel for your build system: - Do you have 200+ developers working on a monorepo? - Are you willing to vendor all of your dependencies and maintain their builds yourself? If so, consider it. The productivity you're losing to unnecessary rebuilding and re-running unchanged unit tests will probably be paid back if you can contort your development process to th…

Another factor: are your languages supported by bazel? If you use the same languages that google uses (C++, Python, go), it's fair to say that those are well supported. For all other languages, even if they are widely used outside of Google (JavaScript, nodejs), you may be out of luck.

Go support is not great either. Bazel can build Go just fine, but you will need to throw away the standard Go tooling and use Bazel instead. There are third-party helpers like Gazelle, but you know you’re in for a bumpy ride when even basic operations require a helper.
Post reply on HN