Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Rust Builds with Bazel

mmapped.blog

Post reply on HN
Print view View original

101–110 of 133 posts

Re: Scaling Rust Builds with Bazel

#101

Post by simplotek »

jmmv wrote: post #47
di4na wrote: post #37

Earlier quoted context omitted.

Because the outside world is diverse and open source does not have the ressources to write bazel rules. Nor does it fit their history. There is definitely a need for better (and Meson is the foss answer, as well as Shake). But Bazel need a context that OpenSource do not have. Time and a clean slate of dependencies cooperating. Also noone fund tools for DX for OpenSource devs :) only for JS and even that...

> Because the outside world is diverse and open source does not have the ressources to write bazel rules. But the open source world has the resources to write new build systems (and ecosystems!) time and time again. This argument does not hold up.

> But the open source world has the resources to write new build systems (and ecosystems!) time and time again.

What leads you to believe that writing new build systems is a popular hobby?

I mean, cmake is the de facto standard for C++ projects and it has been for what? Over a decade? What does this tell you?

Re: Scaling Rust Builds with Bazel

#102

Post by simplotek »

bjackman wrote: post #36
carlmr wrote: post #23

Earlier quoted context omitted.

Question for an insider. Bazel code coverage seems to be one of the roughest edges. At least when running C++ tests. Only lcov is supported, and due to the sandboxing it's impossible to get other tools working since they always spit out extra instrumentation data that's neither cached nor available on the next sandbox. Does blaze support code coverage properly? Does Google just not care about code coverage? Am I just…

Hmm sorry I'm quite ignorant about that. Code coverage is pretty well supported in the internal ecosystem but I would not be surprised to hear this is because we designed Bazel around the system Google was already using for coverage.

> Code coverage is pretty well supported in the internal ecosystem (...)

What you're actually saying is that internally Google managed to get a single and very specific use case to work and forced all projects to either adopt it or have no code coverage.

Re: Scaling Rust Builds with Bazel

#103

Post by simplotek »

lern_too_spel wrote: post #61
habitue wrote: post #12

Hey, what a surprise, ex-google engineers dragging everyone into bazel... Maybe you have some really complicated use case that makes cargo insufficient, but really, it's pretty damn nice and it's integrated with everything. There are languages that need bazel, rust isn't one of them

In addition to the problems stated in the article that you're ignoring here, in any large enough system, you are going to use multiple languages. Cargo isn't built for that. Bazel is.

> In addition to the problems stated in the article that you're ignoring here

What problems, exactly? The author acknowledged cargo works but hand-waves over vague claims of "it does not track dependencies well or support arbitrary build graphs", followed by acknowledging those issues are either non-issues or solved problems whose solution was dismissed without any good reason (caching tools).

The author's claims boil down to "I wanted to shoehorn Bazel but had no good reason to, so I double-down on Bazel's only selling point while ignoring everything".

It's ok if anyone just wants to try stuff without having any rational or coherent argument to justify it, but let's not call that "problems".

Re: Scaling Rust Builds with Bazel

#104

Post by klodolph »

dboreham wrote: post #88
klodolph wrote: post #84

Earlier quoted context omitted.

The experience of using Bazel inside Google with support is different from the experience of using Bazel outside Google. Some ex-Googler may want to use Bazel and may have very good reasons for it, but they’re often unprepared to take over the support duties that, at Google, are provided by the developer tool teams.

"We did X at Google" for me has become an antipattern and a red flag.

In general, yes. Bazel has a lot of potential. It can improve developer productivity, with better correctness guarantees and improved build times. Or it can be a big sink of developer time, with people spending endless hours managing Bazel. It has potential to go both ways.

Re: Scaling Rust Builds with Bazel

#105

Post by kubota »

simplotek wrote: post #91
kubota wrote: post #64

Earlier quoted context omitted.

Bazel isn't about a particular language needing its build facilities - its about ALL of your organizations languages being built with one tool, ideally in one monorepository. Its very attractive when you have lots of microservices that would otherwise live in disparate repositories. Using a monorepo and Bazel allows you to share api definitions (proto files) across your entire code base (rules_proto, rules_grpc) , so…

> its about ALL of your organizations languages being built with one tool, Yeah, that's an effective way to sell someone's promotion, but it's always a promise that is never met and instead it's a constant source of problems. Sometimes reinventing the wheel just buys you a wobbly, squeaky wheel.

When I was at a large money center bank a constant source of problems was trying to deploy the 70+ microservices backing our business unit's main application during "release weekend", or for ad-hoc reasons like the log4shell vulnerability patch. It was very frustrating for our build pipelines to compile / test / static scan the entire millions of lines codebase when only a few libraries have changed. This build often took a few hours for our largest application and at least 10 minutes for each microservice. A few times we ran into breaking api changes and only realized it after it was deployed to production. This is the use case Bazel is serving - god like observability across your organizations code base, quickly compiled.

Re: Scaling Rust Builds with Bazel

#106

Post by 0x457 »

baby wrote: post #69
Smaug123 wrote: post #11

Earlier quoted context omitted.

Wait til you see the eight ways Nix has of packaging Rust applications! https://nixos.wiki/wiki/Rust#Packaging_Rust_projects_with_ni...

And they’re all horrible

Why? As long as it can take `Cargo.lock` as input instead of making you calculate hash yourself - it's all good.

I'm using `crane` and `buildRustPackage` to build wasm front-end that is embedded into server binary (also rust). It works like a charm.

Re: Scaling Rust Builds with Bazel

#107

Post by jvolkman »

simplotek wrote: post #100
jvolkman wrote: post #99

Earlier quoted context omitted.

Yes, Bazel supports both of those things.

You have people in this discussion claiming otherwise, and I also know for a fact that in the past it did not supported either. Can you actually provide a working example?

Not sure if this is what you're looking for exactly, but: https://github.com/jvolkman/bazel-ex-20230324

Re: Scaling Rust Builds with Bazel

#108

Post by habitue »

HervalFreire wrote: post #78
fsociety wrote: post #71

Earlier quoted context omitted.

I think Bazel is actually a good build system, but the issue is that they lack good educational resources. Then others adopt it at smaller companies without a team to manage Bazel and without investing time to onboard others into understanding how it works. There are some new talks coming out external from the Bazel maintainers which do a better job of this, but you still have to hunt for them. This is made worse bec…

Not just that it doesn't integrate well with external tools. Cmake works better here. We had a googler at our company switch everything to bazel and it just slowed everyone down. There really is something about Google engineers, this sort of subtle arrogance that the Google way is superior to everything else is existence. Maybe I'm the ass hole but I wonder if anyone can relate?

I don't think it's arrogance, it's just that blaze inside google is really nice, and they assume that using the tool outside google will also be really nice.

The issue is that to make it really nice:

- you have to convert everything to it, not just almost everything

- you need to set up build servers and lightning fast caching servers

- you need a team dedicated to building bazel rules for whatever doesn't already have rules (see: you must convert everything). This team is a bottleneck

Regular google engineers aren't exposed to these issues, because inside google it just works. It's only once they try to deploy it themselves that they realize it's a sisyphean task.

The thing is, bazel actually is really good at solving the issues it was designed for, it's just most companies do not have those particular problems acutely enough that it's worth paying the cost to switch to bazel.

Re: Scaling Rust Builds with Bazel

#109

Post by di4na »

fsociety wrote: post #73
di4na wrote: post #37

Earlier quoted context omitted.

Because the outside world is diverse and open source does not have the ressources to write bazel rules. Nor does it fit their history. There is definitely a need for better (and Meson is the foss answer, as well as Shake). But Bazel need a context that OpenSource do not have. Time and a clean slate of dependencies cooperating. Also noone fund tools for DX for OpenSource devs :) only for JS and even that...

The standard suite of open sourced Bazel rules are powerful enough for 99% of use cases. I have seen people argue this isn’t true, or that a rules package is fundamentally broken, but then learn that they are holding it wrong. And that holding it wrong reveals some glaring issues with how they setup their build system. It’s not solely the user’s responsibility to hold it right. But Bazel is and has been developed in…

If people hold it wrong so much, the tool is not adapted to the users. By definition

Re: Scaling Rust Builds with Bazel

#110

Post by chatmasta »

kubota wrote: post #64
habitue wrote: post #12

Hey, what a surprise, ex-google engineers dragging everyone into bazel... Maybe you have some really complicated use case that makes cargo insufficient, but really, it's pretty damn nice and it's integrated with everything. There are languages that need bazel, rust isn't one of them

Bazel isn't about a particular language needing its build facilities - its about ALL of your organizations languages being built with one tool, ideally in one monorepository. Its very attractive when you have lots of microservices that would otherwise live in disparate repositories. Using a monorepo and Bazel allows you to share api definitions (proto files) across your entire code base (rules_proto, rules_grpc) , so…

My problem with Bazel - at least based on my understanding when I evaluated it - is that you can't use the language-specific package managers for each module in your monorepo, so it's all or nothing. I'd prefer a tool that makes it much easier to defer build tasks to my package manager of choice in each project (eg Yarn for TS, Poetry for Python, etc.) and that lets me define rules at one level of abstraction above this.

In our monorepo, we ended up just using a Makefile, since it basically does that. Of course the Makefile is now a few hundred lines and we have about 60 *.compose.yml files, but that's another story.

Our current solution still uses the Makefile, but we basically split the monorepo into a Python part and a TypeScript part, and each of those has its own monorepo build system (Yarn workspaces for TS, and Poetry with Pants for Python).

Post reply on HN