Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Rust Builds with Bazel

mmapped.blog

Post reply on HN
Print view View original

71–80 of 133 posts

Re: Scaling Rust Builds with Bazel

#71

Post by fsociety »

bjackman wrote: post #22
habitue wrote: post #12

Hey, what a surprise, ex-google engineers dragging everyone into bazel... Maybe you have some really complicated use case that makes cargo insufficient, but really, it's pretty damn nice and it's integrated with everything. There are languages that need bazel, rust isn't one of them

Lol, Googler here, I can't deny that I have he urge to force Bazel into the world. It's just, when I joined Google I found a build system I didn't hate. This was the first time. I don't actually love Bazel (Blaze internally), but I love that I don't hate it. Every time I have to learn a new build system in open source work I groan. Why are there so many systems for essentially the same thing?

I think Bazel is actually a good build system, but the issue is that they lack good educational resources. Then others adopt it at smaller companies without a team to manage Bazel and without investing time to onboard others into understanding how it works.

There are some new talks coming out external from the Bazel maintainers which do a better job of this, but you still have to hunt for them.

This is made worse because each of the rules packages, which are super useful, have their own idiosyncrasies which only make sense with a solid foundation in what problems Bazel solves, how it works and how to write/read rules.

My other wishlist for Bazel was an easier time setting up a hermetic sandbox without having caveats. This is improving, so I am optimistic on this front.

For context, I am not a Google (or ex-Google) engineer and I’ve had to work with a ton of different build systems on different levels of the stack.

Re: Scaling Rust Builds with Bazel

#72

Post by cryptonector »

jmmv wrote: post #47
di4na wrote: post #37

Earlier quoted context omitted.

Because the outside world is diverse and open source does not have the ressources to write bazel rules. Nor does it fit their history. There is definitely a need for better (and Meson is the foss answer, as well as Shake). But Bazel need a context that OpenSource do not have. Time and a clean slate of dependencies cooperating. Also noone fund tools for DX for OpenSource devs :) only for JS and even that...

> Because the outside world is diverse and open source does not have the ressources to write bazel rules. But the open source world has the resources to write new build systems (and ecosystems!) time and time again. This argument does not hold up.

Not really. Some open source projects go back a long long time and have a large enough build system that rewriting it would require a lot of effort. Other open source projects are small enough that the simplest build system the author(s) could build was good enough.

Re: Scaling Rust Builds with Bazel

#73

Post by fsociety »

di4na wrote: post #37
bjackman wrote: post #22

Earlier quoted context omitted.

Lol, Googler here, I can't deny that I have he urge to force Bazel into the world. It's just, when I joined Google I found a build system I didn't hate. This was the first time. I don't actually love Bazel (Blaze internally), but I love that I don't hate it. Every time I have to learn a new build system in open source work I groan. Why are there so many systems for essentially the same thing?

Because the outside world is diverse and open source does not have the ressources to write bazel rules. Nor does it fit their history. There is definitely a need for better (and Meson is the foss answer, as well as Shake). But Bazel need a context that OpenSource do not have. Time and a clean slate of dependencies cooperating. Also noone fund tools for DX for OpenSource devs :) only for JS and even that...

The standard suite of open sourced Bazel rules are powerful enough for 99% of use cases. I have seen people argue this isn’t true, or that a rules package is fundamentally broken, but then learn that they are holding it wrong. And that holding it wrong reveals some glaring issues with how they setup their build system.

It’s not solely the user’s responsibility to hold it right. But Bazel is and has been developed in the “open source context” for long enough now that your argument doesn’t hold up imo.

Re: Scaling Rust Builds with Bazel

#74

Post by insanitybit »

twunde wrote: post #65
insanitybit wrote: post #60

Earlier quoted context omitted.

> Maybe you have some really complicated use case that makes cargo insufficient, but really, it's pretty damn nice and it's integrated with everything. I feel like our use case at Grapl wasn't particularly crazy. It was just slightly more than "I'm a dev building a library and publishing it" - we had a workspace with multiple devs and CI/CD. CI/CD was really slow - issues with caching, rerunning unnecessary tests, et…

Hey Colin :wave: (I hope you're bringing Rust to DD) To add onto this, Grapl had a moderate sized codebase with less than 10 engineers. I do think we could have done some optimizations in cargo, but it's simply much easier to use a tool like Bazel/pants etc that does the work for you as long as it supports your language of choice.

:D Hey

Re: Scaling Rust Builds with Bazel

#76

Post by carlmr »

Game_Ender wrote: post #54
carlmr wrote: post #23

Earlier quoted context omitted.

Question for an insider. Bazel code coverage seems to be one of the roughest edges. At least when running C++ tests. Only lcov is supported, and due to the sandboxing it's impossible to get other tools working since they always spit out extra instrumentation data that's neither cached nor available on the next sandbox. Does blaze support code coverage properly? Does Google just not care about code coverage? Am I just…

You can get coverage working for almost anything by having each test convert its coverage data to LCOV before stopping, in the same sandbox as the test. I forget the exact details but I think you can do this with a custom run_under test wrapper used during coverage. The wrapper runs your test, with whatever custom coverage system you have, then afterwards it converts to LCOV. Sometimes this might require updates to l…

>I forget the exact details but I think you can do this with a custom run_under test wrapper used during coverage. The wrapper runs your test, with whatever custom coverage system you have, then afterwards it converts to LCOV.

The testing is not all you need to modify. You need to modify every compile and link step, since you get byproducts of the instrumentation on every step.

>Sometimes this might require updates to language specific rules because the test runner itself needs to be configured.

That's what I found, too, however I feel like rewriting the C++ rules is a huge undertaking, since they're not defined in Starlark.

Re: Scaling Rust Builds with Bazel

#77

Post by Game_Ender »

renewiltord wrote: post #63

I think this would have been useful with some code. My experience with Bazel is that you need a BUILD file in each subdirectory or something like that and every time I add a file I have to add it to that BUILD file. Exceedingly tedious to use and has poor support in Clion. It is likely that I don't have the constraints that this company has. The last time I tried this in Java I had to replicate my package dependency…

It is super tedious but for some languages a tool called Gazelle can auto-generate these for you. I know Go works well and there is a Java plugin too [0]. There is definitely some level rust support because that is how external crates are handled.

0 - https://github.com/bazel-contrib/rules_jvm/tree/main/java/ga...

Re: Scaling Rust Builds with Bazel

#78

Post by HervalFreire »

fsociety wrote: post #71
bjackman wrote: post #22

Earlier quoted context omitted.

Lol, Googler here, I can't deny that I have he urge to force Bazel into the world. It's just, when I joined Google I found a build system I didn't hate. This was the first time. I don't actually love Bazel (Blaze internally), but I love that I don't hate it. Every time I have to learn a new build system in open source work I groan. Why are there so many systems for essentially the same thing?

I think Bazel is actually a good build system, but the issue is that they lack good educational resources. Then others adopt it at smaller companies without a team to manage Bazel and without investing time to onboard others into understanding how it works. There are some new talks coming out external from the Bazel maintainers which do a better job of this, but you still have to hunt for them. This is made worse bec…

Not just that it doesn't integrate well with external tools. Cmake works better here.

We had a googler at our company switch everything to bazel and it just slowed everyone down.

There really is something about Google engineers, this sort of subtle arrogance that the Google way is superior to everything else is existence. Maybe I'm the ass hole but I wonder if anyone can relate?

Re: Scaling Rust Builds with Bazel

#79

Post by 19h »

I used bazel for a few internal repositories written in Rust and it's amazing .... until you hit an issue.

Edge cases in Bazel are about as documented as any random fish you might find in a crack at the bottom of the ocean. And the most hilarious thing to deal with is "yeah this was written for a Google project, that other use case didn't apply to our project".

When eventually after the Xth issue I was told "why don't you just fork this standard package" (it was for typescript support inside the Rust repo) I was like "f*** bazel" and never touched it again.

It's amazing when it works. If it doesn't you're f******.

Re: Scaling Rust Builds with Bazel

#80

Post by malkia »

bjackman wrote: post #22
habitue wrote: post #12

Hey, what a surprise, ex-google engineers dragging everyone into bazel... Maybe you have some really complicated use case that makes cargo insufficient, but really, it's pretty damn nice and it's integrated with everything. There are languages that need bazel, rust isn't one of them

Lol, Googler here, I can't deny that I have he urge to force Bazel into the world. It's just, when I joined Google I found a build system I didn't hate. This was the first time. I don't actually love Bazel (Blaze internally), but I love that I don't hate it. Every time I have to learn a new build system in open source work I groan. Why are there so many systems for essentially the same thing?

This was my fist experience with blaze too. First it was, wtf is this shit, and theze blaze-bin/etc folders. Then after I left Google I realized how much I miss it. No Make is not a replacement, or ninja.
Post reply on HN