Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Rust Builds with Bazel

mmapped.blog

Post reply on HN
Print view View original

121–130 of 133 posts

Re: Scaling Rust Builds with Bazel

#121

Post by sam_bristow »

carlmr wrote: post #23
bjackman wrote: post #22

Earlier quoted context omitted.

Lol, Googler here, I can't deny that I have he urge to force Bazel into the world. It's just, when I joined Google I found a build system I didn't hate. This was the first time. I don't actually love Bazel (Blaze internally), but I love that I don't hate it. Every time I have to learn a new build system in open source work I groan. Why are there so many systems for essentially the same thing?

Question for an insider. Bazel code coverage seems to be one of the roughest edges. At least when running C++ tests. Only lcov is supported, and due to the sandboxing it's impossible to get other tools working since they always spit out extra instrumentation data that's neither cached nor available on the next sandbox. Does blaze support code coverage properly? Does Google just not care about code coverage? Am I just…

I've also struggled with the C++ coverage support in Bazel. It looks unloved since around 2018.

The most useful leads I've found so far are:

* The design doc for the (abandoned?) coverage framework [1]

* A prototype of GCOVR support [2]

* Comment on getting llvm-profdata working [3]

[1] https://docs.google.com/document/d/1-ZWHF-Q-qCKf19ik-t33ie58...

[2] https://github.com/ulfjack/bazel/commit/e9f21bbf562c1f6006eb...

[3] https://github.com/bazelbuild/bazel/issues/8178#issuecomment...

Re: Scaling Rust Builds with Bazel

#122

Post by di4na »

lern_too_spel wrote: post #119
di4na wrote: post #109

Earlier quoted context omitted.

If people hold it wrong so much, the tool is not adapted to the users. By definition

These people hold other build tools wrong too. The difference is those other build tools often have no way of being held correctly. That Bazel is possible to use correctly at all explains why it was copied at least twice (Buck and Pants) before it was open sourced.

And yet noone use neither of them in this world. Maybe we should realise that if none of them can be hold right when people use them, it is because they are not adapted to the use.

Re: Scaling Rust Builds with Bazel

#123

Post by bjackman »

simplotek wrote: post #94
HervalFreire wrote: post #78

Earlier quoted context omitted.

Not just that it doesn't integrate well with external tools. Cmake works better here. We had a googler at our company switch everything to bazel and it just slowed everyone down. There really is something about Google engineers, this sort of subtle arrogance that the Google way is superior to everything else is existence. Maybe I'm the ass hole but I wonder if anyone can relate?

> We had a googler at our company switch everything to bazel and it just slowed everyone down. I'd love to hear if anyone can make a case for Bazel knowing that cmake+build cache tools like ccache buy far better performance improvements than any full or incremental build.

FWIW performance is not really in the picture when I say "I wish everything was Bazel". Configured properly, CMake and Make are probably unbeatable for speed.

It's just that I really really hate maintaining Makefiles and CMake configs.

Admittedly in the latter case it's because I have never grokked CMake, but also... I don't want to grok CMake, I grok a perfectly good way of building C++ already. And it can also build Go and Rust and Typescript and Proto and JSonnet and it can run my janky codegen shell scripts and if we migrate to Carbon it will build that too and I won't even have to read the documentation.

Not saying I think everyone should migrate to Bazel (I hope I will not become That Ex-Googler), just explaining where the urge comes from.

Re: Scaling Rust Builds with Bazel

#124

Post by simplotek »

pie_flavor wrote: post #114
simplotek wrote: post #103

Earlier quoted context omitted.

> In addition to the problems stated in the article that you're ignoring here What problems, exactly? The author acknowledged cargo works but hand-waves over vague claims of "it does not track dependencies well or support arbitrary build graphs", followed by acknowledging those issues are either non-issues or solved problems whose solution was dismissed without any good reason (caching tools). The author's claims boi…

The author gave a concrete example of such a build graph and a concrete example of why caching tools can't be used effectively.

> such a build graph

Nonsense. All build systems implement a DAG of build targets.

> why caching tools can't be used effectively

Nonsense. There are a myriad of caching tools freely available, from ccache, sccache, build cache, etc. To get them to work, all you need to do is pass compiler launchers to them, which means setting a single env/build flag.

The author is either trying hard to make a mountain out of irrelevant molehills, or is outright being disingenuous.

Bazel is being sold as a solution desperately grasping at straw-like problems, while being feature incomplete.

Re: Scaling Rust Builds with Bazel

#125

Post by simplotek »

bjackman wrote: post #123
simplotek wrote: post #94

Earlier quoted context omitted.

> We had a googler at our company switch everything to bazel and it just slowed everyone down. I'd love to hear if anyone can make a case for Bazel knowing that cmake+build cache tools like ccache buy far better performance improvements than any full or incremental build.

FWIW performance is not really in the picture when I say "I wish everything was Bazel". Configured properly, CMake and Make are probably unbeatable for speed. It's just that I really really hate maintaining Makefiles and CMake configs. Admittedly in the latter case it's because I have never grokked CMake, but also... I don't want to grok CMake, I grok a perfectly good way of building C++ already. And it can also buil…

> It's just that I really really hate maintaining Makefiles and CMake configs.

CMake is a high level makefile/build system generator.

There is nothing to maintain in modern cmake. You just set what executables/libraries you want to build, set their dependencies, and you're done.

And of all alternatives you chose to push, you pick Bazel of all things? Feature-incomplete and requiring all sorts of low-level maintenance?

Re: Scaling Rust Builds with Bazel

#126

Post by HervalFreire »

simplotek wrote: post #125
bjackman wrote: post #123

Earlier quoted context omitted.

FWIW performance is not really in the picture when I say "I wish everything was Bazel". Configured properly, CMake and Make are probably unbeatable for speed. It's just that I really really hate maintaining Makefiles and CMake configs. Admittedly in the latter case it's because I have never grokked CMake, but also... I don't want to grok CMake, I grok a perfectly good way of building C++ already. And it can also buil…

> It's just that I really really hate maintaining Makefiles and CMake configs. CMake is a high level makefile/build system generator. There is nothing to maintain in modern cmake. You just set what executables/libraries you want to build, set their dependencies, and you're done. And of all alternatives you chose to push, you pick Bazel of all things? Feature-incomplete and requiring all sorts of low-level maintenance…

I don't like bazel, but cmake is a piece of shit too.

The logic is convoluted, documentation is over complicated and the language contains too many primitive functions and ways to shoot yourself in the foot.

There's really nothing good that's available.

Re: Scaling Rust Builds with Bazel

#127

Post by earthling8118 »

baby wrote: post #116
0x457 wrote: post #106

Earlier quoted context omitted.

Why? As long as it can take `Cargo.lock` as input instead of making you calculate hash yourself - it's all good. I'm using `crane` and `buildRustPackage` to build wasm front-end that is embedded into server binary (also rust). It works like a charm.

So one problem we have at the moment in our project is that outdated Cargo.lock will basically break everything. Using `nix develop` will just fail to build the derivation, instead of regenerating the correct Cargo.lock (because it depends on someone not using `nix develop` to build the Rust project). It seems like the ideal way to mix nix and cargo is to simply trust and delegate to cargo once you reach a rust depen…

What do you mean by outdated Cargo.lock?

You definitely should be leaving a sort of boundary between them. You can use fenix in your devshell to get a toolchain and then use crane for building. You just use cargo like normal when developing, including dependent crates, but CI can be handled with nix from that point.

I've got my flakes organized in a way that they are basically modular, so I can easily reuse files from other ones. If I need a rust toolchain, cargo builds, python toolchail, postgres, or whatever I can just import the file and change it up slightly (mostly names)

Re: Scaling Rust Builds with Bazel

#128

Post by pie_flavor »

simplotek wrote: post #124
pie_flavor wrote: post #114

Earlier quoted context omitted.

The author gave a concrete example of such a build graph and a concrete example of why caching tools can't be used effectively.

> such a build graph Nonsense. All build systems implement a DAG of build targets. > why caching tools can't be used effectively Nonsense. There are a myriad of caching tools freely available, from ccache, sccache, build cache, etc. To get them to work, all you need to do is pass compiler launchers to them, which means setting a single env/build flag. The author is either trying hard to make a mountain out of irrelev…

> All build systems implement a DAG of build targets.

And the example they gave is one that doesn't work so well.

> ccache, sccache, build cache, etc

The example they gave included the build cache misfiring. ccache doesn't work with Rust. sccache does, but can't be used with incremental compilation or when building binaries, making it essentially useless for what they needed, i.e. speeding up CI and local dev.

I really have no idea why you're still avoiding reading the article you're dismissing.

Re: Scaling Rust Builds with Bazel

#129

Post by unicornmama »

saidinesh5 wrote: post #93
unicornmama wrote: post #43

Earlier quoted context omitted.

Bazel does not assume statically built. Where do you get this idea?

From their open issues and ugly hacks needed to build anything else... https://github.com/bazelbuild/bazel/issues/1920 https://stackoverflow.com/questions/32845940/symbols-from-st...

Those look like the problems with rules cc for building C++. But it’s not inherent to Bazel, and would not expect problems producing static binaries in other languages.

Re: Scaling Rust Builds with Bazel

#130

Post by saidinesh5 »

unicornmama wrote: post #129
saidinesh5 wrote: post #93

Earlier quoted context omitted.

From their open issues and ugly hacks needed to build anything else... https://github.com/bazelbuild/bazel/issues/1920 https://stackoverflow.com/questions/32845940/symbols-from-st...

Those look like the problems with rules cc for building C++. But it’s not inherent to Bazel, and would not expect problems producing static binaries in other languages.

That could be.

We just had the ugly requirement of having to build a rust library/crate that uses C++ libraries (tensorflow lite + libedgetpu). That forced us to use bazel, on/for platforms that i have little experience / love for (Windows, Apple platforms).

Hitting the platform annoyances of each of those platforms just added to the pain. (On Windows, the 260 character path limit + on iOS Apple messing with ar binary used to create static libraries).

Left a very bad taste of trying to use bazel as a build tool. Even handwritten 1000+ lines of makefiles/experimental build systems like qbs did not cause so many headaches.

Post reply on HN