Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Rust Builds with Bazel

mmapped.blog

Post reply on HN
Print view View original

81–90 of 133 posts

Re: Scaling Rust Builds with Bazel

#81

Post by malkia »

tragomaskhalos wrote: post #45

There is a broad tendency in software to jump onto whatever the big FAANG guys are doing. The trouble with this is often that they are working at a vastly bigger scale than you are, hence their solutions are vastly overfitted to your problem. So it's definitely interesting to read about this stuff but I'm sure Cargo is fine for most folks.

There is also big tendendcy to ignore them "because we are not at that scale" without realizing that you've reached the scale where you need it, even though your scale is magnitudes still less than what FAANG is.

The fallacy here is that just because you are not at the scale of the FAANG does not mean that their tools/libraries/products are not suitable for you - e.g. it's not the case that only if you reach their scale, then maybe you should switch there. This is bs.

Re: Scaling Rust Builds with Bazel

#82

Post by malkia »

fancl20 wrote: post #41
Mesopropithecus wrote: post #28

Earlier quoted context omitted.

In what way?

I guess because package managers (apt, nix, ..) build around the idea of shared libraries, while bazel assume statically built. TBH it’s hard to say which operate model is better anyway. They are simply incompatible :) Related topic: https://youtu.be/Pzl1B7nB9Kc

This is my own fork of OpenTelemetry C++ that builds (using bazel with my changes) OpenTelemetry C++ .dll for Windows - https://github.com/malkia/opentelemetry-cpp

to show how awesome bazel is - in one step it can build (compile) and package them into a .zip file ready to deploy to NuGet-like repo or other place.

Re: Scaling Rust Builds with Bazel

#83

Post by malkia »

illiarian wrote: post #6

One question I have after all these articles and internal company presentations is: how exactly does Bazel track dependencies? Module A importing module B importing module C is a language-level feature, not a build system-level feature.

Not directly to your question, but basically bazel needs a static graph. So if you have a.cpp #include "a.h" - then you need to declare that "a.h" in your BUILD file.

But fear not, (and I'm not sure if BAZEL does this yet, but does it for java). If you've declared only "a.cpp", but forgot "a.h" - then bazel would detect that (/showIncludes in MSVC, or file system sandbox in linux, etc.) - and it'll tell you - you need to add "a.h" to your BUILD file.

Fear not again, it's possible to create a BUILDIFIER command that given what BAZEL told you it can do it automatically for you. Hence the BUILD files needs to be kept simple, for everyone to understand (even non-engineers), while the ugly parts go into the .bzl files and workspaces.

So in tha java world (but any language could do it), anything that was discovered dynamically but not declared in, bazel can offer how to fix it (copy+paste the command offered, or tool makes it for you) - at worst report it and you have to fix it manually

The important thing is - bazel can use only the information provided in the BUILD files to detect - oh hey, things have changed in the CI - so I need to rebuild. Without relying on "cl/gcc/clang" to run this through preproc step to discover actual headers. or keep horrible make-like deps.

Re: Scaling Rust Builds with Bazel

#84

Post by klodolph »

19h wrote: post #79

I used bazel for a few internal repositories written in Rust and it's amazing .... until you hit an issue. Edge cases in Bazel are about as documented as any random fish you might find in a crack at the bottom of the ocean. And the most hilarious thing to deal with is "yeah this was written for a Google project, that other use case didn't apply to our project". When eventually after the Xth issue I was told "why don'…

The experience of using Bazel inside Google with support is different from the experience of using Bazel outside Google.

Some ex-Googler may want to use Bazel and may have very good reasons for it, but they’re often unprepared to take over the support duties that, at Google, are provided by the developer tool teams.

Re: Scaling Rust Builds with Bazel

#85

Post by malkia »

renewiltord wrote: post #63

I think this would have been useful with some code. My experience with Bazel is that you need a BUILD file in each subdirectory or something like that and every time I add a file I have to add it to that BUILD file. Exceedingly tedious to use and has poor support in Clion. It is likely that I don't have the constraints that this company has. The last time I tried this in Java I had to replicate my package dependency…

You don't necesseraily need to, but ideally.

a BUILD file declares a "package" and the demarcation line happens when it sees another BUILD file in a sub-directory. Then that's another "package".

Re: Scaling Rust Builds with Bazel

#86

Post by MuffinFlavored »

19h wrote: post #79

I used bazel for a few internal repositories written in Rust and it's amazing .... until you hit an issue. Edge cases in Bazel are about as documented as any random fish you might find in a crack at the bottom of the ocean. And the most hilarious thing to deal with is "yeah this was written for a Google project, that other use case didn't apply to our project". When eventually after the Xth issue I was told "why don'…

> I used bazel for a few internal repositories written in Rust and it's amazing

Could you explain how it beats for example, a Dockerfile in terms of what you need to write/get your hands dirty with (syntax, tooling, etc.) if your output goal is just an OCI image format to publish/run in container orchestration software? Is that too simple of a usecase?

Re: Scaling Rust Builds with Bazel

#87

Post by saidinesh5 »

19h wrote: post #79

I used bazel for a few internal repositories written in Rust and it's amazing .... until you hit an issue. Edge cases in Bazel are about as documented as any random fish you might find in a crack at the bottom of the ocean. And the most hilarious thing to deal with is "yeah this was written for a Google project, that other use case didn't apply to our project". When eventually after the Xth issue I was told "why don'…

> "yeah this was written for a Google project, that other use case didn't apply to our project"

Not only is that a very narrow set of use cases (Read: they only deploy statically linked binaries on mostly unix-y environments), Even the fixes for those very basic use cases take for ever.

Eg. This 7 year old issue is still open: https://github.com/bazelbuild/bazel/issues/1920 . To be able to create a statically linked library, we had to use: https://github.com/hotg-ai/librunecoral/blob/master/runecora... . Had to use some weird hack to build shared libraries too. Overall, it was just annoying.

Re: Scaling Rust Builds with Bazel

#88

Post by dboreham »

klodolph wrote: post #84
19h wrote: post #79

I used bazel for a few internal repositories written in Rust and it's amazing .... until you hit an issue. Edge cases in Bazel are about as documented as any random fish you might find in a crack at the bottom of the ocean. And the most hilarious thing to deal with is "yeah this was written for a Google project, that other use case didn't apply to our project". When eventually after the Xth issue I was told "why don'…

The experience of using Bazel inside Google with support is different from the experience of using Bazel outside Google. Some ex-Googler may want to use Bazel and may have very good reasons for it, but they’re often unprepared to take over the support duties that, at Google, are provided by the developer tool teams.

"We did X at Google" for me has become an antipattern and a red flag.

Re: Scaling Rust Builds with Bazel

#89

Post by saidinesh5 »

tragomaskhalos wrote: post #45

There is a broad tendency in software to jump onto whatever the big FAANG guys are doing. The trouble with this is often that they are working at a vastly bigger scale than you are, hence their solutions are vastly overfitted to your problem. So it's definitely interesting to read about this stuff but I'm sure Cargo is fine for most folks.

Bigger scale.. or sometimes they just have different requirements.

Re: Scaling Rust Builds with Bazel

#90

Post by dilyevsky »

klodolph wrote: post #84
19h wrote: post #79

I used bazel for a few internal repositories written in Rust and it's amazing .... until you hit an issue. Edge cases in Bazel are about as documented as any random fish you might find in a crack at the bottom of the ocean. And the most hilarious thing to deal with is "yeah this was written for a Google project, that other use case didn't apply to our project". When eventually after the Xth issue I was told "why don'…

The experience of using Bazel inside Google with support is different from the experience of using Bazel outside Google. Some ex-Googler may want to use Bazel and may have very good reasons for it, but they’re often unprepared to take over the support duties that, at Google, are provided by the developer tool teams.

I found when i started using bazel outside of google that my mental model for how its apis really work was really lacking. Once i learned the apis (and the docs are outstanding at helping with that), and with the help of a few examples easily searchable via github, it became quite manageable
Post reply on HN