Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel Fawlty

danmux.com

Post reply on HN
Print view View original

21–30 of 52 posts

Re: Bazel Fawlty

#21

Post by erain »

leg100 wrote: post #14
erain wrote: post #6

For your docker container needs: "Bazel should in theory be able to decide if a docker image should be rebuilt" You can checkout Bazel's docker rule: https://github.com/bazelbuild/rules_docker Also, go already has a very good build sysmtem build-in, and hazel really shines when: - you have a complex codebase with multi-languages: one can build a tool in one language and use it as tool for a another language. - you si…

It eschews docker, and Dockerfiles and their non-determinism, and builds the layers itself, and uses a python lib [1] to push and pull container images. In order to ensure reproducibility/determinism, however, Bazel doesn't have an equivalent of the RUN instruction. You have to use other Bazel rules to fetch and produce artefacts and add them to an image, and there aren't always rules for what it is you want to do (I…

disclaimer: I am one of the rules_docker contributor :D

You are right about the pain that there is no `RUN` equivalent (yet), and I think someone is working on that.

Regarding producing the docker image itself: given Open Container Initiative[1] and its container image spec[2], it is a good thing that multiple tools can produce exchangeable container images and bazel is one of them with its own reproducible properties.

[1] https://www.opencontainers.org/ [2] https://github.com/opencontainers/image-spec/blob/v0.2.0/ser...

Re: Bazel Fawlty

#22

Post by Aissen »

It added a dependency on the JVM. (I probably have an overly painful past with the JVM in production), suffice to say we had to bump up the RAM on the build machines.

This alone is a reason not to use Bazel. It's designed for the enterprise in mind, where JVMs are common, sharing a common build environment in a team is easy (or at least, the setup time isn't an issue).

For a community project, this type of thing wouldn't fly. There's a reason Meson is winning the latest build-systems war: it only depends on python3 & ninja.

Re: Bazel Fawlty

#23

Post by danmux »

elteto wrote: post #18

It seems like Bazel's support for Go is still not first class and a lot of the issues mentioned here come from that. But it strikes me as odd that they are figuring all of this out _now_, as opposed to a year ago when they first switched. Why did they switch to bazel in the first place? Did they do a trade study/test drive? All of the problems mentioned (more complex build files and workflow, increased build times, l…

To give a tool a fair chance I think it has to be used daily to really grok the pleasure / pain. We were aware of some of the problems, but perhaps overly confident about managing them, and underestimated the frustration and lack of agency. Possibly we let it embed too far before we decided we have tried as hard as could be expected. Around 6 months in, once a number of us formally proposed that it be reverted, we took the bold decision that we should persevere no further. Having worked in C codebases in the past - I can imagine it is almost a miraculous improvement there.

Re: Bazel Fawlty

#24

Post by ggambetta »

> Hope is a strategy [...] At some point we have to hope and assume. for example eventually we hope the compiler authors did a good job with the next version we are about to use, or we assume that the kernel fix was good.

Or you can do a very slow, controlled rollout of the new version and see what happens. With all the systems I worked with while at Google, both as a SRE and a SWE, whenever we had a new version to release, we'd update one task in one cluster, let it run for a day or two, check the logs and the metrics... if it was OK, update one job in one cluster, then repeat the process with an entire cluster (the designated canary cluster), and only then release to the rest of the clusters. If any of these went wrong, we'd rollback or patch, depending on the severity. We rarely missed our weekly releases.

I'm sure the same applies to a new compiler version or a new kernel fix. You don't need to assume anything.

Re: Bazel Fawlty

#25

Post by nine_k »

sarabande wrote: post #3

Thanks for the comments on Bazel -- I am evaluating a proper build system for us right now for a data science codebase almost entirely in Python, with a lot of scientific computing dependencies. Bazel doesn't yet seem to have first-class support for Python, so that's out, and I'm running into a lot of problems with Twitter Pants, although their Slack channel is really helpful. Did you ever try using Pants or Buck, or…

(For a mostly-Python shop, I'd consider using [Search domain github.com] https://github.com/waf-project/waf or scons.org)

Re: Bazel Fawlty

#26

Post by danmux »

zellyn wrote: post #20

If your entire codebase is Go, you really shouldn't be using Bazel/Pants/Buck/Please. Go's built-in build system is perfectly adequate for just Go, with light tooling for generated code (eg. protobufs). It's when you have multiple languages and want to be able to build/test only what has changed that you need a Blaze-alike. With regards to Gazelle: is it intended to be run every build, on every file? Or is it intende…

Regarding Gazelle, yes, the CI would break early if it detected a none porcelain repo after running Gazelle itself. I think others may have configured their editors, but for me the occasional 30+s run time (and as mentioned in some workflows 3-4 mins) on file save - plus the already creaking VSCode plugins was too much to bear.

Re: Bazel Fawlty

#27

Post by mkobit »

Aissen wrote: post #22

It added a dependency on the JVM. (I probably have an overly painful past with the JVM in production), suffice to say we had to bump up the RAM on the build machines. This alone is a reason not to use Bazel. It's designed for the enterprise in mind, where JVMs are common, sharing a common build environment in a team is easy (or at least, the setup time isn't an issue). For a community project, this type of thing woul…

> There's a reason Meson is winning the latest build-systems war:

What do you mean by winning? I haven't heard of Meson until the last few weeks, and I haven't seen it consumed by any open source projects on GitHub, although my search biases more towards JVM things anyways.

> it only depends on python3 & ninja

I don't see how depending on both Python3 and Ninja is any less dependencies than the JVM. Python3 isn't really available by default on most distributions (maybe that is changing soon), so there is still the overhead of installing it, right? Public CI systems like Travis CI and Circle CI have images that make having a JDK easy.

Re: Bazel Fawlty

#28

Post by danmux »

ggambetta wrote: post #24

> Hope is a strategy [...] At some point we have to hope and assume. for example eventually we hope the compiler authors did a good job with the next version we are about to use, or we assume that the kernel fix was good. Or you can do a very slow, controlled rollout of the new version and see what happens. With all the systems I worked with while at Google, both as a SRE and a SWE, whenever we had a new version to r…

hmm, yes, this was always going to be a contentious point, and highly subjective. I think the main issue here is it was in a large part also a very human impact, which is much harder to measure. I think it was fair to say that our build system at the time was more rigid and testing fully both flows not within our grasp. I discussed this point about 'hope' with a colleague, I think I agree with his conclusion: "Google actually designs some of the chips it uses! And runs its own power stations - I think if anyone could legitimately say "Hope is not a Strategy" it might be them."

Re: Bazel Fawlty

#29

Post by zellyn »

danmux wrote: post #26
zellyn wrote: post #20

If your entire codebase is Go, you really shouldn't be using Bazel/Pants/Buck/Please. Go's built-in build system is perfectly adequate for just Go, with light tooling for generated code (eg. protobufs). It's when you have multiple languages and want to be able to build/test only what has changed that you need a Blaze-alike. With regards to Gazelle: is it intended to be run every build, on every file? Or is it intende…

Regarding Gazelle, yes, the CI would break early if it detected a none porcelain repo after running Gazelle itself. I think others may have configured their editors, but for me the occasional 30+s run time (and as mentioned in some workflows 3-4 mins) on file save - plus the already creaking VSCode plugins was too much to bear.

Was it 3-4 minutes when updating a single file? Or only running over the whole repo?

Re: Bazel Fawlty

#30

Post by EtDybNuvCu »

Have you tried Nix? It's like Bazel, but without incremental builds. The boons are pretty great.

I'm extremely disappointed, as a career SRE, to see "hope is not a strategy" trampled in the mud. I understand that perhaps the author has succumbed to epistemic helplessness and given up on having computers work without optimism or belief, but computers do not operate on any ruleset which includes hope, and we should not encourage magical thinking.

Post reply on HN