BazelCon 2024 Recap • Hacker News Board

Re: BazelCon 2024 Recap

#31

Post by pjmlp » Thu, Oct 24, 2024, 12:52 PM UTC

evanmoran wrote: post #22

anothername12 wrote: post #7

We have giant Go monorepo at work. Bazel build. The ex-Google guy who set it up, left. No one around who wants to put in the time to learn it. It seems like a general increase in complexity. It seems to replace the go mod stuff so we have something called gazelle that figured it out and third party ide plug-ins. The plug in for IntelliJ is janky. To top it all off, somehow we managed to get a build that is slow local…

This amazes me since I switched to go because its builds are so insanely fast and the module system is quite clean. Out of all the languages, go probably needs it the least! I did use bazel long ago for c++ and it was quite good at it, but we didn’t have very many dependencies.

There are plenty of native languages that compile Go fast, unfortunely the scripting language revolution, and too much focus on C and C++ tooling kind of messed it all up.

Rob Pike even has a talk about how fast ALGOL compilers used to take, naturally one needs to take into account the additional issue with reading punch cards, but still quite fast for early 1960's hardware.

Or Object Pascal and Modula-2 compilers in the early 1990's.

Eventually we (as in industry) started focusing on the wrong points.

Re: BazelCon 2024 Recap

#32

Post by anothername12 » Thu, Oct 24, 2024, 4:00 PM UTC

pjmlp wrote: post #31

evanmoran wrote: post #22

Earlier quoted context omitted.

This amazes me since I switched to go because its builds are so insanely fast and the module system is quite clean. Out of all the languages, go probably needs it the least! I did use bazel long ago for c++ and it was quite good at it, but we didn’t have very many dependencies.

There are plenty of native languages that compile Go fast, unfortunely the scripting language revolution, and too much focus on C and C++ tooling kind of messed it all up. Rob Pike even has a talk about how fast ALGOL compilers used to take, naturally one needs to take into account the additional issue with reading punch cards, but still quite fast for early 1960's hardware. Or Object Pascal and Modula-2 compilers in…

I have warm fuzzies for way back when I coded in Turbo Pascal. Even on the 8086 I had at the time, compilation speed was incredibly fast even on midsized projects. Much faster than my day to day work with Go on an M3 Max.

I had to install it last night in DOSBox-X and compile the Breakout example project. The warm fuzzies are justified.

Re: BazelCon 2024 Recap

#33

Post by shaftway » Thu, Oct 24, 2024, 4:29 PM UTC

shaftway wrote: post #6

dijit wrote: post #2

What's the option if I want to learn Bazel to an intermediate level and I don't work with former Googlers? There's a decent number of Hello World examples around, and I can get moving with some rulesets, but using it for more than toy projects seems to be really hard for me and there's not a lot of intermediate content from what I can tell. There's "Advanced" reference materials, and countless "getting started" tutor…

Most Google engineers don't do anything more complicated than the hello world examples. Probably far less complicated because they don't need to do any repo setup, it all just works out of the box. Most engineers just make simple java build targets. Larger teams would have someone who specializes in tweaking the build and writes Skylark rules, but to other engineers they just look like simple build targets. I'd sugge…

> but to other engineers they just look like simple build targets.

I forgot to mention, this is by design. It is uncommon at Google for a BUILD file to be more than 3 or 4 build targets. Maybe someone will add a for loop to the build so that they can define various flavors of the build easily, but that's it. Anything more complicated and it should be extracted to a separate Skylark file, included, and in the BUILD file it should look like a basic task.

Re: BazelCon 2024 Recap

#34

Post by pjmlp » Thu, Oct 24, 2024, 4:34 PM UTC

anothername12 wrote: post #32

pjmlp wrote: post #31

Earlier quoted context omitted.

There are plenty of native languages that compile Go fast, unfortunely the scripting language revolution, and too much focus on C and C++ tooling kind of messed it all up. Rob Pike even has a talk about how fast ALGOL compilers used to take, naturally one needs to take into account the additional issue with reading punch cards, but still quite fast for early 1960's hardware. Or Object Pascal and Modula-2 compilers in…

I have warm fuzzies for way back when I coded in Turbo Pascal. Even on the 8086 I had at the time, compilation speed was incredibly fast even on midsized projects. Much faster than my day to day work with Go on an M3 Max. I had to install it last night in DOSBox-X and compile the Breakout example project. The warm fuzzies are justified.

Here is Borland's marketing material bragging about Turbo Pascal 5.5 compile times, on those 640 KB MS-DOS PCs.

34 000 lines of code per minute!

https://bitsavers.org/pdf/borland/turbo_pascal/Turbo_Pascal_...

Re: BazelCon 2024 Recap

#35

Post by Arbortheus » Thu, Oct 24, 2024, 4:54 PM UTC

fire_lake wrote: post #11

Arbortheus wrote: post #8

Earlier quoted context omitted.

We had the same setup, ex-Googler SRE demanded builds be done with Bazel for our Go monorepo. He convinced someone to switch it all over, all builds, deploys, CI, testing, etc were all dependent on Bazel. That person left, and the Bazel stuff was left unmaintained as no one had the interest nor time to learn it. Later Bazel decide to kill off rules_docker and replace it with rules_oci, which means we can no longer up…

I’m guessing the build became much slower due to the lack of caching though?

The builds were much faster afterwards, but Go builds always are fast. Maybe in other languages that are slower to build the Bazel cache is more beneficial.

Re: BazelCon 2024 Recap

#36

Post by Arbortheus » Thu, Oct 24, 2024, 5:02 PM UTC

sluongng wrote: post #30

Arbortheus wrote: post #8

Earlier quoted context omitted.

We had the same setup, ex-Googler SRE demanded builds be done with Bazel for our Go monorepo. He convinced someone to switch it all over, all builds, deploys, CI, testing, etc were all dependent on Bazel. That person left, and the Bazel stuff was left unmaintained as no one had the interest nor time to learn it. Later Bazel decide to kill off rules_docker and replace it with rules_oci, which means we can no longer up…

It's a humbling experience that the opening talk of this year BazelCon is about research into projects that FAILED to adopt Bazel. You can check out the slides of the talk here https://static.sched.com/hosted_files/bazelcon2024/41/TheCla... Personally, most cases I have seen are caused by a systematic under-investment into developer toolings and infrastructure at the company. Management layers often don't understand…

I agree in principle, although I’m not sure in our scenario that Bazel was even a value add when it was maintained. It just meant our Golang developers had to use a bunch of obtuse tooling like Gazelle instead of the normal Go tooling they’re familiar with already. It meant a bunch of smart people ended up blocked and having to constantly ask questions in Slack channels to other devs when they otherwise wouldn’t have been. Some teams even avoided joining our monorepo and built out their own CI, builds, etc in another repo solely because we were using Bazel in the monorepo.

The reality is, in a post-layoffs world, practically every company is scrimping on these sorts of “back office” things. If people couldn’t even get these complex build systems to work in the pre-layoffs low-interest-rate tech world where labour was more abundant, how can they get it done and maintained now?

Re: BazelCon 2024 Recap

#37

Post by aaomidi » Thu, Oct 24, 2024, 9:48 PM UTC

dijit wrote: post #2

What's the option if I want to learn Bazel to an intermediate level and I don't work with former Googlers? There's a decent number of Hello World examples around, and I can get moving with some rulesets, but using it for more than toy projects seems to be really hard for me and there's not a lot of intermediate content from what I can tell. There's "Advanced" reference materials, and countless "getting started" tutor…

Join the Bazel slack imo.

Re: BazelCon 2024 Recap

#38

Post by aaomidi » Thu, Oct 24, 2024, 9:52 PM UTC

pjmlp wrote: post #27

Before reading this well written recap, I was convinced that Bazel is a solution to Google problems, now I am fully sure that is indeed the case, and I should only be aware it is something that exists.

Bazel solves a lot of issues for non Google folk.

It has a build and test and coverage cache that actually consistently works. It can run tests in a sandbox to prevent your local environment from getting consumed by the test (this specific thing has actually caused production outages before).

A DAG for your codebase is also super useful as you can automate a lot of “clean up once you’re done” type work.

Re: BazelCon 2024 Recap

#39

Post by aaomidi » Thu, Oct 24, 2024, 9:55 PM UTC

dilyevsky wrote: post #28

anothername12 wrote: post #26

Earlier quoted context omitted.

Go mod inadequate for Go projects?

Yes. Example of the top of my head - on my recent project (only 50 kloc) go build took 30+ seconds to run when you change one line even fairly high up in the dep graph. When you have grpc/proto or large cgo dependencies and a team of people you need to either dockerize all dev workflows (slow and complex) or write massive bash/makefile abominations (fragile and complex)

And some larger third party dependencies take so long to link to the final binary it’s kinda insane.

Re: BazelCon 2024 Recap

#40

Post by johnfr2 » Sun, Oct 27, 2024, 1:35 PM UTC

sluongng wrote: post #30

Arbortheus wrote: post #8

Earlier quoted context omitted.

We had the same setup, ex-Googler SRE demanded builds be done with Bazel for our Go monorepo. He convinced someone to switch it all over, all builds, deploys, CI, testing, etc were all dependent on Bazel. That person left, and the Bazel stuff was left unmaintained as no one had the interest nor time to learn it. Later Bazel decide to kill off rules_docker and replace it with rules_oci, which means we can no longer up…

It's a humbling experience that the opening talk of this year BazelCon is about research into projects that FAILED to adopt Bazel. You can check out the slides of the talk here https://static.sched.com/hosted_files/bazelcon2024/41/TheCla... Personally, most cases I have seen are caused by a systematic under-investment into developer toolings and infrastructure at the company. Management layers often don't understand…

I worked in a huge big tech with infinite resources. They say monorepo using bazel and gazelle is a success there, I personally found it the worst dev experience ever... Everything was so slow, IDE doesn't work properly, there was no easy debugger, intellisence, refactoring, Code generation... Now every enterprise which calls me on LinkedIn, I ask if there is usage of monorepo, if it has, I just answer that I am not interested...