Bazel 2.0

My personal experience is that Bazel cut through a bunch of the problems that I’ve had with CMake, Waf/SCons, etc. Builds were fragile, they were not reproducible, and there were implicit dependencies. This is mostly as someone who’s rewritten a few build systems, rather than as someone who’s been subjected to build systems by others (I mostly inflict these changes on other people). With Bazel, I have much higher confidence that I’ll get consistent results when I check out the repository on different computers or work with other people.

That said, the major sore point with Bazel for me is the general lack of expertise about how to work with it sanely. Depending on what part you’re looking at, it’s somehow both “too opinionated” and “too flexible” at the same time.

I think it will capture a big chunk of the mindshare for build systems over the next few years, and you’ll see more and more of it. Over that time, people will develop the expertise and best practices for different development problems.

For managing third-party dependencies specifically, Bazel gives you a ton of options, including options that only really make sense for huge orgs like Google. Google vendors their third-party libraries directly into the monorepo. If that doesn’t make sense for your org, Bazel lets you work with external Git repos, with artifact repositories, with package repositories like NPM, or with tools like pkg-config.

The thing that makes this hell, right now, is that few people how to use it well and the documentation is rough. I’m personally very happy with it, even for small codebases, but I’ve used it a lot.

Python is not well supported. The official rules_python has an inadequate third-party packaging solution that has led to ~5 open source alternatives existing that each fix a subset of the issues in the official.

Given that the Python community is really oriented around the PyPi registry and pip packaging tool having a good Bazel-native packaging solution is near essential, but right now it’s not quite there.

Last I checked (maybe 6 months ago), it definitely _didn't_ support py3, although it was advertised. I thought I was doing something wrong, but there were half a dozen issues in the tracker that indicated it was critically broken.

I understand that "it should be simpler" is a pretty lazy criticism. It's been a while since I audited Bazel and friends, and I've forgotten which issues apply to which tool. Moreover, because of the awful state of the documentation and the messiness of the code base (or perhaps this is just standard quality for Java projects?), it's really difficult to tell whether any given issue is actually a fundamental shortcoming in the application or whether it's simply a knowledge gap.

As far as what I want, keep the starlark configuration file format; implement all rules as starlark libraries (such that no one needs to write Java to extend, and if you must write Java then for goodness' sake fix the plugin interface or document it better or something such that one doesn't need to be a core contributor to implement a plugin--perhaps this is fine for an enterprise audience, but it's not fine for my use case). The rules should call into a base `mktarget()` or similar that takes args like the target's ID (the package:target_name pair), a target type that identifies the code used to build the target, and a dict of args/params that are passed into the aforementioned code. The args/params can be an arbitrarily nested JSON-like type so long as the leaves are primitives (int, string, etc), references to source files, or other targets and all leaves (and transitively, the whole structure) must be hashable such that we can identify a given execution of the build.

Beyond that core operating model, the code and the user interface should be clean and well documented. Ideally, small and medium-sized projects shouldn't need to run it in daemon mode to get reasonable performance. This is important because a daemon running on local development machines introduces a larger maintenance burden (there's just more that can go wrong). Language-specific plugins (custom rules, whatever you want to call them) should adhere pretty closely to the conventions of the target language. Lastly, there should be good support for building toplevel artifacts--this means I should be able to build a whole CloudFormation package including lambdas, Docker images, etc just like I would build a JAR or a C++ binary.

I realize that those things are easy enough to say, but the devil is in the details. I've actually gone so far as to prototype the implementation, so I'm confident that those goals are achievable. Unfortunately, it's a pretty significant effort (mostly due to the breadth of project types/languages to support and the nuance/expertise required to support any of them), so I'm bound by free time. If anyone is interested in collaborating or discussing more in-depth, hit me up on Twitter @weberc2 or email me (username at gmail.com).

Build systems seem to sit in that category of perenial category of things that keep getting re-invented, and either recapitulate existing problems or create new ones.

I don't think people will ever fundamentally all agree on:

    - static vs dynamic configuration
    - custom language vs piggy back on existing
    - intelligent, deeply integrated / understands code 
      it is building vs "language agnostic" but 
      necessarily shallow integration

All of these are fundamental tradeoffs that mean every tool will have limitations that about 50% of people don't like. And so we will keep re-inventing forever I think.

cross building is even worse with many repos. I've been there, done that and it broke so often. now we have everything in one repo and we barly have problems. btw. we are a small shop with less than 5 people, but have a product on metal that requires multiple services (that sometimes interact with each other)

we don't use bazel (yet), because dotnet is not that supported.