Mill: A fast JVM build tool for Java and Scala

Build tools sit in an unhappy corner of the design space where they provide features not found in the core of regular programming languages, and which are so generally useful that there's a temptation to make them very abstract, but then they often lack some of the features that let regular programs scale well.

The key feature that justifies their existence is parallel and incremental execution of DAGs of world-mutating tasks. This is an awkward fit with most programming languages, hence the prevalence of DSLs. But people don't want their build system to become a general purpose programming language, because they don't want to think about build systems at all and because programmers don't buy programming languages anymore, so, this causes a big design tension between generality (people want to use build systems to automate many things) and deliberately limiting the expressive power to try and constrain the design space and thus tooling investment required.

Java is in an awkward place because the JDK was born in the 90s on UNIX, by people who thought make is a sufficiently good solution. You still see remnants of this belief in the official Java tutorials, in JEPs, and in the fact that OpenJDK itself is compiled using an autotools based build system! (fortunately it's one of the nice make based build systems out there).

The problem with make is twofold:

1. It assumes a CLI that's both powerful and standardized provided by the host OS. Windows violates this assumption but Java is meant to be portable to Windows.

2. "Plugins" are CLI tools or scripts and so make implicitly assumes that subprocess creation is cheap. But process creation on Windows is expensive, and starting up JVM programs is also expensive due to the JIT compiling.

Therefore make just doesn't work well in the JVM ecosystem. At the same time, the Java project wasn't providing any competing solution, so the wider open source community was left to fill in the gaps. These days language developers provide build tooling out of the box as part of the base toolset along with the compiler, but Java still doesn't.

So - you ask, what are people doing with Gradle/Maven that requires all those features. The answer is: everything! Gradle builds frequently orchestrate dozens of different tools as part of a build pipeline, build documentation websites, do upload and deployment, download and manage dependencies, run security scanners and license compliance checkers, analyze dependency graphs, modify compiler behaviors, and so on.

Additionally Gradle isn't specific to Java, or even JVM apps. It can also be used to compile C/C++ programs, run native code compilers like Kotlin/Native, and it abstracts the underlying platform so Gradle builds aren't tied to UNIX.

That's why it's so complicated.

Although a lot of it is generic badness, Kotlin Multiplatform isn't the JVM ecosystem. You don't need CI runners to publish a JVM library. The reason it comes up with Multiplatform is because Kotlin defines "Multiplatform" to mean platforms like JavaScript, or their own LLVM based compiler toolchain that bypasses JVMs entirely.

Author here! Hope you take a look at the project and find it cool. There's a lot of interesting stuff here. In particular, the Video linked on the landing page is a great intros from a Java developer point of view, and the following video is a great intro from a Build Tool Architect point of view:

* https://www.youtube.com/watch?v=UsXgCeU-ovI

While Mill is focusing on JVM for now, it is very extensible and I have a strawman demo of adding a Javascript toolchain in ~100 lines of code https://mill-build.org/mill/0.12.1/extending/new-language.ht...

For those of you who want to learn more about the design principles and architecture of Mill, and what makes it unique, you should check out the page on Design Principles which has links to videos and blog posts where I elaborate on what exactly makes Mill so different from Maven, Gradle, SBT, Bazel, and so on:

* Mill Design Principles https://mill-build.org/mill/0.12.1/depth/design-principles.h...

I've mentioned this in a few places, but the comparisons with other build tools are best-effort. I have no doubt they can be made more accurate, and welcome feedback so I can go back and refine them. Please take them with a grain of salt

I'm also trying to get the community involved, so it's not just me writing code and running the show. To that end, I have set up a bounty program, so pay out significant sums of money (500-2000USD a piece) for people who make non-trivial contributions. It's already paid out about 10kUSD and has another 20kUSD on the table, so if anyone wants to get involved and make a little cash, feel free to take a shot at one of the bounties! https://github.com/orgs/com-lihaoyi/discussions/6

There is basically no DSL. You simply write what a build needs, e.g. you write a function `collectCFiles()` that collects every file with extension `.c`. You then issue a command like `gcc ${collectCFiles()}`. And pretty much that’s it - you can use shell commands, or do anything in scala (or java or whathaveyou). You simply have your functions return either a value (e.g. a checksum) or a location, which is the only mill-specific logic. So your compileC() function will simply invoke your collectCFiles() function, and this invocation implicitly creates a dependency between these tasks. You have written literally the simplest way to describe your build logic. But in the background mill will cache your functions’ inputs outputs and parallelize those that need re-run, which is what a build tool should do.

The implementation may not be the theoretical best, but I think the idea is pretty much the perfect build system out there.

First invocation may be. Subsequent builds are very fast, unless someone decided to write random bullshit into the build scripts that execute at config time, making the config process impure.