Launch HN: Moonrepo (YC W23) – Open-source build system

We agree. We weren't happy with all of the current solutions, at least in the JavaScript space.

In regards to build graph invalidation, we attempt to hash pieces at the granular level. This includes per file content hashing, and for dependencies (those in `package.json`), we parse the lockfile and extract the resolved version/integrity hashes. We can probably improve this further, but this has been working great so far.

As for phantom dependencies, this isn't something moon solves directly. We encourage users to use pnpm or yarn 3, where these phantom problems are less likely. We don't believe moon should solve this, and instead, the package managers should.

If you have any other questions or concerns, would love to hear them.

(for context - I'm not interested in first class node support)

This seems pretty cool. I particularly like how 'gradual' it seems to be relative to things like Bazel, i.e. you can take some shell scripts and migrate things over. I did have a play and hit an initial problem around project caching I think, which I raised at [0].

One comment, from the paranoid point of view of someone who has built distributed caching build systems before is that your caching is very pessimistic! I understand why you hash outputs by default (as well as inputs), but I think that will massively reduce hit rate a lot of the time when it may not be necessary? I raised [1].

Edit: for any future readers, I spotted an additional issue around the cache not being pessimistic enough [3]

As an aside, I do wish build systems moved beyond the 'file-based' approach to inputs/outputs to something more abstract/extensible. For example, when creating docker images I'd prefer to define an extension that informs the build system of the docker image hash, rather than create marker files on disk (the same is true of initiating rebuilds on environment variable change, which I see moon has some limited support for). It just feels like language agnostic build systems saw the file-based nature of Make and said 'good enough for us' (honorable mention to Shake, which is an exception [2]).

[0] https://github.com/moonrepo/moon/issues/637

[1] https://github.com/moonrepo/moon/issues/638

[2] https://shakebuild.com/why#expresses-many-types-of-build-rul...

[3] https://github.com/moonrepo/moon/issues/640

I don't see how that solves the problem mentioned by OP. If the build rule mentions foo.c, we only need to recompile that one object and relink. When you are using globs, changing one file changes the aggregated hash and then would necessitate recompiling every object file.

> Tasks are defined and run as if you were running them in the terminal; no more abstractions like BUILD files.

But abstractions are a good thing! They let you separate a system into layers, so that things programmed at a higher layer don't need to know all the details of the lower layers. It's the way we separate interface from implementation.

Let me give an example from Bazel. Suppose you are compiling some C or C++, and you need to define a preprocessor symbol. There are two ways of doing this, "defines" and "local_defines":

    cc_library(
        name = "my_lib",
        srcs = ["my_lib.cc"],
        hdrs = ["my_lib.h"],
        # This will pass -DFOO when compiling this library
        # *or* any library that depends on it.
        defines = ["FOO"],
    )

    cc_library(
        name = "my_lib",
        srcs = ["my_lib.cc"],
        hdrs = ["my_lib.h"],
        # This will pass -DFOO when compiling this library
        # only.  Libraries that depend on us will *not* get
        # the symbol.
        local_defines = ["FOO"],
    )

You can use "defines" when you have #ifdef statements in headers and "local_defines" when you only test the symbols in your .cc files.

I did not have to think about how defines will propagate to the libraries that depend on me, I just specified what I need and the build system handles the rest. I did not have to define my own CXXFLAGS variable and handle concatenating all of the right things together. The lower layer takes care of that.

What Bazel lets you do is create abstraction boundaries within the build system. People can write rules that define a self-contained set of build logic, then other people can use those abstractions without knowing all the details of how they are implemented.

Bazel is not perfect, and I have found myself banging my head against the wall many times. But overall the ability to define abstractions in a build system is, I think, one of the biggest things it gets right.

Disclosure: I work for Google, but not on Bazel.

Nx is hot garbage imo. Buggy, overly verbose, inflexible, poorly documented, and I'm dubious about their peer review process.

If I don't need build caching I'm not using any tool but PNPM and its workspace toolset - that's literally all most people need for a monorepo. I've looked into Turborepo, and its simplicity versus Nx is a strength. However, it's not the taskrunner that I want.

I now work in a monorepo where build caching is required, so I'm excited about moon and keeping a watchful eye on the project's progress. From my evaluation so far, it fixes all of my gripes about Nx and I'm keen on it not trying to do too much, while allowing me to make it as flexible and extensible as I need. Extending configs is chefs kiss