111–120 of 176 posts
IMHO, if you're targeting the Javascript ecosystem, this area is already fairly crowded, with Turborepo, Nx and various open source tools providing various degrees of functionality (Bazel, Pants, Lerna, etc) already competing in the space. I'm a tech lead for the web monorepo at Uber. We talked to the Turborepo guy a few years ago, and he admitted that he wasn't sure if it could handle our scale in terms of all the b…
> We wanted our system to be enjoyable to use and easy to understand, but also solve the same problems as existing systems. For example, configuration is in YAML, not a proprietary syntax I'm incredibly skeptical of this. I'm ex-meta and have worked a lot with the enterprise solutions you're talking about and the choice of starlark (originally python) as the build definition language is one of the killer features of…
Based on everyone's feedback about YAML (we didn't expect this much), we'll probably reconsider this!
To add to everyone else, please don't use YAML. Starlark is great _precisely_ because it is a readable, well known (nearly Python) language that is limited at the same time (no unbounded for loops, no way to do non-deterministic things like get the current time or `random()`).
Earlier quoted context omitted.
Can you speak to what kind of "functionality" you need a language for? Are you referring to Starlark-like files?
Turing completeness. No, I'm referring to using Python (waf, conan, etc), Javascript (nodejs scripts, esbuild, etc), and other "real" programming languages. It's incredibly frustrating to have to configure a build with something as shitty as YAML. There's a tangible amount of money I have wasted in organizations on it. A build is not actually a static configuration of another system. It's a program and deserves every…
That said, Starlark is way closer to a real language (Python) than YAML.
My advice to anyone making a new build system: Most likely you did this because you felt all the other ones are too complicated. But the reason the “enterprise” ones are so complicated is to serve their enterprise customers, who need “just this one feature” so they can use it. But those customers pay the bills. So basically you have to choose complexity and profit or simplicity and less (or no) profit. Good luck! But…
Are you sure you can't have your cake and eat it too? You can have many configuration options, but give each one a sane default.
Almost none of the complexity comes from what configuration options I've registered ahead of time. It comes almost entirely from, "Well darn, this code depends on this completely unrelated part of the project. I wish it didn't, but now the build tool either needs to sometimes fail to rebuild something correctly, or it needs to build way too much to run quickly."
To deploy the services locally, we use Tilt[2] (K8s for local). We want to be able to reproduce production as much as possible and remove developer overhead on how things work locally and in production.
Then come the issues with Docker and large node code bases:
1 challenge with large monorepos is the huge node_modules folder (rush among other tools put packages into a single large node_modules folder, and symlink every dependency there. It can contain millions of files and GBs, depending on how many 3rd-party libraries you use). On Linux, you can mount it without issues, but Mac has performance issues[3] with large folders.
We pre-build that huge node_modules into a "base" image, and each service in the monorepo pulls that base image and only mounts what's necessary (a few mb). So we can save that npm build time and don't need to copy all those files inside the containers. It is fine because package.json does not change that often. You need to do this pre-build locally then in your CI/CD -> dockerhub, so everyone can get it.
Another challenge is that you need to "watch" files to rebuild them. Watching all your files inside the monorepo isn't really viable. We use a dependency graph to know what services to watch and then copy the built files inside the Tilt containers.
Hope this can help people.
It's built in Rust and does not have full Rust language support? ... how?
Earlier quoted context omitted.
Turing completeness. No, I'm referring to using Python (waf, conan, etc), Javascript (nodejs scripts, esbuild, etc), and other "real" programming languages. It's incredibly frustrating to have to configure a build with something as shitty as YAML. There's a tangible amount of money I have wasted in organizations on it. A build is not actually a static configuration of another system. It's a program and deserves every…
You actually don't want a real programming language because their unbounded loops and standard libraries are sources of non-determinism that can introduce incorrectness in your build. Bazel's correctness is highly contingent on the same set of inputs producing the same outputs, and granularly tracking dependencies, running commands in sandboxes AND having a restricted build language is a key part of it. That said, St…
Earlier quoted context omitted.
Can Prune be used to build a bundle (as in a zip) for, say AWS Lambda, which includes only the dependencies (and not dev dependencies)? I've played around with pnpm's deploy but it felt a bit lackluster. Especially talking about situation where one has a backend package and some shared package. The bundle should contain all dependencies (but not dev dependencies) of the backend package and shared package and of cours…
Yes! Prune then zip the output folder.
Awesome, congrats. I've been an early trier of moon repo and I really fell for the slickness of the website and the name, when evaluating build tools. I've primarily worked in typescript codebases and have used raw yarn workspaces, lerna, nx and recently evaluated moon and turbo. The funky part is I eventually simply went with nx, not just because I've used it before, but also because I felt like the configuration is…
Besides, yes, nx comes with a single configuration file for each project, but alongside of it are jest, eslint, babel and app/lib/spec/ide tsconfig configs - that’s a lot!
All of this shouldn’t be visible to the developer. Initially, when trying to find myself in this mess, I thought that the solution lies in autogenerated ide workspaces - for vscode and sublime. But in practice it wasn’t that helpful, because there is always something which is not handled by autogenerate multiroot structure but is needed, so one needs to have multiple windows open anyways.
Hopefully typescript 5.0 is going to help reduce some of this boilerplate with multiple tsconfig base classes, so lib/app/spec/ide tsconfigs will be able to extend from common base, which currently is not possible.
The worst part of Nx is that there is lack of ssr support, so I had to patch nx with patch-package to generate obfuscated css classes in prod, because currently it’s not possible with their webpack executor! Recently they tried to simplify it a little bit, reducing this webpack boilerplate, which introduced few bugs but it's better than not doing anything!
However, saying all of that and as someone mentioned in previous comments, dx is not primarily improved by lack of config boilerplate, but by having semantic structure of the actual code, clearly knowing what depends on what and where one can find it and put it. Right now we are on our own, because there is lack of information on how to structure cross platform codebase properly. I strongly believe it will change though. Best of luck to the people at moon!
Ps. Nx has a nice blog post why they do t use bazel under the hood https://blog.nrwl.io/on-bazel-support-6be3b3ceba29