Live data from Hacker News

Stripe's Monorepo Developer Environment

blog.nelhage.com

191–200 of 249 posts

Re: Stripe's Monorepo Developer Environment

#191
I love this. I believe I might have even interfaced with your team around that time. I was leading Facebook's (now Meta) Developer Products team and we were building against super similar areas internally.

We ran back then a similar project that I coined "Developer On-Demand" to tackle that same problem space. It's also what eventually lead me to find the magics of Nix and then build Flox.

I also agree with a lot of what was shared in other comments, while the problems we tackled at large orgs such as Facebook, Shopify, Uber, Google (to name a few teams I remember working with) and obviously also Stripe, certain areas of the pain are 100% universal regardless of team size.

On the Flox side, we're trying to help with a few of them today and many more hopefully in the soon future, very open for thoughts! Things like - simple to use Nix for each of your projects + keep deps and config up to date across everyones Macbooks and Linux boxes, etc -- even if you don't have a full AWS team and Language Server team ready to support.

Re: Stripe's Monorepo Developer Environment

#192

We've been building https://devenv.sh for that reason, I expect more companies to go back to local development once they see DX has improved locally.

My small team uses devenv for all our development environments and we really like it. Local DX is really important to me and to our team, which is a big part of why we've chosen Nix and devenv.

As we've started to use it more extensively, we've also found that we want to add some enhancements, work out some bugs, and experiment with our own customizations out-of-tree, etc. I'm happy to report here on HN that devenv is well-documented and easy to extend for Nix users who have some experience with Nix module systems, and that Domen is really responsive to PRs. :)

Re: Stripe's Monorepo Developer Environment

#193
post #67
post #9

Earlier quoted context omitted.

Nix is the right tool for this, developing a tool to make Nix's UX easier is a great idea. Thanks for this!

> Nix is the right tool for this Or Guix, which has the advantage of a more pleasant language.

At my workplace, Guix's lack of macOS support takes away some of the benefit of using something like Nix or Guix as opposed to HVM solutions like Docker Desktop or Vagrant. I imagine this situation is unfortunately common.

For teams where GNU/Linux is the primary development OS, Guix seems like a great choice.

Re: Stripe's Monorepo Developer Environment

#194
post #9

Earlier quoted context omitted.

Nix is the right tool for this, developing a tool to make Nix's UX easier is a great idea. Thanks for this!

What about dev containers?

Containers are a great deployment target, but they're not really a great development environment for a few reasons (e.g., they're Linux-specific, so they require extra virtualization on non-Linux operating systems, the kind of isolation they provide is more of a hindrance than a help when it comes to working on your local filesystem, and for them to be useful you have to set up infrastructure to push and pull your private containers to and from).

Nix is a better fit for this, and when you're using Nix you can also have Nix generated containers for deployment. I think you can also use a container with Nix in to provide the devcontainers interface to devs who don't have Nix installed locally, and have it in turn use Nix against your project's flake to set up its environment.

Re: Stripe's Monorepo Developer Environment

#195

How does a payment service wind up with over a 1000 engineers? I understand that "engineers" may not mean "developers", it could DevOps, site reliability and all the bits and pieces that make up a large service provider, but over a 1000? Can someone please enlighten me?

Surely in 2024 we can't be classifying Stripe as just "a payment service"

Re: Stripe's Monorepo Developer Environment

#196
post #26

Earlier quoted context omitted.

It (obviously) leverages Nix, which in turn means the environment is declarative and fully reproducible (not "reproducible" as in docker). Now, you can use just Nix's devShells, but with devenv you have a middleground between just Nix package manager and a full fledged NixOS module system. Basically, write out one line of code - and you've got your Postgres, another one - full linter set up for whatever language you'…

Can I also get the security/isolation benefits that a duly configured docker/podman can provide (container can only act on mounted volume, non-root user, other seccomp settings?). I feel better doing my "npm install"s in such an environment (of course it's still not a VM – but that's another topic). When I read about nix, reproducibility is a goal, but security/isolation is a non-goal.

> When I read about nix, reproducibility is a goal, but [...] isolation is a non-goal.

Generally, yes.

But you can use or put together something like this to run Nix inside a devcontainer instead of locally: https://github.com/xtruder/nix-devcontainer

So you can use them in conjunction (or alternation, if for some projects you're okay running without a container) without having to specify your development environments twice.

> I feel better doing my "npm install"s in such an environment (of course it's still not a VM – but that's another topic).

There are basically two kinds of integration you can do for a project with Nix, which I'll call deep and shallow. In shallow integration, you just have Nix provide the toolchain and then you build the project (manually, with a script, with a Makefile, whatever). This is pretty common and pretty easy, and gives you no protection from malicious NPM build scripts.

For deep integration, you can actually have Nix build your whole project. This has some downsides, like that it can't really handle incremental builds. It also imposes restrictions, like no network access by anything but Nix at build time, all packages are built by special build users with no homedirs and no perms to access anything, etc. When you do that kind of build/install, you do get some protection from crypto miners lurking in the NPM registry or PyPI or whatever.

Re: Stripe's Monorepo Developer Environment

#197

Earlier quoted context omitted.

Not even once did I want to share my dev. environment, nor did anyone want to share mine. We are talking about 25-odd years of being a developer. Never in my life did I want to scale my dev. environment vertically or horizontally or in any other direction. Unless you work on a calculator, I don't know why would you need that. I have no problems with my environment stopping when I close my laptop. Why is this a proble…

I get a serious feeling that interpreted languages, monorepos, environment orchestration, snapshot ecosystem aggregators, and per-function execution evironments are all pushing software development into the wrong direction. Those things are not bad by themselves. But people tend to do bad things with them, and those bad things spread remarkably well, disrupting every place they infect.

The absurd lengths people will go to avoid learning how computers actually work because they fell for the buy now, pay later promise of 'easy' development.

Re: Stripe's Monorepo Developer Environment

#198
Very insightful blog post!

> Finally: the development experience, of course, is only part of the story: the full lifecycle of code and features continues onward into CI and code review and ultimately through deployment into production, where it will be further observed, debugged, and evolved. Writing about those systems would require further posts at least this long.

In case the author is around: I would love to read those!

Re: Stripe's Monorepo Developer Environment

#199

How does a payment service wind up with over a 1000 engineers? I understand that "engineers" may not mean "developers", it could DevOps, site reliability and all the bits and pieces that make up a large service provider, but over a 1000? Can someone please enlighten me?

It's a fair question, I think you need ~10 or so to do the work and 990 to bikeshed the horrors that come about when you have what should be a few 3mb HTTPS APIs at $50-100pcm split between 1000 resume-hungry engineers and a taste for cloud complexity.

Re: Stripe's Monorepo Developer Environment

#200
post #93

It's always so enlightening to have articles like this one shed light on how companies at scale operate. It goes without saying that many of the problems Stripe faced with their monorepo isn't application to smaller businesses, but there are still bits and pieces that are applicable to many of us. I've been working on an ephemeral/preview environment operator for Kubernetes( https://github.com/pier-oliviert/sequencer…

My main gripe with the dev box approach is that a cloud instance with similar compute resources as a developers MacBook is hella expensive. Even ignoring compute, a 1TB ebs volume with equivalent performance to a MacBook will probably cost more than the MacBook every month.

It’s $250/month for a c6g 2xl 1tb ebs on demand pricing go reserved instances. Given they use AWS and are a major customer, you can expect excellent pricing above the above public pricing quote.

Considering the cost of a developers time, and you can do shenanigans to drive that even lower, this all feels totally reasonable.

Post reply on HN