Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stripe's Monorepo Developer Environment

blog.nelhage.com

Post reply on HN
Print view View original

61–70 of 249 posts

Re: Stripe's Monorepo Developer Environment

#61

Post by bluGill »

srvaroa wrote: post #25

"This scale – the scale of devprod, and in turn the scale of the overall organization, such that it could afford 10 FTEs on tooling – was a major factor in our choices" Is basically the summary for most mono/multi repo discussions, and a bunch of other related ones.

It doesn't matter if you have a mono-rep or multi-repo, you will need engineers on tooling to make it work if your project is large. There are pros and cons to both multi-repo and mono-repo with no one right answer (despite what some will tell you). They are different pros and cons, but which is best depends on your particular context.

Re: Stripe's Monorepo Developer Environment

#62

Post by bluGill »

mhh__ wrote: post #57
srvaroa wrote: post #25

"This scale – the scale of devprod, and in turn the scale of the overall organization, such that it could afford 10 FTEs on tooling – was a major factor in our choices" Is basically the summary for most mono/multi repo discussions, and a bunch of other related ones.

Not sure. I think a lot of this is just type of thing comes because with a monorepo you can actually see the problems to solve whereas you can easily end up with the same N engineers firefighting the same problems K times across all your polyrepos.

You have different problems with both. Some problems are hidden in one, but there is no one best answer. (unless your project is small/trivial - which is what a lot of them are)

Re: Stripe's Monorepo Developer Environment

#63

Post by JasonSage »

yeswecatan wrote: post #60
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

I came to ask the same thing. We use docker-compose to describe all our services which works fine.

This does not scale to a large number of services with a certain amount of RAM/processing per service.

Re: Stripe's Monorepo Developer Environment

#64

Post by aidos »

bastawhiz wrote: post #42
Aeolun wrote: post #16

They decided to keep the code on the local machine, but the language server on the remote one. That seems like a recipe for inconsistency. You only get relevant results from your language server once your code has synced.

I was at Stripe until 2022 and inconsistency with the language server was never an issue

Due to the work that this team put in though, right?

The choice to run dev environment far away from the files puts you in the position of needing to engineer your way past the inconsistency.

Re: Stripe's Monorepo Developer Environment

#65

Post by mleo »

I use syncthing to manage the synchronization of files between local laptop and remote development server. The software code base is upwards of 20 years and has dependencies on Windows for runtime. I can run unit tests locally on very fast MacBook Pro or run it much slower on Windows VM. With syncthing I can easily edit files locally or remotely and they are available locally for source control.

The worst problem is refining the ignore settings to ensure only code is synced preventing conflicts on derivative files and that some rule doesn’t overlap code file names.

Re: Stripe's Monorepo Developer Environment

#66

Post by jvolkman »

reillys wrote: post #37
Maxious wrote: post #19

Earlier quoted context omitted.

I'd suggest you revise your competitor analysis. Bazel definitely has a test command that with remote execution and caching absolutely allows you to run entire test suites in seconds* both locally and in CI eg. https://blog.aspect.build/typescript-with-rbe

This blog post says 2 and a half minutes not seconds. I know Bazel is a build system which distributes builds among remote machines. In fact using any computer language you can achieve these goals - you just need to program it. So yes you could probably do all the things with all the things, but Basel does not solve this problem out of the box. I wonder why stripe didn’t “just use Bazel”.

Stripe does use Bazel.

https://stripe.com/blog/fast-secure-builds-choose-two

Re: Stripe's Monorepo Developer Environment

#67

Post by eadmund »

stavros wrote: post #9
domenkozar wrote: post #2

We've been building https://devenv.sh for that reason, I expect more companies to go back to local development once they see DX has improved locally.

Nix is the right tool for this, developing a tool to make Nix's UX easier is a great idea. Thanks for this!

> Nix is the right tool for this

Or Guix, which has the advantage of a more pleasant language.

Re: Stripe's Monorepo Developer Environment

#68

Post by aidos »

JasonSage wrote: post #63
yeswecatan wrote: post #60

Earlier quoted context omitted.

I came to ask the same thing. We use docker-compose to describe all our services which works fine.

This does not scale to a large number of services with a certain amount of RAM/processing per service.

You could still run the proxy they have that lazy boots services - that’s a nice optimisation.

I don’t think that many places are in a position where the machines would struggle. They didn’t mention that in the article as a concern - just that they struggled to keep environments consistent (brew install implies some are running on osx etc).

Re: Stripe's Monorepo Developer Environment

#69

Post by n0us »

aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

You're limited by the resources available to you on your local laptop and when you close that laptop the dev environment stops running. Remote dev environments are more costly and complicated to maintain but they can be shared, can scale vertically (or horizontally) on demand, can persist when you exit them, and managing access to various internal services from dev environments can in some cases be simpler.

It also centralizes dev environment management to the platform team that owns them and provides them as a service which cuts down on support tickets related to broken dev environments. There are certainly some trade offs though and for most companies a local VM or docker compose file will be a better choice.

Re: Stripe's Monorepo Developer Environment

#70

Post by nxicvyvy »

mleo wrote: post #65

I use syncthing to manage the synchronization of files between local laptop and remote development server. The software code base is upwards of 20 years and has dependencies on Windows for runtime. I can run unit tests locally on very fast MacBook Pro or run it much slower on Windows VM. With syncthing I can easily edit files locally or remotely and they are available locally for source control. The worst problem is…

Try unison, it's built for this use case.

https://www.cis.upenn.edu/~bcpierce/unison/

Post reply on HN