Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stripe's Monorepo Developer Environment

blog.nelhage.com

Post reply on HN
Print view View original

141–150 of 249 posts

Re: Stripe's Monorepo Developer Environment

#141

Post by layer8 »

Daishiman wrote: post #113
recroad wrote: post #100

Earlier quoted context omitted.

If you have 100 services in your org, I don't have to have 100 running at the same time in your local dev machine. I only run the 5 I need for the feature I'm working on.

I've been on this path and as soon as you work on a couple of concurrent branches you end up having 20 containers in your machine and setting these up to run successfully ends up being its own special PITA.

What exactly are the problems created by having a larger number of containers? Since you’re mentioning branches, these presumably don’t have to all run concurrently, i.e, you’re not talking about resource limitations.

Re: Stripe's Monorepo Developer Environment

#142

Post by oftenwrong »

__jonas wrote: post #22
lijok wrote: post #21

Earlier quoted context omitted.

> Won't be surprised to see that many would probably need a safari map or README documentation in every single folder to navigate a repository as large as stripes. No different to having thousands of smaller repos instead. I personally dislike monorepos, for very niche, in-the-weeds operational reasons (as an infra person), but their ergonomics for DX cannot be understated.

The 'ergonomics for DX' benefit is that you can share code across projects without having to go down the path of creating a package / library pushed to some internal registry and pulled by each project right? Or are there any other aspects to the monorepo architecture that make it beneficial for large companies like that? Just curious, I've never worked in such an environment myself.

To put it in the most general terms: It provides the same value that using a VCS has for a project, but applied to the entire company.

In a standalone project, would you accept a change that is incompatible with other code in the project? For example, would you allow a colleague to change a function in a way that breaks the call sites? No, you probably would not.

The attitude within monorepo shops is that this level of rigour should be applied to the entire company. Nobody should be able to make a change anywhere if it would break anything elsewhere, or they should only be permitted to do so with intention. There are caveats to this, but that is the general idea.

Re: Stripe's Monorepo Developer Environment

#143

Post by vlovich123 »

PedroBatista wrote: post #133
physicsguy wrote: post #120

I think for smaller companies, you can get a long way towards a lot of this with judicious use of docker-compose, and convenience scripts in a Makefile. As long as you don't do anything stupid like try and spin up 100 services when you're a team of 8, most laptops these days are sufficiently capable of handling a database, Redis, your codebase, and something like LocalStack.

I would say you can even go a looong way without any Docker at all. And for the large majority of the companies/projects, if your project is so complex and heavy of resources that it doesn't fit on a modern laptop, the problem is not in the laptop, it's in the whole project and the culture and cargo-cult around "modern" software development.

Containers/VMs are a nice way to isolate away any machine configuration discrepancies. Conversely it does encourage the use of non hermetic and deterministic build systems which come with other issues too (eg speed differences surfacing race conditions in the build)

Re: Stripe's Monorepo Developer Environment

#145

Post by hibikir »

bhuga wrote: post #81
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

I work on this at Stripe. There's a lot of reasons: * Local dev has laptop-based state that is hard to keep in sync for everyone. Broken laptops are _really hard_ to debug as opposed to cloud servers I can deploy dev management software to. I can safely say the oldest version of software that's in my cloud; the laptops skew across literally years of versions of dev tools despite a talented corpeng team managing them.…

To provide historical context, 10 years ago there was a local dev infrastructure, but it was already so creaky as to be unreliable. Just getting the ruby dependencies updated was a problem. The local dev was also already cheating: All the asynchronous work that was triggered via RabbitMQ/Kafka was getting hacked together, because trying to run everything that Infra/Queues did locally would have been very wasteful. So magic occurred in the calls to the message queue that instead triggered the crucial ruby code that would be hit in the end.

So if this was a problem back then, when the company had less than 1000 employees, I can't even imagine how hard would it be to get local dev working now

Re: Stripe's Monorepo Developer Environment

#146

Post by uncanneyvalley »

dheera wrote: post #130
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

> By running a Linux VM Or just run Linux on your local machine as the OS. I don't get the obsession with Macs as dev workstations for companies whose products run on Linux.

The year of Linux on the laptop has yet to arrive for most of us. Windows and MacOS both offer better battery life, if for no other reason (and there are usually other reasons, like suspend/wake issues, graphics driver woes, etc.)

Re: Stripe's Monorepo Developer Environment

#147

Post by bhuga »

aidos wrote: post #127
bhuga wrote: post #123

Earlier quoted context omitted.

> I’m referring to running everything inside a single VM that you would have total access to. It could have telemetry, you’d know versions etc. I wonder if there’s some confusion around what I’m suggesting given your points above. I don't think there's confusion. I only have total access when the VM is provisioned, but I need to update the dev machine constantly. Part of what makes a VM work well is that you can make…

So the devs don’t have the ability to ssh to your cloud instances and change config? Other than the size issue, I’m still not seeing the difference. Take your point on it needing to start before you have control, but other than that a VM on a dev machine is functionally the same as one in a cloud environment. In terms of needing to reset, it’s just a matter of git branch, push, reset, merge. In your world that sync c…

> So the devs don’t have the ability to ssh to your cloud instances and change config?

They do, but I can see those changes if I'm helping debug, and more importantly, we can set up the most important parts of the dev processes as services that we can update. We can't ssh into a VM on your laptop to do that.

For example, if you start a service on a stripe machine, you're sending an RPC to a dev-runner program that allocates as many ports as are necessary, updates a local envoy to make it routable, sets up a systemd unit to keep it running, and so forth. If I need to update that component, I just deploy it like anything else. If someone configures their host until that dev runner breaks, it fails a healthcheck and that's obvious to me in a support role.

> Just to be clear, I think it’s interesting to have a healthy discussion about this to see where the tradeoffs are. Feels like the sort of thing where people try to emulate you and buy themselves a bunch of complexity where other options are reasonable.

100% Agree! I think we've got something pretty cool, but this stuff is coming from a well-resourced team; keeping the infra for it all running is larger than many startups. There's tradeoffs involved: cost, user support, flexibility on the dev side (i.e. it's harder to add something to our servers than to test out a new kind of database on your local VM) come immediately to mind, but there are others.

There are startups doing lighter-weight, legacy-free versions of what we're doing that are worth exploring for organizations of any size. But remote dev isn't the right call for every company!

Re: Stripe's Monorepo Developer Environment

#148

Post by aidos »

crabbone wrote: post #94
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

Working in a configuration where your development environment isn't on your computer is always a huge downgrade. Work with VM? -- sooner or later you'll have problems with forwarding your keyboard input to the VM. Work with containers? -- no good way to save state, no good way to guarantee all containers are in sync etc. God forbid any sort of Web browser-based solution. The number of times I accidentally closed the…

Sounds like you’re talking about something else (more like the Citrix / virtual desktop type model - I don’t know the name).

The idea here is that you use a VM (cloud or local) to run your compute. Most people can run it in the background without explicitly connecting to it.

Re: Stripe's Monorepo Developer Environment

#149

Post by marcosdumay »

crabbone wrote: post #98
n0us wrote: post #69

Earlier quoted context omitted.

You're limited by the resources available to you on your local laptop and when you close that laptop the dev environment stops running. Remote dev environments are more costly and complicated to maintain but they can be shared, can scale vertically (or horizontally) on demand, can persist when you exit them, and managing access to various internal services from dev environments can in some cases be simpler. It also c…

Not even once did I want to share my dev. environment, nor did anyone want to share mine. We are talking about 25-odd years of being a developer. Never in my life did I want to scale my dev. environment vertically or horizontally or in any other direction. Unless you work on a calculator, I don't know why would you need that. I have no problems with my environment stopping when I close my laptop. Why is this a proble…

I get a serious feeling that interpreted languages, monorepos, environment orchestration, snapshot ecosystem aggregators, and per-function execution evironments are all pushing software development into the wrong direction.

Those things are not bad by themselves. But people tend to do bad things with them, and those bad things spread remarkably well, disrupting every place they infect.

Re: Stripe's Monorepo Developer Environment

#150

Post by numbsafari »

mootoday wrote: post #77

I've worked with remote dev environments for many years, including some time with one of the providers of such a service. It became clear to me that cloud-only is not the way to go, but instead a local-first, cloud-optional approach. https://mootoday.com/blog/dev-environments-in-the-cloud-are-...

This is my biggest complaint with GitHub CodeSpaces.

I should be able to launch a local VM using the GitHub Desktop App just as easily as I can an Azure-hosted instance.

Post reply on HN