Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stripe's Monorepo Developer Environment

blog.nelhage.com

Post reply on HN
Print view View original

111–120 of 249 posts

Re: Stripe's Monorepo Developer Environment

#111

Post by kccqzy »

n_ary wrote: post #34
pavlov wrote: post #15

Earlier quoted context omitted.

Meta also has a massive monorepo accessed primarily through cloud devservers. When several of the world’s most successful software companies use this approach, it’s hard to argue that it’s inherently bad. Of course it’s sensible to discuss what lessons apply to smaller companies who don’t have the luxury of dedicated tooling teams supporting the monorepo and dev environment.

Just because some successful companies use some approach doesn't make it the best practice. I have seen firsthand nuisance of monorepo, which took almost 15minutes to correctly switch branches on intel machines(and decently spiked the CPU by causing windows defender to panic). It has decent benefit of easy code sharing, but build and test are soul sucking experiences and if someone decides to run some updated formatt…

Nothing you describe is inherent to monorepos. Git is slow yes but go use hg. Build and test are slow? That's a CI problem: you didn't allocate enough resources to the build system. Someone ran a formatter accidentally? That's that someone's mistake.

Re: Stripe's Monorepo Developer Environment

#112

Post by aidos »

bhuga wrote: post #81
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

I work on this at Stripe. There's a lot of reasons: * Local dev has laptop-based state that is hard to keep in sync for everyone. Broken laptops are _really hard_ to debug as opposed to cloud servers I can deploy dev management software to. I can safely say the oldest version of software that's in my cloud; the laptops skew across literally years of versions of dev tools despite a talented corpeng team managing them.…

I want to double check we’re talking about the same thing here. I’m referring to running everything inside a single VM that you would have total access to. It could have telemetry, you’d know versions etc. I wonder if there’s some confusion around what I’m suggesting given your points above.

I’m sure there are a bunch of things that make it the right choice for Stripe. Obviously if you just have too many things to run at a time and a dev laptop can’t handle it then it’s a dealbreaker. What’s the size of the cloud instances you have to run on?

Re: Stripe's Monorepo Developer Environment

#113

Post by Daishiman »

recroad wrote: post #100
JasonSage wrote: post #63

Earlier quoted context omitted.

This does not scale to a large number of services with a certain amount of RAM/processing per service.

If you have 100 services in your org, I don't have to have 100 running at the same time in your local dev machine. I only run the 5 I need for the feature I'm working on.

I've been on this path and as soon as you work on a couple of concurrent branches you end up having 20 containers in your machine and setting these up to run successfully ends up being its own special PITA.

Re: Stripe's Monorepo Developer Environment

#114

Post by cynicalpeace »

bhuga wrote: post #110
delhanty wrote: post #29

>Some caveats: It’s been nearly five years, and I have no doubt that I have misremembered some of the specific details, even though I’m confident in the overall picture. I’m also certain that Stripe has continued evolving and I make no claim this document represents the developer experience at Stripe as of today. Are there any more recently ex-Stripe folks here willing and able to comment on how Stripe's developer en…

Some important differences from 2019: * Code is off of laptops and lives entirely on the dev server in many (but not all) cases. This has opened up a lot of use cases where devs can have multiple branches in flight at once. * Big investments into bazel. * Heavier investment into editor experiences. We find most developers are not as idiosyncratic in their editor choices as is commonly believed, and most want a pre-co…

I'm glad to see that first bullet point. The code living separate from the execution environment seemed like too much overhead and complexity for not enough benefit.

Re: Stripe's Monorepo Developer Environment

#115

Post by aidos »

simonw wrote: post #95
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

In my opinion the single most important feature of any development environment is a reliable “reset” button. The amount of time companies lose to broken development environments is incredible. A developer can easily lose half a day (or more) of productive time. With cloud environments it’s much easier to offer a “just give me a brand new environment that works” button somewhere. That’s incredibly valuable.

For sure, but, a VM has that feature too. They have to run some services directly on the laptop to handle the code syncing. So if you accept a certain amount of “need to do some dev machine setup” as a cost, installing Parallels and running a script to download an iso is a pretty small surface area that allows for a full reset.

I don’t doubt that Stripe have a setup that works well for them them but I also bet they could have gone done a different path that also worked well and I suspect that other path (local VMs) is a better fit for most other smaller teams.

Re: Stripe's Monorepo Developer Environment

#116

Post by cynicalpeace »

nkohari wrote: post #74
delhanty wrote: post #29

>Some caveats: It’s been nearly five years, and I have no doubt that I have misremembered some of the specific details, even though I’m confident in the overall picture. I’m also certain that Stripe has continued evolving and I make no claim this document represents the developer experience at Stripe as of today. Are there any more recently ex-Stripe folks here willing and able to comment on how Stripe's developer en…

I spent 4.5 years at Stripe, and left in March. The biggest difference not mentioned is the article is that code is no longer kept on developer machines. The sync process described in the article was well-designed, but also was a fairly constant source of headaches. (For example, sometimes the file watcher would miss an update and the code on your remote machine would be broken in strange ways, and you'd have to reco…

Glad to see that they moved to code living with the execution environment. The code living separate from the execution environment seemed like too much overhead and complexity for not enough benefit.

Especially given VSCode, or Cursor ;), work so well via ssh.

To the engineers that don't want to use those IDE's it might suck temporarily, but that's it.

Re: Stripe's Monorepo Developer Environment

#117

Post by aidos »

sulam wrote: post #99
aidos wrote: post #68

Earlier quoted context omitted.

You could still run the proxy they have that lazy boots services - that’s a nice optimisation. I don’t think that many places are in a position where the machines would struggle. They didn’t mention that in the article as a concern - just that they struggled to keep environments consistent (brew install implies some are running on osx etc).

I think it’s safe to assume that for something with the scale and complexity of Stripe, it would be a tall order to run all the necessary services on your laptop, even stubs of them. They may not even do that on the dev boxes, I’d be a little surprised if they didn’t actually use prod services in some cases, or a canary at any rate, to avoid the hassles of having to maintain on-call for what is essentially a test env…

I don’t know that’s safe to assume. Maybe it is an issue but it was not one of the issues they talk about in the article and not one of the design goals of the system. They have the proxy / lazy start system exactly so they can limit the services running. That suggests to me that they don’t end up needing them all the time to get things done.

Re: Stripe's Monorepo Developer Environment

#118

Post by adamdecaf »

recroad wrote: post #100
JasonSage wrote: post #63

Earlier quoted context omitted.

This does not scale to a large number of services with a certain amount of RAM/processing per service.

If you have 100 services in your org, I don't have to have 100 running at the same time in your local dev machine. I only run the 5 I need for the feature I'm working on.

We have 100 Go services (with redpanda) and a few databases in docker-compose on dev laptops. It works well when and we buy the biggest memory MacBooks available.

https://moov.io/blog/education/moovs-approach-to-setup-and-t...

Re: Stripe's Monorepo Developer Environment

#120

Post by physicsguy »

I think for smaller companies, you can get a long way towards a lot of this with judicious use of docker-compose, and convenience scripts in a Makefile. As long as you don't do anything stupid like try and spin up 100 services when you're a team of 8, most laptops these days are sufficiently capable of handling a database, Redis, your codebase, and something like LocalStack.
Post reply on HN