Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stripe's Monorepo Developer Environment

blog.nelhage.com

Post reply on HN
Print view View original

211–220 of 249 posts

Re: Stripe's Monorepo Developer Environment

#211

Post by weq »

hibikir wrote: post #145
bhuga wrote: post #81

Earlier quoted context omitted.

I work on this at Stripe. There's a lot of reasons: * Local dev has laptop-based state that is hard to keep in sync for everyone. Broken laptops are _really hard_ to debug as opposed to cloud servers I can deploy dev management software to. I can safely say the oldest version of software that's in my cloud; the laptops skew across literally years of versions of dev tools despite a talented corpeng team managing them.…

To provide historical context, 10 years ago there was a local dev infrastructure, but it was already so creaky as to be unreliable. Just getting the ruby dependencies updated was a problem. The local dev was also already cheating: All the asynchronous work that was triggered via RabbitMQ/Kafka was getting hacked together, because trying to run everything that Infra/Queues did locally would have been very wasteful. So…

Sounds like you made a massive tradeoff in code coupling if your cant easily swap out remote for local queues etc. But i get it, when your thinking cloud first, understanding where your abstractions start or end can be a complex topic that creates flow on effects and often stop the whizz bang cloud demo code from copy/paste working in your solution. Depending on the stage of your company, this could be a feature or a bug. maybe you have so much complexity in your solution from spreading buisness logic across services that your solution only makes sense when your developing against prod-like-infra and in that scenario im seeing a benifit of having cloud first dev infra because keeping that beast tamed otherwise would be a monumental challange given the perchant for cloud-first to be auto-update-everything.

Re: Stripe's Monorepo Developer Environment

#212

Post by hamandcheese »

MetaWhirledPeas wrote: post #182
hamandcheese wrote: post #164

Earlier quoted context omitted.

My main gripe with the dev box approach is that a cloud instance with similar compute resources as a developers MacBook is hella expensive. Even ignoring compute, a 1TB ebs volume with equivalent performance to a MacBook will probably cost more than the MacBook every month.

Wouldn't this be a reasonable alternative? Asking because I don't have experience with this. 1. New shared builds update container images for applications that comprise the environment 2. Rather than a "devbox", devs use something like Docker Compose to utilize the images locally. Presumably this would be configured identically to the proposed devbox, except with something like a volume pointing to local code. I'm in…

At least on a MacBook, docker is still a compromise in many ways since it has to run on a Linux VM (I live in the SF tech bubble where I've only ever been issued a MacBook).

Even on my personal Linux desktop, I don't love developing in containers. It is very tedious to context switch between my local environment and the in-container environment, and I don't even consider myself the type with a super personalized setup.

So I don't consider local docker that much of an improvement over a remote devbox.

Re: Stripe's Monorepo Developer Environment

#213

Post by hamandcheese »

fnordpiglet wrote: post #200
hamandcheese wrote: post #164

Earlier quoted context omitted.

My main gripe with the dev box approach is that a cloud instance with similar compute resources as a developers MacBook is hella expensive. Even ignoring compute, a 1TB ebs volume with equivalent performance to a MacBook will probably cost more than the MacBook every month.

It’s $250/month for a c6g 2xl 1tb ebs on demand pricing go reserved instances. Given they use AWS and are a major customer, you can expect excellent pricing above the above public pricing quote. Considering the cost of a developers time, and you can do shenanigans to drive that even lower, this all feels totally reasonable.

Price out a 1TB ebs volume with similar performance to a local SSD.

For gp3, you only get 3000 IOPS and 125MBps throughout, without paying extra. That is orders of magnitude less than a local consumer-grade SSD.

Spec it up to 100k IOPS and 5000MBps and you're looking at nearly $800 a month. And that's still way less performance than you get with a local SSD.

Spec it up to 500k IOPS (approaching nvme SSD performance) and you're looking at $2500/month. Now you are on par with a consumer grade SSD.

Re: Stripe's Monorepo Developer Environment

#214

Post by dastbe »

hamandcheese wrote: post #213
fnordpiglet wrote: post #200

Earlier quoted context omitted.

It’s $250/month for a c6g 2xl 1tb ebs on demand pricing go reserved instances. Given they use AWS and are a major customer, you can expect excellent pricing above the above public pricing quote. Considering the cost of a developers time, and you can do shenanigans to drive that even lower, this all feels totally reasonable.

Price out a 1TB ebs volume with similar performance to a local SSD. For gp3, you only get 3000 IOPS and 125MBps throughout, without paying extra. That is orders of magnitude less than a local consumer-grade SSD. Spec it up to 100k IOPS and 5000MBps and you're looking at nearly $800 a month. And that's still way less performance than you get with a local SSD. Spec it up to 500k IOPS (approaching nvme SSD performance)…

If you truly need that kind of perf (and at Amazon, we had plenty of dev desktops running on ebs without that kind of performance) then you should really opt for an instance type with local storage.

Re: Stripe's Monorepo Developer Environment

#215

Post by hamandcheese »

dastbe wrote: post #214
hamandcheese wrote: post #213

Earlier quoted context omitted.

Price out a 1TB ebs volume with similar performance to a local SSD. For gp3, you only get 3000 IOPS and 125MBps throughout, without paying extra. That is orders of magnitude less than a local consumer-grade SSD. Spec it up to 100k IOPS and 5000MBps and you're looking at nearly $800 a month. And that's still way less performance than you get with a local SSD. Spec it up to 500k IOPS (approaching nvme SSD performance)…

If you truly need that kind of perf (and at Amazon, we had plenty of dev desktops running on ebs without that kind of performance) then you should really opt for an instance type with local storage.

Yeah, that's a good callout.

Ive been deep in to implementing macOS CI workers on AWS where that isn't an option (or rather, it is an option, but it is unsupported and Amazon only buys Macs with the smallest possible SSD for the given configuration). So your options are to pay an arm and a leg for fast EBS, or pay an arm and a leg for a pro or max instance with larger internal SSD.

But on Linux there are better options.

Re: Stripe's Monorepo Developer Environment

#216

Post by spacemonkey92 »

shepwalker wrote: post #106
spacemonkey92 wrote: post #12

Earlier quoted context omitted.

I also wonder how they handle merge requests in a monorepo, especially when it comes to the code review process.

Hi! I work at Stripe on this. What're you curious about?

When you are working on a large monolith, how exactly do you managed pull request?

Are they split into git-sub modules/ frameworks? Curious to know the development code review flow.

Re: Stripe's Monorepo Developer Environment

#217

Post by dirtbag__dad »

stavros wrote: post #28
dirtbag__dad wrote: post #13

Earlier quoted context omitted.

What about dev containers?

You mean Docker? They tend to rot much more than I'd like, mostly because you forget to pin something at some point. With Nix, you can't forget.

Err — sorry I mean https://containers.dev/

Which has pretty nice support across IDEs. For example - https://code.visualstudio.com/docs/devcontainers/containers

I set this up for my last company where we had all sorts of “works on my machine issues” and a needlessly painful onboarding experience. Local development became streamlined with this tooling BUT pre-apple silicon macs couldn’t handle running Docker like this. Glacially slow. We had a python monorepo with a bunch of services within it.

I am curious whether nix is an alternative / improvement for this. Was going down the nix road at first but an infrastructure team member steered me toward devcontainers instead and I’ve been pretty happy since!

Re: Stripe's Monorepo Developer Environment

#218

Post by Lio »

ahtihn wrote: post #181
jrochkind1 wrote: post #161

Earlier quoted context omitted.

> I'd go as far as to suggest both monorepos and dev-boxes are complex toolchains that many organizations should consider avoiding. I'm not sure "monorepo" means the same thing to you as it does to me? To me, it just means "keep all the code in one repo, instead of trying to split things up into different repos." To me, it's the thing that is the simple solution, it just means "a repo" -- the reason it gets a name is…

How is a mono repo the simple solution compared to one repo per independently releasable component ? All the tooling is much easier to use when each application has its own repo.

> independently releasable component

I think this is the key phrase in what you've written. Quite often I've seen teams insist on separating things that cannot be released independently due to some form of coupling.

You end up with people talking about a particular "release" but not really knowing 100% what's in it and then discovering later that something is missing or included by mistake.

IMHO it's much easier to keep it all in a single repo and use the SHA value as a single source of truth when discussing what's in it. I don't really work on huge codebases though so your mileage may vary.

Re: Stripe's Monorepo Developer Environment

#219

Post by globular-toast »

dheera wrote: post #130
aidos wrote: post #59

Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?

> By running a Linux VM Or just run Linux on your local machine as the OS. I don't get the obsession with Macs as dev workstations for companies whose products run on Linux.

Agreed. It's so much simpler when people run Linux locally too. Most of our dev environment problems are from people who don't. When you run it locally you also get good at using it which, unsurprisingly, helps a lot when you have to figure out a problem with the deployed version. Learning MacOS/Windows is kinda pointless knowledge in the long run.
Post reply on HN