Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stripe's Monorepo Developer Environment

blog.nelhage.com

Post reply on HN
Print view View original

161–170 of 249 posts

Re: Stripe's Monorepo Developer Environment

#161

Post by jrochkind1 »

tmpz22 wrote: post #124
draw_down wrote: post #101

Earlier quoted context omitted.

Right, dev boxes do not need to do double duty as a personal computer plus development target, which allows them to more closely resemble the machine your code will actually run on. They also can be replaced easily, which can be helpful if you ever suspect something is wrong with the box itself - if the new one acts the same way, it wasn't the dev box. I don't recall latency being a big problem in practice. In an org…

I feel like we're not getting the right lessons from this. It feels like we're focusing on HOW we can do something versus pausing for a brief moment to consider if we SHOULD in the first place. To me the root issue is the complexity of production environments has expanded to the point of impacting complexity in developer environments just to deploy or test - this is in conjunction with expanding complexity of develop…

> I'd go as far as to suggest both monorepos and dev-boxes are complex toolchains that many organizations should consider avoiding.

I'm not sure "monorepo" means the same thing to you as it does to me? To me, it just means "keep all the code in one repo, instead of trying to split things up into different repos."

To me, it's the thing that is the simple solution, it just means "a repo" -- the reason it gets a name is because it's unusual for large orgs with enormous codebases to have everything in one repo, it's unusual for them to do the simple thing that works fine for a small org with a normal codebase.

What is it you're suggesting a simple organization should do instead of a "monorepo"?

Re: Stripe's Monorepo Developer Environment

#162

Post by aylmao »

__jonas wrote: post #22
lijok wrote: post #21

Earlier quoted context omitted.

> Won't be surprised to see that many would probably need a safari map or README documentation in every single folder to navigate a repository as large as stripes. No different to having thousands of smaller repos instead. I personally dislike monorepos, for very niche, in-the-weeds operational reasons (as an infra person), but their ergonomics for DX cannot be understated.

The 'ergonomics for DX' benefit is that you can share code across projects without having to go down the path of creating a package / library pushed to some internal registry and pulled by each project right? Or are there any other aspects to the monorepo architecture that make it beneficial for large companies like that? Just curious, I've never worked in such an environment myself.

I'd say there's 4 main advantages, summarizing what other comments are saying but also from my own experience:

- atomic PRs. All changes for a migration/feature living in one spot makes development much easier, especially when dealing with api changes and migrations

- single history. This is useful when debugging. A commit can more easily encapsulate the state of "the whole system" as opposed to a single part of it. This makes reverting, if necessary, easier

- environment consistency. updating the linting tool, formatting tool, UI library, etc is never a priority, so there's always drift, where an old repo gets stuck with old tools, dependencies and an old environment

- not shipping your org chart is easier when everyone can see and work work on the whole codebase, as easily as possible.

Re: Stripe's Monorepo Developer Environment

#163

Post by [deleted] »

crabbone wrote: post #98
n0us wrote: post #69

Earlier quoted context omitted.

You're limited by the resources available to you on your local laptop and when you close that laptop the dev environment stops running. Remote dev environments are more costly and complicated to maintain but they can be shared, can scale vertically (or horizontally) on demand, can persist when you exit them, and managing access to various internal services from dev environments can in some cases be simpler. It also c…

Not even once did I want to share my dev. environment, nor did anyone want to share mine. We are talking about 25-odd years of being a developer. Never in my life did I want to scale my dev. environment vertically or horizontally or in any other direction. Unless you work on a calculator, I don't know why would you need that. I have no problems with my environment stopping when I close my laptop. Why is this a proble…

[deleted]

Re: Stripe's Monorepo Developer Environment

#164

Post by hamandcheese »

p-o wrote: post #93

It's always so enlightening to have articles like this one shed light on how companies at scale operate. It goes without saying that many of the problems Stripe faced with their monorepo isn't application to smaller businesses, but there are still bits and pieces that are applicable to many of us. I've been working on an ephemeral/preview environment operator for Kubernetes( https://github.com/pier-oliviert/sequencer…

My main gripe with the dev box approach is that a cloud instance with similar compute resources as a developers MacBook is hella expensive. Even ignoring compute, a 1TB ebs volume with equivalent performance to a MacBook will probably cost more than the MacBook every month.

Re: Stripe's Monorepo Developer Environment

#165

Post by ParetoOptimal »

numbsafari wrote: post #150
mootoday wrote: post #77

I've worked with remote dev environments for many years, including some time with one of the providers of such a service. It became clear to me that cloud-only is not the way to go, but instead a local-first, cloud-optional approach. https://mootoday.com/blog/dev-environments-in-the-cloud-are-...

This is my biggest complaint with GitHub CodeSpaces. I should be able to launch a local VM using the GitHub Desktop App just as easily as I can an Azure-hosted instance.

But then how will they lock you into paying them monthly?

Re: Stripe's Monorepo Developer Environment

#166

Post by bhuga »

aidos wrote: post #153
bhuga wrote: post #147

Earlier quoted context omitted.

> So the devs don’t have the ability to ssh to your cloud instances and change config? They do, but I can see those changes if I'm helping debug, and more importantly, we can set up the most important parts of the dev processes as services that we can update. We can't ssh into a VM on your laptop to do that. For example, if you start a service on a stripe machine, you're sending an RPC to a dev-runner program that al…

Ah! So that’s a spot where we’re talking past each other. I’d anticipate you would be equally as able to ssh to VMs on dev laptops. That’s definitely a prerequisite for making this work in the same way as you’re currently doing. The only difference between what you do and what I’m suggesting is the location of the VM. That itself creates some tradeoffs but I would expect absolutely everything inside the machine to be…

> I’d anticipate you would be equally as able to ssh to VMs on dev laptops. That’s definitely a prerequisite for making this work in the same way as you’re currently doing.

Our laptops don't receive connections, but even if they could, folks go on leave and turn them off for 9 months at a time, or they don't get updated for whatever reason, or other nutty stuff.

It's surprisingly common with a few thousand of them out there that laptop management code that removes old versions of a tool is itself removed after months, but laptops still pop up with the old version as folks turn them back on after a very long time, and the old tool lingers. The services the tools interact with have long since stopped working with the old version, and the laptop behaves in unpredictable ways.

This doesn't just apply to hypothetical VMs, but various CLI tools that we deploy to laptops, and we still have trouble there. The VMs are just one example, but a guiding principle for us been that the less that's on the laptop, the more control we have, and thus the better we can support users with issues.

Re: Stripe's Monorepo Developer Environment

#167

Post by ParetoOptimal »

0x457 wrote: post #137
earthling8118 wrote: post #71

Earlier quoted context omitted.

The language isn't the problem with nix.

It's not "the problem", but it's a problem. It's better than alternatives, but it's hacky nature shows.

Well, if you believe:

- discoverability is a problem in Nix

- Guix encourages or "shepherds" more discoverable functions, modules, and abstractions

Then the language could be a problem.

Re: Stripe's Monorepo Developer Environment

#168

Post by ParetoOptimal »

otabdeveloper4 wrote: post #107
eadmund wrote: post #67

Earlier quoted context omitted.

> Nix is the right tool for this Or Guix, which has the advantage of a more pleasant language.

That's, like, just your opinion, man. Scheme and/or Lisp is literally the worst language choice for this problem domain.

Why is Lisp the worst for this problem domain?

I will admit I find guix to be much more verbose than Nix.

Re: Stripe's Monorepo Developer Environment

#169

Post by ParetoOptimal »

eikenberry wrote: post #135
bhuga wrote: post #110

Earlier quoted context omitted.

Some important differences from 2019: * Code is off of laptops and lives entirely on the dev server in many (but not all) cases. This has opened up a lot of use cases where devs can have multiple branches in flight at once. * Big investments into bazel. * Heavier investment into editor experiences. We find most developers are not as idiosyncratic in their editor choices as is commonly believed, and most want a pre-co…

That last point has long been a red flag when interviewing. A developer who doesn't care about their tooling also tends to not care about the quality of their work.

Its also a red flag for me when a company mandates an IDE.

Re: Stripe's Monorepo Developer Environment

#170

Post by [deleted] »

hamandcheese wrote: post #164
p-o wrote: post #93

It's always so enlightening to have articles like this one shed light on how companies at scale operate. It goes without saying that many of the problems Stripe faced with their monorepo isn't application to smaller businesses, but there are still bits and pieces that are applicable to many of us. I've been working on an ephemeral/preview environment operator for Kubernetes( https://github.com/pier-oliviert/sequencer…

My main gripe with the dev box approach is that a cloud instance with similar compute resources as a developers MacBook is hella expensive. Even ignoring compute, a 1TB ebs volume with equivalent performance to a MacBook will probably cost more than the MacBook every month.

[deleted]
Post reply on HN