What's the easiest way of sharing things like protobuf definitions across multiple separate repos and making sure things are always in sync?
buf.build
Stripe's Monorepo Developer Environment
131–140 of 249 posts
Re: Stripe's Monorepo Developer Environment
#132Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?
From what I remember (left Stripe in late 2022) much of Stripe's codebase was/is a Ruby tangled "big ball of mud" monorepo due to lack of proper modules. Basically a lot of the core modules all imported code from each other with little layering so you couldn't deploy a lean service without pulling in almost all of the monorepo code. And due to the way imports worked it would load a ton of this code a runtime. This meant that even a simple service would have extremely high memory usage and be unsuitable for a local dev environment where you have N of these bloated services running at the same time. There was a big refactoring effort to get "strict modules" in place to cut down on this bloat which had some promising results. I'm not an expert in this area but I believe this was the gist of it.
Re: Stripe's Monorepo Developer Environment
#133I think for smaller companies, you can get a long way towards a lot of this with judicious use of docker-compose, and convenience scripts in a Makefile. As long as you don't do anything stupid like try and spin up 100 services when you're a team of 8, most laptops these days are sufficiently capable of handling a database, Redis, your codebase, and something like LocalStack.
I would say you can even go a looong way without any Docker at all.
And for the large majority of the companies/projects, if your project is so complex and heavy of resources that it doesn't fit on a modern laptop, the problem is not in the laptop, it's in the whole project and the culture and cargo-cult around "modern" software development.
Re: Stripe's Monorepo Developer Environment
#134Earlier quoted context omitted.
We have 100 Go services (with redpanda) and a few databases in docker-compose on dev laptops. It works well when and we buy the biggest memory MacBooks available. https://moov.io/blog/education/moovs-approach-to-setup-and-t...
Your success with this strategy correlates more strongly with ‘Go’ than ‘100 services’ so it’s more anecdotal than generally-acceptable that you can run 100 services locally without issues. Of course you can. Buying the biggest MacBook available as a baseline criteria for being able to run a stack locally with Docker Compose does not exactly inspire confidence. At my last company we switched our dev environment from…
Have any details on how you've implemented Nix? For my personal projects I use nix without docker and the results are great. However I was always fearful that nix alone wouldn't quite scale as well as nix + docker for complicated environments.
I've used the FROM SCRATCH strat with nix:
https://mitchellh.com/writing/nix-with-dockerfiles
Is that how you implemented it?
Re: Stripe's Monorepo Developer Environment
#135>Some caveats: It’s been nearly five years, and I have no doubt that I have misremembered some of the specific details, even though I’m confident in the overall picture. I’m also certain that Stripe has continued evolving and I make no claim this document represents the developer experience at Stripe as of today. Are there any more recently ex-Stripe folks here willing and able to comment on how Stripe's developer en…
Some important differences from 2019: * Code is off of laptops and lives entirely on the dev server in many (but not all) cases. This has opened up a lot of use cases where devs can have multiple branches in flight at once. * Big investments into bazel. * Heavier investment into editor experiences. We find most developers are not as idiosyncratic in their editor choices as is commonly believed, and most want a pre-co…
That last point has long been a red flag when interviewing. A developer who doesn't care about their tooling also tends to not care about the quality of their work.
Re: Stripe's Monorepo Developer Environment
#136Earlier quoted context omitted.
> Nix is the right tool for this Or Guix, which has the advantage of a more pleasant language.
That's, like, just your opinion, man. Scheme and/or Lisp is literally the worst language choice for this problem domain.
I wouldn't say it's the worst. I don't like Lisp and co, but I think it's alright for this. I don't like Guix for a very different reason.
Re: Stripe's Monorepo Developer Environment
#137Re: Stripe's Monorepo Developer Environment
#138Earlier quoted context omitted.
You can create them with devenv, but they are actually reproducible : https://devenv.sh/containers/ https://devenv.sh/integrations/codespaces-devcontainer/
i missed any description of the actual container content on those examples.
IIRC, it uses what is defined for shell environment. Just instead of activating on your machine, it produces OCI image with that environment.
I have nixOS definitions that I can use to make a SD card image, overtake a running linux system via ssh, deploy to nixos via ssh, or deploy to a local system - all from one definition.
Re: Stripe's Monorepo Developer Environment
#139Earlier quoted context omitted.
You mean Docker? They tend to rot much more than I'd like, mostly because you forget to pin something at some point. With Nix, you can't forget.
did the word rot change meaning recently? pin is what causes rot, not what solves it.
Different kind of rot. With nix and flakes, I can come back to a project 5 years later and as long as external dependencies (i.e. package sources) still available it will bring me back straight to that environment like it was yesterday.
If you have a Dockerfile from 5 years ago...well good luck building it today.
Re: Stripe's Monorepo Developer Environment
#140Maybe a silly question, but why all this engineering effort when you could host the dev environment locally? By running a Linux VM on your local machine you get a consistent environment that you can ssh to, remove the latency issues but you remove all the complexity of syncing that they’ve created. That’s a setup that’s worked well for me for 15 years but maybe I’m missing some other benefit?
> By running a Linux VM Or just run Linux on your local machine as the OS. I don't get the obsession with Macs as dev workstations for companies whose products run on Linux.
Especially when they don’t even deploy to ARM servers.