> Pre-live environments are never at parity with production
Then you fix that particular problem. Infrastructure as code is one idea just off the top of my head.
We don’t use a staging environment
11–20 of 357 posts
Re: We don’t use a staging environment
#12This is a pretty weird article. Their "how we do it" section lists:
- "We only merge code that is ready to go live"
- "We have a flat branching strategy"
- "High risk features are always feature flagged"
- "Hands-on deployments" (which, from their description, seems to be just a weird way of saying "we have good monitoring and observability tooling")
...absolutely none of which conflict with or replace having a staging environment. Three of my last four gigs have had all four of those and found value in a staging environment. In fact, the often help make staging useful: having feature-flagged features and ready-to-merge code means that multiple people can validate their features on staging without stepping on eachother's toes.
Re: We don’t use a staging environment
#13This sounds horrible unless they have a super reliable way to roll back changes to a consistent working state, both in their deployments and their databases.
Agreed, this sounds crazy. One argument raised is because staging is often different from prod. But their laptop are even more different. It seems the main goal was to save money. All this make sense only for a very small team and code base
How much do you save?
We spend like 3-4k/yr tops on staging
Re: We don’t use a staging environment
#14This is pretty common actually At Facebook too there was no staging environment. Engineers had their dev VM and then after PR review things just went into prod That said features and bug fixes were often times gated by feature flags and rolled out slowly to understand the product/perf impact better This is how we do it at my current team too…for all the same reasons that OP states
That would be controlling a lot of feature flags given how many can be switched on at once. How do you control them?
Re: We don’t use a staging environment
#15No post body was provided.
Re: We don’t use a staging environment
#16This is pretty common actually At Facebook too there was no staging environment. Engineers had their dev VM and then after PR review things just went into prod That said features and bug fixes were often times gated by feature flags and rolled out slowly to understand the product/perf impact better This is how we do it at my current team too…for all the same reasons that OP states
That would be controlling a lot of feature flags given how many can be switched on at once. How do you control them?
flag = true
More seriously, at my old company they just never got removed. So it wasn’t really about control. You just forgot about the ones that didn’t matter after awhile.
If that sounds horrible, that’s probably the correct reaction. But it’s also common.
Namespacing helps too. It’s easier to forget a bunch of flags when they all start with foofeature-.
Re: We don’t use a staging environment
#17I don't see how this can scale beyond a single service.
Complex systems are made of several services and infrastructure all interconnected. Things that are impossible to run on local. And even if you can run on local, the setup is most likely very different from production. The fact that things work on local give a little to zero guarantees that they will work in prod.
If you have a fully automated infrastructure setup (e.g: terraform and friends), then it is not that hard to maintain a staging environment that is identical to production.
Create a new feature branch from main, run unit tests, integrations tests. Changes are automatically merged in the main branch.
From there a release is cut and deployed to staging. Run tests in staging, if all good, promote the release to production.
Re: We don’t use a staging environment
#18> We only merge code that is ready to go live
> If we’re not confident that changes are ready to be in production, then we don’t merge them. This usually means we've written sufficient tests and have validated our changes in development.
Yeah I don't trust even myself with this one. Your database migration can fuck up your data big time in ways you didn't even predict. Just use staging with a copy of prod. https://render.com/docs/pull-request-previews
Sounds like OP could benefit from review apps, he's at the point where one staging environment for the entire tech org slows everybody down.
Re: We don’t use a staging environment
#19> Last updated: April 1, 2022
Re: We don’t use a staging environment
#20This sounds horrible unless they have a super reliable way to roll back changes to a consistent working state, both in their deployments and their databases.
Apparently they never roll back, only forwards. That was elsewhere in the article.
Sounds like a miserable idea. If you make a mistake and take down production you have to debug under extreme pressure to find a roll forward solution.