How do you do QA? I mean, staging in our case is accessible by a lot of non technical people that test things automated test cannot test (did I say test?).
We don’t use a staging environment
21–30 of 357 posts
Re: We don’t use a staging environment
#22Let's talk again about this after the next postmortem?
Re: We don’t use a staging environment
#23Not endorsing this point blank but.. One positive side effect of this is that it becomes much easier to rally folks into improving the fidelity of the dev environment, which has compound positive impact on productivity (and mental health of your engineers).
In my experience at Big Tech Corp, dev environments were reduced to low unit test fidelity over years, then as a result you need to iterate (ie develop) in a staging environment that is orders of magnitude slower (and more expensive if you're paying for it). It isn't unusual that waiting for integration tests is the majority of your day.
Now, you might say that it's too complex so there's no other way, and yes sometimes that's the case, but there's nuance! Engineers have no incentive to fix dev if staging/integration works at all (even if super slow) so it's impossible to tell. If you think slow is a mild annoyance, I will tell you that I had senior engineers on my team that committed around 2-3 (often small) PRs per month.
Re: We don’t use a staging environment
#24This article has some very weird trade-offs.
They can't spin up test environments quickly, so they have windows when they cannot merge code due to release timing. They can't maintain parity of their staging environments with prod, so they forswear staging environments. These seem like infrastructure problems that aren't addressing the same problem as the staging environment eo ipso.
They're not arguing that testing or staging environments are bad, they're just saying their organization couldn't manage to get them working. If they didn't hit those roadblocks in managing their staging environments, presumably they would be using them.
Re: We don’t use a staging environment
#25Isn’t the concept of a single staging environment becoming a bit dated? Every recent project I’ve worked on uses preview branches or deploy previews, eg what Netlify offers https://docs.netlify.com/site-deploys/deploy-previews/
Or am I missing something?
Re: We don’t use a staging environment
#26> We only merge code that is ready to go live.
In their perception, is the rest of tech industry gambling in every pull request that some untested code would work in production?
I work at a large company. We extensively test code on local machines. Then dev test environments. Then small roll out to just a few data centers in prod bed. Run small scale online flight experiments. Then roll out to the rest of prod bed.
And I've seen code fail in each of the stages, no matter how extensively we tested and robustly code ran in prior stages.
Re: We don’t use a staging environment
#27I don't see how this can scale beyond a single service. Complex systems are made of several services and infrastructure all interconnected. Things that are impossible to run on local. And even if you can run on local, the setup is most likely very different from production. The fact that things work on local give a little to zero guarantees that they will work in prod. If you have a fully automated infrastructure set…
[deleted]
Re: We don’t use a staging environment
#28They mention database as a factor not to have a staging env due to different size, but they don’t mention how they test schema migrations and any feature which touches the data which usually produce multiple issues, or even data loss.
Re: We don’t use a staging environment
#29Earlier quoted context omitted.
Agreed, this sounds crazy. One argument raised is because staging is often different from prod. But their laptop are even more different. It seems the main goal was to save money. All this make sense only for a very small team and code base
How much do you save? We spend like 3-4k/yr tops on staging
Or, on the flip side, how much do you lose by deploying an 'oops', resulting in customers having a bad experience and posting "This thing sux!" on social media?
I can sympathize with the costs in both time and money to maintain a staging environment, but you're going to pay for those bugs somehow - either in staging or in customer satisfaction.
Re: We don’t use a staging environment
#30Isn’t the concept of a single staging environment becoming a bit dated? Every recent project I’ve worked on uses preview branches or deploy previews, eg what Netlify offers https://docs.netlify.com/site-deploys/deploy-previews/ Or am I missing something?
no you're right, "staging" is gradually being replaced with per-commit "preview". but at enterprise scale when you have distributed services and data, and strict financial controls, and uncompromising compliance standards, it can often be unrealistic to transition to that until a new program group manager comes in with permission to blow everything up