We duplicate the production environment and sanitize all the data to be anonymous. We run our automated tests on this production-like data to smoke test. Our tests are driven by pytest and Playwright. God bless, I have to say how much I love Playwright. It just makes sense.
We don’t use a staging environment
131–140 of 357 posts
Re: We don’t use a staging environment
#132This article has some very weird trade-offs. They can't spin up test environments quickly, so they have windows when they cannot merge code due to release timing. They can't maintain parity of their staging environments with prod, so they forswear staging environments. These seem like infrastructure problems that aren't addressing the same problem as the staging environment eo ipso. They're not arguing that testing o…
Having staging always encourages this. It’s really difficult to replicate prod in any non trivial way that exceeds what can be created on a workstation. Eg. Even if you buy the same hardware you can’t replicate production load anyway because it’s not being used by 5 million people concurrently. Your cache access patterns aren’t the same, etc. It’s far better to have a fast path to prod than a staging environment in m…
I think it's too much to expect staging to match the load and access patterns of your prod system.
I find staging to be very useful. In various teams I have been a part of, I have seen the following productive use cases for staging
1. Extended development environment - If you use a micro-services or serverless architecture, it becomes really useful to do end-to-end tests of your code on staging. Docker helps locally, but unless you have a $4,000 laptop, the dev experience becomes very poor.
2. User acceptance testing - Generally performed by QAs, PMs or some other businessy folks. This becomes very important for teams that serve a small number of customer who write big checks.
3. Legacy enterprise teams - Very large corporations in which software does not drive revenue directly, but high quality software drives a competitive advantage. Insurance companies are an example. These folks have a much lower tolerance for shipping software that doesn't work exactly right for customers.
Re: We don’t use a staging environment
#133If you are saying you don't have a staging environment, what you are really saying is that your company doesn't have any QA process. If your QA process is just developers testing their own shit on their local machine then you are not going to get as much value out of staging.
I’ve worked with multiple teams where QA tests in prod behind feature flags, canary deploys, etc. Staging environments and QA don’t always go hand in hand.
Re: We don’t use a staging environment
#134If you are saying you don't have a staging environment, what you are really saying is that your company doesn't have any QA process. If your QA process is just developers testing their own shit on their local machine then you are not going to get as much value out of staging.
That's absolutely not true.
You can just compartmentalise important changes behind feature flags / service architecture and test things later.
Re: We don’t use a staging environment
#135I dunno if I'm getting older or if this is as silly as it seems.
You don't like pre-live because it doesn't have parity with production, so you use a developers laptop? What???
I stopped reading at that point because that's pretty indicative of either a specific niche or a poorly thought out problem/solution set
Re: We don’t use a staging environment
#136If you are saying you don't have a staging environment, what you are really saying is that your company doesn't have any QA process. If your QA process is just developers testing their own shit on their local machine then you are not going to get as much value out of staging.
> If you are saying you don't have a staging environment, what you are really saying is that your company doesn't have any QA process.
Come on - this is nonsense.
Feature flags for example?
Re: We don’t use a staging environment
#137This is good insofar as it forces you to make local development possible. In my experience: it's a big red flag if your systems are so complex or interdependent that it's impossible to run or test any of them locally.
That leads to people only testing in staging envs, causing staging to constantly break and discouraging automated tests that prevent regression bugs. It also leads to increasing complexity and interconnectedness over time, since people are never encouraged to get code running in isolation.
Re: We don’t use a staging environment
#138Isn’t the concept of a single staging environment becoming a bit dated? Every recent project I’ve worked on uses preview branches or deploy previews, eg what Netlify offers https://docs.netlify.com/site-deploys/deploy-previews/ Or am I missing something?
I imagine you missed the same thing I did: the last update time.
April 1st, 2022
Re: We don’t use a staging environment
#139If you are saying you don't have a staging environment, what you are really saying is that your company doesn't have any QA process. If your QA process is just developers testing their own shit on their local machine then you are not going to get as much value out of staging.
> If you are saying you don't have a staging environment, what you are really saying is that your company doesn't have any QA process. Come on - this is nonsense. Feature flags for example?
Feature flag systems don’t magically prevent a new feature from causing a bug for other existing features, or even taking the whole site down.
Re: We don’t use a staging environment
#140Earlier quoted context omitted.
We often deploy to production directly because a customer wants a feature right now. I was thinking of changing the staging server to be called beta. Customers can use new features directly, but at their own risk.
Staging environments should be separate from production environments. If the Beta is expected to persist data in the long term, then it's not staging. Staging environments should be nukable. You don't want a messy Beta release to corrupt production data or to have customers trying to sue you if you reset staging. I don't know about your customer but wanting a feature yesterday may be a sign of some dysfunctional oper…
What I found with customers is that they really like it if they talk to you about a feature, and next week it's there, although it's a preview version of the feature. After that they forget about it a bit and you've got plenty of time to perfect it.