271–280 of 357 posts
This is pretty common actually At Facebook too there was no staging environment. Engineers had their dev VM and then after PR review things just went into prod That said features and bug fixes were often times gated by feature flags and rolled out slowly to understand the product/perf impact better This is how we do it at my current team too…for all the same reasons that OP states
Earlier quoted context omitted.
> company that deploys directly from a developer's laptop to production Luckily, there's no sign of doing that here. There's no mention of how their CI/CD works, probably because it's out of scope for an already long article, but that's clearly happening.
"We only have two environments: our laptops, and production. Once we merge into the main branch, it will be immediately deployed to production." Maybe my reading skills have completely vanished but to me, this exactly says they deploy directly from their developers' laptops to production. Those are literally the words used. The rest of the article goes on to defend not having a pre production environment. They litera…
Good monitoring, logs, metrics, feature flagging (allowing for opening a branch of code for a % of users), blue/green deployment (allowing a release to handle a % of the user's traffic) and good tooling for quick builds/releases/rollback, in my experience, are far better tools than intermediate staging environments. I've had great success in the past with a custom feature flags system + Google's App Engine % based tr…
And yes, you need blue/green deployments in addition to feature flags, as it is not easy to feature flag certain things, such as a language runtime version update or a third party library upgrade, among many other things.
This article has some very weird trade-offs. They can't spin up test environments quickly, so they have windows when they cannot merge code due to release timing. They can't maintain parity of their staging environments with prod, so they forswear staging environments. These seem like infrastructure problems that aren't addressing the same problem as the staging environment eo ipso. They're not arguing that testing o…
> They're not arguing that testing or staging environments are bad, they're just saying their organization couldn't manage to get them working. That is exactly what I got from reading this article. Their staging process was poorly set up and they simply abandoned ship. Additionally, I was getting poor software culture vibes.
Similarly, "Poor ownership of changes" and "People mistakenly let process replace accountability" just don't seem staging-related to me. I've been in environments where people throw code over the fence straight into production.
I don't think you can take infrastructure seriously without a staging environment. For many companies that is fine - they don't have significant infrastructure to maintain (or just don't maintain the infrastructure they have). I work on a team that maintains our database layer and the lack of a staging environment is incredibly painful. Every test has to be done in production and massive effort needs to be taken to p…
I don't think you can take infrastructure seriously without a staging environment. For many companies that is fine - they don't have significant infrastructure to maintain (or just don't maintain the infrastructure they have). I work on a team that maintains our database layer and the lack of a staging environment is incredibly painful. Every test has to be done in production and massive effort needs to be taken to p…
imho this only applies for teams with average or below average devs
We used to believe staging environments are not important enough. If you believe that then I would argue that you have not crossed a threshold as an org where your product is critical enough for you consumers. The staging environment or any for that matter just acts as a gating mechanism to not ship crappy stuff to customers. You cannot have too many gates, then you would be shipping lates but with less number of gat…
This reminds me of the "bake time" arguments I've had. There's some magical idea that if software "bakes" in an environment for some unknowable amount of time, it will be done and ready to deploy. Very superstitious. what is the actual value gained from staging specifically? Once you have a list of those, a specific list, figure out why only staging could do that and not testing before or after. And "it's caught bugs…
Earlier quoted context omitted.
This is how my current place does it. The only issue we are having is library / dependency updates have a tendency to work perfectly fine locally and then fail in production due to either some minor difference in environment or scale. It's a problem to the point that we have 5 year old ruby gems which have no listed breaking changes because no one is brave enough to bump them. I had a go at it and caused a major prod…
Do you have a replicate of the production environment codified somehow, like into a VM? It's rarely perfect, but I usually try and develop locally on the same stack I deploy to which can help with the environment differences. It's also why I think it's smart to rebuild the environment on deploy if it makes sense for your pipeline, so that you wipe any minor differences that have been accruing over time. Working on a…
No disrespect but you can do this for an analytics dashboard or a content web site with canaries (Facebook). May not be the best for high liability sites like financial systems.
People hold up banking as the pinnacle of serious and responsible high-quality high-reliability software engineering and operations, but my bank is the only web property or mobile app I use that's routinely unavailable for hours at a time.
Earlier quoted context omitted.
Multiply that by the number of employees who need it ... staging is cheaper.
Multiply that by the lost hours waiting for compilation, etc and whether you have staging or not doesn’t matter.