Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We don’t use a staging environment

squeaky.ai

Post reply on HN
Print view View original

311–320 of 357 posts

Re: We don’t use a staging environment

#311

Post by aniforprez »

aniforprez wrote: post #291
drewcoo wrote: post #76

Earlier quoted context omitted.

> I've seen code fail in each of the stages How many of the failures caught in dev would have been legitimate problems in production? How about the ones in staging? If your environments are that different are you even testing the right things? And if yes, if you need all of those, then why not add a couple more environments? Because more pre-prod environments means more bugs caught in those, right? /s

What? What even is this The whole point of a dev environment is so you can catch errors you wouldn't in production. Keeping it as close as possible to production is the point. If you're catching errors in dev, you'll see them in production More environments don't catch more bugs. What is this corollary? More testing catches more bugs. A dev environment allows free integration testing without act users of production b…

I believe they edited the post to indicate sarcasm after I saw it which is a relief. I got pretty fired up

Re: We don’t use a staging environment

#313

Post by jillesvangurp »

WYepQ4dNnG wrote: post #17

I don't see how this can scale beyond a single service. Complex systems are made of several services and infrastructure all interconnected. Things that are impossible to run on local. And even if you can run on local, the setup is most likely very different from production. The fact that things work on local give a little to zero guarantees that they will work in prod. If you have a fully automated infrastructure set…

The problem with staging environments is that replicating the functionality is easy but replicating the data, interactions, and behavior of people in a real environment is not. It's better to think in terms of early access releases and some kind of controlled roll out of new software so you catch bugs and issues before they impact most of your users.

I've seen many projects where the staging environment is a bad joke and where most real testing happens in production anyway. These days alternative strategies are being more clever about how you work with rolling out software to your production environments. There are various ways of doing this but it always boils down to having both the old and the new software running in the same environments and controlling who gets to see what using feature flags, dns, routing, etc. Also, if you run any kind of AB tests, this is what you would need. I've seen some companies do that but mostly this is more of an aspirational thing than an actual thing of course.

For the SAAS company I'm a CTO of, I actually stumbled on a nice mechanism when I realized that our customers' desire for dedicated setups lead us to a natural state where we update those last, thus making our multi-tenant environment a natural place to test / provide early access. Likewise our webapp rolls out immediately from our master branch but we package it up for Android/IOS less frequently because of the release bureaucracy Apple and Google impose. So that branch effectively is our stable release. And we have a matching web server for that branch as well that updates only when we merge to our production branch. The other server uses the same infrastructure (database, redis, etc.) but updates straight from our master branch. So, our staging server is part of our production environment and serves the same data, is exposed to the same user behavior, etc.

That also makes it easier to verify that old and new client software needs to work with both our latest server as well as the production servers for our dedicated setup.

Re: We don’t use a staging environment

#315

Post by skerit »

I currently have this with a client. When I was the only backend developer, the staging server setup worked perfectly because it was basically the master branch + 1 or 2 queued changes. Now that other people have joined the mix it's become more of a nuisance.

Re: We don’t use a staging environment

#316

Post by xorcist »

tharkun__ wrote: post #51
fishtoaster wrote: post #12

This is a pretty weird article. Their "how we do it" section lists: - "We only merge code that is ready to go live" - "We have a flat branching strategy" - "High risk features are always feature flagged" - "Hands-on deployments" (which, from their description, seems to be just a weird way of saying "we have good monitoring and observability tooling") ...absolutely none of which conflict with or replace having a stagi…

FWIW I don't think it is weird at all. Maybe a little short on details of what ready really means for example. While I don't think going completely staging-less makes a lot of sense, going without a shared staging environment is a good thing. It is absolutely awesome to be able to have your own "staging" environment for testing that is independent of everyone else. With the Cloud this is absolutely possible. Shared s…

> "staging" environment for testing that is independent of everyone else

That's not usually what people mean by staging. Staging is a type of pre-production test environment where several different features that are continously developed by different teams can be tested together. Third party integrations such as logistics, ordering and payment systems can also have their integration testing here.

> Shared staging environments are really bad

That sounds dangerously close to "testing is hard, let's go shopping". That it can be a logistical challenge to test code that touches many parts of a multi stakeholder system does not mean we shouldn't do it.

Having to wait weeks to test a feature sounds like the process has broken down, not that the process is unnecessary.

Re: We don’t use a staging environment

#318

Post by dimgl »

ohmanjjj wrote: post #65

I’ve been shipping software for over two decades, built multiple successful SaaS companies, and have never in my life written a single unit test.

If it's an environment I'm comfortable with, writing unit tests is faster than testing by hand.

If it's not an environment I'm comfortable with... dear lord it is absolutely brutal to write any kind of test.

Re: We don’t use a staging environment

#319

Post by butlerm »

freedomben wrote: post #190
_3u10 wrote: post #177

Earlier quoted context omitted.

dev workstations should cost at least $4000. Like how much productivity is being wasted because their machine is slow. $4000 workstations are cheap compared to staging.

when I worked for big corp, the reason we were told in engineering for getting $1,000 laptops was that it wasn't fair to accounting, HR, etc for us to have better machines. In the past people from these departments complained quite a bit. The official reason (which was BS) was "to simplify IT's job by only having to support one model"

Who cares what is "fair"? A decision like that should be based on an elementary productivity calculation. If not the inmates have taken over the asylum.

Re: We don’t use a staging environment

#320

Post by makach »

What works for you works for you. If you can't have a staging environment you obviously found a work around. There are many ways to deploy. Basically, you decide what risk you want to accept when you define a lifecycle.
Post reply on HN