Ask HN: Testing Code Changes Against Production

#1

Post by psp219 » Tue, Jul 25, 2023, 5:35 AM UTC

I’m currently reaching out here to see if anybody has any advice on testing code prior to pushing to production. I write unit / integration tests to make sure code is as correct as possible, but I don’t have any way of actually testing it against real, production traffic to know for sure.

Writing a unit / integration test for every potential edge case doesn’t seem scalable - wondering if there’s a way to test code changes directly against production traffic without exposing users to potentially breaking changes?

A canary comes to mind but serving real production traffic would expose users to potentially breaking changes?

How do you guys handle this at your companies? Would appreciate any thoughts.

Re: Ask HN: Testing Code Changes Against Production

#2

Post by ssss11 » Tue, Jul 25, 2023, 10:34 AM UTC

I suppose it depends what type of code you’re developing but in general I’d expect a dev environment and unit tests, then a test environment and integration tests, a staging environment and UAT (or UAT in Test) and then production.

Test or staging would usually have a copy of old prod data.

It sounds as though you can’t predict the prod traffic though. Is there no sample traffic or even stats on types and volume of traffic over a period of time you can take to then feed to your test environment?

Re: Ask HN: Testing Code Changes Against Production

#3

Post by ericalexander0 » Tue, Jul 25, 2023, 1:45 PM UTC

There is no one size fits all answer. It depends on the tolerance for failure in the system. If testing in production stays within the tolerance, then it's likely more efficient to test there. If not, then it's more efficient to test pre-prod.