I've never done much testing in production. A long time ago I was too lazy to put my website into git, so I would just ssh into the webserver and edit the HTML files with mg. Not particularly productive or enjoyable, honestly. I am sure the search engines also liked index.html~ being very similar to index.html; was also too lazy to turn off backup files ;)
My priorities with production are getting as much information recorded as possible; if there is ever a bug that occurs and isn't detected by monitoring and debuggable by looking at the telemetry, that's a big problem that is a priority to fix. It is always a work in progress, but something that you can chip away at gradually over time. (Add them as postmortem action items.)
The provided articles mention weird quirks that only happen in production, like network card firmware issues that drop a particular bit pattern. I've definitely seen things like this (at a higher level); I add the bit patterns to my test suite and make the test suite runnable as an "application" in the production environment and then collect my data. As for straight-up hardware problems, that's happened exactly once in my career. I used to maintain a several-thousand replica application; one day one replica was crash looping. I looked at the stack traces, different each time, and couldn't figure out what was possibly wrong with the code. A nearby coworker suggested "just restart that replica with --avoid_parent to schedule it on a different machine". The problem went away and never came back. Shrug. Sometimes the computer doesn't faithfully run the instructions that you put into memory, but it is pretty rare. Detect it and remove the faulty computer, I guess.
For less quirky things, I like the ability to simulate resource constraints, rather than trying to run into them with physical hardware. For example, it's pretty hard to write a load test that makes S3 slow, but it's pretty easy to hack up Minio to sleep for a second every MB of data and now your load tests can see what blows up when S3 is slow. Then you can edit your code to be resilient against that. (etcd on low iops disks has also been a problem in my work; that is easy enough to simulate without changing the code, cgroups provides a mechanism. Now you don't actually have to generate enough load to make your disk slow.) Adjusting network latency with "tc qdisc add dev X netem ..." has also been useful for debugging slow file uploads over high-latency links without actually going through the hassle of renting a server far away to upload things to. I will say the disadvantage there is that the less you know about the full stack, the less you trust your simulations. You'll end up with a lot of pushback along the lines of "that's not a real scenario", and it is true that calling Write() slowly versus the OS not returning from the write() syscall because the disk is busy is a slightly different codepath and there can always be side effects that you're missing. But often the black box model is a worthwhile tradeoff for improved development cycle times; just make sure you add the instrumentation to real production so you can get data about how good your simulation is.
I'm willing to use error/latency budget for unusual production deployments to collect real-world data, for example, running 10% of requests through a build with the race detector enabled. That now accounts for your worst 10% response latency (and errors if you do have data races in hot paths!), but if it's within the budget, it's worth it because you get a stack trace pointing at a critical correctness error in your code, and you can go add that case to your unit tests and never have the problem again. Sometimes you can't think of everything, which is why telemetry from production is so important to me. (This kind of data is important for more than just the mechanics of the code, of course. Talk to your users and see if they like the new icon set. If they don't, your test in production failed and you should fix your app.) Finally, I also like fuzz testing on top of all of this; have a beefy computer generating the most corrupt possible data billions of times a second and see how your app behaves. Every fuzz test I've ever written has exposed a boneheaded subtle mistake in the code, even in code with 100% test coverage.