The blame should be on whoever decided a junior engineer should be given permission to delete the production database on day one. One time at my first job my boss accidentally deleted a staging database, and even he said he never should have had permission to do that.
Accidentally destroyed production database on first day of a job
11–20 of 31 posts
Re: Accidentally destroyed production database on first day of a job
#12Previous post on HN: https://news.ycombinator.com/item?id=14476421
I didn't notice until now this is 3 years old. I wonder what ended up happening; I hope he/she is okay.
Re: Accidentally destroyed production database on first day of a job
#13The experience of reading this on reddit today versus what it was like when this was originally posted is astonishing. The comment are what make this post valuable and the view to them is abysmal.
Try using old.reddit.com:
https://old.reddit.com/r/cscareerquestions/comments/6ez8ag/a...
(And there are browser extensions that will turn all reddit links into old.reddit.com links).
Re: Accidentally destroyed production database on first day of a job
#14Lots of red flags there:
- Dev installation guides with credentials to prod.
- Tests that delete everything, not just what they create.
- The obvious of giving access to delete production on the first day.
- Where they lacking backups?
Re: Accidentally destroyed production database on first day of a job
#15Lots of red flags there: - Dev installation guides with credentials to prod. - Tests that delete everything, not just what they create. - The obvious of giving access to delete production on the first day. - Where they lacking backups?
Seems from the post that there were backups but they had never actually tested restoring from them.
Re: Accidentally destroyed production database on first day of a job
#16Maybe this shouldn't be posted here, but any new employee at my company is required to see this and because my company is small, typically everyone watches as well to see his reaction: https://www.youtube.com/watch?v=1aEqd4bl6Bs (subtitles required).
Re: Accidentally destroyed production database on first day of a job
#17For us, one of our junior Devs managed to wipe out all environments, all datacenters for one of our Elastic instances. They got handed a task to modify an index. The Dev Lead and Sr. Devs were 'too busy' and she stack overflowed how to do it and it was rubber stamped. What she did worked - but it wipes out all the existing documents when she dropped and recreated the indexes. Issue was it was a large enough collectio…
Nice story.
Your dev lead/senior engineer should have never been "too busy" for letting someone do that on a first day unsupervised
Re: Accidentally destroyed production database on first day of a job
#18I fondly remember accidentally deleting around half of a warehouse's bulk inventory once during a previous job; I was trying to wipe out test data from a test database, and stupid Azure Data Studio was showing the IP address of the test DB server in the status bar even though the active tab was using a prod connection.
Luckily we were pulling full snapshots of the relevant tables into CSV exports (for ingestion into Redshift for reporting), so it was just a matter of grabbing the most recent of those, reinserting the data, and having some of the warehouse workers do some cycle counts to spot check. Still was a nerve-wracking and awkward conversation with my boss, though, lol
Re: Accidentally destroyed production database on first day of a job
#19Any chance this story was made up?
Re: Accidentally destroyed production database on first day of a job
#20Any chance this story was made up?
If not, I was imagining this could be an elaborate scam. Although the clues seem to point that it was an in-person job, I can imagine in today's climate "hiring" people remotely and pulling a scam like this. The result being to get some cash payment from the victim to avoid legal trouble.