I'm getting internal server error.
Me too. Here's the cached version: http://webcache.googleusercontent.com/search?q=cache:http://...
How I Fired Myself
51–60 of 424 posts
Re: How I Fired Myself
#52Whoever cancelled the backups was equally responsible
This. Times 1000. I would even go as far as to fire the CTO because if the data is your bread and butter, the you protect the data.
Protecting the data is called a disaster recovery (DR) plan in those big outdated companies that people like to make fun of.
The reason that these companies have a DR plan is to tell the CEO that 'when' (not 'if') all of the data goes away, (a) how long will it take to get it back and (b) how out-of-sync will this data be (2 minutes from freshness? 6 hours?).
Re: How I Fired Myself
#53You did them more good than harm. 1) Not having backups is an excuse-less monumental fuckup. 2) Giving anyone delete access to your production db, especially a junior dev through a GUI tool, is an excuse-less monumental fuckup. Hopefully they rectified these two problems and are now a stronger company for it.
I think it's a bit extreme to say he did more good than harm. He might have done some long-term good by having the company re-examine permissions and environments, but he probably did a lot of long-term harm by alienating current and future customers.
Better that it happened 2 months after backups were canceled than 6 months or later. If you're going to cancel your backups you're begging for disaster.
Re: How I Fired Myself
#54I would love to see some reflection on this story from OP. What do you think you learned from this experience? Do you think your response was appropriate? What would you have done differently? Are you forever afraid of Prod env? Many, many , many of us have been in this situation before, whether as 'monumental' or not. So it is interesting to hear how others handle it.
OP here.
I realize that the dev environment was a recipe for disaster, and I was simply the one to step on the mine .. but I believe my guilt about leaving the company is 'quite right'. Thankfully I'm not forever afraid of Prod env - I still do a lot of risky stuff .. but I always have nightly backups, and other 'recreate the data' strategies in place.
Re: How I Fired Myself
#55This was a "sword of Damocles" situation. No backups, no recovery plan, and now clue how important any of these things were.
A thousand things can make an SQL table unreadable. "What do we do when this happens" is what managers are for, not finding someone to blame for it.
Re: How I Fired Myself
#56I find it disgusting that the "game designers" are the so called overlords. Fuck them. If you're a developer and a gamer then you're practically a game designer. What ever "education" they had is bullshit. You can go from imagination to reality with just you alone. And perhaps an artists to do the drawing. All those "idea" fuckers a.k.a game designers are just bullshitters.
And yeah this wasn't your fault. It was the CTO's fault. He holds responsibility.
"They didn't ask those questions, they couldn't take responsibility, they blamed the junior developer. I think I know who the real fuckups are."
Re: How I Fired Myself
#57If you ever notice that your employer or client isn't backing up important data, take a tip from me: do a backup, today, in your free time, and if possible, and again in your free time, create the most basic regular backup system you can.
When the time comes, and someone screws up, you will seem like a god when you deliver your backup, whether it's a 3-month-old one-off, or from your crappy daily backup system.
Re: How I Fired Myself
#58News flash, If you are a CEO you should be asking this question: "How many people in this company can unilaterally destroy our entire business model?" If you are a CTO you should be asking this question: "How quickly can we recover from a perfect storm?" They didn't ask those questions, they couldn't take responsibility, they blamed the junior developer. I think I know who the real fuckups are. As an aside: Way back…
Well said!
Re: How I Fired Myself
#59Whoever was your boss should have taken responsibility. Someone gave you access to the production database instead of setting up a proper development and testing environment. For a company doing "millions" in revenues, it's odd that they wouldn't think of getting someone with a tiny bit of experience to manage the development team.
Even worse, they didn't have a backup.
Re: How I Fired Myself
#60You did them more good than harm. 1) Not having backups is an excuse-less monumental fuckup. 2) Giving anyone delete access to your production db, especially a junior dev through a GUI tool, is an excuse-less monumental fuckup. Hopefully they rectified these two problems and are now a stronger company for it.
I think it's a bit extreme to say he did more good than harm. He might have done some long-term good by having the company re-examine permissions and environments, but he probably did a lot of long-term harm by alienating current and future customers.
There's an argument to be made that the company is doing harm to customers just by existing in such a precarious state. Anything that forces the technical leadership of the company to do the right thing or fail completely is actually better for customers in the long term.