91–100 of 424 posts
The guy may be absentminded, but that is a trait of some of the brighest people on earth.
Something similar anyway (was deleting rows from production and hadn't selected the where clause of my query before I ran it).
It was on my VERY FIRST DAY of a new job.
Fortunately they were able to restore a several hours old copy from a sync to dev but the wasn't a real plan in place for dealing with such a situation. There could have just as easily not been a recent backup.
This was in a company with 1,000 employees (dev team of 50) and millions in turnover. I've worked other places that are in such a precarious position too.
At least my boss at the time took responsibility for it - new dev (junior), first day, production db = bad idea.
Earlier quoted context omitted.
I think it's a bit extreme to say he did more good than harm. He might have done some long-term good by having the company re-examine permissions and environments, but he probably did a lot of long-term harm by alienating current and future customers.
Better that it happened 2 months after backups were canceled than 6 months or later. If you're going to cancel your backups you're begging for disaster.
Some experiences are non-transferable. This identical conversation has taken place millions of times, but noooo: every penny-wise-pound-foolish CEO wants to experience the real thing, apparently.
Earlier quoted context omitted.
That is good advice, just make sure that it doesn't look like you are stealing your customers/clients data.
That is good advice, just make sure that it doesn't look like you are stealing your customers/clients data. Excellent point! Any tips on how to avoid that, other than not taking the data home / copying to personal Dropbox-type things?
I would document the backup process and communicate it to my manager/client with a mail like "hey, I set up backups, they are stored at , docs are in the wiki".
Other potential issues: causing unauthorized costs ("who stored 10TB on S3?") or privacy violations, e.g. when working with healtcare or payment data.
Earlier quoted context omitted.
We sell middleware to a number of customers with millions of dollars in revenue who don't have backups, don't have testbeds for rolling out to "dev" before pushing to "prod" and don't have someone with any expertise in managing their IT / infrastructure needs. My experience is that this is the norm, not the exception.
As a college student, this fucking horrifies me. Is there anyway I can guarantee I don't end up at someplace as unprofessional as this? I want to learn my first job not teach/lead.
Also, you have a choice of leaving if you don't like the job, and or don't find the practices in place to be any good, or you can fix them.
Earlier quoted context omitted.
We sell middleware to a number of customers with millions of dollars in revenue who don't have backups, don't have testbeds for rolling out to "dev" before pushing to "prod" and don't have someone with any expertise in managing their IT / infrastructure needs. My experience is that this is the norm, not the exception.
As a college student, this fucking horrifies me. Is there anyway I can guarantee I don't end up at someplace as unprofessional as this? I want to learn my first job not teach/lead.
More than anything else, this describes an appalling failure at every level of the company's technical infrastructure to ensure even a basic degree of engineering rigor and fault tolerance. It's noble of the author to quit, but it's not his fault. I cannot believe they would have the gall to point the blame at a junior developer. You should expect humans to fail: humans are fallible. That's why you automate.
News flash, If you are a CEO you should be asking this question: "How many people in this company can unilaterally destroy our entire business model?" If you are a CTO you should be asking this question: "How quickly can we recover from a perfect storm?" They didn't ask those questions, they couldn't take responsibility, they blamed the junior developer. I think I know who the real fuckups are. As an aside: Way back…
>If you are a CEO you should be asking this question: "How many people in this company can unilaterally destroy our entire business model?" This is a question that the person in charge of backups needs to think about, too. I mean, rephrase it as "Is there any one person who can write to both production and backup copies of critical data?" but it means the same thing as what you said. (and if the CTO, or whoever is in…
Which also means they can't fix something in case of a catastrophic event. "Recover a file deleted from ext3? Fix a borked NTFS partition? Salvage a crashed MySQL table? Sorry boss, no can do - my admin powers have been neutered so that I don't break something 'by accident, wink wink nudge nudge'." This is, ultimately, an issue of trust, not of artificial technical limitations.
> one case where outsourcing is very likely to be better than anything you could do yourself.
Hm. Your idea that "cloud is actually pixie dust magically solving all problems" seems to fail your very own test. Is there a way to prevent the outsourced admins from, um, destroying something when they are actively hostile? Nope, you've only added a layer of indirection.
(also, "rouge" is "#993366", not "sabotage")
More than anything else, this describes an appalling failure at every level of the company's technical infrastructure to ensure even a basic degree of engineering rigor and fault tolerance. It's noble of the author to quit, but it's not his fault. I cannot believe they would have the gall to point the blame at a junior developer. You should expect humans to fail: humans are fallible. That's why you automate.
Exactly what you said. Rigor is truly the right word to use here. Cancelling your db backups is basically asking for a disaster. I'm not sure i have ever been at job that didnt require a db backup for some reason at some time.
If there were only two junior folks, what were the senior folks doing?