Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We deleted the production database by accident

keepthescore.co

Post reply on HN
Print view View original

331–340 of 456 posts

Re: We deleted the production database by accident

#331

Post by qz2 »

rawoke083600 wrote: post #326
qz2 wrote: post #312

Earlier quoted context omitted.

That's a tad ironic is it not? All I am calling for is people to take responsibility.

What is the future value in that ? From a system and reliability point of view ? Genuine question - not trying to be a d k

The point is that if you take responsibility then you're taking pride in your work, are invested in it and willing to invest in self-improvement and introspection rather than doing the minimum to get to a minimum viable solution. The outcome of this is an increase in quality and a decrease in risk.

Re: We deleted the production database by accident

#332

Post by ClumsyPilot »

vox17 wrote: post #315

This line's the winner for me : "Thankfully nobody’s job is at risk due to this disaster. The founder is not going to fire the developer – because they are one and the same person."

To be fair he could, and least in theory - he could get someone else to do development, for money or equity, for the project and do something else himself.

Re: We deleted the production database by accident

#333

Post by Jugurtha »

neillyons wrote: post #265
t0mas88 wrote: post #243

Earlier quoted context omitted.

The software sector needs a bit of aviation safety culture: 50 years ago the conclusion "pilot error" as the main cause was virtually banned from accident investigation. The new mindset is that any system or procedure where a single human error can cause an incident is a broken system. So the blame isn't on the human pressing the button, the problem is the button or procedure design being unsuitable. The result was a…

This sounds quite interesting. Any books you could recommend on the "pilot error" topic.

"The Checklist Manifesto", by Atul Gawande, dives into how they looked at other sectors such as aviation to improve healthcare systems, reduce infections, etc. Interesting book.

Re: We deleted the production database by accident

#334

Post by Agentlien »

kungtotte wrote: post #263
Agentlien wrote: post #228

Earlier quoted context omitted.

Sometimes ambulances are occupied and taking a taxi goes faster. Especially if it's something which isn't immediately life threatening. I once dislocated my shoulder while on a large trampoline and was unable to get up from my hands and knees due to the intense pain whenever the trampoline wobbled. The ambulance was redirected to more serious injuries three times. I was stuck in that position waiting for two hours be…

Sure, that's true. In that scenario it would also be appropriate to wait for a driver to sober up before driving you to the hospital if neither ambulance nor taxi were available (or delayed). One glass of wine would be out of most people's systems after two hours. Thus poking hole in the "drunk drive someone to the hospital" argument, which is what this was all about in the first place.

I did argue, in my original comment, that drunk driving should not be an option. I certainly stand by that. My original comment also mentioned a taxi, to which you replied about ambulances.

In my previous comment I just meant that sometimes ambulances can take a good while and a taxi might not.

In the unfortunate case of the trampoline there were several sober people with driver's license and cars available and a taxi would have been there immediately.

Unfortunately,they failed to get me out of there, meaning I still had to wait until an ambulance was available. It was beyond painful and exhausting both physically and mentally. But it was still technically not an emergency.

Re: We deleted the production database by accident

#335

Post by t0mas88 »

quietbritishjim wrote: post #324
janoc wrote: post #269

Earlier quoted context omitted.

It is not only that but also realizing that there is never a single cause to an accident or incident. Even when it was a suicidal pilot flying the plane into a mountain on purpose. Someone had to supervise him (there are two crew members in the cockpit for a reason), someone gave him a medical, there is automation in the cockpit that could have at least caused an alarm, etc. So even when the accident is ultimately ca…

> Even when it was a suicidal pilot flying the plane into a mountain on purpose. Someone had to supervise him (there are two crew members in the cockpit for a reason), someone gave him a medical, there is automation in the cockpit that could have at least caused an alarm, etc. There was indeed a suicidal pilot that flew into a mountain, I'm not sure if you were deliberately referencing that specific time. In that cas…

And as a result of that incident the procedures have changed, now a cabin crew member (or relief pilot in long haul ops) joins the other pilot in the cockpit if one has to go to the bathroom.

A similar procedure already exists for controlled rest in oceanic cruise flight at certain times, using the cabin crew to ensure the remaining pilot was checked to be awake every 20 minutes.

Re: We deleted the production database by accident

#336

Post by t0mas88 »

neillyons wrote: post #265
t0mas88 wrote: post #243

Earlier quoted context omitted.

The software sector needs a bit of aviation safety culture: 50 years ago the conclusion "pilot error" as the main cause was virtually banned from accident investigation. The new mindset is that any system or procedure where a single human error can cause an incident is a broken system. So the blame isn't on the human pressing the button, the problem is the button or procedure design being unsuitable. The result was a…

This sounds quite interesting. Any books you could recommend on the "pilot error" topic.

It's part of the Human Performance subject in getting an ATPL (airline license), it was one of the subjects that I didn't hate as much when studying. You can probably just buy the book on Amazon, they're quite accessible.

Re: We deleted the production database by accident

#337

Post by rawoke083600 »

qz2 wrote: post #331
rawoke083600 wrote: post #326

Earlier quoted context omitted.

What is the future value in that ? From a system and reliability point of view ? Genuine question - not trying to be a d k

The point is that if you take responsibility then you're taking pride in your work, are invested in it and willing to invest in self-improvement and introspection rather than doing the minimum to get to a minimum viable solution. The outcome of this is an increase in quality and a decrease in risk.

Wow - "...invest in self-improvement and introspection.."

I would hate for that to be our system reliability improvement methodology.

Ok fine now I'm being slightly a "dk" but really ?

Re: We deleted the production database by accident

#338

Post by t0mas88 »

watwut wrote: post #262
t0mas88 wrote: post #243

Earlier quoted context omitted.

The software sector needs a bit of aviation safety culture: 50 years ago the conclusion "pilot error" as the main cause was virtually banned from accident investigation. The new mindset is that any system or procedure where a single human error can cause an incident is a broken system. So the blame isn't on the human pressing the button, the problem is the button or procedure design being unsuitable. The result was a…

Yeah, but "pilot was drinking alcohol" would be considerate issue, would lead to fired pilot and would lead to more alcohol testing. I understand what you are taking about, but aviation has also strong expectations on pilots.

Believe it or not, even "pilot is an alcoholic" is still part of the no blame culture in aviation. As long as the pilot reports himself he'll not be fired for that. Look up the HIMS program to read more details.

Re: We deleted the production database by accident

#340

Post by ricksharp »

Are you sure it was the production database that was affected?

If you are not sure how a hard coded script that was targeting localhost affected a production database, how do you know you were even viewing the production database as the one dropped?

Maybe you were simply connected to the wrong database server?

I’ve done that many times - where I had an initial “oh no“ moment and then realized I was just looking at the wrong thing, and everything was ok.

I’ve also accidentally deployed a client website with the wrong connection string and it was quite confusing.

In an even more extreme case: I had been deploying a serverless stack to the entirely wrong aws account - I thought I was using an aws named profile and I was actually using the default (which changed when I got a new desktop system). I.e. aws cli uses —profile flag, but serverless cli uses —aws-profile flag. (Thankfully this all happened during development.)

I now have deleted default profiles from my aws config.

Post reply on HN