Earlier quoted context omitted.
Whenever this happens to someone it's always a horrible feeling where one feels very guilty and ashamed no matter what people say to you and unfortunately mistakes like these are almost the bread and butter of any extremely experienced grey beard so it's kind of normal that something like this happens to someone at some point sooner or later. The only people who never make costly mistakes are those who were never tru…
> the cost which often gets quoted with those mistakes is not a real cost Oh that depends entirely on the industry.. in social media maybe not, in banking and Fintech those can most certainly be real costs. And can tell you - that feels even worse
Why you've never been in a plane crash
141–150 of 310 posts
Re: Why you've never been in a plane crash
#142This concept of a blameless culture reminds me of one time when I was talking to a SWE at Facebook around 2010. I don’t know whether the story is actually true or just folklore, but apparently someone brought down the whole site on accident once, and it was pretty obvious who did it. Zuckerberg was in the office and walked up to the guy and said something along the lines of “Just so you are aware, it would probably t…
Re: Why you've never been in a plane crash
#143Earlier quoted context omitted.
There are several ways we could write a list of best practices. But the simplest would be to simply attach a financial cost to leaking any personal data to the open internet. This is essentially how every other industry already works: If my building falls down, the company which made it is financially liable. If I get sick from food poisoning, I can sue the companies responsible for giving me that food. And so on. We…
Software is in a very unique position. It can be attacked all the time with completely impunity. Buildings are completely vulnerable even to simple hammer breaking windows, locks have been basically broken for decades Food processor is easily vulnerable to poisoned potato. Only the software has to take all attacks humans can come up with and withstand it. What other industry has to deal with that? Maybe military, arm…
Yes, things are constantly evolving in computer engineering, but how many of the failures that make the news are caused by novel new attack vectors, versus not following best-practices?
Re: Why you've never been in a plane crash
#144In "blameless" there's no personalities mentioned at all but in reality personality and individuals matter. EG if we had an outage because I put code into production w/o review, yes it's a systemic failure but it's also something about me, something about my manager, etc. By talking about those things as well, you make it easier for both the system and the individuals to evolve.
Re: Why you've never been in a plane crash
#145Survivorship bias?
Re: Why you've never been in a plane crash
#146Earlier quoted context omitted.
> the Captain of the Titanic ... Any decent accident investigation should highlight that the Captain was under strict instructions from the ship owners to break a record, wealthy influential owners that could and would destroy his career if he failed to push the ship. Good investigations identify causes with a view to prevent repetition of circumstances. Cases like these (Titanic, UK Post) strengthen the case for whi…
Among other things, the investigation into the Titanic accident gave us the requirement of having enough life boats for everyone and standards in how to evacuate a vessel. Captains being pressured by mamagement / organization to do hazardous stuff is a thing still, in aviation as well as in seafaring.
Unfortunately the thing most people remember (and which you highlighted) is life boats and, perhaps those are actually a bad idea, at least for most ships.
Here's how that goes: SOLAS requires life boats, but almost always you won't use them. Titanic is a rare example of a situation where life boats are very useful, an ocean linear breaks apart in the middle of the ocean. In most cases you're not very far from land, and so almost always you just get the people onto land and maybe the ship is damaged/ destroyed or maybe not, that's just stuff and it's insured. Fire? Control the fire, go to port. Hole in the ship? Pumps control sinking, go to port. Engine failure? Tow the ship to port. So in all these cases you don't use the life boats, doing so is basically a last resort.
But, even though you would very rarely need them, likely never for a vessel which operates close to shore, they must be maintained periodically because SOLAS, and maintenance of lifeboats is pretty dangerous because they're on the outside of a ship. So you may end up killing or seriously injuring more people by having lifeboats.
Re: Why you've never been in a plane crash
#147Earlier quoted context omitted.
Whenever this happens to someone it's always a horrible feeling where one feels very guilty and ashamed no matter what people say to you and unfortunately mistakes like these are almost the bread and butter of any extremely experienced grey beard so it's kind of normal that something like this happens to someone at some point sooner or later. The only people who never make costly mistakes are those who were never tru…
> the cost which often gets quoted with those mistakes is not a real cost It is still money they would have made that now weren't made. It is very important to explain to people how much value is lost during these events so that we also correctly value the work to prevent such events in the future.
Some usage (sales, ad views, whatever) will be delayed, some usage will be done somewhere else, some usage will be abandoned.
But costs are likely down too. If there's any paid transit, that usually drops during an incident. If you're paying for electricity usage, that drops too.
And significant outages can generate news stories which can lead to more usage later. Of course, it can also contribute to a brand of unreliability that may deter some use.
Re: Why you've never been in a plane crash
#148This concept of a blameless culture reminds me of one time when I was talking to a SWE at Facebook around 2010. I don’t know whether the story is actually true or just folklore, but apparently someone brought down the whole site on accident once, and it was pretty obvious who did it. Zuckerberg was in the office and walked up to the guy and said something along the lines of “Just so you are aware, it would probably t…
First of all, the "expensive training now you'll be the guy we trust the most blabla" story has been around for decades. Second, if this one is true it's a terrible way to phrase/present it.
The other story you are referencing is "this was an expensive education in which you have learnt to not do stupid stuff".
This is framed as "this was an expensive learning opportunity for us to learn that we have a gap in our systems that allowed this downtime to happen".
These are different sentiments! To me the above quote is very explicitly the latter and directly refutes the notion of "this was expensive training for you" by stating that it's impossible for an individual to apply that learning in a way that would recoup the loss.
Re: Why you've never been in a plane crash
#149This concept of a blameless culture reminds me of one time when I was talking to a SWE at Facebook around 2010. I don’t know whether the story is actually true or just folklore, but apparently someone brought down the whole site on accident once, and it was pretty obvious who did it. Zuckerberg was in the office and walked up to the guy and said something along the lines of “Just so you are aware, it would probably t…
Look to your left, look to your right, count the heads. Now divide the money that was lost through the number of heads. This is the theoretical ceeling- how much you could make if there were no shareholders and you had your own company - or had a union.
Re: Why you've never been in a plane crash
#150This concept of a blameless culture reminds me of one time when I was talking to a SWE at Facebook around 2010. I don’t know whether the story is actually true or just folklore, but apparently someone brought down the whole site on accident once, and it was pretty obvious who did it. Zuckerberg was in the office and walked up to the guy and said something along the lines of “Just so you are aware, it would probably t…