Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accountability Sinks

250bpm.substack.com

Post reply on HN
Print view View original

251–260 of 407 posts

Re: Accountability Sinks

#251

Post by jldugger »

satisfice wrote: post #24

I am deeply suspicious of "blameless" post mortems. I agree that we should work in ways that minimize fear. We should, to some degree, celebrate the learning we glean from our failures. But I keep seeing "blameless" being construed as lying about why something happened. It's construed in such as way that anyone can hide from their misdeeds. People screw up, and we need to hold them accountable, and THEY need to hold…

I'm nearly wrapping up Sydney Dekker's book _Just Culture_, and Allspaw has a few pages in it. And preceding that is a section titled "blame-free is not accountability-free."

Accountability under Dekker's restorative justice model means providing a complete record of what happened, so the justice system can focus on who was harmed and who needs to repair that harm. In some ways I think they can end up mirroring the typical punitive justice system, when the person who needs to repair harm matches what we would call a guilty party in other circumstances. But the idea is not to lie about what happened! It's to expand the network of causality beyond a simple thought terminating "Bob did it" so we can address the systemic problems that led to Bob doing the wrong thing.

> Not necessarily with "punishment" (what does that even mean in a professional context)

A few options depending on profession:

1. Demotion 2. Pay cuts or fines 3. Firing 4. Loss of certification, thus preventing this person from ever working in the field again 5. Jail time, preventing this person from even being in society for some time, perhaps forever.

Dekker's book is full of examples of professionals facing all of the above consequences. If you don't think these punishments are applied to the SRE community Allspaw addressed when originally describing "blameless postmortems" then you probably want to read the all time highest upvoted post to /r/cscareerquestions, "Accidentally destroyed production database on first day of a job, and was told to leave, on top of this i was told by the CTO that they need to get legal involved, how screwed am i?"[1]

[1]: https://www.reddit.com/r/cscareerquestions/comments/6ez8ag/a...

Re: Accountability Sinks

#252

Post by Wobbles42 »

finnh wrote: post #162
diognesofsinope wrote: post #122

Earlier quoted context omitted.

> "we've implemented all best practices, contracted out the hard parts to world-renowned experts, and had third party audits to verify that - there was nothing more we could do, therefore it's not our fault" The amount of (useless) processes/systems at banks I've seen in my career that boil down to this is incredible, e.g. hundreds of millions spent on call center tech for authentication that might do nothing, but th…

What's funny is that checklists in hospitals have been shown, empirically, to be massive life-saving devices. cyber perhaps not so much...

Checklists solve the problem of forgetting specific details. They work very well in situations where all possible problems have been enumerated and the only failure mode is forgetting to check for one.

They do not solve the problem of getting people to think things through and recognize novel issues.

There are some jobs you can't do well. You can do them adequately or screw them up. Checklists are helpful in those jobs.

Re: Accountability Sinks

#253

Post by Wobbles42 »

lucianbr wrote: post #155
TeMPOraL wrote: post #66

My go-to example of a whole mesh of "accountability sinks" is... cybersecurity. In the real world, this field is really not about the tech and math and crypto - almost all of it is about distributing and dispersing liability through contractual means. That's why you install endpoint security tools. That's why you're forced to fulfill all kinds of requirements, some of them nonsensical or counterproductive, but necess…

I wonder what the difference is between cybersecurity and civil aviation safety. At a glance they both have a lot of processes and requirements. Somehow on one side they are as you said, a way to deal with liability without necessarily increasing security, while on the other safety is actually significantly increased.

Aviation safety is mostly about learning from past experience. You mitigate known hazards that, once mitigated, stay mitigated.

Cybersecurity is about adversarial hazards. When you mitigate them they actively try to unmitigated themselves.

It is more analogous to TSA security checks than to FAA equipment checklists. The checklist approach can prevent copycats from repeating past exploits but is largely useless for preventing new and creative problems.

Re: Accountability Sinks

#254

Post by atoav »

t_luke wrote: post #153

The conclusion of Davies' second extract — about e.g. being bumped off a flight — is recognisable but the conclusions are actually wrong. The situation in these cases is actually more subtle. The person you're speaking to does normally have some capacity to escalate in exceptional cases. But they can't do it as a matter of course, and have to maintain publicly that it's actually impossible. The people who get what th…

As someone who worked in support as a youngling:

If you behave unpleasant enough I'll go out of my way to make sure your behavior does not pay off. I will note your abrasive behavior in the ticket or might even mark your mail as spam. On telephone our line will suddenly experience technical difficulties. And throughout I will remain as friendly and patient as ever.

I will warn superiors about you, so once you escalate they already have a colorful 3D image of your wonderful personality in mind. Whether that 100% is in your favor, you can guess.

Play asshole games? Win asshole prices.

Behave like a decent person with empathy instead, press the right buttons and I might even skip some of the company rules for you. Many people in support do not give a single damn if they lose their job over you and you might just be worth it.

These are not sfter-the-fact shower thoughts, these are actually lived experiences from the trenches and I know how other people in those roles think.

Persistence pays off, being an asshole not so much

Re: Accountability Sinks

#255

Post by Wobbles42 »

TeMPOraL wrote: post #175
lucianbr wrote: post #155

Earlier quoted context omitted.

I wonder what the difference is between cybersecurity and civil aviation safety. At a glance they both have a lot of processes and requirements. Somehow on one side they are as you said, a way to deal with liability without necessarily increasing security, while on the other safety is actually significantly increased.

I think a big part of it is that failures in aviation safety cost lives, often dozens or hundreds per incident, in quite immediate, public and visceral fashion. There also isn't much gradation - an issues either causes massive loss of life, or could cause it if not caught early, or... it's not relevant to safety. On top of that, any incident is hugely impactful on the entire industry - most people are fully aware how…

> I think a big part of it is that failures in aviation safety cost lives

This is an interesting point, and it certainly affects the incentives involved and the amount of resources allocated to mitigating the problems.

I do think cyber security incidents with real consequences are likely to become more common going forward (infrastructure etc). We haven't experienced large state actors being malicious in a war time footing (yet).

Will we able to better mitigate attacks given better incentives? I think that is an open question. We will certainly throw more resources at the problem, and we will weight outcomes more heavily when designing processes, but whether we know how to prevent cybersecurity incidents even if we really want to... that I wonder about.

Re: Accountability Sinks

#256

Post by throwaway7783 »

steveBK123 wrote: post #167
t_luke wrote: post #153

The conclusion of Davies' second extract — about e.g. being bumped off a flight — is recognisable but the conclusions are actually wrong. The situation in these cases is actually more subtle. The person you're speaking to does normally have some capacity to escalate in exceptional cases. But they can't do it as a matter of course, and have to maintain publicly that it's actually impossible. The people who get what th…

Yes unfortunately I've observed this in some support systems. The best way is to thread the needle between being extremely personally polite to the other human on the line, but going through the required machinations on their runbook to trigger an escalation. That is - you don't really have to behave unpleasant (raise voice, swear, be impolite, threaten) but you should just refuse to get off the line, demand escalati…

Doordash tier 1 is so extreme that they terminate conversations unilaterally. One of the worst trashy customer services I've ever seen. Then you yell in the email and you get the right response from a "manager". Waste of everyone's time

Re: Accountability Sinks

#257

Post by setr »

amluto wrote: post #230
macNchz wrote: post #195

Earlier quoted context omitted.

Having been on both sides of this—working behind a counter and answering phones at various jobs long ago, and being someone who often surprises family and friends with my ability to extract good outcomes from customer service—I think it’s somewhat of a misconception that being as unpleasant as possible is actually effective at getting results. I fully understand that the godawful CS mazes many companies set up wind u…

As an exception to the exception, a lot of automated telephone systems have a tree of options, and they try really hard to avoid giving you a real person, and none of the options are helpful. But some of them are programmed to detect swearing and direct users to a representative. So a valid strategy is to swear at the automated system and then be polite to the real human that you get.

There’s generally no repercussions to bullying robots — or being nice to one. Aggressively direct, if not outright unsympathetically cruel, is probably the best approach in all scenarios

Re: Accountability Sinks

#258

Post by GeneralMayhem »

UncleMeat wrote: post #194
EasyMark wrote: post #180

Earlier quoted context omitted.

There is no connection between Ab Ghraib and 24, a fictional TV series. If you think this stuff didn't happen before 24 then I'd like some proof. TV reflects reality (or a very stretched version of it), not the other way around, and 24 also wasn't the first version of such a thing. It's just that Abu G they used people who were young and not professionals so it leaked. It has probably been happening as long as the US…

There is a direct connection. Antonin Scalia was one of the architects of substantial limitations on the 8th amendment and was a key figure in a number of cases specifically about extraordinary rendition and "enhanced interrogation." Scalia has multiple times in public referenced Jack Bauer as an argument for why prohibitions on torture are unworkable. At a panel on the very topic, Scalia responded to "Thankfully, se…

Christ, what a ghoul.

"The ends justify the means" is a horrific way to run a society in any case, but of course it skips over the question of whether the means actually caused the ends, let alone were the only way to do so. Even if torture did save lives, it isn't a great justification - but then pile on top that your only evidence that it actually does work is fiction and it starts to look like the means were what you really wanted in the first place.

Re: Accountability Sinks

#259

Post by hkon »

atoav wrote: post #254
t_luke wrote: post #153

The conclusion of Davies' second extract — about e.g. being bumped off a flight — is recognisable but the conclusions are actually wrong. The situation in these cases is actually more subtle. The person you're speaking to does normally have some capacity to escalate in exceptional cases. But they can't do it as a matter of course, and have to maintain publicly that it's actually impossible. The people who get what th…

As someone who worked in support as a youngling: If you behave unpleasant enough I'll go out of my way to make sure your behavior does not pay off. I will note your abrasive behavior in the ticket or might even mark your mail as spam. On telephone our line will suddenly experience technical difficulties. And throughout I will remain as friendly and patient as ever. I will warn superiors about you, so once you escalat…

If you are helping, why would they be assholes?

Re: Accountability Sinks

#260

Post by dkbrk »

The discussion near the end about how leadership taking responsibility can beneficially relieve accountability reminded me of the story of the Naval Tactical Data System (NTDS) [0].

[1]:

> When NTDS was eventually acclaimed not only a success, but also one of the most successful projects in the Navy; it amazed people. Especially because it had stayed within budget and schedule. A number of studies were commissioned to analyze the NTDS project to find why it had been so successful in spite of the odds against it. Sometimes it seems there was as much money spent on studying NTDS than was spent on NTDS development.

[2]:

> ...the Office of the Chief of Naval Operations authorized development of the Naval tactical Data System in April 1956, and assigned the Bureau of Ships as lead developing agency. The Bureau, in turn, assigned Commander Irvin McNally as NTDS project “coordinator” with Cdr. Edward Svendsen as his assistant. Over a period of two years the coordinating office would evolve to one of the Navy’s first true project offices having complete technical, management, and funds control over all life cycle aspects of the Naval Tactical Data System including research and development, production procurement, shipboard installation, lifetime maintenance and system improvement.

[1]:

The Freedom to Fail: McNally and Svendsen had an agreement with their seniors in the Bureau of Ships and in OPNAV that, if they wanted them to do in five years what normally took 14, they would have to forego the time consuming rounds of formal project reviews and just let them keep on working. This was reasonable because the two commanders were the ones who had defined the the new system and they knew better than any senior reviewing official whether they were on the right track or not. It was agreed, when the project officers needed help, they would ask for it, otherwise the seniors would stand clear and settle for informal progress briefings.

The key take-away is that the NTDS was set up as a siloed project office with Commanders McNally and Svendsen having responsibility for the ultimate success of the project, but other than that being completely unaccountable. There were many other things the NTDS project did well, but I believe that fundamental aspect of its organization was the critical necessary condition for its success. Lack of accountability can be bad, in other circumstances it can be useful, but diffusion of responsibility is always the enemy.

How many trillions of dollars are wasted on projects that go overbudget, get delayed and/or ultimately fail, and to what extent could that pernicious trend be remedied if such projects were led from inception to completion by one or two people with responsibility for its ultimate success who shield the project from accountability?

[0]: https://ethw.org/First-Hand:No_Damned_Computer_is_Going_to_T...

[1]: https://ethw.org/First-Hand:Legacy_of_NTDS_-_Chapter_9_of_th...

[2]: https://ethw.org/First-Hand:Building_the_U.S._Navy%27s_First...

Post reply on HN