Live data from Hacker News

How Completely Messed Up Practices Become Normal

danluu.com

181–190 of 267 posts

Re: How Completely Messed Up Practices Become Normal

#181

Earlier quoted context omitted.

tl;dr Thank goodness for sociopathic jerks

I've long held the theory that humans have differing degrees of rudeness because it breaks deadlocks like "after you", subdivide-or-finish the divisible food item, and who takes the last piece. (In places that were too polite, I have seen the last piece of a cake go stale uneaten.)

My friend and I used to have a serious case of "after you" deadlocks during our studies; we would get stuck in front of a door insisting the other goes through first. We eventually figured out a simple solution - if we found ourselves about to get stuck in another "after you" loop, we'd play rock-paper-scisors, and the loser had to go first. It quickly became almost a second nature for us, and it looked pretty confusing/hilarious to people stuck behind us in the queue.

Re: How Completely Messed Up Practices Become Normal

#182
post #180

Earlier quoted context omitted.

It's a bit like when there's two cash machines, with a long queue at the first but no-one at the second. You can see people looking at it, wondering whether it's worth the humiliation of finding it's broken and having to go to the back of the queue, or the joy of finding it is working and getting one over everyone else. Often nobody cracks.

Why would you have to the back of the queue after reconning the other queue? What kind of jerks do you queue with?

Depends on a queue, I guess. In case of cash machines one could feasibly rejoin the queue at the original position; in a mall however, reserving yourself a spot in a queue and continuing to shop is considered rude.

Re: How Completely Messed Up Practices Become Normal

#184

Most of what the author is complaining about boils down to business needs being more important than the needs of the engineering team. To be blunt, they're paying you to do a job, not to make the organization better. That's what they pay the leadership for. You want to be part of the leadership, work your way through the ranks or start your own business. My life got much, much easier once I learned to stop straining…

>>To be blunt, they're paying you to do a job, not to make the organization better. If this dichotomy exists -- if doing your job does not by definition make the organization better -- then that's a dysfunctional organization.

They all are.

Re: How Completely Messed Up Practices Become Normal

#185
post #98

Earlier quoted context omitted.

Leadership leads. This is a tautology. It is unhealthy to pretend otherwise. Lets take for Leadership decides to incur technical debt to allow a faster entry into market, Development Team Lead decides this is unacceptable, and does not take on the Technical Debt, but instead does things 'The Right Way'. This leads to competitors entering the market first, snagging customers and mindshare, which eventually leads to th…

GP is not pretending anything. Further, taking a position on technical tradeoffs (your example) is an orthogonal concern to defining leadership structure, which is also distinct from the possibility of making bottom-up changes. No one is proposing anarchy here, just people fixing what they can.

OK, I think there are some unspsoken givens that I need to spell out to make my point clearer.

Typically people resolve the issues that directly cause them pain pretty quickly if it is in their control. Further "control" requires understanding how the levers a party influences impact the obstacles that cause the pain.

WTF level issues typically occur when the above conditions aren't met, usually that means the 'pain' doesn't impact the party that has the control to end the pain. next most common is the party that has the control doesn't understand how its levers affect its obstacles.

just fixing what they can is not going to resolve WTF level issues...

Re: How Completely Messed Up Practices Become Normal

#186
post #183

Getting Things Done When You’re Only a Grunt , Joel Spolsky, 2001: http://www.joelonsoftware.com/articles/fog0000000332.html

This is a much better version of the original article. Here's solutions to situations that arise when you have access to information that you feel should be acted upon that doesn't have broad organizational support already.

Re: How Completely Messed Up Practices Become Normal

#187

There seems to be a bit of a false dichotomy underpinning this article that companies value feature growth above all else and this directly results in poor operational performance. However, you can get terrible availability without delivering any features whatsoever for months and months. Two 9s of availability? Half the customers I've had would be ecstatic to have even ONE 9 of availability. And those guys hardly ev…

> And those guys hardly ever ship any code due to how encumbered developers typically are in those places and release maybe once every 6 months to a year perhaps.

I've worked in places where some teams ship with this three or six month frequency. They consider it completely normal and find ideas like continuously delivery or even weekly deployments as not just abnormal but risky and irresponsible! This is the very point OP is trying to make, the _normalization of deviance_.

Re: How Completely Messed Up Practices Become Normal

#188
post #19

Earlier quoted context omitted.

This is my thinking also... it came to me over a long time, and typically has a lot to do with why I switch jobs. At a certain point i give up struggling against the 'WTF' and just start going along with the flow, then one day i look at the stuff i am doing, realize working at the place is making me a worse professional, and move along... However I do want to note that in general these things do have real concrete ne…

> However I do want to note that in general these things do have real concrete negative impacts on the bottomline of the organization. Oh absolutely. But your duty to that organization is to raise these issues to the person best-equipped to see the bigger picture, and to do your best to convince him it's a real problem. Once you've done that, your work is done , you have just done more to help your organization than…

I completely disagree with this.

To expand a little, I don't think the point of having a positive effect on the WTF parts of an organization has anything to do with my effect relative to other employees.

If I see a WTF security or ops practice, I don't sleep better at night by telling myself, "Welp, I noticed it and said something to someone, and that's a lot more than most people do."

Maybe I think of my job too broadly, but my job as a developer is, at least in part, to protect the business, to find the WTFs and get them sorted out.

The idea that IT is just there to solve the "business problems" and they should shut up about anything that isn't directly related to making money is absurd to begin with. But even if I'm being completely naive about that, it still implies a really shallow understanding of "business problem."

Security is a business problem. Stupid ops habits that result in downtime are a business problem. Groupthink that results in acceptance of worst practices is a business problem. All the whatthefuckery the article talks about boils down to business problems. Many of these problems are such that the people primarily concerned with running the business are not in a position to recognize as problems.

It absolutely is my job and every developer's job to find these things and take care of them.

If I ever found my self in a situation where I saw some WTFs going on and I went to my boss or a coworker and got no explanation other than 'This is how we've always done it.' I would be concerned. I would go back to my desk and finish what I was supposed to be working on. Then I would go home that evening and write out the clearest and most concise explanation of why this is a serious business problem with citations and take it back to the coworker or boss.

In my experience, the WTFs don't usually come from this is how we've always done it. They had some reason back at some point in the past where someone really needed it to be that way for some reason, and usually temporarily.

Here's an example from several jobs ago, one of my first in the industry, actually:

Why does this set of boxes still have password auth enabled? Why isn't it locked down to ssh key only? Why is port 22 accessible outside of the VPN?

Oh, it's because our CEO likes to get his hands dirty with code every once in a while, but he didn't want to mess with pub/private keys, so we just left it open because these boxes weren't that important. They were 'just' dev machines pointed at test DBs with no real data in them.

But it got written into a setup script or Wiki somewhere, and when those dev boxes got repurposed for production, people followed the scripts or rules or whatever that were specific to those machines. So now you have prod machines running with password access with SSH exposed to the public. Ones that are pointed at live DB servers with creds for them.

That's a serious WTF.

I'm not even close to a security expert of any kind. I wouldn't claim to be in a million years. I work with databases and Python, almost exclusively. Though I dabble in DevOps when I need/want to. Even I know that the above is a serious WTF.

What did it take to get that fixed after my coworkers said, "Yeah, that's how it is."? Going straight to my boss, who also said, "Yeah, that's how it is." Then spending maybe an hour of my time writing up how much of a business problem this is and going back to my boss, who said, "Yeah, I know it's a problem, but I don't even know why it's like that. It just is."

So then I ask him who he can think of who might know why that is. Oh, quelle suprise, his boss might know.

And indeed, his boss did know. But he had long since stopped trying to get the point across to the CEO who liked to dabble, so he just went with it. Here comes my very brief paper explaining why this is--you guessed it--a business problem. The CEO responded within a couple of hours requesting that someone come setup SSH keys on his computer and shut down the passwd auth and close port 22 on all prod machines immediately.

That's a lot more work than mentioning it to someone. But I think that it's my job to do that, even though I've never worked in DevOps or Security teams.

My work was absolutely not done when I asked a coworker about it once, and then asked my boss.

----------------------------------------------- Edited to add:

Astute readers will note that there was a lot more WTF-ery going on than just auth and port exposure on prod. Like, for example, that the non-technical CEO who liked to dabble was doing so on a production box because he was not made aware that the boxes he was logging into to play with had been repurposed.

There was a lot of cascade in tracking down that one particular WTF, and many other WTFs were solved because of it.

At the time for that company, my job description was C# middleware for a web app. I maintain that it was absolutely part of my job to pursue that WTF to its endpoint, and that doing anything less would have been completely irresponsible.

Re: How Completely Messed Up Practices Become Normal

#189
post #147
post #98

Earlier quoted context omitted.

Leadership leads. This is a tautology. It is unhealthy to pretend otherwise. Lets take for Leadership decides to incur technical debt to allow a faster entry into market, Development Team Lead decides this is unacceptable, and does not take on the Technical Debt, but instead does things 'The Right Way'. This leads to competitors entering the market first, snagging customers and mindshare, which eventually leads to th…

"Lets take for Leadership decides to incur technical debt to allow a faster entry into market, Development Team Lead decides this is unacceptable, and does not take on the Technical Debt, but instead does things 'The Right Way'. This leads to competitors entering the market first, snagging customers and mindshare, which eventually leads to the company going under." To be honest, how often does this actually happen? I…

> Most startups aren't doing anything that special, and so this whole, "We have to move at twice the speed of light or we're all dead!" attitude doesn't belong anywhere.

burn rate/runway is a critical survival factor to startups

Re: How Completely Messed Up Practices Become Normal

#190

Earlier quoted context omitted.

This is exactly a perfect example of WTF worthy behavior being treated as totally normal; that it's encoded that there's "leadership" and "not leadership" as binary job distinctions with distinct responsibilities. I've been in both types of cultures and, by far the healthier culture was the one in which "making the organization better" was treated as everyone's responsibility. I've seen very junior engineers drive qu…

> If you seriously believe that all (or most) companies disallow anyone not in leadership to improve things, I'd consider getting out of your current situation and seeing things again with clear eyes. I've been around a bunch, including the US Air Force, and I can confidently tell you that heavily hierarchical organizational culture is the norm, individuals contributors who want to have effects beyond their job scope…

[deleted]
Post reply on HN