Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Infrastructure Debt

littlehart.net

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Infrastructure Debt

#21

Post by brd »

dirtyaura wrote: post #11

I think the author has a different interpretation of "technical debt" than how it was originally intended. Technical debt is not accidental and it doesn't consist of small mistakes, but it's due to a deliberate decision to cut corners when trying to get product or feature launched. Just like financial debt is not accidental, but a deliberate decision. (although you could argue that credit cards etc can create acciden…

"Technical debt is not accidental" is a pretty bold statement in my opinion. Technical debt could be defined as all which needs refactoring. Making a mistake in your original system design because of an unforeseen feature, scope creep, optimization requirements as you scale, all are things that can drive up your technical debt. Conscious shortcuts are only the most obvious path to debt and they're most likely the easiest to pay back as well.

Technical debt is a concept you become intimately familiar with working in the enterprise space.

Re: Infrastructure Debt

#22

Post by raganwald »

grumpycanuck wrote: post #14
raganwald wrote: post #9

I like the article, but am thinking about a bike shed. For each thing, like manual deployments or failure to use source control, I ask myself: Is it “debt?" Or "friction?” Debt and friction both accumulate, but debt must be "repaid" in a lump sum. If "technical debt" or "infrastructure debt" means your velocity is falling over time, it's friction, not debt. Infrastructure debt would be something that is eventually go…

(Blog post author here) Upon reflection your use of the term "friction" is a good one. Although I tend to look at "debt" as something you pay back as quickly as possible but sometimes you can only pay it back in installments. I guess you could say having too much infrastructure friction eventually leads to one humungous infrastructure bonfire. :)

Well, “friction” is on ongoing cost but doesn’t result in some catastrophic event down the road, whereas debt may or may not have an ongoing cost but carries with it some non-trivial probability of disaster. Like real debt, the longer you wait to eliminate it, the more expensive it is to fix and the worse the catastrophe if you on’t fix it.

So... I tend to think of source control problems as being infrastructure debt, because you are definitely going to crash and burn eventually, and the longer you wait, the worse the problem will be. I am open to rethinking this, but I would classify automated deployment as being friction. If you can deploy by hand, and everybody knows how to deploy by hand... It seems that deploying by hand is probably friction while you are in development and then debt once the product is in “actual” production with end users. In development, you might make a mistake, forget a library, and fixing it is work but not catastrophic. But once you have actual users, making a deployment mistake could produce irrevocable disaster.

Anyways, just to be clear, I’m only bringing up the distinction for the sake of discussion. I like the post just the way it is.

Re: Infrastructure Debt

#23

Post by karlmdavis »

raganwald wrote: post #22
grumpycanuck wrote: post #14

Earlier quoted context omitted.

(Blog post author here) Upon reflection your use of the term "friction" is a good one. Although I tend to look at "debt" as something you pay back as quickly as possible but sometimes you can only pay it back in installments. I guess you could say having too much infrastructure friction eventually leads to one humungous infrastructure bonfire. :)

Well, “friction” is on ongoing cost but doesn’t result in some catastrophic event down the road, whereas debt may or may not have an ongoing cost but carries with it some non-trivial probability of disaster. Like real debt, the longer you wait to eliminate it, the more expensive it is to fix and the worse the catastrophe if you on’t fix it. So... I tend to think of source control problems as being infrastructure debt…

Keep in mind, though, that even "friction" will eventually cause a team's productivity to drop down to 0. The amount of friction that a given team can tolerate depends on the team members, of course, but especially on the size of the team (see: Mythical Man Month).

There comes a point that even a tiny bit of friction, spread out over enough developers, eventually causes the marginal productivity boost of adding new devs to reach 0. That can happen even without friction, but the friction exacerbates the trend.

Re: Infrastructure Debt

#25

Post by dells »

gizmo wrote: post #3

You have to walk a fine line. If you neglect your infrastructure for too long you get in a situation where you get bogged down by all the circumstantial complexity in your infrastructure. If you are too zealous about debt you automate things that don't need automating and you end up with an over-engineered infrastructure that is completely automated but never works quite right. As with the design of software projects…

+1

Re: Infrastructure Debt

#26

Post by Retric »

gizmo wrote: post #3

You have to walk a fine line. If you neglect your infrastructure for too long you get in a situation where you get bogged down by all the circumstantial complexity in your infrastructure. If you are too zealous about debt you automate things that don't need automating and you end up with an over-engineered infrastructure that is completely automated but never works quite right. As with the design of software projects…

It's not a static trade off either. If you see two world class developers debating some tradeoff it's about as relevant to the average coder as what two Olympic level swimmers discussing their diet has to do with someone trying to lose some weight. AKA avoid holes to big for your team to get themselves out of.

Re: Infrastructure Debt

#28

Post by jwatte »

I work with 40,000+ unit tests. I deploy across a live cluster with a single command. My co-workers and I deploy to production dozens of times a day. Every new hire gets a VM image sandbox to develop in. We still have infrastructure debt. The truth is, debt leverages the future into the present, and as long as your net productivity keeps increasing, you're winning. You have to pay off enough debts along the way to not grind to a halt, but you will always be in technology debt. If you're not, you paid too much for what you have.

Re: Infrastructure Debt

#29

Post by endlessvoid94 »

grumpycanuck wrote: post #27
endlessvoid94 wrote: post #24

I must advocate reading James Hamilton's paper: On Designing and Deploying Internet-Scale Services. http://www.usenix.org/event/lisa07/tech/full_papers/hamilton...

Just took a look, that is an awesome paper.

It's invaluable. I can't believe it's not more well-known.
Post reply on HN