Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

blog.hut8labs.com

Post reply on HN
Print view View original

31–40 of 66 posts

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#31

Post by nevergetenglish »

To estimate the time required to design your system you need to estimate the variance of the time required. A well known result is that the variance of a sum of independent variables associated to independent problems is the sum of the variances of the subproblems.

What this mean in practice? It means that you should try to decompose your problem in independent and dependent subproblems (perhaps associated to features). Independent subproblems their variance, the formula for dependent problems depends of the correlation between them.

What I am trying to say is that if you know a little math and have an intuitive knowledge of the structure of the problem divided in sub-problems you could make a much better estimate of the variance (and this mean there is much less uncertainty in the delivery time). One could design a program that constructs a graph and a knowledge base of previous programs that you have designed. The graph edges should be weighted in accordance with the correlation factor of dependency of those components.

Now that I am writing this, it seems very likely that someone has implemented such a scheme to estimate time delivery. Perhaps there are start-ups using this system to estimate delivery time, since all this is basic math.

Edited: Mostly grammar.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#32

Post by markbnj »

rayiner wrote: post #11

One of the best things you can do in your organization is create a culture that treats obstacle discovery as forward progress. There is nothing more disheartening for an engineer than banging her head against the wall trying to debug some race condition only to get flak from management about why it isn't fixed and why it's holding up some milestone. The root of this problem is management that doesn't understand the p…

Dead on, imo.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#33

Post by sporkenfang »

rayiner wrote: post #11

One of the best things you can do in your organization is create a culture that treats obstacle discovery as forward progress. There is nothing more disheartening for an engineer than banging her head against the wall trying to debug some race condition only to get flak from management about why it isn't fixed and why it's holding up some milestone. The root of this problem is management that doesn't understand the p…

Having experienced both sides of this, I concur.

When your manager is annoyed and replies "oh man, how much will I have to push back the next release deadline for your bug?!" even though you just discovered it and didn't necessarily cause it, it's much more disheartening than something like

"Good! Fix it! I'm giving you two weeks until the next release (instead of releasing tomorrow) and if it's still an issue we'll get you some backup."

Race conditions can in some cases break the entire product or render some of its results useless (depending on scale and location of the bug, naturally). I definitely think in most cases they shouldn't be taken lightly.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#34

Post by lnanek2 »

kator wrote: post #12

I've been developing for 30 years. When asked for an estimate I spend time thinking about it and then when I come up with a number I multiply it by 3. Usually this works but sometimes I can be off. My typical process is to score stuff from "totally get it" to "don't have it all in my head". I then guess on hours and in the end multiply by this 3 factor. When my clients complain about the estimate I often say we "writ…

When I'm writing a contract I usually put this in the milestones I'm paid by. E.g. one week for feature 1 delivery, next week for client requested adjustments to it, next week for feature 2 delivery, week after for client requested adjustments, etc..

Generally I only pick something I could finish in half a week for the 1 week milestone too, so there is plenty of time if it turns out tougher than I thought.

This is also a good way to communicate that you won't tweak a feature every week for months after without adding another milestone (and more pay) to compensate, something every client always asks for in the end, because you don't really learn anything until a feature is put in front of an end user.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#35

Post by rayiner »

sporkenfang wrote: post #33
rayiner wrote: post #11

One of the best things you can do in your organization is create a culture that treats obstacle discovery as forward progress. There is nothing more disheartening for an engineer than banging her head against the wall trying to debug some race condition only to get flak from management about why it isn't fixed and why it's holding up some milestone. The root of this problem is management that doesn't understand the p…

Having experienced both sides of this, I concur. When your manager is annoyed and replies "oh man, how much will I have to push back the next release deadline for your bug?!" even though you just discovered it and didn't necessarily cause it, it's much more disheartening than something like "Good! Fix it! I'm giving you two weeks until the next release (instead of releasing tomorrow) and if it's still an issue we'll…

> even though you just discovered it and didn't necessarily cause it

Whether you caused it or not is irrelevant. The only issue on the table when confronted with a defect should be: 1) how do we fix it so the customer does not get shitty product; 2) how do we improve the process so that such defects are not introduced in the first place.

If there are people who do careless work and consistently introduce defects, that's a wholly separate issue to be handled through separate channels. If you mix the engineering issue, with the HR issue you will create a culture where people would rather let defects get to the customer than get in trouble for raising them internally.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#36

Post by Ntrails »

anthay wrote: post #30
kator wrote: post #12

I've been developing for 30 years. When asked for an estimate I spend time thinking about it and then when I come up with a number I multiply it by 3. Usually this works but sometimes I can be off. My typical process is to score stuff from "totally get it" to "don't have it all in my head". I then guess on hours and in the end multiply by this 3 factor. When my clients complain about the estimate I often say we "writ…

But you risk your manager (if you have one) telling you you are "sandbagging." And you might see others on your team tell your manager "it's no big deal - an hour or so of work". And they slap some crap in there that whacks the mole down long enough for the manager to make a mental note to give your team mate a bigger raise than you.

If you want shitty work that creates a mountain of technical debt, I can do that (no, really, I'm a natural at it).

If you want a thought through process that covers all of the points as best I can, I will do that.

Align my incentives. But don't complain because I do the optimal thing

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#37

Post by kator »

anthay wrote: post #30
kator wrote: post #12

I've been developing for 30 years. When asked for an estimate I spend time thinking about it and then when I come up with a number I multiply it by 3. Usually this works but sometimes I can be off. My typical process is to score stuff from "totally get it" to "don't have it all in my head". I then guess on hours and in the end multiply by this 3 factor. When my clients complain about the estimate I often say we "writ…

But you risk your manager (if you have one) telling you you are "sandbagging." And you might see others on your team tell your manager "it's no big deal - an hour or so of work". And they slap some crap in there that whacks the mole down long enough for the manager to make a mental note to give your team mate a bigger raise than you.

If you can't have a good relationship with your stakeholders then you should find a new stakeholder not sweat what someone else is willing to do to get ahead.

In the end if you're reasonable and can show you've hit the nail on the head often enough to be trusted then you won't have conversations about sand bagging.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#38

Post by michaelochurch »

I tend to have a darker, more cynical view.

Asking for an estimate is often a power play or "microaggression". It's more often about showing dominance than any legitimate business need to know this unknowable quantity. It plays a programmer's present self against her future self.

Her future self wants an accurate or even pessimistic estimate to be made, so there are no unpleasant surprises to management and painful conversations resulting from inflated expectations. But her present self wants the guy standing at her desk to go away so she can get back to work. An unreasonably optimistic estimate gives the powerful, annoying people what they want so they go away. A realistic estimate is going to lead to obnoxious follow-on questions. "Why's it going to take so long?" "I don't know, but experience leads me to think..." "But you see no specific reason why this can't be done in 2 weeks?" "Well, no, because there are unknown unknowns..." "Great! Two weeks!"

This degenerates for two reasons. First, managers tend to believe that, even if overly optimistic estimates are more inaccurate, the work is done more quickly with unreasonable estimates (which become "milestones", then "deadlines"). So they take this as an incentive to be more irritating because it "makes people work faster". Second, it leads to poor planning and brittle schedules and much more undesirable variance.

Obviously, the good software managers don't play these games, but they're probably 1 in 10. Because there is so much money in software, and because programmers refuse to organize and allow themselves to be underpaid, that creates lots of room for mismanagement. Developers are also to blame for some of this; because they've been in mismanaged environments for years, many of them have developed a distrust of management that has led them to miscommunicate and (inappropriately) simplify, hence the "estimates" that evolve into deadlines. Overconfidence and miscommunication are, for sure, substantial components.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#39

Post by kazinator »

> “We should just be more careful at the specification stage”. But this turns out to fail, badly. Why? The core reason is that, as you can see from the examples above, if you were to write a specification in such detail that it would capture those issues, you’d be writing the software.

Not true; a specification can precisely describe the properties of the solution of a (sub-)problem, rather than the process of solving it.

Re: Coding, Fast and Slow: Developers and the Psychology of Overconfidence (2013)

#40

Post by yarrel »

rayiner wrote: post #35
sporkenfang wrote: post #33

Earlier quoted context omitted.

Having experienced both sides of this, I concur. When your manager is annoyed and replies "oh man, how much will I have to push back the next release deadline for your bug?!" even though you just discovered it and didn't necessarily cause it, it's much more disheartening than something like "Good! Fix it! I'm giving you two weeks until the next release (instead of releasing tomorrow) and if it's still an issue we'll…

> even though you just discovered it and didn't necessarily cause it Whether you caused it or not is irrelevant. The only issue on the table when confronted with a defect should be: 1) how do we fix it so the customer does not get shitty product; 2) how do we improve the process so that such defects are not introduced in the first place. If there are people who do careless work and consistently introduce defects, tha…

1) By spending the time to do so.

2) Use oracles.

Post reply on HN