Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Planning and estimating large-scale software projects

tomrussell.co.uk

Post reply on HN
Print view View original

101–110 of 138 posts

Re: Planning and estimating large-scale software projects

#101

Post by tikhonj »

Seems like there's a structural issue at play here: the team is accountable for building a specific set of features at a specific time. High-level functionality is fixed and the timeline is fixed. If the estimate is off or something unexpected comes up, what's going to give?

Answer: everything else. Anything that the planners and executives didn't consider ahead of time. Other aspects of the work: code quality, product design, accessibility, performance, robustness, edge-case-handling... Team learning, culture and satisfaction. At the limit, the team will compromise on anything that you don't need to claim a feature is "done" with a reasonably straight face.

It's simply not a good system even when the estimates work out reasonably well—and, empirically, they usually don't.

To be fair, this isn't entirely the fault of estimates in general or this estimation approach in particular; I believe those contribute, but it's primarily a reflection of how the company and culture are structured. If you're already in a system like that and you can't change it, trying to do estimates well might be the best option forwards, but only because you're already in a corner.

Re: Planning and estimating large-scale software projects

#103

Post by igouy »

https://rightingsoftware.org/

"Based on first principles in software engineering and a comprehensive set of matching tools and techniques, Löwy’s methodology integrates system design and project design. First, he describes the primary area where many software architects fail and shows how to decompose a system into smaller building blocks or services, based on volatility. Next, he shows how to flow an effective project design from the system design; how to accurately calculate the project duration, cost, and risk; and how to devise multiple execution options."

Re: Planning and estimating large-scale software projects

#104

Post by bsder »

ColinHayhurst wrote: post #10

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

I find that if you take the estimates and roll them up statistically, you get a pretty good result.

The problem is that everybody schedules to the estimate assuming it's the 90% point (project done) when it's actually a pretty accurate mean (50% point). In reality, the number is probably closer to the 33% probability because software can be an unbounded amount of time late but only a bounded amount of time early.

The primary problem is that when you feed people the 90% number up front they freak out. The secondary problem is that nobody goes back after the project was done and checks whether the 90% estimate was accurate (it generally is pretty close).

Re: Planning and estimating large-scale software projects

#105

Post by afarrell »

sarks_nz wrote: post #97
skohan wrote: post #95

Earlier quoted context omitted.

Honestly it's not developers' job to hit their estimates. If we're talking about a long-term estimate, as in "this project will be finished in 6 months", it's your job in management to find a way to do this. You've got to break the goal down into achievable sub-goals, and monitor progress along the way. Long term estimates will be wrong. Software projects generally take longer than expected, so it's up to you as a ma…

Thanks for that. I'm talking about the difference between hitting the deadline on Monday versus Friday. How to incentivise that? ie, do half an hour more work for a week, or skip the table-tennis when someone asks, etc. As a developer, I was into making sure I hit my goals, and at work to work. As a manager I do struggle with how to emphasise that ownership of product, quality, time. Why should developers care about…

This sounds like you need to be asking very different questions.

Re: Planning and estimating large-scale software projects

#106

Post by chii »

mason55 wrote: post #88
setr wrote: post #81

Earlier quoted context omitted.

My rule of thumb is that if you give me an honest estimate, then it’s my job to make it work. Negotiate scope, change implementation, move resources, increase cost, whatever. But I need estimates that at least reasonably correspond to reality in order to make my case successfully. The biggest issue I have is developers making business decisions without realizing it or without saying it — saying no to something becaus…

Yeah, from the product management side this is the way to approach it. Make priorities clear and make it clear whether the important part is the date or some set of features. If the date is what's important then start the conversation with "here's the date, can we get this all done by then? If not then what can we get done? If we can't do it all, here's the stuff that I think is important." And make sure to allow som…

> the decision maker to be clear about which parts really matter

which is good and all, but the reason software projects have such a high failure rate is _because_ these decision makers aren't clear about which part really matter! And not to blame them solely, because it's a hard problem to know.

Re: Planning and estimating large-scale software projects

#107

Post by sarks_nz »

kalleth wrote: post #2

Author here. I've been lucky enough in my career to hold some senior positions, and I thought I'd give a step-by-step on an approach I took with an "enterprise-scale" software project, and how I stole some techniques from university project management courses to meet with some success. Happy to answer any questions :)

Thanks for the article. Curious how you ensure dev outputs match up with other (external) deadlines, for example, the sales/marketing teams firing up a promotion of your new functionality.

How do you incentivise devs to hit those timeframes?

Re: Planning and estimating large-scale software projects

#108

Post by b0bby »

setr wrote: post #81
commandlinefan wrote: post #57

Earlier quoted context omitted.

> I won’t be mad about a long estimate So... my experience with this mindset is that you won't be mad , but you'll just say "that's too long/expensive" and cancel the project entirely. Then the same project will come up again in two months with different wording, again and again, until I give you the estimate you think you can afford. And then the thing will end up taking longer than the original "too expensive" esti…

My rule of thumb is that if you give me an honest estimate, then it’s my job to make it work. Negotiate scope, change implementation, move resources, increase cost, whatever. But I need estimates that at least reasonably correspond to reality in order to make my case successfully. The biggest issue I have is developers making business decisions without realizing it or without saying it — saying no to something becaus…

My favorite PM to work with never wanted any estimates to be optimistic. Things had to be within reason, but he preferred very conservative to very lean. Once the numbers for a given proposal had been collected, it was his job (and anybody else from the team who went into the pricing/costing delegation) to go back and forth with segment management:

"We can't win with these numbers. Are you trying to lose this business for us?"

"Nope. I'm letting you know what we believe is necessary to finish the work on schedule. We can be more aggressive in some areas and, if so, we'll add items to our risk register."

"You need to hit XYZ target or we're not even going to send a bid."

"Roger."

The internal cost debate usually ends up with a good budget and the PM/team have an "I told you so" card in their back pocket in case the team can't meet the delegated EBIT. It isn't perfect but this friction (PM sticking up for his team and management pushing for competitive numbers) is better than nothing. I imagine it is similar at most engineering shops.

edit: The fun part is that everybody knows its a game. PM wants as much cushion as possible, management wants great profit at a low price, and the team wants to keep getting work so as to stay employed. Sometimes the costing/pricing stuff goes a little over the top and the infamous "death spiral" gets tossed out by management. Always a hoot.

Re: Planning and estimating large-scale software projects

#109

Post by HillRat »

satyrnein wrote: post #55
ColinHayhurst wrote: post #10

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

Joel Spolsky's team actually built that approach into a feature: https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...

Fogbugz really was (is!) an underappreciated product. Jira’s become a necessary evil, but I really wish FB had managed to make a bigger dent in the market.

Re: Planning and estimating large-scale software projects

#110

Post by BowBun »

The more projects like these I plan and execute, the more I find the need to account for external parties affecting timelines.

In the case of the author's payment system, say they go with a vendor. At that scale, there may be weeks of contract negotiations for fees, rates, minimums, etc. What if some piece of documentation is flawed and you need to have a back-and-forth with their support? What if they need time to onboard you into their systems?

It makes me appreciate Apple's supply chain mastery that allows them to deliver exactly on time because they own their whole process inside and out and demand similar rigor from any vendors that supply them. If we could imitate that in software, we could eliminate a huge source of uncertainty in many projects.

Post reply on HN