Live data from Hacker News

Planning and estimating large-scale software projects

tomrussell.co.uk

111–120 of 138 posts

Re: Planning and estimating large-scale software projects

#111
> Engineering does not work in a vacuum, and there are commercial contracts that will be signed, at a high level, without your involvement. Deals that will have been agreed before you joined the organization.

Indeed. And they’ll have been built on lies, damned lies and wishes.

Re: Planning and estimating large-scale software projects

#112

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

I don't trust multipliers. I often find those rules of thumb to be very poor predictors. Since if you don't know how can you predict some fixed fraction of the work?

I led a small team to deliver software over a multi-year project. The software was used to control a machine. The machine was contracted as part of a multi-million dollar deal. We delivered software on time and with good quality. It was very far from trivial. The key to this is that the team had experience in doing similar projects, so we generally knew how long things take. We knew what needed to get done. We knew we would be debugging the machines. We knew we would need to work around issues. We knew we'd be limited until the first prototypes are assembled. We knew what existing tools and libraries we could reuse. And we also knew what's the minimal viable product and what are nice to haves. For me personally this was the third consecutive project of building very similar machine control software, for very similar applications, using very similar technologies. This wasn't exactly the same but I went through this process and I knew how it works. The first of those actually failed because the software wasn't ready on time (and other reasons, but that was the main one).

A team with no experience, building something new, has zero chance of correctly estimating anything. Not only will they likely not deliver on time, they might not deliver at all, ever. Or they'll deliver something that doesn't work. Someone external who has experience with these teams might be able to provide the right estimate (e.g. they'll say it'll take them 3 years and they'll deliver nothing). In my current job I asked a junior engineer for an estimate, he broke down stuff to tasks, and came back with something like "a month". Ten months later it wasn't done. Much simpler technically than the other project but the engineer didn't really see all the details, didn't have experience with similar projects, so he basically just made stuff up.

You always have to plan for surprises and you need to have contingencies. An estimate is always some sort of distribution but for a large enough project these distributions do form some sort of coherent picture. Sure, you might have a surprise, but you do the risky things first to reduce those risks.

Re: Planning and estimating large-scale software projects

#113
post #106
post #88

Earlier quoted context omitted.

Yeah, from the product management side this is the way to approach it. Make priorities clear and make it clear whether the important part is the date or some set of features. If the date is what's important then start the conversation with "here's the date, can we get this all done by then? If not then what can we get done? If we can't do it all, here's the stuff that I think is important." And make sure to allow som…

> the decision maker to be clear about which parts really matter which is good and all, but the reason software projects have such a high failure rate is _because_ these decision makers aren't clear about which part really matter! And not to blame them solely, because it's a hard problem to know.

100% agree. They aren’t clear because they don’t know. They might not be able to think in abstracts, or simply don’t have their priorities straight.

The same would happen with ANY other type of project.. marketing, home renovation, building an aircraft, or even baking a cake

Re: Planning and estimating large-scale software projects

#114

Earlier quoted context omitted.

I think this attitude is too fatalistic. A good approach would be to see if there’s some way to break up the feature in smaller pieces, into smaller milestones, etc. If you have a manager that is unable to deal with this kind of stuff, then your problem is the manager, not the estimates. Estimates are extremely useful, and you’re doing yourself a disfavor if you think that they are never meaningful.

If I was surrounded by people who could put together accurate software estimates that made upper management happy, and I was the only one who was always got it wrong, I'd hang up my hat and see if I could get a job selling life insurance. Hell, if 10% of my peers could put together accurate software estimates that made upper management happy, I'd lobby to have them elevated to senior positions and spend all my time b…

Just get more experienced developers, who understand that there needs to be done a lot more than writing some code. Pff wait, you’ll have to pay more, and you’ll have to trust their experience. Money and status…

It’s probably easier to get a younger dev who’ll go into crunch mode bc they made estimates based on incomplete specs and lack of experience, but who feel the responsibility to deliver. They’re more easily pressured into working more, when the PM should step in and get more resources or do damage control for the mess they created.

Re: Planning and estimating large-scale software projects

#115

Earlier quoted context omitted.

If I was surrounded by people who could put together accurate software estimates that made upper management happy, and I was the only one who was always got it wrong, I'd hang up my hat and see if I could get a job selling life insurance. Hell, if 10% of my peers could put together accurate software estimates that made upper management happy, I'd lobby to have them elevated to senior positions and spend all my time b…

> But they don't. Nobody does. They don't just get the timelines wrong, they get the tasks and milestones wrong - which makes sense, because the people asking for the estimates don't actually know what the goals are, usually even after the software is delivered. I definitely agree that the people asking don't actually know what the goals are. They usually have a very fuzzy picture there. But I've always found the est…

You’ll spend more than half the total time on meetings and discussing. Client will not be happy, but at least they can’t deny it was needed.

Still it’s the only way that works.

They pay for someone who types code, but need someone who understands their business.. oh wait, isn’t that a consultant?

Consultants get a bad name because they usually don’t understand either the business or the tech. But they have good verbal skills

Re: Planning and estimating large-scale software projects

#116
Call me crazy, but if we're going to call these goals, "estimates," they should actually be based on real data. Otherwise we're just making up numbers.

If the relevant data aren't tracked or stored anywhere, that kind of tells you how serious the org is about making accurate estimates.

Re: Planning and estimating large-scale software projects

#117
post #95

Earlier quoted context omitted.

Honestly it's not developers' job to hit their estimates. If we're talking about a long-term estimate, as in "this project will be finished in 6 months", it's your job in management to find a way to do this. You've got to break the goal down into achievable sub-goals, and monitor progress along the way. Long term estimates will be wrong. Software projects generally take longer than expected, so it's up to you as a ma…

Thanks for that. I'm talking about the difference between hitting the deadline on Monday versus Friday. How to incentivise that? ie, do half an hour more work for a week, or skip the table-tennis when someone asks, etc. As a developer, I was into making sure I hit my goals, and at work to work. As a manager I do struggle with how to emphasise that ownership of product, quality, time. Why should developers care about…

You can’t and you shouldn’t. Pressing isn’t how you get more or better software engineering work done.

What can change the game is spending as lavishly as possible on the factors which make a project take extra days. Bad interfaces. Missing documentation. Sharp edges. De-prioritized bugs. Deferred refactors. Commit the greatest number of the most expensive people to work which is not connected to customers, deadlines, or business metrics, but to mitigating their own frustrations and sensibilities. Of course that is anathema to business culture so it’s rarely done.

Re: Planning and estimating large-scale software projects

#118
post #110

The more projects like these I plan and execute, the more I find the need to account for external parties affecting timelines. In the case of the author's payment system, say they go with a vendor. At that scale, there may be weeks of contract negotiations for fees, rates, minimums, etc. What if some piece of documentation is flawed and you need to have a back-and-forth with their support? What if they need time to o…

Apple also cuts scope and delays committing to a release until they know they can deliver...

Re: Planning and estimating large-scale software projects

#119
post #88
post #81

Earlier quoted context omitted.

My rule of thumb is that if you give me an honest estimate, then it’s my job to make it work. Negotiate scope, change implementation, move resources, increase cost, whatever. But I need estimates that at least reasonably correspond to reality in order to make my case successfully. The biggest issue I have is developers making business decisions without realizing it or without saying it — saying no to something becaus…

Yeah, from the product management side this is the way to approach it. Make priorities clear and make it clear whether the important part is the date or some set of features. If the date is what's important then start the conversation with "here's the date, can we get this all done by then? If not then what can we get done? If we can't do it all, here's the stuff that I think is important." And make sure to allow som…

> And make sure to allow some time for people to think about things and give an honest estimate.

I like to tell people to give an estimate for when they can provide an estimate.. and estimate for that, if needed :)

Eventually someone knows something

Post reply on HN