Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Planning and estimating large-scale software projects

tomrussell.co.uk

Post reply on HN
Print view View original

91–100 of 138 posts

Re: Planning and estimating large-scale software projects

#91

Post by mason55 »

commandlinefan wrote: post #57
mason55 wrote: post #50

Earlier quoted context omitted.

One thing I’ve found to be helpful is to make it clear that I won’t be mad about a long estimate If that’s how long it’s going to take then that’s how long it’s going to take. I think there’s a group of people who are used to getting lots of pushback on their estimates and so they estimate low to avoid that conflict. The future conflict of things being late is not something they have to deal with right now and maybe…

> I won’t be mad about a long estimate So... my experience with this mindset is that you won't be mad , but you'll just say "that's too long/expensive" and cancel the project entirely. Then the same project will come up again in two months with different wording, again and again, until I give you the estimate you think you can afford. And then the thing will end up taking longer than the original "too expensive" esti…

> my experience with this mindset is that you won't be mad, but you'll just say "that's too long/expensive" and cancel the project entirely

Well, as far as this stuff goes, I kind of have it on easymode because I'm a VP Eng who also does a bunch of product management work. When I'm doing product design, I have the benefit of knowing approximate engineering LoE/order of magnitude, so I can make sure that I design products where the engineering effort fits into the product development cycle.

But it's not unusual for something to be harder than I thought because of some detail that I'm just too far from to realize, and in those cases it's clear to the team that they should be honest with challenges and we'll work around them together, vs. some bullshit because they're afraid I won't like their estimate. The last part is built with trust though and basically never "shooting the messenger" when someone tells me that there's a challenge.

Re: Planning and estimating large-scale software projects

#92

Post by kalleth »

justin_oaks wrote: post #85
kalleth wrote: post #2

Author here. I've been lucky enough in my career to hold some senior positions, and I thought I'd give a step-by-step on an approach I took with an "enterprise-scale" software project, and how I stole some techniques from university project management courses to meet with some success. Happy to answer any questions :)

How are deadline dates assigned? Is the deadline exactly the same as the estimated completion date? Realistic estimates aren't padded, but they still have significant probability of being inaccurate. After all, they're estimates, not information from the future transmitted to the past.

This is where the ideal meets the annoying reality of The Enterprise (tm).

I can't talk in too much detail, but in general, the deadline date was fixed through commercial contracts signed at a high enough level that engineering didn't have sight of them. The concept and commercial case was sound, but the implementation hadn't been worked out yet, when a date was set.

My strong preference would be for estimation to come first, of course, before a deadline is picked (and even then, only picked if it is really a necessary deadline), which is then based on reality, and also include some slack for unintended discoveries.

Re: Planning and estimating large-scale software projects

#93

Post by kalleth »

galaxyLogic wrote: post #90

Doesn't this assume you have the "spec" before you start implementing it? But then how do you estimate how long it will take to come up with that spec? If you have a very good detailed spec you have already done much of to work to make the implementation easy. If the spec is high-level and "fuzzy" it leaves the work of "resolving the spec" to the programmer. So trying to estimate the time it takes to code a system de…

If you don't have the spec, then you likely don't have a deadline, or a "project", really -- and something like this would be the wrong choice for an approach to follow.

I'd say leaving the overall roadmap (which is all this produces, at the end of the day, if you ignore the estimation piece) fuzzy and allowing the team to work that out with users/subject matter experts is the right approach, imo.

Re: Planning and estimating large-scale software projects

#94

Post by ChrisMarshallNY »

ColinHayhurst wrote: post #10

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

Good management advice. I was a first-line manager for a Japanese company (hard estimates, with lots of international frowny-faces, if the estimate was missed). That was pretty much what I did.

We usually came relatively close to our estimates, but would sometimes encounter anomalous situations that would force us to re-evaluate. I found that my managers were pretty good at accepting these, as long as they were not too frequent, and as long as I didn't do the "Lucy in the Chocolate Factory" thing. They set hard plans, and we were not to deviate from them.

But our customers often did not like the software that we delivered on-time, and to-spec. Since we were a hardware company, it was considered OK, but we hated it.

So we delivered yesterday's technology; tomorrow. :(

But I also ran a small team of really high-functioning C++ engineers. They stayed with me for decades, and I got to know them quite well.

I can't even imagine running one of these shops with half a million inexperienced engineers that flow in and out like guppies in a pond. My management experience would be worthless for that environment.

Since leaving, I have taken a very different tack with the software that I write.

I assume that my estimates are basically fiction. I am going to make big changes, to suit changes in the deployment environment, customer expectations, competitive landscape, platform tech, etc.

I tend to write very flexible software, in layers and modules. It allows me to react to these changes fairly quickly, and to maintain my very high quality standards, throughout.

I often toss out huge gobs of code, when I hit one of these challenges. That actually makes me happy. The code I don't write, is the best code of all.

Flexible software is a very double-edged sword. Most quality processes advise against it; for good reason.

But I tend to know what I'm doing. I've been at this a long time, and have been very humbled by many, many mistakes. I would probably have a difficult time trusting developers with lesser experience to take the kinds of chances that I take.

So my personal process depends heavily on me being me. I know myself, fairly well; warts and all. This means that I can trust the Principal engineer on my projects.

I manage scope. My projects are small-batch, artisanal, projects. I don't bite off more than I can chew. That said, my "small batches" are bigger than a lot of folks' personal projects. I often spend months, writing my code, so change comes almost invariably, during my projects.

Works for me. YMMV.

Re: Planning and estimating large-scale software projects

#95

Post by skohan »

sarks_nz wrote: post #82

How to make developers want to hit their deadlines with quality? Startup land. There was no feedback loop that rewarded developers to meet the estimates. Stock options weren't an option, and I didn't want them to do a sloppy job just to hit the 'deadline'.

Honestly it's not developers' job to hit their estimates.

If we're talking about a long-term estimate, as in "this project will be finished in 6 months", it's your job in management to find a way to do this. You've got to break the goal down into achievable sub-goals, and monitor progress along the way.

Long term estimates will be wrong. Software projects generally take longer than expected, so it's up to you as a manager to anticipate this and communicate to stakeholders with the correct degree of uncertainty.

If it's an external deadline which must be met, firstly you should engineer enough extra time into the timeline to handle inevitable delays. And if at any point you feel like the timeline is unachievable, it's up to you to renegotiate with stakeholders, or adjust the scope to make it achievable.

And if you have the feeling your team is slacking off and not getting work done, honestly this sounds like a lack of leadership skills. It's up to you to have the kind of relationship with your developers so that they are motivated to meet the team's collective goals and take responsibility. That's basically all that being a manager is.

Re: Planning and estimating large-scale software projects

#96

Post by ColinHayhurst »

eikenberry wrote: post #84
ColinHayhurst wrote: post #10

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

Did you ever come across a person who was If the answer is never or rarely then that lends a reasonable amount of credence to the idea of just multiplying your estimates. Because you most likely should and the chance of overestimating is low.

Not in my experience

Re: Planning and estimating large-scale software projects

#97

Post by sarks_nz »

skohan wrote: post #95
sarks_nz wrote: post #82

How to make developers want to hit their deadlines with quality? Startup land. There was no feedback loop that rewarded developers to meet the estimates. Stock options weren't an option, and I didn't want them to do a sloppy job just to hit the 'deadline'.

Honestly it's not developers' job to hit their estimates. If we're talking about a long-term estimate, as in "this project will be finished in 6 months", it's your job in management to find a way to do this. You've got to break the goal down into achievable sub-goals, and monitor progress along the way. Long term estimates will be wrong. Software projects generally take longer than expected, so it's up to you as a ma…

Thanks for that. I'm talking about the difference between hitting the deadline on Monday versus Friday. How to incentivise that? ie, do half an hour more work for a week, or skip the table-tennis when someone asks, etc.

As a developer, I was into making sure I hit my goals, and at work to work. As a manager I do struggle with how to emphasise that ownership of product, quality, time. Why should developers care about hitting Monday with effort, instead of coasting to Friday?

Re: Planning and estimating large-scale software projects

#98

Post by twelvechairs »

ColinHayhurst wrote: post #10

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

Estimation depends on the unknowns which you can typically put down to overall task time and complexity of the task. If its is something they have done before and a short task (hours), x1 is fine. If it is something new and long (weeks), x5 may be underselling it.

Breaking down a long task to be as granular as possible then rating each component against a scale of how much experience/confidence you have is the best approach to managing risk. As a manager, if you aren't asking people to explain this, you should be.

I've never seen anyone do it (I also haven't worked anywhere massive), but monte-carlo analysis with a tornado chart etc. would be the best way of managing this kind of risk at a complex-systems scale. It is a system used by the top real-estate developers.

Re: Planning and estimating large-scale software projects

#99

Post by ska »

ColinHayhurst wrote: post #10

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on th…

Individual factors are useful, but I've also found it very effective to ask for more that one confidence level, e.g. tell me how long you think it will take with 95% confidence, and with 50%. The spread on these is informative.

Re: Planning and estimating large-scale software projects

#100

Post by tikhonj »

majormajor wrote: post #67
andruby wrote: post #53

Earlier quoted context omitted.

Do you mind explaining why you say estimating is fundamental? I don’t disagree that it is important, I probably wouldn’t label it as fundamental, at least in a context of a company developing its own software internally.

Even when developing your own stuff internally, you have a limited resource: your dev team's capacity to do work. The most important question for your internal software team is still "what should we work on" and estimates are a fundamental part of answering that. It's someone's job to figure out what things that team could do to provide the highest value. Usually some combo of product/sales/marketing/tech. But that's…

There's a qualitative difference between the sort of informal estimates you mentioned ("it'll be hard") vs the sort talked about here—people don't treat the informal estimates as commitments!

As soon as an estimate is really a commitment it totally changes the dynamics around it and actively impacts autonomy, so my impression is that what people mean by "no estimates" is more like "no timeline commitments".

Post reply on HN