Live data from Hacker News

Software estimation is hard – do it anyway

jacobian.org

121–130 of 231 posts

Re: Software estimation is hard – do it anyway

#121
One strategy I've found useful when asked for an estimate is to ask back: "What do you need the estimate for?" Often, this leads to a useful discussion, and we can discover things like:

- They don't actually need to estimate, because the task can very obviously be completed by the previously window

- They're simply trying to prioritize two different features, so the estimate doesn't need to account for who will be working on the project, known vacations, meetings, etc.

- The business is trying to use the estimate for strategic planning, so high-confidence, or multiple estimates (optimistic/normal/conservative) are actually needed.

It's similar to when someone comes to engineering and asks "Please build this button for me" - it's always crucial to ask "Why?" and understand the problem they're trying to solve, since often what they've asked for is not what they need.

Re: Software estimation is hard – do it anyway

#122
post #38
post #23

Here's my beef with estimates. I can give you a really really accurate estimate, but in order to do so we're going to have to spend a lot of time going through the request, building and verifying actual requirements, designing the solution and then validating it. The process will require dev resources, business resources and probably people from the support team and will take a lot of time. I'm happy to do it. It's a…

Well said. My current rule based on experience is that estimates should only be hours/days/weeks/months/quarters/years with NO numbers. It is to give a sense of scale and effort so it could be prioritized and/or modified. If they want exact dates, then it is like you said, several days/weeks to go get an accurate date. I only wish that the sales team had to commit to closing dates the same way software teams do.

That's actually a good hack. I tend to take a similar approach, but using approximate days in a fibonacci sequence.

Start at the high end, will it take... years: NO quarters: NO months: erm... NO. weeks: maybe days: Unlikely hours: Ha. No.

So you end up with a range (days?-weeks-months). That's too broad, what could go wrong to avoid making it months long project (well we could investigate X, Y, and watch for Z). What needs to go perfect for it to be days? (well we could... wait, days is unlikely).

Those discussions about the high and the low to get "reasonable" confidence are super important.

Re: Software estimation is hard – do it anyway

#123

Earlier quoted context omitted.

> Demoing every day only works if your work is easily visible I don't much like front-end work, but seeing how easy it is for front-end devs, designers, and UX folks to get noticed, makes me seriously reconsider my priorities, sometimes. For them, it's practically effortless, just something that happens .

It's just as easy to get noticed reproducing some ugly piece of shit EMR application for the open source world in some new language just to show how efficient of a programmer you are in a weekend project. (UI/UX can generalize about the other side, too ;D)

Sure, but the difference is that no-one gives a shit about that unless you are attached to, and high up in, a project with massive, successful PR (e.g. React). Showing up in a meeting with some at-least-competent design mock-ups and getting lots of positive reactions and excitement, meanwhile, is the norm, in my experience, even on fairly mundane parts of mundane projects.

Ugly but technically-impressive weekend code projects may impress programmers and gain visibility there. Meanwhile, designs routinely impress non-technical management, stakeholders, product managers, and clients. I mean, the degree to which that's true is so well-known that it's practically a cliché. There's a huge difference in how hard it is to get people who matter (in terms of career advancement, comp, and even just staying off your back about how much work you're doing) to notice your work. It's not at all comparable, and it's entirely to do with how legible one's work is to the rest of an organization.

The down side is that where non-UI developers meet confusion and ignorance ("so... what is it you still need to do? Why will it take so long? What do you mean it doesn't work yet? Oh you made the query finish in 3% the time it took before? That's nice, thanks. Moving on...") designers instead get endless suggestions, because every dumb-ass thinks their ideas about UI are good, and some of those dumb-asses really, really want to influence the design (why? Because it's so high-visibility, so it's something they can point to for higher-ups or in a portfolio and say "I did that"—they want to acquire some of that natural designer/UI/UX legibility-of-work for themselves)

Re: Software estimation is hard – do it anyway

#125
post #93
post #48

Estimation that isn't based on previous data - I think the article that follows this one refers to it as "Evidence-Based Scheduling" - is almost entirely a waste of time. We analyzed our five+ year history of estimates vs actual time, and our standard deviation was larger than our mean. It was ridiculous how wrong our estimates were. The problem was in what was being estimated - coding time. Developers would get aske…

Kanban lends itself well to this. Breaking down work into similarly sized tickets/units can, over time, be used to predict delivery/capacity (which one can use Cycle time to calibrate) Even neater with enough data it becomes possible to use Monte Carlo simulations to give you confidence intervals on how much can you do or how long you will take to do X amount of work. https://kanbanize.com/kanban-resources/kanban-ana…

    Breaking down work into similarly sized tickets/units can,
    over time, be used to predict delivery/capacity
IMO "can break up work into similarly-sized units" is equivalent to "can estimate accurately".

Re: that article - I can't imagine many things LESS accurate than "we have 104 tasks on the board and each team member's cycle time is 2 days so we can finish all the tasks with 10 people working for 20.8 days". Yeah, it makes for a nice graph - but it omits important details like dependencies...

Re: Software estimation is hard – do it anyway

#126
post #23

Here's my beef with estimates. I can give you a really really accurate estimate, but in order to do so we're going to have to spend a lot of time going through the request, building and verifying actual requirements, designing the solution and then validating it. The process will require dev resources, business resources and probably people from the support team and will take a lot of time. I'm happy to do it. It's a…

That's why I try to give Ballpark first, not estimates if I can get away with it. I would say "This can cost anywhere from $x to $y units (time/money)" where $y-$x is usually a big range.

Then we start breaking it down further if there is interest and for that we need everyone involved like you said. Not always easy but sometimes it works.

Re: Software estimation is hard – do it anyway

#127
post #23

Here's my beef with estimates. I can give you a really really accurate estimate, but in order to do so we're going to have to spend a lot of time going through the request, building and verifying actual requirements, designing the solution and then validating it. The process will require dev resources, business resources and probably people from the support team and will take a lot of time. I'm happy to do it. It's a…

> But the business invariably doesn't want to spend the time and money to do that.

Most engineers can actually estimate things fairly well when they take the time to iterate on a PoC and gather all sorts of details. Estimation fails when management has unrealistic expectations, e.g. asking for an estimate immediately after a proposal, or some set of initial documents are written.

Re: Software estimation is hard – do it anyway

#128

At the risk of being downvoted, I would argue that working on a project without first estimating the work is not engineering. It's coding. A thoughtful estimate shows care in understanding the project and how it fits into the greater whole of the existing platform. It allows all members of the company to trust in the timelines of the engineering team and align their work to meet the milestones. An estimate is by no m…

here is the problem: estimating is also work. breaking down the work is hard work. sometimes it takes more to break down and think things through than to do the work.

the fundamental problem is that management does not want estimates. they want quick estimates (ie close to zero effort) and after that they turn around and use those estimates as deadlines.

now as a developer what are you supposed to do? you’re gonna get burned a couple of times and be forced in death marches. after that you’ll: take your time estimating. you will pad your estimates to mitigate risk. ruthlessly dissolve complains about how big the estimates are by pointing out all the things that you need to think about and do. reestimate everything when anything but the most trivial thing changes.

everyone loses. really. management believes that they are squeezing the maximum amount of value but they’re not even close. developers end up doing the bare minimum and will take absolutely zero risks even if it would make the product better. fuck all that agility we claim to have.

welcome to software development in the 21st century. oh… I know. I’ll use copilot to write my code and I’ll also update it to estimate stuff! glorious!!!

Re: Software estimation is hard – do it anyway

#129

I used to do real estimates and was borderline prophetical on them. Didn't matter. I stopped and now just do a rule of thumb plus two weeks, two months, two years depending on the project. Professionally it changed nothing. For me it made my life much better and now projects come in "early" and make customers happy. Instead of people frothing at the mouth because it was a "day late" Why do we as an industry put up wi…

Agile, is in practice, a way to have the dev team under more strict scrutiny than anyone else in the org. It doesn't happen anywhere else, mostly because other industries have a long standing unions and working rights instead of free "professionals" doing services.

Re: Software estimation is hard – do it anyway

#130
> Many Agile methodologies involve arbitrary scoring systems – story points, t-shirt sizing, etc. – deliberately designed to help avoid giving estimates in time-scale units.

I can't tell if there's a really deep misunderstanding of what the author calls "no-estimate" systems, or broad agreement but with a small/superficial difference in preference on an implementation detail.

> However, sooner or later, someone’s going to ask “when will Feature X ship?”

Story points let you do this.

As I see it, the key idea with what the author calls "no-estimate" scoring systems is to psychologically decouple the act of estimating from "real time units" into "abstract work units", which (the claim is) are more accurate than "real time units". Most engineers are bad at producing time estimates for things, but if you ask them for a "points estimate", they are more likely to compare the new task to representative examples of past work (which tends to be more accurate), whereas asking for a "time estimate" they are more likely to envision themselves completing the task at hand (which leads to overly-optimistic estimates).

Given a set of points estimates for upcoming tasks, you look at your team's velocity of "abstract work units" per unit time, and you can project timelines for your backlog. The goal with scrum story points / t-shirt sizes is not to avoid estimating when a feature will ship, it's to make that process more accurate.

Scrum suggests that you try to keep a few sprint's worth of tasks finely-groomed, and keep the rest of the backlog coarsely groomed (i.e. rough estimates at epic-level, where you might have blocks of work that are multiple developer-months in size). This is using the "lean manufacturing" principle; don't spend time grooming/analyzing/estimating work that you're not going to use immediately, as it takes time to do so, and the backlog is subject to changes which would invalidate the preparation you did. But if you have a specific need to forecast 3-6 months of backlog, then of course you would do so, and points-based systems are capable of doing so without any modification.

There's nothing more to it - if you follow this process you end up with a roadmap/backlog that gives predictions for when everything you've estimated is going to land (i.e. "when will feature X ship"), with uncertainty naturally increasing the further in the future that you are looking.

To be clear though -- if you prefer using "days" as your estimate unit, that's completely fine. One of the key principles about doing lower-case-A agile software development is that you need to experiment and figure out what works for your team. I'd recommend that you retrospect on how many "days estimated" of work you actually complete per day though, because it's likely not to be a 1:1. And then, if you're regularly completing 7 "days" of work per 10-day sprint, wouldn't it be more sensible to forecast that you'll complete 7 "days" per sprint, instead of constantly claiming you'll complete 10 days of work every sprint, and only finishing 7 of them? Now you've re-implemented points. Of course, I think the author would prefer to say "fix your estimates and stop saying you'll do 10 when you only do 7", but in my experience the actual amount of work delivered is very lumpy, and so it's hard to close this feedback loop accurately.

A middle-ground here is to distinguish between "burdened" and "unburdened" days, where an unburdened day is the mythical "if I had no other tasks, how long would this take me?" estimate. These are closer to what an average developer will give if you ask them for an estimate. Then you can convert unburdened=>burdened by some ratio, depending on how much time you allocate to non-task time. These are things like devops work, on-call, code review, architecture review, etc. You can improve the unburdened/burdened time ratio, so it can be nice to be able to keep all your old estimates valid as you remove/add burden from your engineering team. In this terminology, the author advocates for asking developers for fully-burdened estimates, i.e. the estimator is responsible for folding in all of the complexity of non-sprint tasks. In my experience, few engineers (very few below staff level) are good at this process, as it's hard, and is fairly orthogonal to most of the normal task work that non-managers participate in.

Now, the case for "the author is making a superficial disagreement" - if you hop over to the author's technique for estimating (https://jacobian.org/2021/may/25/my-estimation-technique/) you'll see a very sensible process that is to my eyes structurally isomorphic to the standard best-practice "agile" techniques, including using time-boxed spikes to reduce implementation uncertainty, and proactively breaking up large tasks into more easily-estimatable chunks. The main differences I see are that the author estimates in fully-burdened days instead of points, and is more explicit about communicating the uncertainty on the estimates given. (In standard points-based approaches you just decline to give an estimate with "high uncertainty", or would give the pessimistic worst-case estimate, and would prefer scheduling a spike before starting to work on something that's highly uncertain. In some cases I can see where an explicit uncertainty range would be more useful to external stakeholders, so I like the author's process. I also can see that asking engineers to be explicit about their uncertainty might be a good way of achieving the same sort of decoupling-from-the-happy-path that story points are aiming to achieve. So overall it seems a good system.)

Post reply on HN