Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software effort estimation is mostly fake research

shape-of-code.coding-guidelines.com

Post reply on HN
Print view View original

171–180 of 318 posts

Re: Software effort estimation is mostly fake research

#171

Post by IanCal »

lixtra wrote: post #143
koonsolo wrote: post #136

Earlier quoted context omitted.

I once had a manager that thought he could negotiate estimates. That is like negotiating with the weatherman about the weather. Sure, maybe in the end you can convince them it will be finished sooner. But the fact is that it really doesn't change reality.

Typically there are many dimensions to a solution, the negotiation can be about trading one dimension for another. - does it have to be instant or can we run it as a batch - maybe it’s enough to have this tool in English only Etc.

I think this is a vital point. Estimates help narrow down what is important and what isn't.

Maybe you want features x, y and z. If it takes a day then X is worthwhile, if it's going to take a month don't bother. Z must be done so if that's going to take a long time we need to drop y.

One of the best product owners I've worked for would often ask for estimates and base plans and features from those. Some things would be easy technically and valuable, others harder and just a nice to have (but worth it if it only pushed things back a day).

Re: Software effort estimation is mostly fake research

#172

Post by mxcrossb »

I need to get a check list for the ever so popular criticisms of academia article. This one surely would hit plenty of boxes: scientists don’t pay attention to industry, researchers only care about grants, etc. It’s a lot of words just to say “they need bigger datasets”, which is also a check box.

The article ends by suggesting using GitHub. When I was a PhD student I recall open source projects being a standard data source for people who did software engineering research. But it’s not my field, so I’m not sure if effort estimation really has missed this.

Re: Software effort estimation is mostly fake research

#173

Post by tikhonj »

xyzelement wrote: post #132
didibus wrote: post #129

Earlier quoted context omitted.

I think you're talking about goals, which isn't the same as estimates. A well run sales team, like a well run dev team do benefit from goals. Like increase X by Y. That's because it can guide your decision process of what you'll spend your time on and what you won't. It can also push for focused innovation around a particular area. But I think OP meant what if you asked the sales team: How much time will it take you…

Sales team certainly understand the sales cycle and different velocities involved. Eg - two jobs we ago sold to finance playera from small hedge funds to huge asset managers to central banks and finance ministries. You could sell more small hedge funds by knocking on more doors but you understood that the big players took months and years to decide. And of course we always tried to hone the sales org and process to a…

> the only limit to your velocity is you

I have never seen this to be the case outside small, self-contained projects. Even with a fixed scope (its own can of worms), major slow-downs often clearly come from outside the team—unexpected or unreasonable infrastructure limitations, problems with the products of other teams, shortcomings of internal or external tools and libraries, unclear or inconsistent evaluation of the work by its eventual users... etc.

Moreover, in cultures tied to estimates-as-quotes (or estimates-as-deadlines), there is often significant pressure for "product" teams not to try to fix these problems, even for themselves. It would be net faster for us to write our own version of X than to try to integrate the work of team Y? But they've already done the work, why would you want to do something redundant?

In a world where technical teams have clear ownership, this is less of a problem. Team Y giving you problems? Work around it the best way you can. Infrastructure not supporting your needs? Hack together your own on top of lower-level pieces until it's ready. But organizations with cultures that give technical teams the space to do this also have far less issues with estimates—both in having better estimates and being able to handle estimate uncertainty gracefully.

Re: Software effort estimation is mostly fake research

#174

Post by tikhonj »

lixtra wrote: post #143
koonsolo wrote: post #136

Earlier quoted context omitted.

I once had a manager that thought he could negotiate estimates. That is like negotiating with the weatherman about the weather. Sure, maybe in the end you can convince them it will be finished sooner. But the fact is that it really doesn't change reality.

Typically there are many dimensions to a solution, the negotiation can be about trading one dimension for another. - does it have to be instant or can we run it as a batch - maybe it’s enough to have this tool in English only Etc.

Negotiating scope is a different mindset than negotiating estimates.

Even better would be making sure the team understands both the real goal they're trying to achieve and how time plays into it, letting the team make scope trade-offs like that as they go along.

Re: Software effort estimation is mostly fake research

#175

Post by avdlinde »

randomsearch wrote: post #111
BowBun wrote: post #100

It's unfortunate that this HN thread has been reduced to the generic discussion about software estimates when the article is specifically talking about research done on the topic of software estimates. According to the article, proper research remains a struggle due to outdated datasets from before modern agile methodologies, and that the modern datasets from industry are hard if not impossible to gather. If industry…

I'd say the problem is more from the academic side. If good data isn't available, then academics should not be publishing papers on toy data. It's meaningless. The goal is not to publish papers but to advance science.

Well, the metric is #papers and #citations to get funding. So yes that's true, ideally, but...

Re: Software effort estimation is mostly fake research

#176

Post by andrekandre »

virgilp wrote: post #37
durandal1 wrote: post #21

Earlier quoted context omitted.

Apple executes massive feature releases on a yearly waterfall-like schedule across hundreds of teams. It requires honesty, transparency, strong leadership (ruthless prioritization), strong cross-team goodwill, strong cross-team collaboration as well as a solid body of experienced top-tier engineers. Other than that there's no magic. While there are teams within apple that does scrum/agile, none of the core OS/framewo…

Adobe executes (executed) massive releases on a well-defined cadence (1.5 yr per "creative suite"). Sure, I won't say they don't have collaboration, or top-tier engineers etc. But in the end, the success of this well defined cadence boiled down to the simple strategy of "we release what is ready". Everybody knew the deadlines well in advance, and everybody strives to 'make it' - but, (not so) occasionally, some teams…

> "we release what is ready"

this really is key, if you can do it

alot of companies though, are "business driven" which means releases and schedules are set beforehand, which makes this hard... (but if you can do it, its great!)

Re: Software effort estimation is mostly fake research

#177

Post by Ma8ee »

snarf21 wrote: post #98
didibus wrote: post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

The issue is that you are being asked to estimate something that has never been done before. Even houses always go over time and money and that is fairly straight forward. These days, I only give estimates in terms of units but without numbers. Hours, days, weeks, months, quarters or years. Some relatively small number of those units. If you want a quote it will take an extra 1/4 of the estimate worth of time for an…

I used to work for a company that sold custom business systems. When the customers did their parts, i.e., signed of specifications and did their acceptance testing within schedule, we usually delivered within estimate and in time. This was of course applications built with boring technology where 80% was simple CRUD. But the market for this should not be underestimated.

Re: Software effort estimation is mostly fake research

#178

Post by TeMPOraL »

gugagore wrote: post #169
meheleventyone wrote: post #140

Earlier quoted context omitted.

It doesn’t even have to be a big construction project to hit budget and schedule issues. Rebuilding our house had us going right back to the drawing board having to get new plans drafted and approved. It worked out much better in the end but the unforeseen cost and time overrun was quite anxiety inducing at the time. But yeah the idea anyone has magically solved accurately predicting the future is completely bonkers.

I like to remember "plans are useless but planning is indispensable". The plan is useless because of the challenge of predicting the future. But the planning is useful because trying to predict the future gets you asking good questions.

Also because planning gets you to make your desires explicit and to put them in a useful framework. Unexpected problems, disruptive as they may be, don't usually derail the whole plan. Some parts may need to be rethought, but others are still valid, and the whole thing keeps you focused.

Re: Software effort estimation is mostly fake research

#179

Post by lesdeuxmagots »

snarf21 wrote: post #98
didibus wrote: post #82

The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…

The issue is that you are being asked to estimate something that has never been done before. Even houses always go over time and money and that is fairly straight forward. These days, I only give estimates in terms of units but without numbers. Hours, days, weeks, months, quarters or years. Some relatively small number of those units. If you want a quote it will take an extra 1/4 of the estimate worth of time for an…

Sales teams do EXACTLY that. Go into salesforce and you’ll see the estimates.

Potential deal with client Foo will be for $X, to be signed on Y day, with Z% chance of closing.

On an individual level, it may be more precise than it is accurate, but it’s a fairly standard and important part of sales operations. And yes, the estimate dates are often just ways to express orders of magnitude (put next Friday as estimate dates not because it’ll definitely close next Friday, but bc it’ll probably take about 2 weeks). The % chance of closing are typically pre determined by what stage of the deal you’re in.

However, when applied across dozens, hundreds, thousands of potential deals, it allows the organization to understand their sales pipeline and sales velocity fairly accurately.

Of course the numbers are gamed a bit. And some sales people are much worse at estimating than others, or less consistent about keeping the numbers up to date. But the benefit of these practices are massive.

Re: Software effort estimation is mostly fake research

#180

Post by discreteevent »

benhurmarcel wrote: post #141
koonsolo wrote: post #136

Earlier quoted context omitted.

I once had a manager that thought he could negotiate estimates. That is like negotiating with the weatherman about the weather. Sure, maybe in the end you can convince them it will be finished sooner. But the fact is that it really doesn't change reality.

I find it much less stressful to deal with since I've made peace with the fact that some managers want to be lied to.

Be careful because those lies become expectations for everyone that manager speaks to.
Post reply on HN