[deleted]
Software effort estimation is mostly fake research
231–240 of 318 posts
Re: Software effort estimation is mostly fake research
#232It's unfortunate that this HN thread has been reduced to the generic discussion about software estimates when the article is specifically talking about research done on the topic of software estimates. According to the article, proper research remains a struggle due to outdated datasets from before modern agile methodologies, and that the modern datasets from industry are hard if not impossible to gather. If industry…
I could imagine getting data from tools like Jira, but there is so little consistency on how data is entered and updated, I would have a hard time swallowing any conclusions from that data.
A firm like Pivotal Labs (which, I just discovered, is now VMware Tanzu Labs, which is frankly the saddest thing I have read all day) would have a dataset and a good enough understanding of the assumptions in their Pivotal Tracker software to do good research here. I'm almost certain that they have internal papers that are far beyond the state of the art, so to speak, in the broader academic literature.
Re: Software effort estimation is mostly fake research
#233It's unfortunate that this HN thread has been reduced to the generic discussion about software estimates when the article is specifically talking about research done on the topic of software estimates. According to the article, proper research remains a struggle due to outdated datasets from before modern agile methodologies, and that the modern datasets from industry are hard if not impossible to gather. If industry…
I'd say the problem is more from the academic side. If good data isn't available, then academics should not be publishing papers on toy data. It's meaningless. The goal is not to publish papers but to advance science.
While I tend to agree, the research we environment demands this nonsense. Also, they have to start somewhere to set a record to get future funding. Should probably do more survey studies of what data exists and where the state of things are then suggest new studies and seek funding for it.
Short of large and completely government funded development projects, I think it would be a struggle to get data. Few businesses would be willing to offer up development processes and surrounding data due to potential IP lost. Any organization that has good processes have it in their interest to striffle others from discovering how to improve theirs or learn from the success of others.
Part of the issue is that academia largely just doesn't pay for software development which they can leverage as an accessible cheap data source, it's done as a completely privatized exercise. Any research that requires highly protected commercial processes is pretty difficult to get any traction on unless you're inadvertently rediscovering the same processes (from my experience).
With that said, I feel like the amount of empirical data you'd need is going to be incredibly high, much if it not even currently being collected.
Re: Software effort estimation is mostly fake research
#234Earlier quoted context omitted.
Have you noticed their declining OS release quality in recent years? I have, and I now wait 6 months to upgrade. The triangle is real[1]. Something's gotta give when a deadline can't move, and as we know from Brooks, software development speed doesn't increase linearly with more people. So it's primarily quality or scope that must suffer to hit a software deadline. [1] - https://en.wikipedia.org/wiki/Project_manageme…
People have been saying that since OS X first came out, and OS 9 was so buggy it had a famous cute bomb icon for when it crashed.
Don't the bomb icon and "Sad Mac" go back to the very beginning?
Re: Software effort estimation is mostly fake research
#235The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…
I call it the "What's the earliest date you can't prove you won't be finished by?" management question.
Re: Software effort estimation is mostly fake research
#236I've been in this business long enough to know that point estimates are always wrong. A proper estimate is a range with a confidence interval. When forced to do a fixed bid, you have to raise the price even higher to the upper end of the cone of uncertainty.
This agrees with my experience as well. Stop giving point estimates.
Re: Software effort estimation is mostly fake research
#237The issue with estimates are expectations. While nobody acknowledges it, you're not actually asked for an estimate, you're being asked for a quote. The difference is when you're asked for a quote, you're asked how much you will be charging, with the expectations that you'll be willing to eat into your own margins to give a lower quote. That's why it's a negotiation, where you negotiate how much extra effort, time and…
The issue is that you are being asked to estimate something that has never been done before. Even houses always go over time and money and that is fairly straight forward. These days, I only give estimates in terms of units but without numbers. Hours, days, weeks, months, quarters or years. Some relatively small number of those units. If you want a quote it will take an extra 1/4 of the estimate worth of time for an…
Estimating in terms of units is mostly for internal teams with a fixed payroll. It can work really great in that context. But imagine that you are asking for project funding. The person across the table logically wonders: how much do you need? You yourself logically wonder: how much do I ask for? In that environment, can either party really accept “who knows” or “5.5 units” as the answer unless money is not a concern for anybody?
Re: Software effort estimation is mostly fake research
#238My problem with this is all estimation is hard. Period. Quantitative discussions of something that has been done? Fairly easy, if still not accurate. Discussing something that hasn't been rehearsed before? You can really only discuss how long you are willing to work on it. Not how long it will take to get done. Fun examples. How long would it take you to clean out your fridge? How long would it take you to learn to p…
Estimation is hard if you are stuck in the mindset that you have to make a point estimate. It's not hard at all if you are willing to be (and are allowed to be) honest about the uncertainty of the inputs and calculate the uncertainty of the final result based on that. It's true that people may demand precision that you can't give them. But at the same time, you know something and it is simple to compute what you know…
Even distributions can fail in execution. I can't remember the name, but the joke rule that work expands to fit the time given to it. Such that, if all work expands to the max of the distribution, your time is easily set at the point of the max.
Of course, this has been tried and doesn't work. The max time on many distributions is effectively infinity. Which, fails. For obvious reasons. And is why I assert you can really only talk of how long you are willing to work on something. Not how long it will take.
Re: Software effort estimation is mostly fake research
#239Earlier quoted context omitted.
The issue is that you are being asked to estimate something that has never been done before. Even houses always go over time and money and that is fairly straight forward. These days, I only give estimates in terms of units but without numbers. Hours, days, weeks, months, quarters or years. Some relatively small number of those units. If you want a quote it will take an extra 1/4 of the estimate worth of time for an…
Estimating in terms of units is mostly for internal teams with a fixed payroll. It can work really great in that context. But imagine that you are asking for project funding. The person across the table logically wonders: how much do you need? You yourself logically wonder: how much do I ask for? In that environment, can either party really accept “who knows” or “5.5 units” as the answer unless money is not a concern…
In my opinion, the estimate gives them a sense of scale. If some small number of months would cost too much, then you know and can start to have an investigation project of a week to better help define what should be in and what shouldn't.
Re: Software effort estimation is mostly fake research
#240Earlier quoted context omitted.
> Even houses always go over time and money and that is fairly straight forward. This is an incredibly important point. I was "raised" in commercial software at a time when there was a lot of condescension internally towards ourselves as an industry because we couldn't do estimation and planning, and building was the favorite comparison. A $100 million building couldn't run late or over budget. That would be unimagin…
I've seen a lot of industries compared to programming, like construction and manufacturing. For example, people trying to apply lessons from the Toyota Production System to programming. This doesn't make sense, because we have systematized the construction of executable programs with extremely high reliability and predictability: we call it "Copy File" It's the design that is the hard part, and the process of coding…
I like to compare programming to the marketing industry.
Highly creative work and when deadlines are involved the work becomes spurty (sprints) with corner cutting (scope reduction) to meet unrealistic and over committed deadlines.