Live data from Hacker News

An epic treatise on scheduling, bug tracking, and triage

apenwarr.ca

51–60 of 67 posts

Re: An epic treatise on scheduling, bug tracking, and triage

#51
Interesting article, I have a few comments/questions

Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get thought of in time units. What techniques have you used to solve this problem?

Defects - some good points were made but you never really discussed how these are managed as work. Stories and defects are getting worked on at the same time, but how exactly? Are defects just lumped in with stories and the PM prioritizes them? I do not think so, but you are not clear in this area. If a PM prioritizes a story but the team spends all of its time resolving defects then how is value being delivered as expected? You seem to have left this out completely, or I missed it.

Finally, also on defects, while your points on not estimating them makes some sense, someone has to decide what to work on, and generally you need some idea how long things will take to make that decision. Even in your wording, you acknowledge some defects take a long time to fix, where as others are super-quick and in the end they all average out. Still, someone in the organization still cares about dates and delivery. It made a lot of sense to me where you indicated that the story point estimate can be valuable to the PM in terms of prioritizing. When they see a story with a high estimate that might cause them to lower its priority compared to other stories they can get delivered quicker. But seemingly defect fixing would have to factor into this somewhere too, and then wouldn't the same concept apply? If defects are not estimated, how can the time it will take to resolve the defect inform the decision making process?

Re: An epic treatise on scheduling, bug tracking, and triage

#52

Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get tho…

The whole "point" of story points is that they are meaningless until after something has been actually accomplished. After working for a few weeks, it's pretty easy to say something along the lines of "the team got 52 points finished and deployed last week" - this is useful to know, and over time you'll develop an intuition that may be useful for forecasting what's likely to get done over the next few weeks or months on this particular project with this particular team. That's it, that's the most you can do. Most of the other stuff people try to do is so noisy that it makes for nice spreadsheets of the "plan" but bears very little resemblance to reality.

Re: An epic treatise on scheduling, bug tracking, and triage

#53

Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get tho…

The whole "point" of story points is that they are meaningless until after something has been actually accomplished. After working for a few weeks, it's pretty easy to say something along the lines of "the team got 52 points finished and deployed last week" - this is useful to know, and over time you'll develop an intuition that may be useful for forecasting what's likely to get done over the next few weeks or months…

I get that, but when you have that first estimation meeting with the team they inevitably ask what a point is. They get the relative sizing idea but you still need to size the first stories.

Re: An epic treatise on scheduling, bug tracking, and triage

#54

Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get tho…

I actually reject the idea of story points as a unit-less number, mostly because people can't help but think about them in terms of time anyway (whether they want to or not) [1].

Rather, I like to use a discrete list of story point values as scalar value that represents a probability distribution for how long the task might take. As the story point gets bigger, not only does the mean time get bigger, but the variance grows as well.

For example, a 1 is 2-4 hours, but a 13 is 2-3 weeks, and a 40 is 1-2 months. The idea is that not only do more complex tasks take more time, but the precision of our estimates goes down.

This makes engineers happy because they get to be more honest about estimates, and it makes managers happy because they only have to deal with one number.

[1] Unit-less measures might be easier in an environment that is more concerned with true productivity than with deadlines, but that is not most environments.

Re: An epic treatise on scheduling, bug tracking, and triage

#55
post #6

I’m the author of the article. I apologize for its massive length but wasn’t able to make it shorter without losing my favourite bits :) I’m happy to answer any questions people have here.

Very interesting article, thanks for writing it and sharing it, I forwarded it to my PM :) . I do think it could have been edited without losing your favourite bits though :)

Re: An epic treatise on scheduling, bug tracking, and triage

#56

Earlier quoted context omitted.

I think we agree here, except on definitions. To me, potential "methods" are the list of things you describe: taxes, classrooms, teachers, home life, baseball bats. And also the ones I described: teach to the test, fraudulent scoring. A manager that just says "teachers will be evaluated based on their students' standardized test scores" will get nothing useful, because they did not provide a method, and the teachers…

> A manager that just says "teachers will be evaluated based on their students' standardized test scores" will get nothing useful, because they did not provide a method There is no method for a teacher to solve those problems. It's a systemic issue.

He didn't say anything that was contrary to that. Note that it's the "manager" that didn't "provide a method" in his framing, and note that "manager" is a shorthand for "the teacher's boss, the school board that created the policies the teacher's boss follows, the federal/state governing bodies that created the legislation that governs the school board's decisions, etc etc"

Re: An epic treatise on scheduling, bug tracking, and triage

#57

Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get tho…

I actually reject the idea of story points as a unit-less number, mostly because people can't help but think about them in terms of time anyway (whether they want to or not) [1]. Rather, I like to use a discrete list of story point values as scalar value that represents a probability distribution for how long the task might take. As the story point gets bigger, not only does the mean time get bigger, but the variance…

Thanks and interesting. If you have written up your full system anywhere I would be interested to read more.

Re: An epic treatise on scheduling, bug tracking, and triage

#58
post #47

Earlier quoted context omitted.

Sure, we could treat all stories as being of a single size and rely on the central limit theorem, but that takes far, far longer to converge. On the order of years, I suspect, not weeks or months, which is what you get out of pointing. "We should have this done sometime between now and 2022" is not a useful metric for a PO. We add story points because given consistent incentives, estimates tend to be consistent. Mayb…

Your claims are outrageous with no supporting evidence. If you just want to throw out anecdotes I'll give you my own. I collected data on all the pointed stories at one previous job for three years and found a negative correlation between story point and time from a story being started to being completed. Sure, you might claim that it all averages out in the end, except that our velocities were wildly in flux for tha…

Outrageous? Seriously? Hyperbole much?

I doubt that you did, if I understand you. Because it sounds like you're saying that overall 3 pointed stories took the most amount of time, and 13 (or whatever your max) took the least.

But let's say it did. I'd look to see why your estimates fluctuated all over the place. Or whether stories were being closed when they were actually finished (i.e., being accurately reported). Did you have deadlines? Did you have delivery pressures? Because that right there is a good reason; as you near a deadline you start padding estimates more. Did you keep having things come up that broke the sprint? Etc. All manner of things can cause estimates to be wrong. But not negatively correlated, -especially- with velocities constantly in flux (and I mean seriously in flux; you take an average because it can and will vary, especially if there's unexpected stuff, like someone getting sick, that you didn't account for when planning); that to me, yes, definitely sounds like you were doing something very, very wrong.

Re: An epic treatise on scheduling, bug tracking, and triage

#59
post #44

Earlier quoted context omitted.

If you look at the first Central Limit Theorem slide in the original article, it compares story precision vs precision of unestimated bugs. The short answer is that if your stories are big, not estimating them causes more error (weeks or months) than most people are willing to accept. Not so with small tasks (bugs) which average out, as you suggest. However, it’s hard to make long-range estimates using only many tiny…

Except you can't, as I've said, engineers are not good at estimating. They get significantly worse when you start estimating months out instead of days out. The problem is not that they're engineers, no one is good at estimating work they've never done before. This is a well known problem in pretty much every single software shop I've ever been in. Teams never deliver what was planned on time, only functional teams c…

[deleted]

Re: An epic treatise on scheduling, bug tracking, and triage

#60

Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get tho…

You're right, those two points are inadequately covered in the article. Thanks for reading so carefully :)

Story point size: the usual thing to do is to have "baseline tasks" (which the whole / almost the whole team did in the past) that you choose way back at the beginning, and then continue to use them as reference whenever estimating in the future. To do this, a couple of people who know the "approximate size" of a point estimate some, say, 2-point and 5-point past tasks, but don't tell anyone else how big a point was. And after that, they try to forget how big a point was during the baselining process. But you never, ever let people ask about time; you just say "was it bigger or smaller than the baseline 5-point task"?

Defects: in the model as it's being discussed, we assume (perhaps too optimistically) that we generally fix bugs before adding new features. This is why, in the second simulation slide [1], each subsequent feature takes longer than the last. Eventually, you cannot sustain this method if you still want to launch new features and your team size hasn't grown and you haven't contained the number of new bugs somehow; I can't tell you what to do when that happens. It's hard.

If you have mostly small bugs (which is common; just fix them) and a few large bugs (also common), then if the bugs are important, they can probably be described by writing a story. At that point you can elevate them to the estimation and PM prioritization process.

Beware, however, that in general, if a bug is introduced by adding a new feature, you should almost always fix it before launching that feature. Otherwise you have basically lied about how long it took to implement that feature, and as that gets worse over time, it progressively upsets your estimates.

[1] http://apenwarr.ca/log/?m=201712#slide16

Post reply on HN