Why I'm not a big fan of Scrum • Hacker News Board

Re: Why I'm not a big fan of Scrum

#351

Post by speby » Tue, Aug 09, 2016, 12:49 AM UTC

This article is, rightly so given the author, written from the perspective of an individual contributor on a Scrum team.

First, the author refers to Scrum as a methodology. It is not. Scrum is a framework, and being based in agile principles, is much more about a way of thinking and working. It should not be used as a purely do-this-then-that prescriptive approach to getting software built. As we now can see, there many organizations that tout being "agile," when their true behaviors and output are merely agile window dressing. Also known as "fragile" instead of "agile."

I disagree that Scrum is not useful to many organizations and teams. In particular, organizations that basically lack any process (trust me, I've been in a number of them) and it's kind of a free-for-all or based on whichever executive screams the loudest gets what they want when they want it.

What is missing here is why Scrum can provide useful, especially in terms of velocity. A team's velocity can serve as an underlying basis for projecting how long things can take. Like it or not, internal "users" or "stakeholders" in an organization as well as many external ones expect to get some idea of when things are going to start happening and when things are going to be done happening, for particular features or commitments. As expected, this involves being able to at least intelligently (and based on historical data) make a reasonable prediction at dates.

Entire books have been written about software estimation and no framework gets it right.

Re: Why I'm not a big fan of Scrum

#352

Post by timv » Tue, Aug 09, 2016, 1:30 AM UTC

kpil wrote: post #321

superuser2 wrote: post #267

Earlier quoted context omitted.

> technical debt is part of our KPIs. If it's not captured, it's not actionable This is where I suspect manager-types and developers have a vigorous divergence in values. Professionals routinely encounter situations where something is wrong and needs to be "actioned" but its wrongness isn't effectively measured by any metric (other than the opinions of the experienced people looking at it). There is a certain species…

On the other hand, at the end of the road - or the rewrite - something should have been gained in terms of money, risk or time. Being shitty is not reason enough, but it's almost always possible to reason and quantify and weigh the cost versus the benefits.

Senior staff in pretty much all organisations are given some latitude with respect to performing tasks that they believe will be beneficial without needing to produce a specified cost-benefit.

We have one-on-one catch-ups with staff, we have all-hands meetings, we visit clients, read books and articles, attend industry functions, issue press releases, meet with potential investors, etc.

There's value in all those things, but very rarely do we need to account for the fact that we spend time on them.

Senior engineering staff need that same freedom - it should be totally appropriate for them to make that call that "I'm accountable for the technical quality of this code, and I've decided that this needs to be done".

That requires some sense of business acumen - there needs to be an appropriate balance of technical investment to business investment - but that's part of the role of being a senior dev / tech lead / architect / whatever-your-organisation-calls-them.

Re: Why I'm not a big fan of Scrum

#353

Post by taneq » Tue, Aug 09, 2016, 1:36 AM UTC

lmm wrote: post #68

thackerhacker wrote: post #52

Earlier quoted context omitted.

It's a long time since I've had to deal with "strict" Scrummers. But I do remember being utterly baffled as to the insistence on user features being ready by the end of a sprint, even for quite complex, technical components. Why can't we make some sub-component this sprint then the UI bit the next? I tried various ways of reframing it such that the developer of the UI be the "user" but it didn't wash.

> Why can't we make some sub-component this sprint then the UI bit the next? Because that's how you get bad UI. The user-facing design needs to drive the API interface, not the other way around.

Depends what you're building. Which leads us to the worst (meta-)aspect of Agile/Scrum these days, which is that it's the industry's current favourite hammer and so it gets used to bash every single problem. Now, hammers are quite versatile, but you have to know when to bash, when to use the claw, when to lever or just nudge things instead of swinging.

The moment you start getting dogmatic about your process is the moment your decisions start being driven by something other than your actual needs at hand. And that's the moment when you start to produce a bad product.

Re: Why I'm not a big fan of Scrum

#354

Post by JoeAltmaier » Tue, Aug 09, 2016, 3:20 AM UTC

vpeters25 wrote: post #325

JoeAltmaier wrote: post #251

Earlier quoted context omitted.

And that's where the 'incremental tasks with no architecture' part comes in. When the scheduler needs rewritten to support whatever robustness; when the server feeds need to be redundant; maybe only 1 team member has any real idea what it takes. Coding isn't always about putting up another web widget or making another database entry.

That's the point: while there might be just 1 team member with a clear understanding of the effort, everybody gets to pick an estimate based on their own knowledge of the subject matter, even if they know nothing. The exercise of just enumerating in a couple minutes all the steps to complete such a story gives everybody, including the one who is going to do the job a clear idea of the effort or whether they need more…

Again, for a simple story where everything is known, sure. But some stories are all about figuring out how and what. The actual doing may be simple. The Scrum process fails.

Re: Why I'm not a big fan of Scrum

#355

Post by protomyth » Tue, Aug 09, 2016, 3:20 AM UTC

TeMPOraL wrote: post #345

protomyth wrote: post #332

Earlier quoted context omitted.

I knew some folks in the 90's that prototyped in Shockwave. They did UI and animated uses cases. I was impressed by how quick people got their blob-type people animated use cases that were basically little cartoons. Seemed like way too much work, but I guess it worked for them.

I've learned the hard way just how effective such prototyping tools can be if you only care about... prototypes. Or the visual stuff. Seeing a designer whipping up a running example in 15 minutes in Construct2 that was equivalent to what me and two of my friends spent last 8 hours coding has taught me to respect those tools, at least in particular use cases. And my point is, if we're focusing only on short-term clien…

To be fair to them, they were hella good C++ programmers and used Shockwave to prototype and lock down the business requirements. You can do an amazing amount of specification like that. It was basically CRC cards put to animation as people and things interacting.

They provided quite a bit of short and long term value by being better at giving clients an understanding of what they were actually getting. They cared about the internals and making sure their clients understood the logic the internals would use.

If I was skilled in some animation tool like they were I would do the same.

Re: Why I'm not a big fan of Scrum

#356

Post by redler » Tue, Aug 09, 2016, 4:06 AM UTC

abritinthebay wrote: post #300

redler wrote: post #228

Earlier quoted context omitted.

The problem with this is that it imposes a membrane on the process that is only permeable to work that contains a shippable piece of user-facing software. It presupposes that all work can be divided into such pieces. And since that's not true, it leads to contorted stories to squeeze necessary work through that membrane.

> It presupposes that all work can be divided into such pieces. No, it states the reasonable conclusion that all valuable features can be broken down into such (because otherwise if it adds no value to the product). There are also Scrum tasks around Research, Tech Debt, etc that are perfectly fine to create and work on but they have an affect on your overall time to build new features and that's ok . Because it needs…

You italicized "valuable features" and I understand why, but I think the trouble spot is actually the word "all"; your parenthetical about adding no value merely begs the question. I'd argue that, indeed, some valuable features can be organically broken down into two-week user-facing deliverables, but certainly not all and probably not even most. There's nothing magical about being able to partition a deliverable into a sprint's time frame that, by such distinction alone, makes it valuable versus valueless.

Systems like scrum try to wrangle into a manageable bolus a process that -- if we're to make good softare -- necessarily includes creativity, inspiration, and the traveling of paths yet unseen. It's like writing a novel and having two-week deliverables like "complete the arc of the Alice character", versus "write approximately 100 pages". It's not valueless to write 100 pages, despite not finishing the Alice section, and perhaps specifically because we discover that Alice's emerging story turns out to intersect perfectly with what we want to do with the Bob arc later on.

So there's writing and engineering and lines of code versus plotting and architecture and inspiration. We need all of it, right? Does everything that's not a "valuable feature" have to be shunted into the cul de sac of a research spike, doomed to be frowned upon by the management for whom the system otherwise provides the sheen of predictable velocity? Does the critical work of "dreaming" of what to do at both macro and micro levels become a casualty of the banal necessity of marching equal-sized boluses through the development tract?

I'm making a florid point, but to me this feels like the essential tension.

Re: Why I'm not a big fan of Scrum

#357

Post by kpil » Tue, Aug 09, 2016, 5:54 AM UTC

timv wrote: post #352

kpil wrote: post #321

Earlier quoted context omitted.

On the other hand, at the end of the road - or the rewrite - something should have been gained in terms of money, risk or time. Being shitty is not reason enough, but it's almost always possible to reason and quantify and weigh the cost versus the benefits.

Senior staff in pretty much all organisations are given some latitude with respect to performing tasks that they believe will be beneficial without needing to produce a specified cost-benefit. We have one-on-one catch-ups with staff, we have all-hands meetings, we visit clients, read books and articles, attend industry functions, issue press releases, meet with potential investors, etc. There's value in all those thi…

I agree.

I am not saying that you always need a detailed balance sheet, just that it's good to reason about the benefits.

It's especially hard to quantify risks - that's where you really need the expertise.

What I wanted to point out is that there seems to be quite many developers that actually do not have a sense of business acumen, and are willing to spend alot of money on low priority tasks, just for their own personal pleasure.

Re: Why I'm not a big fan of Scrum

#358

Post by crdoconnor » Tue, Aug 09, 2016, 9:11 AM UTC

Udik wrote: post #281

crdoconnor wrote: post #266

Earlier quoted context omitted.

>And if I am under pressure by the management is because I spent time developing without understanding what I was doing; a better developer would have gotten it right at the first try, and there would be no technical debt. This is possibly the most wrongheaded comment you've made. 1) There is no such thing as "no technical debt". It asymptotically trends to zero but never, ever gets there. If you think that you oe an…

Ok for your first point - although I'd argue that technical debt is something that usually asks to be repayed within months. A two year old technical debt is just an improvable software - that is, it didn't yet show problems serious enough as to call for a refactor at unchanged requirements . As for your second point, what should I do? Try to get it wrong? To get it working how ? We're not talking of premature optimi…

>Ok for your first point - although I'd argue that technical debt is something that usually asks to be repayed within months. A two year old technical debt is just an improvable software - that is, it didn't yet show problems serious enough as to call for a refactor at unchanged requirements.

I've worked on five year old technical debt. It meant that bugs were far more common and fixes/new features took 10-15x as much effort as they would have otherwise.

It wasn't that it didn't 'call' for a refactor - it's that the team didn't respond to the problems by refactoring. They tried the following instead: heavy manual regression testing before release (once in two years), waterfalling, longer and longer feature/code freezes, keeping multiple branches around for different customers.

Managerial response was to hire additional mediocre developers, making the problem worse, but it wasn't like hiring better developers made developing immediately quicker and less risky. Paying that debt down to a reasonable level was impossible with mediocre developers would take ~36 months with good developers (also working on bugs/features).

>As for your second point, what should I do? Try to get it wrong? To get it working how?

You should do red->green->refactor.

After writing a failing test, your only priority should be to make the test pass. Not elegant. Just passing. Once it's passing, then make it elegant.

The reasons for this are twofold:

1) You're solving fewer problems at the same time. Something you want to avoid as much as possible as a developer is to have to juggle 40 different competing problems at the same time.

2) Refactoring-driven architectural decisions are ~95% of the time better decisions than those made during up-front design.

>We're not talking of premature optimization here

It's a closely related problem but it's not identical.

Re: Why I'm not a big fan of Scrum

#359

Post by crdoconnor » Tue, Aug 09, 2016, 9:32 AM UTC

Udik wrote: post #287

crdoconnor wrote: post #259

Earlier quoted context omitted.

Because the same developer with the same skills can often ramp up technical debt to get a feature out in half an hour instead of a day, and there aren't any developers who haven't felt the pressure to do exactly that. Ramping up technical debt isn't always about speed, either. It's sometimes about risk - it's often less risky in the short term to copy and paste a block of code than it is to change a block of code and…

True. The amount of technical debt produced is a function of the time constraints and the person skills. In turn, the time constraints can depend on the skills of other people in the organization at planning, designing, figuring out requirements, managing the team and the process, etc. Saying "ah sorry, we'll have to work one more week/ month on this because, you know, technical debt" is sweeping all these possible i…

I don't see why. Technical debt is just a measure of how much crap there is in the code. It doesn't preclude having a discussion about how much that is to do with skills and how much that is to do with pressure/time constraints/existing technical debt.

The point of the dial is just to make the trade off between quality and speed that individual developers are making every day both explicit and management's responsibility.

It means if the dial is turned up to 100% management have no excuse for asking the question "why is our product a pile of crap?". It means also if the dial was at 60% for a year and a half the developers have no excuse for why the product is still riddled in technical debt, meaning that skills problems are distinguished from time constraints and managerial pressure comes with a cost attached.

Re: Why I'm not a big fan of Scrum

#360

Post by crdoconnor » Tue, Aug 09, 2016, 9:40 AM UTC

ScottBurson wrote: post #333

gearhart wrote: post #99

Earlier quoted context omitted.

When I read this I assumed you meant a dial that represents the current level of technical debt, rather than a dial that represents how much technical debt you're allowed to accrue . The other interpretation might be a neat solution. Allow developers to indicate publicly what proportion of the time their last tasks took, they should have taken had there been no technical debt. It would handle the communication from d…

At the very least you need both dials! If you have only the one that management sets, saying how much time to spend refactoring, it will always be at 0. (Okay, they don't literally have to be dials, but there has to be some communication of the kind you describe, so management has some idea how much time is being lost.)

>At the very least you need both dials! If you have only the one that management sets, saying how much time to spend refactoring, it will always be at 0.

Management decisions are usually CYA based, and leaving the dial set at 0 both exposes them and gives developers a get out of jail free card.

i.e. leaving it at zero is suicide. That's the whole point of making it a trackable dial.

Probably what would happen in most cases is it would fluctuate between 30% during average times and 0% during crunch times.