Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The drivers behind software delivery inefficiency

dl.acm.org

Post reply on HN
Print view View original

41–50 of 58 posts

Re: The drivers behind software delivery inefficiency

#41

Post by jamesfinlayson »

QuercusMax wrote: post #18
vasco wrote: post #3

It's hard to believe something when I saw the opposite work in practice. Removing QA makes developers care about production. You can say now they are wasting part of their time doing what QA did but to me it's not wasted time. Incidents went down both times I saw it happen, and stayed down.

Really depends on what you're having QA folks do . Writing automated end-to-end tests? Those are very often best done by those writing the software, with assistance from experts who know how to effectively build test infrastructure. I've very rarely seen them done effectively by QA or "test engineering" folks. Running manual test scripts and torture testing the UI to break it in new and unusual ways? Definitely get d…

> Writing automated end-to-end tests? Those are very often best done by those writing the software

This so much - but most of the places I've worked at have insisted that manual testers with zero programming experience should build and maintain the test frameworks. Talk about trying to push a square peg into a round hole. Granted some of the testers have embraced it and not done a terrible job but by and large most of the testers have had no idea and done very little.

Re: The drivers behind software delivery inefficiency

#42

Post by Corvyn »

vasco wrote: post #3

It's hard to believe something when I saw the opposite work in practice. Removing QA makes developers care about production. You can say now they are wasting part of their time doing what QA did but to me it's not wasted time. Incidents went down both times I saw it happen, and stayed down.

I struggle with this question a lot: when does specialization make a team better, and when does it start to erode ownership?

When I first joined Amazon, my team had no dedicated QA, Product Manager, Technical Program Manager, or frontline support team. I had come from a startup where none of those roles existed either, so it felt normal. Engineers also carried the pager and dealt directly with the operational consequences of what they built.

Since then, I have worked with teams of very different sizes and levels of product complexity: internal and external products, Tier 1 services and much less critical systems. Some had every specialization—QA, PM, TPM, and a dedicated support organization.

On a few of those teams, I encountered a very different engineering culture. Product understanding was the PM’s job. Quality was QA’s job. Delivery coordination was the TPM’s job. Customer problems were support’s job. Collectively they had created enough distance that engineers no longer felt accountable for the whole outcome.

For me it is less about whether a team has QA, PM, TPM, or support line, and more about whether those roles increase the team’s capability or partition its accountability.

The litmus test I use is the 3 a.m. page: does the engineer have enough situational awareness to understand the customer impact, navigate the system, make a credible mitigation, and help drive the permanent fix? They do not need to be experts in every discipline, but they should understand the product and system well enough to own the outcome.

Specializations are necessary as systems, products, and organizations grow. But it has gone too far when engineers can no longer operate effectively without every specialist standing beside them.

I am also exposing my own bias here: I believe engineers should carry pagers. That is how I grew up professionally, and I think direct exposure to production consequences produces materially better engineers.

Re: The drivers behind software delivery inefficiency

#43

Post by QuercusMax »

jamesfinlayson wrote: post #41
QuercusMax wrote: post #18

Earlier quoted context omitted.

Really depends on what you're having QA folks do . Writing automated end-to-end tests? Those are very often best done by those writing the software, with assistance from experts who know how to effectively build test infrastructure. I've very rarely seen them done effectively by QA or "test engineering" folks. Running manual test scripts and torture testing the UI to break it in new and unusual ways? Definitely get d…

> Writing automated end-to-end tests? Those are very often best done by those writing the software This so much - but most of the places I've worked at have insisted that manual testers with zero programming experience should build and maintain the test frameworks. Talk about trying to push a square peg into a round hole. Granted some of the testers have embraced it and not done a terrible job but by and large most o…

Building high-quality test infrastructure often requires more skill than building the software it supports. Some of the most impactful work I've done in staff/principal roles has been test infra.

Re: The drivers behind software delivery inefficiency

#44

Post by jamesfinlayson »

QuercusMax wrote: post #43
jamesfinlayson wrote: post #41

Earlier quoted context omitted.

> Writing automated end-to-end tests? Those are very often best done by those writing the software This so much - but most of the places I've worked at have insisted that manual testers with zero programming experience should build and maintain the test frameworks. Talk about trying to push a square peg into a round hole. Granted some of the testers have embraced it and not done a terrible job but by and large most o…

Building high-quality test infrastructure often requires more skill than building the software it supports. Some of the most impactful work I've done in staff/principal roles has been test infra.

Yep, very very true. Unfortunately some of the places I've been at more recently have believed that mock servers are the best way to do end to end testing. It was a battle I couldn't win, but no one was willing to see that testing critical UI flows against something that spits back canned JSON could hardly be considered thorough testing.

Re: The drivers behind software delivery inefficiency

#45

Post by yetihehe »

retatop wrote: post #31
onion2k wrote: post #16

I'm pretty certain that the one factor above all others that makes software delivery slow down is doing too many things at once. You should aim to be working on one project at a time in your team. Taking on another project, or workstream, or idea, or investigation in a team is pretty much entirely downside. - more context switching, which means people get cranky faster - lower bus factor for any piece of work, until…

I've never really understood this opinion. For me there's plenty of times that some problem needs to be solved and simply banging my head against it isn't going to make it happen any faster. Having some easy low priority stuff to do for a bit (or a high priority short task that interrupts) seems to let my subconscious solve the issue in the background and keeps me happier and more motivated. If it was two difficult t…

> low priority stuff to do for a bit (or a high priority short task that interrupts) seems to let my subconscious solve the issue in the background and keeps me happier and more motivated.

For a bit is ok. I had one month task, that was interrupted daily for over a year, that's how I got demotivated and burned out so much that I needed neurologist.

> two difficult tasks that were supposed to be done in parallel maybe that would be different, but I've never seen anyone asking for that.

Probably that's why you say interruptions are good, no experience of how bad can interruptions be.

Re: The drivers behind software delivery inefficiency

#46

Post by antihero »

NichoPaolucci wrote: post #33
antihero wrote: post #21

Earlier quoted context omitted.

I’ve genuinely found that context switching is a skill like any other. When I first started using LLMs and context switching doing 5-10 different work streams at once I was utterly, utterly exhausted at the end of each day. Now many months in, it doesn’t feel nearly as taxing. Maybe this is in no small part due to LLMs improving enough you don’t have to spend as much mental energy reviewing their output and them bein…

5-10 feels like a lot. I can reliably maintain hands on 2-4 separate work streams. I still need to maintain a pretty close eye on changes to our system, as I’m one of 2 devs. I’ve tried doing 5+ things at once and I just cannot get it to work. I think I move faster when I’m more involved anyhow, since I have adequate “mini-context” of what each operation is.

Our org has a really nicely set up review bot which has access to a repo of really strict standards to check new code against. It means that the reviews are much more useful and catch a lot of things that a plain bot wouldn’t. I think that’s a huge factor in being able to trust the reviews to catch more. You can also check the repo out locally.

Re: The drivers behind software delivery inefficiency

#47

Post by ares623 »

Out of curiosity, why is this a PDF on acm.org? It reads like a blog post, but it's presented like a scientific paper? There's a few LLM-isms too, so I suspect some level of slop.

Don't get me wrong, the subject and abstract is a topic I relate with so I'll probably still read it. But curious if this is normal. I just want to know how much salt I need to use.

Re: The drivers behind software delivery inefficiency

#48

Post by regularfry »

4lx87 wrote: post #32
regularfry wrote: post #23

For us, review latency was obviously a problem so we eliminated it (for most tickets, anyway) by making the handover synchronous. Get on a call with the reviewer when it goes into the review column. The win from eliminating review latency massively outweighs the context switch cost for the reviewer. All the more so when the patch is small enough to actually read through on the call, so if there is anything immediate…

> Get on a call with the reviewer when it goes into the review column. The context switch cost is simply moved to the reviewer, who presumably was working on something. It’s net the same result.

That is the context switch I'm referring to. It's worth the reviewer suffering a 15 minute (say) flow state loss to avoid the multiple hours the ticket would otherwise have been waiting to be picked up.

Re: The drivers behind software delivery inefficiency

#49

Post by dickersnoodle »

regularfry wrote: post #48
4lx87 wrote: post #32

Earlier quoted context omitted.

> Get on a call with the reviewer when it goes into the review column. The context switch cost is simply moved to the reviewer, who presumably was working on something. It’s net the same result.

That is the context switch I'm referring to. It's worth the reviewer suffering a 15 minute (say) flow state loss to avoid the multiple hours the ticket would otherwise have been waiting to be picked up.

Worth it to who?

Re: The drivers behind software delivery inefficiency

#50

Post by jurgenburgen »

QuercusMax wrote: post #30
transitorykris wrote: post #27

Earlier quoted context omitted.

Single Piece Flow fits with other Toyota ideas of how to get high quality work done. If something is frequently blocking the line, the solution is to put resources toward the bottleneck, not spin up more work in parallel.

Building software isn't the same as building cars. Some ideas needs months or years to be realized, and there's only so fast you can realistically move.

When is multi-tasking ever productive? The biggest time sink is understanding the problem. I can’t understand two different problems at the same time.
Post reply on HN