The Epistemology of Software Quality

> we usually omit it as too costly as if doing the first and only independent validation with our own users is not

Users will accept absolute shite and that is definitely a cost saving. Unfortunately. End users complain a lot but in the end they'll just work around bugs and this is why I blame a lot of the deficiency in current software development on end users.

1) pair up (from my POV) randomly (not everybody included in this is a coder)

2) (from my POV) flip a coin to decide who picks an item to review

3) pick an item you've touched recently or if there isn't one something you're going to touch in the next week (could be doc, build scripts, other things depending on what your job is; you might adjust your selection depending on your peer's duties and responsibilites, no sense having them review something they will never ever even look at).

4) flip a coin to decide who is going to explain what the code does

5) one person explains what the code does / doc means to the other person; the other person asks clarifying questions

Repeat the above steps weekly.

There was a sheet that was filled out that recorded defects found in categories such as logic, (failure of) abstraction, requirements, doc. We talked about style but there wasn't really an enforced style guide.

It can be uncomfortable at first to talk about somebody else's code or have them explain your code. You learn to listen and have a sense of humor and be patient. Perhaps the most poignant lessons learned had to do with doc in code (generally more is better but watch out for zombie comments) and sometimes the most clever way of doing something isn't worth the potential for misunderstandings. There was usually something which needed more doc! Improving tests was also a common outcome. We did find logic errors occasionally, which would inevitably lead to discussion around improving tests. Requirements issues and abstraction / architecture issues came up infrequently and typically spawned email or a meeting which came later.

> instead of ... asynchronously

Yes.

> What was the process if the artifact was "rejected"

We only did it once a week and all code was not reviewed. Well, I should say not all code was formally reviewed. And we had tests, and some of them came from "real" Engineering (level 1). We didn't have 100% coverage, but "more tests" was a frequent outcome, and it was implicit that this wasn't limited to the specific artifacts which were reviewed. There might be a timing or integrity issue and so there might be some attempt to determine what the appropriate scope of testing was (the emails / meetings I mentioned).

I don't recall that anything was ever rejected in one of these. [Edit - added] This was mostly front-loaded, because we never wrote code until there was some consensus on what change was needed. So some of these were somebody bringing a "problem child" and either walking through it or enjoying the entertainment as some rando did it.

The process existed with the implicit goal of levelling everybody into some shared framework of discussing code. We would reach out to peers all the time for informal review and it was pretty frictionless (email somebody with a branch and particular range of LOC). About the only time you wouldn't reach out like that was if you had pretty close to 100% test coverage. 8-)

> neither ... of the pair had enough understanding to fix it

In another fine example of "moral hazard for good" that never happened! Less facetiously those ended up being the requirements / architecture issues. So as far as the process was concerned the pair reached a consensus.

> synchronously asks clarifying questions of someone who is not the author

Yes, about half the time the person explaining the code was the one who'd never seen it before that moment. LOL. Brutal. OTOH I don't know if it's more brutal than explaining code to someone whose job doesn't actually include coding... Just different.

Like I said, it improves people's ability to talk about code and creates a culture that makes it easier to seek help the rest of the time.

No I do not. I've got an unindexed web server where I post random stuff from time to time, and occasionally I write articles on LinkedIn.

I'm heartened you care. I care about quality, what gets me up in the morning is helping people up their game. But like security (which is largely a quality problem) quality is something you do not something you buy (baseball comment farther upthread) and it's been an even harder sell since the VCs were given zero interest money to throw around.

The disciplines where people (are supposed to) care about quality and measurement seem to be captured by vendors with checklists and pseudo managers with alphabet soup after their names. I'm in a niche market which largely gets papered over by those parties. Most of what I'd write has been or would be co-opted and therefore would sound the same as what they'd write and I'd just be lost in the noise.

I'm open to smart ideas for finding the people who care enough about quality to realize that it's actually something you have people do utilizing the expensive toys you buy.