Title is editorialized. Original: "This code is CRAP" referring to code in review as Change Risk Anti Pattern. Also, (2011)
Isn't that what this title is? Which part is editorialized?
This Code Is CRAP (2011)
41–50 of 55 posts
Re: This Code Is CRAP (2011)
#42When a measurement becomes a target, it ceases to be a good measure.
Re: This Code Is CRAP (2011)
#43Wait, I thought all hand-curated enterprise code was godly pre-LLMs?
Re: This Code Is CRAP (2011)
#44The pendulum has swung too far in the direction of class, function, cyclomatic complexity (and here, CRAP) and similar idiotic metrics. This reminds me of a talk Sandi Metz did called "All the Little Things" where she covers the Gilded Rose kata. In the talk, she reworks her solution until there's almost nothing left showing the essence of the problem being solved. The cyclomatic complexity metric is touted at each s…
Reasonable cyclomatic complexity is useful (at least) for testability. No metric is perfect and no metric should be a primary goal, but it has its use, and if you read e.g. a bit of leaked Windows you certainly will understand why it matters (and not because it has a particularly low cyclomatic complexity...)
Of course if you introduce new methods of dispatch and do not take them into account into a metric, you end up with something less... precise? useful? But given the primary intent and why and how the metric was created this seems a pretty trivial observation.
Now I agree it is also retarded to attempt to get only very short functions or extremely low cyclomatic complexity everywhere (even if you try to adapt it to count new kind of dispatch), because the only effect that produce is that it moves the complexity in another more abstract place we are less well equipped to manage.
"Short functions used to be hot. Uncle Bob [...]": well yes, Internet and sometimes group of people inspired ultimately by Internet and group effects can be pretty idiotic, but honestly Uncle Bob ideology was never considered serious in actual studies, and it is now even widely recognized mostly bullshit. It is just a kind of tech influencer if you want. Computer science and/or software engineering has more serious branches, where cyclomatic complexity can have its use.
Re: This Code Is CRAP (2011)
#45> CRAP1(m) = comp(m)^2 * (1 – cov(m)/100)^3 + comp(m) and > Here’s why we think that CRAP1 is a good anti-pattern to detect. Writing automated tests (e.g., using JUnit) for complex and convoluted code is particularly challenging, so crappy code usually comes with few, if any, automated tests. This is so wrong. The formula uses code coverage as a fundamental metric, when in reality, a lot of people write code "correct…
If the code is simple, the tests aren't much extra work.
High test coverage doesn't mean your code is good, but it at least reduces the rate at which you accidentally break stuff.
Re: This Code Is CRAP (2011)
#46The pendulum has swung too far in the direction of class, function, cyclomatic complexity (and here, CRAP) and similar idiotic metrics. This reminds me of a talk Sandi Metz did called "All the Little Things" where she covers the Gilded Rose kata. In the talk, she reworks her solution until there's almost nothing left showing the essence of the problem being solved. The cyclomatic complexity metric is touted at each s…
Oh god, yes. The powers that be in my workplace are obsessed with cyclomatic complexity, so now we’re reviewing a flood of LLM diffs that take perfectly good code and extract each loop into its own function with a dozen arguments. The code is now more maintainable on paper and far, far less maintainable (by a human) in practice.
Re: This Code Is CRAP (2011)
#47I'm sure this method has evolved and/or been supplanted over the last 15 years, but one thing that struck me reading this is how much the dynamics of unit test coverage have changed in recent history, with AI-generated commits containing 10x as many unit tests (many of them kind of silly and tautological) as in the olden days. Gonna need to update some of those coefficients in their CRAP1 formula... Or maybe test cov…
[dead]
Re: This Code Is CRAP (2011)
#48I'm sure this method has evolved and/or been supplanted over the last 15 years, but one thing that struck me reading this is how much the dynamics of unit test coverage have changed in recent history, with AI-generated commits containing 10x as many unit tests (many of them kind of silly and tautological) as in the olden days. Gonna need to update some of those coefficients in their CRAP1 formula... Or maybe test cov…
Humans have been hand-crafting artisanal test cases of equal banality for years, often to meet some metric no one is verifying deeper than an automated report etc. I'm more inclined to believe the null hypothesis here, that the formula doesn't need any change and the signal is still equally valid. If anything changed, the signal is probably more useful now, since you'll never catch a good AI writing test cases that do literally nothing except call a function in a try catch with an empty catch block. (Models may have done that a few years ago but I haven't witnessed that behavior in a long time.) I've witnessed myriad examples of this in pre-AI production code.
Re: This Code Is CRAP (2011)
#49I'm sure this method has evolved and/or been supplanted over the last 15 years, but one thing that struck me reading this is how much the dynamics of unit test coverage have changed in recent history, with AI-generated commits containing 10x as many unit tests (many of them kind of silly and tautological) as in the olden days. Gonna need to update some of those coefficients in their CRAP1 formula... Or maybe test cov…
Anecdotally, I’ve found that codebases that enforce code coverage metrics often have worse behavior coverage than ones that don’t. It’s a classic example of Goodhart’s Law in action. Code coverage metrics only measure what percentage of code the test suite causes to run. But it’s very, very easy to write tests that run code without actually confirming that it produces correct output for all possible inputs. And it’s…
Djikstra seemed like he was mostly against testing. But only because he was for proofs. A unit test is a single example. The real way to demonstrate the absence of errors is to prove they aren’t there (vis a vis axioms and assumptions).
But most developers don’t have the mathematical sophistication nor the time.
It’s not that unit testing is useless. Just good to know what their limitations are and to plan your testing strategy accordingly.
Re: This Code Is CRAP (2011)
#50I'm sure this method has evolved and/or been supplanted over the last 15 years, but one thing that struck me reading this is how much the dynamics of unit test coverage have changed in recent history, with AI-generated commits containing 10x as many unit tests (many of them kind of silly and tautological) as in the olden days. Gonna need to update some of those coefficients in their CRAP1 formula... Or maybe test cov…
> Or maybe test coverage has/will become too noisy a parameter to use at all. It already is, ive banned unit tests via ci checks from our codebases, they were not particularly useful before LLMs and now they are a net negative. We require int and some e2es and that does all that units do and more.
Are they slow and brittle?
I like having both. Unit tests are a decent indicator for local development. Fast, quick to iterate on. Integration tests are slow as molasses and I can’t iterate with them when it takes 20 minutes to setup the suite and run everything. Too slow. But still useful as you say.