Avoid Indirection in Code

> We should extract small functions when it helps understanding the program.

This. If I may reiterate your point:

`is_foolike(x)` may not be much better than `x.startswith("foo")` but, say, `is_superuser(x)` is much easier to understand than `x.username.starts_with("su_")`. And note how the mental burden is decreased in this case as well, as a first-time reader would not be wondering why we're checking usernames in the middle of a function way past login.

It's the principle of checking separated from using which is wrong in almost all cases. Instead you should cause and/or handle errors!

(Also known in Python as duck typing when applied to types.)

When the author used generic "foo" examples, it's hard to tell if there's meaning behind it or not. In a real example, there might very well be meaning behind it, perhaps his "foo" examples in his actual life are more like your "checkHasPermissions" example.

But I think you're right that "abstraction" is the important concept here, which "indirection" is either a synonym for or one part of, depending on what you mean by "indirection".

I think the main challenge of organizing your code ("architecture") is figuring out the right abstractions.

And "too much abstraction" is definitely a thing.

In my experience there aren't any magic rules or guidelines you can just follow to get the right level and kinds of abstraction. There are guidelines and things to think about, and while getting more experience writing (and maintaining!) code is the only way you get better at it, reading and thinking and discussing it helps too.

Some things that have been attempts to guide people to the right levels and kinds of abstraction, like "design patterns", have proven to be no magic bullet either.

In actual experience, I think most of us end up erring on the side of too much/too complicated abstraction ("over-engineering"), so when it doubt, it makes sense to try erring on the side of less abstraction, including duplicating code at times (do not let "DRY" be your only guide).

Which, in the end, is what I think OP is telling us.

"While it's useful to have it split up into logically discrete chunks with sensible naming these 2-4 line functions far increase the mental load of the code."

I don't do this a lot, but in those cases where I'm tempted to pull out "helper functions" that I just call once, simply so I can give them a name, I often use blocks instead. That is, instead of:

    func DoTheThing(...) {
         x = doTheFirstPart(...)
         y = doTheSecondPart(...)
         return doTheThirdPart(x, y)
    }

    func doTheFirstPart(...) { ... }
    func doTheSecondPart(...) { ... }
    func doTheThirdPart(...) { ... }

I'll do:

    func DoTheThing(...) {
        // Do the first part
        var resultType1 x
        {
            // basically the contents of doTheFirstPart here
        }

        // Do the second part
        var resultType2 y
        {
            // basically the contents of doTheSecondPart here
        }

        // Do the third part
        {
            // doTheThirdPart here
            return final result
        }
    }

If you're literally going to call the function only once, and it just disrupts the flow, this has a number of advantages. In most brace-using languages, the braces will confine a scope just like a function would have, so you still have clear specification of what can flow out of the brace functions via the previous variable declarations. (They can stack up a bit if you have a lot of these sections but I've not had too much trouble with that.) If you do need to pull it out into a function later, it's a fairly mechanical process instead of a surgical process, because you're already 90% of the way there. But if you never need to do that, you retain the ability to simply read through a function and see everything it's doing.

I've noticed a lot of my "payload" functions sometimes end up long, but if you look carefully, you'll see I'm still using a lot of tools for scoping and simplification to confine action-at-a-distance within the function itself.

You do get a couple of weird looks and possibly some weird review comments if you use this; people are not used to seeing scopes used solely for isolation like this. I've had people ask me whether this is a syntax error or not in the review. But usually once you explain yourself it passes through.

This seems to mostly end up in my testing code, where I may need a chunk to set up the environment for a particular chunk, and then may have some tests I want mostly isolated from each other, but to use the same environment, or in a recent case, I had a chunk that setup the unique environment for the test, the code under test was basically "testEnvironment.DoIt(...)", and then I had half-a-dozen blocks verifying that the DoIt call had done all the things it was supposed to do and had all the exact side effects it was supposed to have, one class of them at a time, confined in braces so it's syntactically enforced they can't have leaking values. (This was an integration-level test.)

There are definitely other cases where simply naming things and breaking them into functions can be helpful, so you end up with an "orchestration" function that reads cleanly, and all the little bits also read cleanly. I do that plenty too.

Well, good abstractions/indirections are hard.

The wrong abstraction can be worse than no abstraction at all. And it can be non-obvious if you've got the "wrong" abstraction until much later when it's been in use for a while and had to be maintained -- and sometimes even then, we aren't good at recognizing that our pain is coming from the use of the wrong abstractions.

We often are, without necessarily realizing it, reluctant to refactor existing abstractions once there, preferring to alter them slightly or even more add new ones on top (which can make things worse in the long-run). And this isn't necessarily irrational, refactoring/removing existing abstractions is expensive, and with no guarantee you'll get it "right" this time either.

Sandi Metz says:

> prefer duplication over the wrong abstraction

https://www.sandimetz.com/blog/2016/1/20/the-wrong-abstracti...

Mechanically extracting things as in OP can often lead to "wrong" abstractions, which is what I get as the takeaway from the OP, which is a good reminder.

You are right that abstraction is the main tool we use as computer programmers, one way or another. Thinking in abstractions is the main thing we do that characterizes computer programming in contrast to other endeavors. So we certainly can't give it up entirely.

But this fact, I think, from my observations over my career, often leads us to over-abstraction ("over-engineering"), and the wrong abstractions. Because we are so taken with abstraction (it is neat, and you probably think so if you enjoy computer programming). It is good to be reminded that there is such a thing as too much as well as too little abstraction. Sometimes it's the "wrong" abstraction, but sometimes the "right" abstraction and the time it would take to arrive at it were unnecessary, and less/no abstraction would have served you just fine.