From design patterns to category theory

There is a flaw in this type of thinking which I think is not addressed here, but should be. (I took the quote from the summary, but I think it's fair.) The usual issue with ad-hoc informally specified things is that the ad-hocness and the informality leaves something out, making sure that if you understand the informal thing, but not the formal, you end up understanding only some part of the whole thing that you could understand if you tried.

So in the context of design patterns, that means there should exist situations where the extra abstraction, the non-ad-hoc non-specialized understanding, allows you to write code that is better than otherwise, and (more importantly) that you would not have otherwise written. This would show explicitly the gap between the ad-hoc ideas and the formal ideas instead of leaving you to guess.

Whereas the way this post tries to explain it, as I understand it, is that it shows you things (design patterns) that you would have otherwise written, and shows how it can be talked about in more general terms. But the things that you wouldn't have written are missing.

E.g., in some monad tutorials at some point one of the exercises is the nondeterministic choice monad, which is totally something you might not have written yourself, which makes it a way of showing why monads are a useful concept. If the only monad you had was IO, it'd be a much more useless idea (it would merely be a different, not even necessarily better, way of writing things that you already knew).

But many of the CS pioneers dealt with the issue of reading input, processing it, and giving a meaningful output. Recently I've stumbled upon Jackson's Principles of Program Design and it has really helped me in writing a parser. IMHO general understanding of structuring a program goes a long way than arbitrary design patterns. I now am a propenent of modular programming. For me it resembles functional programming and TDD. Basically it states your program should be made up of numerous self-contained modules that can be independently tested.

For data science workflow it could be an import package, further divided in modules such as general helpers and implementations for different file types; a computation package etc.