The Complexity Trap: Think Before You Leap
danielwestheide.com
The Complexity Trap: Think Before You Leap
1–10 of 47 posts
Re: The Complexity Trap: Think Before You Leap
#2Re: The Complexity Trap: Think Before You Leap
#3Has somewhat similar intro to the Rich Hickey's famous "Simple made easy" talk. A good read nevertheless.
Re: The Complexity Trap: Think Before You Leap
#4Has somewhat similar intro to the Rich Hickey's famous "Simple made easy" talk. A good read nevertheless.
https://www.infoq.com/presentations/Simple-Made-Easy
My favorite part is the knitted castle.
Re: The Complexity Trap: Think Before You Leap
#5> Doing that certainly means that you scrap quite a bit of boilerplate and reduce the complexity in your code. However, if you look at the bigger picture, this is the opposite of removing complexity. Exposing your domain model like this almost always leads to strongly coupled systems that are very difficult to evolve. If you ask me, this is one of the most promising ways to create a distributed monolith. It really amazes me how many people are totally okay with simply making their internal model their public API, especially since functional programmers are usually crazy about abstractions.
Good functional programmers are happy not to introduce an extra layer of indirection yet, because they know that their code is easy to refactor. The article itself seems to come to that point later, yet it gets it completely backwards here.
Generate JSON formats the easy, zero-boilerplate way for now. As and when you have a requirement for a JSON representation that looks different from your domain representation - perhaps because you want to evolve your internal domain model while keeping the JSON representation the same - you can introduce manual JSON codecs (or DTOs, or whatever you prefer) at that time.
> Object-oriented programmers call this technique stubbing, and they get mocked for doing it, mostly by people like us.
They get mocked for doing it because it breaks all the rules of the language. It usually relies on reflection, bytecode manipulation, or other such horrors. It's often thread-unsafe. Having tests that you can't refactor the way you normally refactor code is a huge, mostly-hidden cost.
> But you have no clue if your production interpreter, which is talking to an external system, is working correctly at all. So abstracting over the effect monad and using the Id monad in tests doesn’t even help you to write meaningful tests.
Tests need to be decomposed like any other code. There's a huge value in testing your business logic separately from testing your interactions with external systems. Functional core, imperative shell achieves that, but at a higher cost: you can't interleave your interactions with external systems and your logic in the natural way in code, you have to explicitly, manually push all the interactions to the edge instead.
> What if we did server-side rendering of HTML instead? There wouldn’t be any need to design an evolvable JSON API, which, unlike simply exposing your domain model, is not trivial, and there would definitely be no need for defining any JSON codecs. And that’s just what you get on the server side. On the frontend side, you would have one piece of software less to maintain, you’d get faster load times and you could get rid of duplicated business logic.
You'd get slower load times as you had to do page transitions, and more duplicate logic as things like form validation had to be done on the frontend and backend with separate code.
> At some point, we realize that we have an architectural problem at the macro level. Maybe, if we continue to ask why long enough, we will come to the conclusion that all these services we need to communicate with exist in exactly this way because of how the company is organized.
And which is easier to fix? At some point you decide that the entirety of western civilization is fundamentally flawed, and you can either start ranting about how we need to rebuild society from the ground up, or you can put a bit of complexity in your software and make something actually useful.
> I was recently talking to someone about Rust, and that person said, “A language without higher-kinded types is completely useless!”. Yes, certain abstractions are not possible without them, but I have never seen a project fail due to a lack of higher-kinded types.
That may have been me, and I absolutely have seen projects fail due to lack of higher-kinded types. Not in an obvious way; rather it felt like a general sludginess to development, a need to repeat the same tasks over again, until the project eventually dies from a thousand cuts. (Hell, for all I know the company I left several years ago may still be trying to make that project work).
> Projects fail because the problem domain is not well understood, or because the architecture is not a good fit for the requirements. They also fail because of how the organisation is structured and because of politics, be it intra- or inter-organizational. Projects fail because of people, and because of people not talking to each other.
This is facile. Projects fail because of people talking to each other too much. Projects fail because people tried to fix their organisational politics instead of implementing technical workarounds. Projects fail because the technology choices were simply bad.
Reading this article was weird, because I found myself nodding along with most of the woolly principles but then massively disagreeing with every single example. It makes me think the article is arguing at the wrong level of abstraction somehow: no-one thinks you should use a complex solution when a simple one will do. Very few people will claim that the problem with today's software is that programmers think things through too much and should rush to write code instead (though for the record I think that's actually the case). But those consensus principles don't actually help us at all when it comes to discuss specific decisions.
Re: The Complexity Trap: Think Before You Leap
#6As one who frequently deals with Scala code, I have to say that this guy is right: the Scala community in particular seems to relish in complexity.
Scala is a wonderful language and well-designed, especially in contrast to Java. The learning curve requires a good deal of effort, but with experience, it becomes incredibly fun to write Scala code. Once I got past the learning curve, Scala became my go-to language.
That said, there’s an underlying arrogance of Scala developers who, in my experience, tend to often abuse the language to show off their cleverness at the expense of users and maintainers. I don’t know why this is. I suspect it’s because Odersky set the bar so high with his obvious brilliance that some feel insecure.
One take away quote for me:
“Unfortunately, we don’t like boring. We like to take the language and the compiler to their limits. We also like to adopt the latest fad. But the software systems we work on in our jobs should not be a playground or a test bed for the latest fad.”
Scala developers aren’t doing anything new here. People have abused fads in programming for decades. “Design Patterns” and “Test Driven Development” and “RESTful” come to mind. However, it’s unfortunate that Scala developers in particular do this. Because, Scala seems to be a language specifically designed to avoid past mistakes.
Re: The Complexity Trap: Think Before You Leap
#7As one who frequently deals with Scala code, I have to say that this guy is right: the Scala community in particular seems to relish in complexity. Scala is a wonderful language and well-designed, especially in contrast to Java. The learning curve requires a good deal of effort, but with experience, it becomes incredibly fun to write Scala code. Once I got past the learning curve, Scala became my go-to language. That…
I worked professionally for a year with Scala. For me the biggest problem were functional fanatics that design solutions only top 20% of devs can understand. The rest comes from the object oriented realm and can't deal with monad transformers etc. That might work in small teams, but when you want to scale a team to 100 people you can't ignore this effect. And functional fanatics are strong in ignorance.
Re: The Complexity Trap: Think Before You Leap
#8>> Hence, we try to isolate business logic from effects and implement the former as pure functions.
This is a valuable insight. Especially if your main programming language supports both functional and procedural and/or OOP approaches; This statement offers a good guideline as to where the boundary should be.
I find that functional programming is great for business logic but not as great for more concrete logic where performance is important. For example, when dealing with low level concrete objects like sockets.
Re: The Complexity Trap: Think Before You Leap
#9“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?”
--Brian Kernighan
How many times have I bitten myself this way? Dear past me: stop being so smart.
Re: The Complexity Trap: Think Before You Leap
#10As one who frequently deals with Scala code, I have to say that this guy is right: the Scala community in particular seems to relish in complexity. Scala is a wonderful language and well-designed, especially in contrast to Java. The learning curve requires a good deal of effort, but with experience, it becomes incredibly fun to write Scala code. Once I got past the learning curve, Scala became my go-to language. That…
I think the equivalent to this in Python is "How many list comprehensions can I nest!?"
Solving things with list comprehensions is very idiomatic python, so the thought process goes "the more of these I can stuff inside each other, the more idiomatic my code is."