61–70 of 162 posts
Since this seems to be the place for personal views, my views, after 20 years of experience, is that people do not notice the complexity of anything that they have digested as normal. As a result doing anything other than what they are used to seems very complex to them. Whether that other thing is actually complex to a neutral third party is situation and observer dependent. I have seen this with procedural programm…
Since this seems to be the place for personal views, my views, after 20 years of experience, is that people do not notice the complexity of anything that they have digested as normal. As a result doing anything other than what they are used to seems very complex to them. Whether that other thing is actually complex to a neutral third party is situation and observer dependent. I have seen this with procedural programm…
When learning functional, I had a haha moment as I was coming from OO.
When learning React, I had another haha moment as I was coming from regular JS libraries.
There is an objective truth that is there or not.
> What if there is no "complexity trap", just disagreements about how best to design programs? While I think this post asks reasonable questions, and has some healthy skepticism, the complexity trap is very real, in my experience. After 20 years of professional software development, I can safely say I've watched literally tens of millions of dollars wasted over-engineering solutions to problems that were anywhere fro…
I'm fairly young for my new role and architecting (from the ground up) some fairly big systems. I'm really struggling not to over complicate problems and I think it's really easy to do for a couple reasons.
The selfish architect I'm working very hard not to be would rather have his work be complex. This builds ego if it works and you can always argue "it's a complex problem" if it doesn't. He'd rather face the shame of maintaining software that's overengineered as that's a very low visibility cost. No one bats an eye at estimates to update an existing system if it's needed. On the other hand, failure to account for a use case you deemed 'a bit complex for MVP' is super visible, even if it's a miscommunication between product and development. This selfish guy doesn't have eyes for business needs, he just falls in love with solving a problem, when he starts to elaborate on capabilities that it could have, product manager will just nod and say "oh yea, that might be nice", so you over-engineer to make sure that pathway is open, even if the product lady forgot about that possible feature before she left the meeting. He's also afraid of showing up at a meeting saying we solved the problem with a boring tweak on an existing implementation and having someone show him up by asking questions about why he didn't just use a software platform or language that felt foreign and complex when he read a few articles about it on reddit.
Idk, felt good to vent some of this stuff, but those are the things I have to make sure I'm aware of, otherwise I'm just gonna build shit to the ceiling.
A big part of this is that programmers have different programming worldviews. Many people seem to be unaware that worldviews are not perfect rational easily modifiable constructions but are instead foundational structures that thinking builds upon automatically. Programmers are about as unlikely to easily change their programming worldviews as they are to change their political beliefs. Which is not to say that progr…
This is a great comment, and very succinctly put. Somewhat incidentally, but I believe this is a large part of why, to me, software development is not an engineering discipline. At the end of the day, engineers (I think; not actually being one myself) have to answer to physics. The 'possible model space' is relatively constrained, and everyone is working under, roughly, the same set of assumptions, rules, and standar…
As a software engineer who calls what I do "engineering", I have to answer to physics every day. The difference is that the relative cost of everything is skewed so far out of proportion to what other engineers work with that it's hard to account for the differences, and software engineers work with so many orders of magnitude. We are still working with physical stuff, basically refined sand and metal, which consumes electricity and other resources, just like other engineers.
Imagine that you were a civil engineer tasked with building a bridge. Except instead of using concrete and steel, I handed you some adamantium at the low, low cost of $0.01 per cubic meter, and a bunch of robots to machine it for you. Designing bridges is easy now, and you can basically make it look like whatever you want, just because adamantium is so amazing.
But just because we've made bridge-building trivially easy doesn't mean that civil engineering is obsolete. Adamantium is nice, and all, but now I'm asking you to build a Dyson shell, and you have to tell me that it's not physically possible, even with adamantium.
I would love to just handwave the design of the systems I work on, I would love to just arrange pieces however I felt like. Instead I have all these constraints like the reliability of components (hard drives fail), physical laws (speed of light constrains network latency), the memory hierarchy (the software manifestation of constraints on circuits), and all sorts of other problems that I can't just pretend don't exist. I have to choose algorithms that will complete with the computational resources that I have available. Maybe an exact algorithm is not physically realizable, maybe I have to choose some kind of approximation.
This is not true in mathematics. In mathematics, algorithms are not necessary (at least, for most mathematicians). In mathematics, I can prove that some object exists without concerning myself with whether some physical system is capable of constructing that object.
At the end of the day, I am writing software, and that software is one part of a real, physical system which obeys physical laws. This makes me an engineer. Maybe I play around with 20 orders of magnitude in my designs and the civil engineer plays around in 8, but so what?
Earlier quoted context omitted.
When you're fighting for speed on your code, it ends up feeling like what you're describing of that engineering world. Your techniques have to adapt to what is actually happening in the lower layers. When you're fighting for ease of use, you can't make your users forget their preconceptions, you have to adapt to these externalities. I think sometimes engineering and software development have different degrees of free…
I think programming has more trade-offs to choose from, and thus more freedoms. You can make a program slow, fast, small, large, limited, comprehensive, quickly written for a specific case or carefully designed for the generic case, or anything else at the expense of something else. Building a bridge doesn't have much of that: the bridge must withstand loads from the traffic, loads from the elements, material wear ac…
Many software projects are profitable without being anywhere near physical limits. Other software projects are not, if you are working for an HFT, then you're counting microseconds and measure distances by how long it takes light to travel. If you're storing 10^10 bits of data you don't need to even think about it, at 10^15 bits of data you can go to the store and buy a bunch of hard drives, and at 10^20 bits of data you need a team of experts to make it even possible, never mind cost-effective.
If civil engineering looked like software engineering, a big chunk of our civil engineers would be building popsicle-stick bridges to carry occasional featherweight loads across 1-inch gaps, but there would still be some real bridges out there.
Earlier quoted context omitted.
Have you seen the mkdir world view? It's the one where all the code is spread out over as many directories as possible, and leaf directories typically have no siblings just yet and one child.
Is this a particularly egregious worldview that should not be tolerated?
A big part of this is that programmers have different programming worldviews. Many people seem to be unaware that worldviews are not perfect rational easily modifiable constructions but are instead foundational structures that thinking builds upon automatically. Programmers are about as unlikely to easily change their programming worldviews as they are to change their political beliefs. Which is not to say that progr…
Amen to this, and by the way, it doesn't even have to be in the context of a discussion about a particular issue, let's say issue A. You can blunder into someone else's kneejerks unknowingly, on a whole other topic called issue B, and they will MAKE it be about issue A.
Earlier quoted context omitted.
That's funny because I have become, and observe the exact opposite. The notion of trying to make things more complex than they need to be I find deeply problematic. Junior engineers often end up devising solutions which are in fact too complex as they try to over engineer everything. The more experience I have, the more I accept that code is complexity and cost and that it should be avoided. Always take the easy path…
I tend to observe that junior engineers simply misplace the level of complexity that a system needs in order to be as simple as possible. Meaning, I've seen errors to the side of being too simple, and errors to the side of being too complex; the unifying theory here is that its a mis-estimation of "necessary complexity". Two examples: In designing an email templating system, I've seen junior engineers devise really c…
Was there a discussion about high-level design of the templating system at all before the junior started banging out code? This feels more like a failure of the senior/lead.
> But, a counterpoint: For a long while, on some node.js backend API we were writing, much of the core business logic was implemented as simple functions across hundreds of files. Worked great when we had four API routes. Now, with hundreds, its impossible to find anything, because there's no structure, patterns, or organization. So, we needed someone to step in and introduce rules and structure in order to make the system more simple, and our job would have been easier if we'd had the foresight to see that necessary complexity coming.
This feels like a false dichotomy to me. Your system existed in a state somewhere between between 4 routes and hundreds, didn't it? At the point something starts being painful is when it should be addressed, imo. That was probably somewhere in the teens of routes would be my guess. At that point the system is still small and understandable enough to make a large architectural change, but it's big enough that you can be reasonably certain you're making the correct large architectural change.
Couldn't read because of the terrible font but I'm sure I disagree with all he says.
A big part of this is that programmers have different programming worldviews. Many people seem to be unaware that worldviews are not perfect rational easily modifiable constructions but are instead foundational structures that thinking builds upon automatically. Programmers are about as unlikely to easily change their programming worldviews as they are to change their political beliefs. Which is not to say that progr…
I feel like most of the strife caused at work is different value judgements on things. So I've been trying to tease out truths vs value judgements. 1 + 1 will always equal 2, no one debates that. AWS offers both Postgres and MySQL, no one debates that. But which one is better, that will be informed by possibly irrational value judgements ("Postgres bit me once, never again!") I think people are irrational in terms of…
Challenge accepted.
* In boolean algebra, 1 + 1 = 1
* Over the field Z2 (the integers modulo 2), 1 + 1 = 0
* In binary, 1 + 1 = 10
* In a language that allows overloading +, 1 + 1 = usually 2, but potentially almost anything else
In conclusion, context matters