This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…
The collapse of complex software
191–200 of 304 posts
Re: The collapse of complex software
#192This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…
We already have had, for a long time, a 'universal' model for a specific slice of information processing that consists of a very small set of components: Graphic User Interfaces.
One problem is the diversity of domain semantics and all the associated nuances. In GUI programming, this is the tedious bit of naming the components and mapping them to processing elements.
Another problem is engineering culture (du jour). A reductionist approach that attempts to leverage structural commonalities requires what is poo poo'd as "boiler plate" and all the associated warts of component oriented programming (including factory-factories). A revisionist shift in mindset is required.
Re: The collapse of complex software
#193Earlier quoted context omitted.
What would be counterexamples to that, i.e. great engineering? FOSS possibly?
seL4. Simple and formally verified. I think it's time to pour VC money on a fully verified stack. Perhaps not everything needs to be proven, integrity for some components can be ensured more informally with e.g. refinement types. Lots of governments and organizations need more robust software.
Re: The collapse of complex software
#194This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…
> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering alway…
Re: The collapse of complex software
#195Earlier quoted context omitted.
GNU utils are incredibly primitive and their text-driven APIs are fragile and cause tons of issues in shell scripting. I love Emacs, but it's a slow, crufty mess. Vim is a huge pile of code with a lot of complexity in its codebase and hilariously bad design decisions like VimScript. Apache is a mess from a security perspective (1.7k listed CVEs[1]), and still slower than Nginx. Want to guess at the amount of lost pro…
What would be counterexamples to that, i.e. great engineering? FOSS possibly?
Re: The collapse of complex software
#196Earlier quoted context omitted.
> Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now! NO! This mentality is short sighted by definition and leads to stagnation. It may work within large financially healthy companies that aren't trying to rock to the boat. Making assumptions about the future is incredibly important for us to move forward. But with these caveats: 1)…
I think you and the parent are talking about two different things. Building products that are looking to the future is valuable, if risky. But I believe the parent comment was talking about software abstractions. i.e., don't build abstractions because you might need them later. Instead, build the software in the simplest possible way to solve the problems it needs to handle now. This is because you'll likely be wrong…
> don't build abstractions because you might need them later ... [snip] ... This is because you'll likely be wrong about your assumptions about which abstractions you'll need, and ripping those abstractions out is going to be more expensive than just modifying simple code to do new things.
At worst, this is dangerously wrong. At best, it creates a straw man that people can use as a tool to shut down innovative ideas.
What you're describing is a very narrow/specific strategy that one might apply at a company. An example would be when a company is doing well financially and you don't need to risk amassing technical debt for some experimental side projects. Just build the MVPs, see what sticks, and only then decide whether or not to commit serious architecture to the problem.
If you're designing new systems that need to scale quickly (just one example) then you NEED to place bets on abstractions. A well designed abstraction could save you weeks (even months!) of time when you most need it.
I've seen what happens when a successful product is comprised of stitched together MVPs. It's too late for a rewrite and all of that "simple code" is creating a death-by-a-thousand-cuts nightmare.
Bad abstractions will always be bad. Bad system design will always be bad.
It's an aged/jaded mentality to claim that we should just stop designing systems ahead of time because we suck at it. Not everyone does. Some people are actually very good at it. It's possible to design an abstraction defensively so that it isn't more expensive to rip it out later.
Want a concrete example? Payment systems. If I'm writing a billing system then you bet I'm going to abstract away the interface with the payment processor. I would do this irrespective of our plans to expand to other payment processors. It shouldn't even be a debate because there's a way to build that abstraction with minimal overhead to the team.
On the other hand I would never blindly introduce an ORM into a system that doesn't need it. That would be a bad abstraction.
Re: The collapse of complex software
#197Earlier quoted context omitted.
> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering alway…
If history is a guide, then you are correct. But I believe complexity is fractal (I think we all know this: "turtles all the way down..."), and that you can structure and manage complexity with recursive rules, so that you only see the resolution that you care about, for the areas that you care about, and everything else is a low frequency representation. Frameworks approximately do this, but for specific domains: th…
Actually, "combinatorial" or "exponential" would be better words to describe complexity explosion. The number of possible cases/scenarios/logic flows/etc grows exponentially as a system's complexity increases. It is a curse and a blessing at the same time keeping so many SWEs gainfully employed.
Re: The collapse of complex software
#198This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…
Re: The collapse of complex software
#199Earlier quoted context omitted.
I think that's a big factor. This is what RSUs and bonuses are supposed to solve: You entice employees to stay rather than job hop. However, companies have gotten too greedy and RSUs amount to a pittance, a vestigial perfunctory play act mimicking a bygone era.
The developers have also gotten too greedy and most of them will job hop every year for the maximum income.
Re: The collapse of complex software
#200Earlier quoted context omitted.
I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that conven…
As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the s…
Type signatures don’t tell you which should be the “then” clause vs the “else” clause in any conditional.