This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…
The collapse of complex software
171–180 of 304 posts
Re: The collapse of complex software
#172Earlier quoted context omitted.
But that's an interesting facet in itself. The more common refrain you hear in corporate settings is "we need a ground-up rewrite" - how did these projects manage not to need that? I have my suspicions, but it's a very interesting question nonetheless.
As a big fan of rewrites (but not as big a fan as some), I think Linux has avoided needing big rewrites with a lot of very clever modularity in its design. Many corporate software programs do not have that level of modularity. Modularity has performance costs, is slower to develop, and "looks ugly" on a block diagram, but it pays dividends for open source projects and projects that need upgrades over time. Individual…
Re: The collapse of complex software
#173This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…
I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that conven…
struct Screenname {
Matches _;
// TODO: ensure it is illegal to ask for users' real names
// for now we'll just do the right thing unilaterally
}Re: The collapse of complex software
#174Earlier quoted context omitted.
GNU utils are incredibly primitive and their text-driven APIs are fragile and cause tons of issues in shell scripting. I love Emacs, but it's a slow, crufty mess. Vim is a huge pile of code with a lot of complexity in its codebase and hilariously bad design decisions like VimScript. Apache is a mess from a security perspective (1.7k listed CVEs[1]), and still slower than Nginx. Want to guess at the amount of lost pro…
What would be counterexamples to that, i.e. great engineering? FOSS possibly?
I think it's time to pour VC money on a fully verified stack.
Perhaps not everything needs to be proven, integrity for some components can be ensured more informally with e.g. refinement types.
Lots of governments and organizations need more robust software.
Re: The collapse of complex software
#175Earlier quoted context omitted.
Maybe. I've never worked in an organization where engineers were driving the complexity, nor have I worked in an organization where microservices introduced complexity (I've heard people complain in vague terms about microservices, but I don't relate to their experiences). However, I have worked in an organization that transitioned from monoliths to microservices, and the overall complexity was reduced because (I did…
"Microservices" (ie. SOA) absolutely adds complexity, the question is whether the benefits are worth the overhead. The answer varies according to domain, but is largely a function of team size.
The net complexity may well be (and often is IME) less for a microservice than for an otherwise similar monolith. In addition to this net complexity discussion, microservices also have security, scalability, and reliability benefits.
Re: The collapse of complex software
#176Earlier quoted context omitted.
I'm curious how big of a factor a consistent core team is to having long-term sustainable and stable software. When companies have people rotating on projects constantly and the original vision and design principles of the product are lost or changed, I can only imagine that contributes to the problem.
I think that's a big factor. This is what RSUs and bonuses are supposed to solve: You entice employees to stay rather than job hop. However, companies have gotten too greedy and RSUs amount to a pittance, a vestigial perfunctory play act mimicking a bygone era.
Re: The collapse of complex software
#177Earlier quoted context omitted.
I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that conven…
> What's the "right" way to store person names? struct Screenname { Matches _; // TODO: ensure it is illegal to ask for users' real names // for now we'll just do the right thing unilaterally }
Re: The collapse of complex software
#178Re: The collapse of complex software
#179Earlier quoted context omitted.
This assumes that complex software problems are decomposable into discrete components defined and analyzable solely by their public interfaces. Complex software frequently cannot be modeled in this way -- it is what makes the software "complex". In reality, components have complex interactions far outside of what is captured in the component interfaces (see also: Hyrum's Law). Surfacing these implicit interactions at…
There are many embellished, hand wavy statements in this comment. Could you give real world examples of: * "In reality, components have complex interactions far outside of what is captured in the component interfaces" * How template metaprogramming "precisely express[es] behavioral contracts... that are difficult to do in other languages" I think you contradict your poetry with "Any non-trivial software system has to…
Re: The collapse of complex software
#180This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…
> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering alway…