Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The collapse of complex software

nolanlawson.com

Post reply on HN
Print view View original

141–150 of 304 posts

Re: The collapse of complex software

#141

Post by nradov »

perlgeek wrote: post #37
daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that conven…

Falsehoods programmers believe about names:

https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...

As a practical matter, the HL7 FHIR data model for names can work well enough for the vast majority of applications in almost any country.

https://www.hl7.org/fhir/datatypes.html#HumanName

Re: The collapse of complex software

#142

Post by mananaysiempre »

jrochkind1 wrote: post #35

I appreciated this essay. Among other things, it suggests that there are some incentives for complexity: That engineers just enjoy complexity, and that making simple designs might not be good for the respect of your peers/bosses/career: "A simple design might make it seem like you’re not really doing your job." While that's true, I think the essay isn't clear enough about the fact that simplicity is just plain hard .…

There is the well-worn Pascal quote about having no time to write a shorter letter[1]. There is also, I think, an important point hidden in the “domain knowledge” part: the freedom to change the spec; the effusive texts on Charles Moore’s approach to Forth programming[2,3], however much scepticism they deserve otherwise, are very direct about that freedom being essential.

But that means simplicity is a cross-cutting concern, like security, and does not particularly fit into any environment that would attempt to separate design and implementation. It needs “redraw all the module boundaries” to be a plausible (if improbable) response to “this one 300-line part of this one module is awkward and three times longer than it really needs to be”. Aside from the obvious administrative problems, suggests that the complete edifice needs to fit into the mind of a couple of people at most or the iteration simply will not converge most of the time.

I don’t see a good solution to this.

[1] https://quoteinvestigator.com/2012/04/28/shorter-letter/

[2] http://www.ultratechnology.com/forth.htm

[3] There was an example about changing a negative-feedback control loop, written into the spec by hardware engineers emulating analog circuitry, into a couple of fixed-point operations that behaved differently but did the actual job just as well; I can’t find the reference right now but this a minor domain-specific change among the kind I have in mind.

Re: The collapse of complex software

#143

Post by a_e_k »

henning wrote: post #36

> engineers like complexity. Admit it: as much as we complain about other people’s complexity, we love our own No. Speak for yourself. I am happiest when my solution to a problem is simplest. I am least happy when a solution requires introducing new infrastructure or major new ideas.

I came here to say this.

I have no idea who first said it, but one of my favorite sayings is: "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."

I strongly believe that this applies to computers as well. I love it when a system neatly solves its requirements in a way that looks effortless. When an algorithm looks like it just barely works for all the potential edge cases without any special handling because those edge cases just miraculously happen to work out as they fall through the happy path just like any other case. That's when a solution "feels" right.

If I can present my solution/code/explanation to a junior engineer, a C.S. student, or even a non-technical person and they get it, then that's when I feel like I've really done my job. It's music to my ears if I can make a problem look so easy that someone asks "Is that it all it takes?" (until they've tried to solve it on their own :-)

Re: The collapse of complex software

#144

Post by throwaway894345 »

jacobsenscott wrote: post #107

This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…

> It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly.

Even if you have plenty of skilled engineers, it doesn't mean that the business won't impose unrealistic timelines (and insist the timelines be met even if it means taking on loads of tech debt which never gets paid down) or change the requirements abruptly or even demand things that are inherently complex (without understanding the complexity).

In other words, there are plenty of ways that management can create complexity, and in my experience management-induced complexity is the most common kind (note that I'm faulting "management" for things that include "communication between engineers and management" because management ultimately owns the software development process).

Re: The collapse of complex software

#146

Post by datavirtue »

hinkley wrote: post #24
parsley27 wrote: post #4

Earlier quoted context omitted.

That assumes operating costs go up with cloud providers. Some orgs see savings, but obviously not all.

Some developers are merely bad at math. Most are awful. Others take this flaw to epic proportions. Be afraid of the developer who confidently tells you the math works out. It's true that a lot of organizations had operational teams that have become money pits, or pushed back on all quality of life improvements because they don't have the talent, the budget, or the imagination to pull it off. When we move to the cloud…

We used to have devs bringing in frameworks and libraries without reading the manual. Now they spin up entire services or subsystems without reading the manual. You should read through the docs for MongoDB Atlas...which plainly illustrate that you now need a DBA unless you're cool with people pressing buttons in a panic during some type of self-induced performance issue.

Re: The collapse of complex software

#147

Post by jinwoo68 »

jacobsenscott wrote: post #107

This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…

In the corporate world, "leaders" are incentivized by having more people in their orgs. It makes them have more "power". So they constantly broaden the scope and try to hire more people. The leaders in software orgs never benefit from finishing projects early with less engineers. Nor do they benefit from maintaining stable and robust software. Making the project take longer with more engineers with complicated architectures makes them gain more power. Yeah, this makes me sad.

Re: The collapse of complex software

#149

Post by protomyth »

I always thought it was because we really couldn't break the applications down without massive dependencies and the complexity of human procedures brought to the computer's logical world.

It seems like folks who've built some form of common bus (shared data types and communications channels) do better scaling applications. If I can effectively split off a part of the application and plug a new part in its place, I can scale. OOP seems to love dependencies that accumulate until objects cannot be replaced because of all the connections. Clean separation is a hard problem.

Computers are logical machines. Human laws and regulation are not. Modeling physics is an easier problem (programming not computing need wise) than calculating taxes. At one point in the US, every gallon of bio-diesel had to have its own serial number[1], this adds a level of complexity that goes beyond simple programs.

My own thoughts (or my longtime hobby project) is that Agent oriented programming hasn't really been given a workout, and that decoupling data structures might lead to some improvements. I honestly think being able to replace parts of a running program in an orderly manner might be a indicator of a good direction.

1) https://mnbiofuels.org/media-mba/blog/item/1495-rins-101 - as a programming exercise, write a program to add, subtract gallons from a storage unit - it gets fun.

Re: The collapse of complex software

#150

Post by amenghra »

phendrenad2 wrote: post #125
sethetter wrote: post #111

Earlier quoted context omitted.

I'm curious how big of a factor a consistent core team is to having long-term sustainable and stable software. When companies have people rotating on projects constantly and the original vision and design principles of the product are lost or changed, I can only imagine that contributes to the problem.

I think that's a big factor. This is what RSUs and bonuses are supposed to solve: You entice employees to stay rather than job hop. However, companies have gotten too greedy and RSUs amount to a pittance, a vestigial perfunctory play act mimicking a bygone era.

It's common for tech companies to give RSUs which vest over 4 years with annual refreshers being comparatively quite small. The financial incentives can dwindle after just 4-6 years at a given job.
Post reply on HN