Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The collapse of complex software

nolanlawson.com

Post reply on HN
Print view View original

91–100 of 304 posts

Re: The collapse of complex software

#91

Post by commandlinefan »

atoav wrote: post #86
js8 wrote: post #80

I feel like it's a bit backwards. The software complexity increases because people lack the patience to tackle the existing complexity, or learn why things are complex, and prefer to believe that it can be simple, or get job done somehow, and in order to do that, they (somewhat egoistically) go their own way, which actually - barring occasional good insight - mostly increases the total complexity. The same goes for s…

I believe many software developers often tend to write software which has precisely the segree of complexity they can just barely manage. There is not many people who have the guts to build a simple straightforward step for step script. Software developers that fall into love with the beauty of their own abstractions can be a real problem.

> precisely the segree of complexity they can just barely manage

"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" -- Brian Kernighan

Re: The collapse of complex software

#92

Post by nradov »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

I am skeptical of visual programming in general. Some of us just aren't visual thinkers. I find it much easier to conceptualize complex concepts in terms of text and narrative.

Re: The collapse of complex software

#93

Post by go_elmo »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

I am sure every SE agrees that somehow things are always the same. How about making code pieces algebraic in order to reason and write with them much easier? Like using Haskell?

Re: The collapse of complex software

#94

Post by nradov »

daenz wrote: post #68
davidgay wrote: post #59

Earlier quoted context omitted.

> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering alway…

If history is a guide, then you are correct. But I believe complexity is fractal (I think we all know this: "turtles all the way down..."), and that you can structure and manage complexity with recursive rules, so that you only see the resolution that you care about, for the areas that you care about, and everything else is a low frequency representation. Frameworks approximately do this, but for specific domains: th…

The complexities at different levels of the hierarchy are so qualitatively different that I doubt whether a unified set of abstractions could ever be practically useful. We're going all the way from CPU microcode to loosely-coupled distributed systems with multiple dependencies on third-party services. If you want to make any progress you'll need to narrow down your focus.

Re: The collapse of complex software

#95

Post by bell-cot »

Back on the "Collapse of Complex Societies" side, my impression is that a fair number of such collapses are now blamed on climate change, and/or resource depletion. The Anasazi thrived from ~700 to ~1130 due to an unusually rainy spell in SW North America. When the period of heavier rains ended, so did the agricultural productivity that supported their large population and complex society. Similar reasons may hold for the fall of the Khmer Empire ( https://en.wikipedia.org/wiki/Khmer_Empire ). A number of sharp, grim setbacks for European civilizations occur suspiciously close to major volcanic eruptions, which could have hit their agricultural foundations hard.

...and "yes", I read second ~half of the article. My point here is that the author's analogy works at least as well where he did not bother to apply it as where he did.

Re: The collapse of complex software

#96

Post by commandlinefan »

henning wrote: post #36

> engineers like complexity. Admit it: as much as we complain about other people’s complexity, we love our own No. Speak for yourself. I am happiest when my solution to a problem is simplest. I am least happy when a solution requires introducing new infrastructure or major new ideas.

I feel the same way, but we both have to accept that we're tilting at windmills. Complexity can't be measured, so there's no way to demonstrate, much less prove, that the time you spent reducing complexity was time well spent. You can demonstrate a run time or memory performance improvement. You can demonstrate a new feature. Unless you can reduce complexity in zero time and do so without introducing any unintentional side effects/bugs, you're just hurting yourself when you try.

Re: The collapse of complex software

#97

Post by appleflaxen »

loudmax wrote: post #28

I highly recommend Rich Hickey's "Simple Made Easy" talk as a way to think about software complexity. We all know that complex systems are hard to work with, but I've found his presentation very useful in understanding the nature of complexity. Understanding complexity helps being able to avoid or minimize it. For example, easy is not the same as simple, and simple changes to one part of a system can introduce great…

I've watched that video at least three times, but I still can't articulate the exact distinction he is making between simple and easy :/

Re: The collapse of complex software

#98

Post by pebcakID10T »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

I have been developing software for over 20 years and leading teams of developers for the last 10. Please allow me to give you some insight from experience.

To your first point, in my experience, the best software, including complex platforms that handle massive amounts of traffic and data, can be built and maintained by small teams. I mean less than 20 developers. The larger the number of developers working on an application or platform has an almost inverse correlation to the speed in which new features are built or bugs fixed.

Software architecture has been done visually since perhaps its inception (tools like UML). At most places I've worked, every new project or large feature is diagrammed visually to use as a guide in breaking down the project into component parts.

In my experience complexity arises, not from lack of tools or industry knowledge, but from 3 main causes:

1. Inexperienced developer asked to create project, who just starts building without planning beforehand.

2. The main one - business demands features built that were never expected or planned for, and built as fast as possible. This causes developers to take shortcuts, make inelegant and difficult to maintain design choices. And leads to often inscrutable code that becomes technical debt especially after the original developer leaves the company. This will always happen as long as software is used to make a business money.

3. High team turnover - I've seen places where developers came and went so often that there was a myriad of things half started and never finished.

How I've solved or helped alleviate these issues: Make the business case to company owners or management that technical debt will be an ever increasing impediment to development velocity and the dev team will need a percentage of work in any given sprint to tackle tech debt issues (as opposed to having everybody work 100% on new features and bug fixes all the time).

I have successfully taken a platform that was bug-ridden, difficult to maintain, and where new feature development had slowed to a crawl due to the over-complexity of the software, to a place of stability and ease of development, simply by allowing our team to chip away at tech debt over the course of several years. Tech debt issues were rated by level of complexity, risk to the business in change (regression bugs), and impact on team velocity. We worked on the highest impact, lowest risk items first and kept going until there wasn't much left on that list.

Re: The collapse of complex software

#99

Post by jldugger »

> One thing working in complexity’s favor, though, is that engineers like complexity.

Honestly, I hate complex solutions. Even my own. What I like, and what impresses me, is the simple solution that takes a lot of work or math to prove it works. Like, emergent behavior type shit.

Control theory, for example, runs systems with very simple runtime behavior. A few open loops with some parameters. The analysis to chose those parameters is where the engineering happens.

Microservice practices are not yet that simple, but some day I hope they can be.

Re: The collapse of complex software

#100

Post by _gabe_ »

danielovichdk wrote: post #63

I will tell you this. It's based on my 20+ years, 50 different teams and many hundreds, perhaps thousands of encounters with software developers. It's almost never the domain complexity that is high. Sending a rocket to Mars is complex. And perhaps complex every time. Dealing with people that has to work together can only be simplified when agreements are made and agreements are kept. Otherwise you cannot trust and h…

> Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now!

Yes! 100% this. We need less fortune tellers and more pragmatists.

Post reply on HN