The collapse of complex software • Hacker News Board

Re: The collapse of complex software

#151

Post by theteapot » Thu, Jun 09, 2022, 8:40 PM UTC

jacobsenscott wrote: post #107

This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…

> Look at software that has been around for decades, continues to improve, never "needs" a re-write, and will never be re-written. It exists and is not uncommon. Things like the linux kernel, all the GNU utils, emacs, vim, apache, nginx, postgres.

They don't get re-written because they can't be re-written without tearing down everything built on top. In every case though there is some newer software out there is inspired by but aiming to "fix" all the short comings of the predecessor.

Re: The collapse of complex software

#152

Post by lytefm » Thu, Jun 09, 2022, 8:41 PM UTC

throwaway894345 wrote: post #144

jacobsenscott wrote: post #107

This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…

> It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Even if you have plenty of skilled engineers, it doesn't mean that the business won't impose unrealistic timelines (and insist the timelines be met even if it means taking on loads of tech debt which never gets paid down) or change the requirements abruptly or even demand things that are…

Engineer induced complexity is also common. I've never heard about management pushing for a scalable, event-driven, microservice based architecture when a simple monolith serving a few thousand daily users would be just fine.

Re: The collapse of complex software

#153

Post by LeonenTheDK » Thu, Jun 09, 2022, 8:43 PM UTC

kitanata wrote: post #108

This is crazy. I just gave a nearly identical presentation for a series of interviews. One at AKASA, one at Scribd, and one at Panorama. Like I gave this presentation, with the exact same title 2 weeks ago to a bunch of junior engineers at these various companies. It is super timely. I’m consistently surprised how multiple people can arrive to the same point, via the same inspirational sources at the same time and re…

Please do! I'd love to see more takes and discussion on the matter.

Re: The collapse of complex software

#154

Post by p0nce » Thu, Jun 09, 2022, 8:48 PM UTC

I think Software education is in its infancy, instead of teaching people how to make debt decisions to keep complexity and costs low, we actively teach them to write disposable code. That is the content of most courses: code that will be thrown away, and that has to be written under tight time constraints.

And when it comes to keeping complexity low, there is zero theory to back up anything, so it's not taught.

Our trade is somewhat similar to reading/writing, and if you were to teach someone to read/write, you would probably insist on good writing, not just "finish your novel at this date, I won't read it anyway". And you would have good material for teaching to read, material that is currently missing.

Re: The collapse of complex software

#155

Post by Glavnokoman » Thu, Jun 09, 2022, 8:51 PM UTC

theteapot wrote: post #151

jacobsenscott wrote: post #107

This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…

> Look at software that has been around for decades, continues to improve, never "needs" a re-write, and will never be re-written. It exists and is not uncommon. Things like the linux kernel, all the GNU utils, emacs, vim, apache, nginx, postgres. They don't get re-written because they can't be re-written without tearing down everything built on top. In every case though there is some newer software out there is insp…

Half of software you mentioned is just garbage inside. And it will never be rewritten simply because it happens to work good enough to discourage any serious investment in rewriting it.

Re: The collapse of complex software

#156

Post by tasuki » Thu, Jun 09, 2022, 8:52 PM UTC

nradov wrote: post #92

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

I am skeptical of visual programming in general. Some of us just aren't visual thinkers. I find it much easier to conceptualize complex concepts in terms of text and narrative.

Don't worry - I am a visual thinker and I'm skeptical of visual programming. While it's cute for toy examples, I haven't seen it handle complexity well.

Re: The collapse of complex software

#157

Post by likortera » Thu, Jun 09, 2022, 8:59 PM UTC

jacobsenscott wrote: post #107

This "software grows too complex" is certainly a culture issue, and not a software problem. It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Of course it will outgrow the ability of that team to work with, reason about, keep clean, and extend. They will get frustrated and move on, and a new group comes in, re-writes it from scratch, and s…

Best comment I've ever read on HN.

Re: The collapse of complex software

#158

Post by regularfry » Thu, Jun 09, 2022, 8:59 PM UTC

pclmulqdq wrote: post #131

turminal wrote: post #118

Earlier quoted context omitted.

> Things like the linux kernel, all the GNU utils, emacs, vim, apache, nginx, postgres. The list goes on. We know how to write software that lasts for decades, continuously improves, and doesn't need constant re-writes. At least half of the items on that list are still here purely on momentum and reputation from the time there were no alternatives, not because they were really that marvelous. I use and even enjoy som…

Also, most of them have gotten a rewrite worth of upgrades during their life. The Linux kernel is a good example: most of the code you use today didn't exist in the kernel 15 years ago. A lot of code from back then still exists for compatibility purposes, but is rarely used in modern systems.

But that's an interesting facet in itself. The more common refrain you hear in corporate settings is "we need a ground-up rewrite" - how did these projects manage not to need that? I have my suspicions, but it's a very interesting question nonetheless.

Re: The collapse of complex software

#159

Post by miscaccount » Thu, Jun 09, 2022, 9:00 PM UTC

lytefm wrote: post #152

throwaway894345 wrote: post #144

Earlier quoted context omitted.

> It mostly comes from businesses throwing a bunch of under-skilled labor at the problem, and demanding more and more too quickly. Even if you have plenty of skilled engineers, it doesn't mean that the business won't impose unrealistic timelines (and insist the timelines be met even if it means taking on loads of tech debt which never gets paid down) or change the requirements abruptly or even demand things that are…

Engineer induced complexity is also common. I've never heard about management pushing for a scalable, event-driven, microservice based architecture when a simple monolith serving a few thousand daily users would be just fine.

Well probably you have never met management that is keyword drive. We need 100 percent scalable system with zero downtime upgrade and support thousand customers update. Current number of customers : 0 Number of customers in pipeline: 0

Re: The collapse of complex software

#160

Post by regularfry » Thu, Jun 09, 2022, 9:01 PM UTC

phendrenad2 wrote: post #125

sethetter wrote: post #111

Earlier quoted context omitted.

I'm curious how big of a factor a consistent core team is to having long-term sustainable and stable software. When companies have people rotating on projects constantly and the original vision and design principles of the product are lost or changed, I can only imagine that contributes to the problem.

I think that's a big factor. This is what RSUs and bonuses are supposed to solve: You entice employees to stay rather than job hop. However, companies have gotten too greedy and RSUs amount to a pittance, a vestigial perfunctory play act mimicking a bygone era.

The gotcha to that, of course, is that while you're enticing people to stay in the company, you're doing nothing necessarily to keep them on a single project within that company.