Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The collapse of complex software

nolanlawson.com

Post reply on HN
Print view View original

51–60 of 304 posts

Re: The collapse of complex software

#51

Post by yhoiseth »

michaelbuckbee wrote: post #44
daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

What you're describing is one approach, the other is that you pay a bunch of SaaS/PaaS/IaaS companies to handle a lot of the workload for you so you and a small team can just focus on the business logic of your particular niche.

Agree that this helps in some ways, but it also adds complexity. You can e.g. subscribe to a billing SaaS, but then you have to link your system to the billing system somehow

Re: The collapse of complex software

#52

Post by nonrandomstring »

groby_b wrote: post #12

Our industry's "solution" to complexity is somewhat unique: We sweep it under the carpet. The reason that actually works is that the marginal cost to reproduce an existing pile of complexity is zero. Once we've got a sufficiently complex pile that actually has promise, everybody duplicates the pile. And then puts an additional layer on top. These many pile+layer combos then co-evolve (and co-erode ;) At some point, s…

"Don't worry. We'll fix it when it goes bad" -- Dolittle (Dark Star)

Re: The collapse of complex software

#53

Post by chrisweekly »

WaxProlix wrote: post #10
_xnmw wrote: post #6

I'm waiting for the day that AWS collapses under its own weight. Heard horror stories from backend engineers working on the infrastructure/dashboard code there.

If anything were going to do it, it'd be the labor crunch that we're seeing here. Without a constant influx of fresh-faced college coal lumps to burn in the tech debt engines Amazon would have a harder time of it. They've done a good job of pivoting though, and I think they'll weather the storm here (and be better off for it). See the increased comp to boost hiring rates - especially of non college hires - and an int…

> "constant influx of fresh-faced college coal lumps to burn in the tech debt engines"

made me literally LOL. That turn of phrase is gold.

Re: The collapse of complex software

#54

Post by feoren »

nightski wrote: post #42
perlgeek wrote: post #37

Earlier quoted context omitted.

I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that conven…

As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the s…

The problem is "pulling in a tried and tested solution" means pulling in a slice of the whole stack: CSS, Javascript, client/server communication, API endpoints, data-flow logic, and database schema. People are very very bad at this because they believe that software engineering principles apply only to little tiny portions of this stack. If you actually apply your software engineering principles to the whole stack, you end up with something that looks very different than what everyone else is doing (so there's not a lot of tooling to help with). I have a better way to do this than anything else I've ever seen, but even with that I struggly very strongly with the JS/API side.

Re: The collapse of complex software

#55

Post by swatcoder »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

This is an admirable and worthy pursuit, but keep in mind that any process you make streamlined:

1. becomes a post from which clever and ambitious people can build complex things. You invent docker to simplify application deployments and then somebody builds a n-dimensional microservice cloud on one side and starts commercializing new hardware architectures on the other. You don't remove the complexity, you just let it move around into new domains. (not a bad thing!)

2. is more temporal than you expect. The "finite and fundamental" problems of yesterday, today, and tomorrow are of different sets -- partly because (1) opens up new problem classes and partly because most problems are inescapably cultural and therefore subject to fashion cycles.

Re: The collapse of complex software

#58

Post by notriddle »

nightski wrote: post #42
perlgeek wrote: post #37

Earlier quoted context omitted.

I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that conven…

As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the s…

> Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service.

The type signature of GPT-3 is `string -> string`

Re: The collapse of complex software

#59

Post by davidgay »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers.

Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle.

So in that view, you can't succeed at making software engineering always simple. Instead, you can make previously intractable problems tractable.

Re: The collapse of complex software

#60

Post by mark242 »

chasd00 wrote: post #38
mark242 wrote: post #14

Earlier quoted context omitted.

They absolutely do reduce complexity! For example, think about building, testing, and deployments. In pre-cloud Internet times, you'd have an untold number of extremely brittle bash scripts, cron jobs, rsync ssh key setup, fleets of build + test boxes to manually worry about disk space, pre-provisioned dev/QA database servers with also untold brittle sql startup/teardown scripts, and all of the requisite people whose…

idk, in the enterprise world all those things still exist but they're applied against AWS/other instead of your servers in the datacenter. Instead of "build+test boxes" you have "build+test instances". It takes the same level of redtape/approvals to get an AWS instance as it took to get a server in the old datacenter. All my enterprise clients have the dedicated infrastructure teams they always had, only now they're…

I am going to go out on a limb and guess that this is almost entirely down to your finance department attempting to understand and control the monthly cloud spend. This is obviously fine, but rather than "oh, another 2xlarge, we have to add a couple hundred bucks to our budget" the reverse should be true; you have $x to spend per month on average with a reasonable growth plan built in.

In the build and test example, the answer to "how much compute is running?" is based off developer velocity and so "it depends" is a fair answer. When nobody is shipping any builds, your cost should be $0. (I've found that this is kind of hard for enterprise-y finance departments to wrap their heads around and is why all those esoteric billing notifications AWS services even exist)

Pre-cloud services days, finance departments had a much easier time. You had racks of physical boxes that had static costs attached to them, you had a static monthly bandwidth bill that let you run at a certain speed, and you had salary costs which are also pretty static month-to-month. The idea of "scale to 0" was completely unheard of. What do you mean your QA environment doesn't cost anything on the weekends when nobody's doing anything? etc etc etc, you get my point.

Post reply on HN