Live data from Hacker News

The Software Crisis

wryl.tech

161–170 of 201 posts

Re: The Software Crisis

#161
post #116

Earlier quoted context omitted.

This looks pretty cool! I agree -- a big reason I like Unix, and find it productive, is that it's "shallow and composable". But GUIs have long been a weakness. I guess because there are many "global" concerns -- whether a UI is nice isn't a "modular" property. Personally I would more of a UI, but still retain automation That is, the property of shell where I can just saved what I typed in a file, and then run it late…

I’m going to play around a little bit with this headless mode and see if I can come up with anything interesting! Is there a place y’all hang out or a way you communicate? I foresee needing some guidance in places.

Oh also, my feeling of why this may be interesting is if you think "I want to write a GUI for shell, but I can't modify the bash source code to do what I want"

That is, the shell itself is a limitation

OSH is the most bash-compatible shell by a mile, AND we can easily add new hooks and so forth.

I think most stuff can be already done with FANOS, but I think completion is one area that needs some testing / feedback. For example, I imagine that when constructing a pipeline, you might want to complete exactly what's available -- like shell functions that the user has defined ?

Re: The Software Crisis

#162
post #101

Earlier quoted context omitted.

Our society / societies may well be able to afford to build this quantity of software well. We choose not to.

I don't know how/haven't seen an attempt to approach this question by a method other than "my hunch", but as a software engineer "my hunch" is it would cost at least 10-50x as much human labor (not just engineers but designer and UX researchers as well as all the other support roles like project managers etc) to build the software "well" (including more customization for individual enterprises or uses), and that it w…

> (not just engineers but designer and UX researchers as well as all the other support roles like project managers etc)

Oh! I thought you were going to say "testing teams, design reviewers, trainers".

I'm not on-board with this "10-50x" claim for the amount of effort. I'd say maybe 3x the effort, given the developers have been well-trained, the methodology is sound, and the managers' focus is exclusively quality. That last item is one I've never experienced; the closest I came was on a banking project where the entire development team was subordinated to the Head of Testing.

Re: The Software Crisis

#163
@wryl:

I can't find a definition of the title term, "Software Crisis," anywhere in the post.

Is it "...[the] growing complexity of software..."?

It's difficult to reason about something with no definition.

Re: The Software Crisis

#164

Earlier quoted context omitted.

> how many human endeavours have one or all of these things? You want a binary answer as in "it never happens with X"? Because for most kinds of project, we say it didn't go well when 1 or 2 of those happen, while we declare a victory in software if we manage to avoid 2 of them. > And then how much software is actually pretty great? And now I'm curious about your bar for "pretty great" software. This is obviously sub…

> This is obviously subjective, but most people wouldn't look at any practical software and call it great. This what I think is insane. We interact with probably thousands of pieces of software every day (and I'm probably off by an order of magnitude) and none of it is great? What is the bar here? It's funny that something that was absolutely amazing yesterday is no longer even so much as great today. I can instantly…

> generate a pretty good poem about my cat

I do not consider a text generator to be the pinnacle of "great software", regardless of how well the code was written. I don't think "meh" is the wrong word.

Re: The Software Crisis

#165
post #139

Earlier quoted context omitted.

Windows 3.1 and Word easily fit on a 40MB hard drive with plenty leftover. Word operated on systems with as little as 2MB RAM on a single core 16MHz 80386. Modern microcontrollers put this to shame! Word then didn’t lack for much compared to today’s version. Windows and Office require now, 50-100GB disk just to function? 1000X for what gain? This is sheer insanity, but we overlook it — our modern systems have 5000-10…

You mean the Word where i had to save the document every 20 seconds just to be sure i didn't loose my school assignment from crashes at the most inconvenient moment. It left scars in my spline, to the point that I still have to actively hold myself back from the reflex of hitting Ctrl+S halfway through this comment.

Yes, that Word. Imagine if we'd been focused on improving reliability this whole time...

Re: The Software Crisis

#167

I don't think we have a software crisis. Millions of programmers are able to create more or less useful programs all over the word; everything including your toaster is running software on it succesfully enough; and the community was able to build programs that are accessible to all from a 5 year old kid all the way to your grandparents. Where is the crisis in it? However, we have a project management crisis, which i…

> everything including your toaster is running software

Doesn't that just support the author's claim that there's too much software?

FTR, my Dualit toaster doesn't run software.

Re: The Software Crisis

#168
I agree with the statement that there's a software crisis, but completely disagree with the author about what the problem is caused by.

The software crisis, if there is one, is caused by complexity. Complexity is the single enemy of a software developer. I would say that reducing complexity is the whole purpose of the software engineering field. I have many small hobby projects where I am the sole developer, and I still struggle with complexity sometimes... I've tried many languages and programming paradigms and still haven't found one that actually "solves" complexity. I am convinced, for now, that the only solution is developer discipline and, guess... good abstractions.

Because complexity doesn't necessarily come from abstractions. In fact, it's the exact opposite: the only way we know to make things "look" simpler, so that we can make sense of it, is to abstract away the problem! Do you need to know how the network works to send a HTTP request?? No!!! You barely need to know HTTP, you just call something like "fetch url" or click a link on a browser and you're done. This is not something we do because we are stuck on some local maximum. Whatever you do to "hide" complexity that is not crucial to solving a certain problem, will be called an "abstraction" of the problem, or a "model" if you will. They always have downsides, of course, but those are vastly offset by the benefits. I can write "fetch url" and be done, but if something goes wrong, I may need to actually have a basic understanding of what that's doing: is the URL syntax wrong, the domain down, the network down, the internet down, lack of authorization?? You may need to dig a bit, but 99% of the time you don't: so you still get the benefit of doing in one line what is actually a really complex sequence of actions, all done behind the layers of abstractions the people who came before you created to make your life easier.

> Various efforts have been made to address pieces of the software crisis, but they all follow the same pattern of "abstract it away"

Of course they do. Abstracting away is the very definition of addressing complexity. I believe what the author is actually trying to say is that some of the abstractions we have come up with are not the best abstractions yet. I would agree with that because as hardware evolves, what the best abstraction is for dealing with it also should evolve, but it hardly does so. That's why we end up with a mismatch between our lower level abstractions (Assembly, C) and the modern hardware we write software for. Most of the time this doesn't matter because most of us are writing software on a much higher level, where differences between hardware are either irrelevant or so far below the layers we're operating on as to be completely out of sight (which is absolutely wonderful... having to write code specific for certain hardware is the last thing you want if all you're doing is writing a web app, as 90% of developers do), but sure, sometimes it does.

> We lament the easy access to fundamental features of a machine, like graphics and sound. It is no longer easy to build software, and nothing comes with a manual.

I have trouble to take this seriously. Is the author a developer? If so, how can you not know just how wonderful we have it these days?? We can access graphics and sound even from a web app running on a browser!! Any desktop toolkit has easy to use APIs for that... we even have things like game engines that will let you easily access the GPU to render extremely complex 3D visualisations... which most of the time working on most Operating Systems in use without you having to worry about it.

Just a couple of decades ago, you would indeed have to buy a Manual for the specific hardware you were targeting to talk to a sound board, but those days are long gone for most of us (people in the embedded software world are the only ones still doing that sort of thing).

If you think it's hard to build software today, I can only assume you have not built anything like even 10 years ago. And the thing is: it's easy because the "hard problems" are mostly abstracted away and you don't even need to know they exist! Memory allocation?? No worries, use one of a million language that come with a GC... even if you want the most performant code, just use Rust, still you don't need to worry (but you can if you must!!! Go with Zig if you really want to know where your bytes go). Emit sound? Tell me which toolkit doesn't come with that ready off the box?? Efficient hash table? Every language has one in its standard lib. Hot code reloading so you can quickly iterate?? Well, are you using Lisp? If so, you've had that since the early 80's, otherwise, perhaps try Smalltalk, or even Java (use jdb, which lets you "redefine" each class on the go, or the IntelliJ debugger, just rebuild it while it's on, specially if you also use DCEVM which makes the JVM more capable in this regard) or Dart/Flutter, which has that out of the box.

Almost any other problem you may come across , either your language makes it easy for you already or you can use a library for it, which is as easy to install as typing "getme a lib"). Not to mention that if you don't know how to do something, ask AI, it will tell you exactly how to do it, with code samples and a full explanation, in most circumstances (if you haven't tried in the last year or so, try again, AI is getting scary good).

Now, back to what the problem actually is: how do we create abstractions that are only as complex as they need to be, and how many "layers" of abstractions are ideal? What programming model is ideal for a certain problem?? Sometimes OOP, sometimes FP, sometimes LP... but how to determine it? How to make software that any competent developer can come and start modifying with the minimum of fuss. These are the real problems we should be solving.

> Programming models, user interfaces, and foundational hardware can, and must, be shallow and composable. We must, as a profession, give agency to the users of the tools we produce.

This is the part where I almost agree with the author. I don't really see why there should be a limit on how "shallow" our layers of abstractions should be because that seems to me to limit how complex problems you can address... I believe the current limit is the human brain's ability to make sense of things, so perhaps there is a shallow limit, but perhaps in the future we may be able to break that barrier.

Finally, about user agency, well, welcome to the FOSS movement :D that's what it is all about!!

Re: The Software Crisis

#169

Earlier quoted context omitted.

Yes, shallow, wide, and composable. That's how our abstractions should be. But the larger problem is not GUIs. GUIs are a problem, but they are necessarily at the top of the abstraction stack, so the problem doesn't compose any further. (What interestingly means they are so much of a problem that they aren't anymore.) The elephant in the room nowadays are distributed systems.

I would love to hear your thoughts on how distributed systems lead to deep, narrow and isolated abstractions. I can think of some obvious reasons but it seems like you’ve put more thought into this than I!

I haven't put anywhere near enough thought to decide on something simple.

There are two main dynamics that I noticed:

- Each distributed component has a high minimal cost to create and maintain, what leads people to consolidate them into more complex, fewer ones;

- You'll invariably need to interact with some component that doesn't share your values. So you need security, contracts (the legal kind), consumer protection, etc... and that creeps into your abstraction.

Outside of that, there's one thing that isn't inherent but does happen every single time is information erasure. Distributed interfaces are way less detailed than the ones on monolithic systems.

Re: The Software Crisis

#170
“Modularity based on abstraction is the way things get done” --Barbara Liskov

Something as big and complex as the internet, which covers technologies from email to fiber, is held together by layered abstractions.

Also, software has gotten incredibly better since the 70s. We've built so much better tooling (and I believe tooling can help close the gap on growing complexity). When I was learning how to program, I had to double check I had every semicolon in the right place before hitting compile. I simply don't have to do that anymore. I can even get an entire API auto-completed using something like copilot on VSCode.

Nonetheless, a very thought-provoking article. Thank you for sharing!

Post reply on HN