Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The collapse of complex software

nolanlawson.com

Post reply on HN
Print view View original

291–300 of 304 posts

Re: The collapse of complex software

#291

Post by mbrodersen »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

The problem with software engineering is not technical complexity. The problem with software engineering is the complexity of humans and the real world. Have you ever worked on software that needs to encode 30+ years of government and employee/union agreements into rules that can be used to automatically decide how much a specific employee needs to be paid overtime in a given month? If you had you would know that no amount of tooling or software will make that problem simple.

Re: The collapse of complex software

#292

Post by 7thaccount »

timeserious wrote: post #290
7thaccount wrote: post #280

Earlier quoted context omitted.

I had trouble following your message. You've stated that Kdb+ looks good on a resume, Haskell requires explanation, and IP is bad? I'm not a big fan of closed source myself (and Kx Systems sounds like it would have a lot of red tape), but the technology is supposedly very fast and I find the language to be very simple and charming. I'd love for our analytics database to migrate , but I think the technology would real…

Get in touch! I work within KX as a Solutions Architect and would love to see how we could help you out. [redacted]

Will do at some point. Thanks for the invite!

Re: The collapse of complex software

#293

Post by CRConrad »

js8 wrote: post #80

I feel like it's a bit backwards. The software complexity increases because people lack the patience to tackle the existing complexity, or learn why things are complex, and prefer to believe that it can be simple, or get job done somehow, and in order to do that, they (somewhat egoistically) go their own way, which actually - barring occasional good insight - mostly increases the total complexity. The same goes for s…

> The software complexity increases because people lack the patience to tackle the existing complexity, or learn why things are complex, and prefer to believe that it can be simple, or get job done somehow, and in order to do that, they (somewhat egoistically) go their own way, which actually - barring occasional good insight - mostly increases the total complexity.

Easy to implement for me --> but not so good for overall structural simplicity --> worse for everyone in the long run, kind of thing?

Re: The collapse of complex software

#294

Post by infinite8s »

pketh wrote: post #277

A counter example from societies and software that seems to grow and scale well are biological systems like tissues. Maybe we could design better software and better teams using patterns from biology instead of the romans of old.

Also the idea of architecture (and discardable scaffolding) being important. To build a roman arch you need scaffolding in place until you can drop in the keystone. To build a (mammalian) body you need scaffolding in the form of a womb until the body is self-sufficient.'

There's also the matter of different types of architecture at different levels of scale. For cellular biology the foundational properties are statistical mechanics and physical chemistry. What are the foundational properties of computation?

Re: The collapse of complex software

#295

Post by agumonkey »

daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

Out of curiosity, have you watched/read Alan Kay's team work at VPRI ? They managed to make a kind of desktop OS in less than 1MLoC, tangling DSLs to generate the boilerplate from very succint rules (or parsing RFC specs for the network stack). They wanted a 100kLoC but couldn't do it, but I believe it's still doable.

Re: The collapse of complex software

#296

Post by ItsMonkk »

pjscott wrote: post #287
ItsMonkk wrote: post #269

Earlier quoted context omitted.

The number of channels is exponential, that's just Metcalfe's law. The ability to suss-out meaning behind those channels and come to a shared understanding pushes it to factorial. If A is speaking to B and C, A needs to also think about what communication is happening between B and C, and this is different to what B knows about C and C knows about B. In a carefully balanced classroom you might manage to get everyone…

> The number of channels is exponential, that's just Metcalfe's law. Metcalfe's law is quadratic rather than exponential -- O(n^2), not O(2^n). And if n people all need to be aware of communication happening between every pair of them (which is a worst-case), then that should just add a factor of n, bringing it up to cubic. (On an unrelated and less pedantically nitpicky note, one of the most valuable professional sk…

One of the surprises I got when doing that was realizing that Y's code was terrible because of how X was interacting with it, and a rewrite of X allowed a rewrite of Y, and I was able to delete half the code in each. This has occurred countless times.

But that doesn't stop black-boxes from being created faster than I can fix them, unfortunately.

Re: The collapse of complex software

#297

Post by mbrodersen »

Software complexity is 95% a software developer problem. That’s why 10x developers exist. All they have to do is to not add complexity while adding features and they are already 10x more productive long term than the average developer who is busy adding more dependencies to “fix” problems with the dependencies they added last week.

Re: The collapse of complex software

#298

Post by CRConrad »

winternett wrote: post #240
galaxyLogic wrote: post #235

There is no solution. Software is isomorphic to theorem proving (Curry-Howard). Trying to get rid of complexity in software would be like trying to get rid of complexity in math in general. Yes you can often find a simpler proof for some theorem, but that depends on the theorem, on the problem. Just because you can discover a simpler proof for something doesn't mean you can discover a simpler, less complex proof to a…

Think of software like car making. Cars once ran off steam, very simple and mechanical... Now cars run on combustion or electronics, but relatively none run off steam now, because the industry forgot to make steam engines evolve... The same goes for development... It's not impossible to rethink things in a more simple manner, it's just that we've stopped listening to the right size shoe for each foot that needs one b…

Your analogy certainly works, but probably not quite in the way you intended:

1) Cars are transitioning to electric propulsion, not electronic. Electronics have long been present in engine management of internal combustion engines too.

2) This is actually not new, but just a resurgence of something very old -- at the turn of the century (19th/20th, not 20th/21st), electric propulsion was quite common[1], the land speed record was held by an electric car[2], and the first "series-hybrid" system[3] was introduced[4].

3) Steam engines -- discounting steam turbines -- are also just combustion-driven piston engines; the only difference is that the combustion is external, not internal. OK, so it's not the combustion gases themselves that are expanding within the chamber, but steam heated (created) by them. But that's a minor detail, IMO.[5]

What does this mean for software engineering (or "engineering", in scare quotes)? Not sure... Probably something like:

A) Old tech doesn't go away. It just falls out of use and gets forgotten.

B) "New tech" often isn't. It's just old tech being uncovered or independently rediscovered.

C) What we think is a huge innovation is often just a return to older tech. ("The Cloud" ~= the mainframe?)

A + B + C = "COBOL, here we come!"? :-)

Idunno.

.

EDIT -- oops, footnotes:

___

[1]: https://en.wikipedia.org/wiki/Electric_car#Early_development...

[2]: https://en.wikipedia.org/wiki/La_Jamais_Contente

[3]: Where an internal combustion engine drives a generator that drives the wheels via electric motors and charges the main battery. https://en.wikipedia.org/wiki/Hybrid_vehicle_drivetrain#Seri...

[4]: https://en.wikipedia.org/wiki/Lohner%E2%80%93Porsche

[5]: In fact, AFAICS a four-stroke (preferably diesel?) internal combustion engine could fairly easily be converted into another kind of hybrid, an "external combustion - internal evaporisation" engine. I've been thinking about whether the (alleged?) greater efficiency of continuous combustion in comparison to intermittent would make such a conversion worthwhile. (Just one more of the many half-baked inventions bopping around in my head. :-)

Re: The collapse of complex software

#299

Post by CRConrad »

ngcc_hk wrote: post #213

Not complexity guy, competition and innovation. So far the internet is still in its early stage, yes, it is not your “system” but the whole (internet, user, those tidy systems like hbo, Facebook, …). Complexity of the individual system just like any eco-system comes and goes. But what worry is whether the fundamental change (like before and after internet). Just like you observed individuals like us die is not a (per…

> You do not much use basic to do computing or pascal.

Not you, perhaps, but some do: https://www.lazarus-ide.org/

Post reply on HN