Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The collapse of complex software

nolanlawson.com

Post reply on HN
Print view View original

281–290 of 304 posts

Re: The collapse of complex software

#281

Post by wtetzner »

eranation wrote: post #129

> Simplicity of design sounds great in theory, but it might not win you many plaudits from your peers. A complex design means more teams to manage more parts of the system, more for the engineers to do, more meetings and planning sessions, maybe some more patents to file. A simple design might make it seem like you’re not really doing your job. “That’s it? We’re done? We can clock out?” And when promotion season come…

Yeah, the thing that seems to be missed here is that designing simple things is hard. Things are complicated because the first version of pretty much everything ends up being more complicated than it needs to, and most engineers don't get the chance to revise/rewrite the first version into something simpler.

Re: The collapse of complex software

#282

Post by galaxyLogic »

winternett wrote: post #240
galaxyLogic wrote: post #235

There is no solution. Software is isomorphic to theorem proving (Curry-Howard). Trying to get rid of complexity in software would be like trying to get rid of complexity in math in general. Yes you can often find a simpler proof for some theorem, but that depends on the theorem, on the problem. Just because you can discover a simpler proof for something doesn't mean you can discover a simpler, less complex proof to a…

Think of software like car making. Cars once ran off steam, very simple and mechanical... Now cars run on combustion or electronics, but relatively none run off steam now, because the industry forgot to make steam engines evolve... The same goes for development... It's not impossible to rethink things in a more simple manner, it's just that we've stopped listening to the right size shoe for each foot that needs one b…

That is a good point. In software development momentum is everything. Why because you got to have many people working on it to allow it to become better. And "better" is a holistic thing: You got to be COMPATIBLE with other popular development tools, to be "better".

Re: The collapse of complex software

#283

Post by coip »

daenz wrote: post #68
davidgay wrote: post #59

Earlier quoted context omitted.

> I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. Not that I want to discourage you, but my view is that anything that makes software engineering simpler just leads us to tackling more complex problems until the complexity reaches the limit that people can handle. So in that view, you can't succeed at making software engineering alway…

If history is a guide, then you are correct. But I believe complexity is fractal (I think we all know this: "turtles all the way down..."), and that you can structure and manage complexity with recursive rules, so that you only see the resolution that you care about, for the areas that you care about, and everything else is a low frequency representation. Frameworks approximately do this, but for specific domains: th…

You seem like a fun person to work with

Re: The collapse of complex software

#284

Post by leonroy »

I think as software is growing increasingly complex (and it only grows more complex) it's reaching the level where it will become difficult for a human brain to fully grasp the system.

We can only keep going back to the drawing board so much before the cost of rebuilding the system becomes prohibitive.

What do I see instead happening?

I think tools like Github Copilot which assist user's writing code will become ever more sophisticated and broad in their application. That they will not only perform simple bug fixes or write blocks of repetitive code, but even automatically build integrations with third party systems with just an API specification, refactor code to make it cleaner, less repetitive and more decoupled and be able to break apart complex monolithic code bases into smaller libraries or services.

Of course building such a system is a complex undertaking in its own right, but much like how a compiler eventually eaches a point where it can compile its own binary so too such a system might reach a point where it can extend and improve itself taking over from its human creators.

Re: The collapse of complex software

#285

Post by ozim »

mathgladiator wrote: post #48
daenz wrote: post #11

This is a problem I want to focus my life solving. I believe that software engineering can be made so simple and clear that it can be wielded by an extremely small team of engineers. I believe that there are finite and fundamental classes of problems, that underpin the vast majority of problems, that have a distinct visual representation, and that by representing these problems visually, you let your "visual coproces…

Welcome to the club! The fundamental problem of simplifying software is humans. Just consider date formats, time zones, and tax codes. Humans love to make complicated things. My philosophy on this has been to take the complex human stuff and stick it in a black box. A professional feather in cap with this approach is called BladeRunner ( https://dl.acm.org/doi/10.1145/3477132.3483572 ) which radically simplified the…

If you look from perspective of software creator that wants to handle all kind of timezones or post codes - yes humans complicate things.

If you look from a person or user perspective - time zones or date formats are used in one location or in some context. People living in that context have it a lot easier because most of the time they don't care about other time zones.

I would say people simplify things but on local scale. If you want to go global that is your problem not humans that live in one place and use single time zone all their life.

Re: The collapse of complex software

#286

Post by Heyso »

vlunkr wrote: post #109
loudmax wrote: post #28

I highly recommend Rich Hickey's "Simple Made Easy" talk as a way to think about software complexity. We all know that complex systems are hard to work with, but I've found his presentation very useful in understanding the nature of complexity. Understanding complexity helps being able to avoid or minimize it. For example, easy is not the same as simple, and simple changes to one part of a system can introduce great…

I hadn't heard that before, it's great. I like how he states that simpler software doesn't necessarily mean fewer components (roughly quoting). Where the article argues for fewer boxes and arrows. It's hard to argue specifics, because the article isn't getting into them, but sometimes more boxes and arrows makes things simpler. If you don't have multiple boxes, what do you have? One box of spaghetti? I also disagree…

Well, if you compare react and angular2+, react has less stuff, and is less complex. But on the opposite side, a function that tries to do too much (like lodash pick function) look simple at first glance, but is actually complex.

Re: The collapse of complex software

#287

Post by pjscott »

ItsMonkk wrote: post #269
pjscott wrote: post #202

Earlier quoted context omitted.

> As teams get bigger, communication sales factorially, That sounds a bit drastic. Surely at worst the scaling is quadratic?

The number of channels is exponential, that's just Metcalfe's law. The ability to suss-out meaning behind those channels and come to a shared understanding pushes it to factorial. If A is speaking to B and C, A needs to also think about what communication is happening between B and C, and this is different to what B knows about C and C knows about B. In a carefully balanced classroom you might manage to get everyone…

> The number of channels is exponential, that's just Metcalfe's law.

Metcalfe's law is quadratic rather than exponential -- O(n^2), not O(2^n). And if n people all need to be aware of communication happening between every pair of them (which is a worst-case), then that should just add a factor of n, bringing it up to cubic.

(On an unrelated and less pedantically nitpicky note, one of the most valuable professional skills I've developed is an ability and willingness to dive into black boxes. It's remarkable how many bugs arise from the interaction of two or more pieces of software with no overlap between their developers. Debugging those can involve a long, agonizing back-and-forth between people who know how X works and people who know how Y works -- or it can be over in a jiffy if you can quickly familiarize yourself with the basic internal workings of both. Past a certain point nobody can know everything, it's true, but this doesn't need to be as crippling for productivity as a lot of people allow it to be.)

Re: The collapse of complex software

#288

Post by 0xFACEFEED »

danielovichdk wrote: post #249
0xFACEFEED wrote: post #196

Earlier quoted context omitted.

We're talking about the same thing. Perhaps I was just speaking more broadly. > don't build abstractions because you might need them later ... [snip] ... This is because you'll likely be wrong about your assumptions about which abstractions you'll need, and ripping those abstractions out is going to be more expensive than just modifying simple code to do new things. At worst, this is dangerously wrong. At best, it cr…

I totally agree on validating ideas but that's IMO not looking in to the future. It's trying to find a fit for something you believe is a good idea. There is a big difference in testing whether something is needed in the near future or you simply stick to your guns and blindly dictate how you believe the future will look like. Your ORM example is good and exactly what i am referring to. Too much abstraction/gold plat…

> There is a big difference in testing whether something is needed in the near future or you simply stick to your guns and blindly dictate how you believe the future will look like.

There are innumerable successful examples of committing blindly toward a future few people believe in. These are often cases where the objective evidence of success is very limited. If the objective evidence existed then "everyone would be doing it".

This is the crux of my disagreement. Of course we'll both agree that applying the wrong/stupid abstraction is wrong. But that's not what I'm discussing.

I'm talking about this quote specifically:

> Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now!

Perhaps you need to reframe this point to make it more clear?

> Don't look into the future. Because you cannot.

This is silly for all the aforementioned reasons. Especially when innovating in tech!

> And you are both naive and arrogant if you believe you can.

Calling someone arrogant for attempting to predict the future in technology. Come on, man.

> Deal with what is known now!

What is known now is what everyone else knows now. Unless you're dealing with the _extremely_ rare case where you have access to information no one else does, innovation requires one to commit to the unknown.

Re: The collapse of complex software

#289

Post by 0xFACEFEED »

wtetzner wrote: post #275
0xFACEFEED wrote: post #196

Earlier quoted context omitted.

We're talking about the same thing. Perhaps I was just speaking more broadly. > don't build abstractions because you might need them later ... [snip] ... This is because you'll likely be wrong about your assumptions about which abstractions you'll need, and ripping those abstractions out is going to be more expensive than just modifying simple code to do new things. At worst, this is dangerously wrong. At best, it cr…

I'm certainly not claiming you should never design anything up front. And I'm also not saying you should never design abstractions. As with everything, the devil is in the details. What I am trying to get at is that I've seen many times where a complex abstraction was introduced, and then it was only ever used for a single implementation of something. I've also seen many times where an abstraction was introduced, to…

I think you and I are generally on the same page, but we're reading this statement by the root commenter very differently:

> Don't look into the future. Because you cannot. And you are both naive and arrogant if you believe you can. Deal with what is known now!

See my additional response to him elsewhere in the thread where I tried to break it down sentence by sentence.

Everything you're saying is on point for good system design IMO. What he was saying was totally different and IMO is a "dangerous" way of thinking when innovation is important.

Re: The collapse of complex software

#290

Post by timeserious »

7thaccount wrote: post #280
retcon wrote: post #266

Earlier quoted context omitted.

KDB is unique for translating "in house" tech to marketable CV entry status. Having Haskell and CS(FB) experience requires educating the recruitment agencies, for comparable recognition. Personally I consider every ill in computing as symptoms of the intellectual property system, from making reading potential prior art (aka learning) verboten to non disclosures making the Peter Principle look like the common cold to…

I had trouble following your message. You've stated that Kdb+ looks good on a resume, Haskell requires explanation, and IP is bad? I'm not a big fan of closed source myself (and Kx Systems sounds like it would have a lot of red tape), but the technology is supposedly very fast and I find the language to be very simple and charming. I'd love for our analytics database to migrate , but I think the technology would real…

Get in touch! I work within KX as a Solutions Architect and would love to see how we could help you out. [redacted]
Post reply on HN