Live data from Hacker News

Software Development Has Diseconomies of Scale

allankelly.blogspot.com

71–80 of 89 posts

Re: Software Development Has Diseconomies of Scale

#71
post #49

Earlier quoted context omitted.

> These are all very suitable to such decomposition into communicating processes. That doesn't make them cheap to build; only possible.

Better expensive than not for sale.

Sure, but I think the author's point about "diseconomy" of scale still stands even when the large system is composed of small parts (and pretty much all large software systems in at least the last 20 years have been composed of small parts).

Re: Software Development Has Diseconomies of Scale

#73
post #51

Bootloaders are small, but very important software. k/q is small but a very useful interpreter. There are so many examples, but it appears that to "the market" the most valued software development is large scale. The sentiment is create and contribute to large projects or go home. Stupid, but true. "Do one thing well" is more than just a UNIX philosophy. It is an essential truth. Most programs are lucky if they can d…

> "Do one thing well" is more than just a UNIX philosophy. It is an essential truth. Yes, thanks. My eye detects light and passes the signal on to my brain. It does it quite well. My liver removes toxins from my blood. It does it quite well. The big ball o mud made by BigCo is a hammer that reads the ambient temperature, weighs my nails, tells me the time, and predicts the outcome of futures markets. Is it any wonder…

Oh please. Your eyes are a system rife with bugs. You have a blind spot that exists for absolutely no good reason - the bundle of nerves can just as well connect to the back of the retina rather than the front. About 50% of people have a defective lens, which can only be corrected by putting yet another lens in front of it (in frames, directly on the eyeball or by ablating the eyeball in order to turn it into a lens). You can be near-sighted AND far-sighted at the same time. Your eyeballs contain imperfections called floaters that your body has no way of removing, so your brain learns to filter them out. You constantly see your own nose. Some people's eyes are misaligned and they have no depth perception. The colour violet doesn't exist, your red cones are actually activating in response to your blue cones detecting high-frequency blue light. And I don't mean they activate in response to the light, physically the red cones don't detect any of the wavelengths of violet light - they're just wired up in such a way that a low activation of the blue cones triggers activation of the red cones. Some people have a defect that makes their red cones activate in response to wavelengths too close to green light, so they can't tell the two apart. Some are just completely colour-blind. Oh, and the image you get in your eyes is rotated 180 degrees.

Your eyes most definitely don't do just one thing and they most certainly don't do it all that well. But they do an important job reasonably well, which is what strive for in any large software system.

Re: Software Development Has Diseconomies of Scale

#74

1 (British pound per liters) = 5.69500061 U.S. dollars per US gallon Milk is a lot cheaper in the US. I usually pay about $4/gallon.

You'd have to be buying organic goat milk to pay that much. From the figures in the article:

Buy it by the pint: 0.49 (British pounds per Imperial pint) = 4.90836249 U.S. dollars per US gallon

Buy it by the 2 pints: (0.85 British pounds) per (2 Imperial pints) = 4.25725318 U.S. dollars per US gallon

Buy it by the 4 pints: (1 British pound) per (4 Imperial pints) = 2.50426658 U.S. dollars per US gallon

Re: Software Development Has Diseconomies of Scale

#75
post #64

Earlier quoted context omitted.

"But you need to control complexity as you develop." One way to control the complexity is by consistent development of automatic regression tests which cover >90% of functionality. Then you are much less afraid if you remove something or make a big change in production grade software used by many customers.

This is one of those seductive statements that is true, but it's also such incredibly bad advice that just this one piece of theory has the potential to completely destroy your company if you follow it. But it's seductive for a reason : it's true. Doing regression tests on this scale will protect you from quite a sizable class of bugs. BUT there is no way to write lots of tests without locking down the implementation…

I've been thinking about this a lot, and I think a way out might be to drop the phrase 'black box' in your comment. I've been exploring white-box testing by making assertions on the log emitted by a program. Not only does it allow greater flexibility in large-scale reorganizations without needing to mess with tests, it also allows us to write tests for things that we don't currently write tests for, like performance (make sure the number of swaps in this sort function don't exceed this threshold), fault tolerance, race conditions, and so on.

More details: http://akkartik.name/about. An early write-up on white-box testing: http://akkartik.name/post/tracing-tests

Re: Software Development Has Diseconomies of Scale

#76

Every time software economies of scale come up, I can't help but be reminded of Jira's pricing model ( https://www.atlassian.com/software/jira/pricing?tab=host-in-... ): (Per month:) Users Total Per user ----- ------- -------- 1 $10 $10 5 $10 $2 10 $10 $1 15 $75 $5 25 $150 $6 50 $300 $6 100 $450 $5 500 $750 $2 2000 $1500 $1

It is certainly odd, but I'm guessing this is more about price discrimination. I think price must be greater than or equal to cost at some level, but certainly not proportional to it.

Re: Software Development Has Diseconomies of Scale

#77
post #71

Earlier quoted context omitted.

Better expensive than not for sale.

Sure, but I think the author's point about "diseconomy" of scale still stands even when the large system is composed of small parts (and pretty much all large software systems in at least the last 20 years have been composed of small parts).

You could make the exact same claim about any bespoke piece of technology. What does it cost to make one new car? one aircraft? one piece of electronics? one moon rocket? one satellite?

It's all economies of scale that come into play as soon as you can ramp up the volume. And that's where software wins hands down, in spite of all the up-front costs.

Re: Software Development Has Diseconomies of Scale

#78
post #27

Earlier quoted context omitted.

While some small, simple programs are great, big economic contribution is mainly in large software: manufacturing control, ERP, air-traffic control, power-plant control etc.. Small software is great, but the big economic impact belongs to big software.

There is no reason why 'big software' can not be made up out of small software pieces inter-operating. In fact those are the strongest and most maintainable systems.

Big software is made up of inter-operating small pieces of software: subroutines and objects ;)

Breaking something into multiple processes doesn't magically reduce complexity. You can have the same architecture within one process or spread out among 50, and in the latter case you've actually added the complexity of some IPC layer and lifetime management for all of the different processes.

That's obviously not what you meant to say. You meant: break things down into the right abstract components such as to decrease coupling and increase cohesion. You meant: do it so well that Component A can be maintained by a team that doesn't even know about the existence of Component B. And that routine changes in the requirements for one don't require changes in the other. That's really hard, especially if your codebase is a coupled mess because you wanted to SHIP IT fast. It's very hard to make a business case for big refactoring ("So we're going to spend all of this money and at the end it's going to work the same way it does now, but maybe a bit better? Can't we just spend that money fixing bugs instead?"). Also, strong boundaries between components can ironically make it harder to refactor, when it turns out that a different breakdown makes more sense. And God-forbid that other projects have taken dependencies on the components, further tying your hands.

If this was like traditional engineering, you'd have some massive upfront planning effort, dozens of committees, reviews by regulators, massive fault tolerance, etc. You'd then have a (hopefully) mini version of this same process every time requirements change or technical limitations get in the way of your original plan. Five years later you release your shiny piece of modular enterprise software. But your competitor shipped their big monolithic alternative that GetsTheJobDone in a quarter of the time and price. A good example of this is that all of the successful operating systems have monolithic kernels, even though academics and researchers favor microkernels.

So you're right, there's no reason why big software can't be made up of a bunch of small software, but there are a lot of reasons why it sometimes isn't.

Re: Software Development Has Diseconomies of Scale

#79
post #73
post #51

Earlier quoted context omitted.

> "Do one thing well" is more than just a UNIX philosophy. It is an essential truth. Yes, thanks. My eye detects light and passes the signal on to my brain. It does it quite well. My liver removes toxins from my blood. It does it quite well. The big ball o mud made by BigCo is a hammer that reads the ambient temperature, weighs my nails, tells me the time, and predicts the outcome of futures markets. Is it any wonder…

Oh please. Your eyes are a system rife with bugs. You have a blind spot that exists for absolutely no good reason - the bundle of nerves can just as well connect to the back of the retina rather than the front. About 50% of people have a defective lens, which can only be corrected by putting yet another lens in front of it (in frames, directly on the eyeball or by ablating the eyeball in order to turn it into a lens)…

You're right. It's only one of the most complex and intricate pieces of machinery in the universe, enabling billions of organisms around the planet to build a realistic, 3-dimensional picture of the world around them by picking up hundreds of millions of bits of information from the fastest moving particles known to man.

It's about on par with Microsoft Word.

Re: Software Development Has Diseconomies of Scale

#80

Earlier quoted context omitted.

There is no reason why 'big software' can not be made up out of small software pieces inter-operating. In fact those are the strongest and most maintainable systems.

Big software is made up of inter-operating small pieces of software: subroutines and objects ;) Breaking something into multiple processes doesn't magically reduce complexity. You can have the same architecture within one process or spread out among 50, and in the latter case you've actually added the complexity of some IPC layer and lifetime management for all of the different processes. That's obviously not what yo…

> Breaking something into multiple processes doesn't magically reduce complexity.

Actually, it does.

If you break up a complex system into communicating processes then for each and every process you have a clearly defined set of inputs and outputs, memory protection, the ability to run multiple instances seamlessly without mucking around with threads and a very much reduced scope.

Decomposition into multiple communicating processes is an extremely powerful tool to reduce complexity.

Post reply on HN