Live data from Hacker News

Software Development Has Diseconomies of Scale

allankelly.blogspot.com

81–89 of 89 posts

Re: Software Development Has Diseconomies of Scale

#81

Earlier quoted context omitted.

Big software is made up of inter-operating small pieces of software: subroutines and objects ;) Breaking something into multiple processes doesn't magically reduce complexity. You can have the same architecture within one process or spread out among 50, and in the latter case you've actually added the complexity of some IPC layer and lifetime management for all of the different processes. That's obviously not what yo…

> Breaking something into multiple processes doesn't magically reduce complexity. Actually, it does. If you break up a complex system into communicating processes then for each and every process you have a clearly defined set of inputs and outputs, memory protection, the ability to run multiple instances seamlessly without mucking around with threads and a very much reduced scope. Decomposition into multiple communic…

Yes, it reduces complexity considerably, but nearly all big systems are already built this way and are still very complex. You cannot use this approach to eliminate complexity, i.e. make the cost linear with the system size.

Some domains where software is used to yield a tremendous economic impact are so complex that even with all complexity-management approaches at our disposal, the cost of creating them still grows super-linearly with their size.

If there are n abstract features to the system, all dependent on almost all others, you have an order of n^2 interactions. Whether you make every function its own simple communicating process, or bundle a few of them together (shifting complexity into a process) the abstract complexity of the system is still n^2.

Re: Software Development Has Diseconomies of Scale

#82
post #71

Earlier quoted context omitted.

Sure, but I think the author's point about "diseconomy" of scale still stands even when the large system is composed of small parts (and pretty much all large software systems in at least the last 20 years have been composed of small parts).

You could make the exact same claim about any bespoke piece of technology. What does it cost to make one new car? one aircraft? one piece of electronics? one moon rocket? one satellite? It's all economies of scale that come into play as soon as you can ramp up the volume. And that's where software wins hands down, in spite of all the up-front costs.

I think that all the author was trying to say is that the cost of a single software system grows super-linearly with its size.

Re: Software Development Has Diseconomies of Scale

#83
post #81

Earlier quoted context omitted.

> Breaking something into multiple processes doesn't magically reduce complexity. Actually, it does. If you break up a complex system into communicating processes then for each and every process you have a clearly defined set of inputs and outputs, memory protection, the ability to run multiple instances seamlessly without mucking around with threads and a very much reduced scope. Decomposition into multiple communic…

Yes, it reduces complexity considerably, but nearly all big systems are already built this way and are still very complex. You cannot use this approach to eliminate complexity, i.e. make the cost linear with the system size. Some domains where software is used to yield a tremendous economic impact are so complex that even with all complexity-management approaches at our disposal, the cost of creating them still grows…

Yes, it reduces complexity considerably, but nearly all big systems are already built this way and are still very complex.

complexity is a function of interactions. multiple processes won't do much for you if every one communicates with everybody else.

If there are n abstract features to the system, all dependent on almost all others [...].

the discussion has been about using process separation as an aid in preventing accidental complexity arising from improper coupling introduced as a shortcut so this looks a little disingenuous. my response is that if you separate every function into its own simple process, a misbehaving function need not take down the whole group.

on the improper-coupling-reduction front, process separation is apparently paramount to success, see eg. sendmail vs postfix.

Re: Software Development Has Diseconomies of Scale

#84
post #83
post #81

Earlier quoted context omitted.

Yes, it reduces complexity considerably, but nearly all big systems are already built this way and are still very complex. You cannot use this approach to eliminate complexity, i.e. make the cost linear with the system size. Some domains where software is used to yield a tremendous economic impact are so complex that even with all complexity-management approaches at our disposal, the cost of creating them still grows…

Yes, it reduces complexity considerably, but nearly all big systems are already built this way and are still very complex. complexity is a function of interactions. multiple processes won't do much for you if every one communicates with everybody else. If there are n abstract features to the system, all dependent on almost all others [...] . the discussion has been about using process separation as an aid in preventi…

Let me put it another way. Many of the software systems that make the biggest economic contributions have a very high essential complexity, and so their cost grows super-linearly with their size.

Re: Software Development Has Diseconomies of Scale

#85

Every time software economies of scale come up, I can't help but be reminded of Jira's pricing model ( https://www.atlassian.com/software/jira/pricing?tab=host-in-... ): (Per month:) Users Total Per user ----- ------- -------- 1 $10 $10 5 $10 $2 10 $10 $1 15 $75 $5 25 $150 $6 50 $300 $6 100 $450 $5 500 $750 $2 2000 $1500 $1

It is certainly odd, but I'm guessing this is more about price discrimination. I think price must be greater than or equal to cost at some level, but certainly not proportional to it.

Price discrimination is an important aspect to consider but I don't think it's directly relevant to the post.

From a business perspective, price discrimination probably has more to do with accounting, support, and long-term sustainability.

A transaction that includes 1000 seats requires 1 account to manage them and will likely have in-house support staff to help solve with technical issues.

A 1000 transactions for 1 seat requires 1000 accounts to manage and 1000 potential points of contact that need to be supported when things go wrong.

In addition, companies that have the resources to buy a lot of seats up front are more likely to be more 'stable'. Those are the customers who will standardize their internal processes to use your software, thereby guaranteeing consistent payment over the long term, and higher likelihood of return sales in the future.

That's probably why a lot of SaaS sites offer free accounts for individual/small-team usage. Early adopters tend to be more technically inclined, will evangelize the product if they like it, will be more sympathetic when minor bugs are encountered, and the 'free' aspect means there's no guarantee of support. They're essentially buying a community of QA testers with free seats.

The 1000 seat customers cover the bills (ie CAPEX+Fixed-Costs). The lesser customers cover the rest (ie OPEX+Variable Costs). The latter keeps things in motion, the former signals it's safe to scale up.

To remain profitable it's important to minimize CAPES+Fixed-Costs as much as physically possible because when feast turns to famine they're the hardest to reduce.

In accounting terms, favoring financial liquidity means favoring flexibility to adjust to changes in the market. This is a key deciding factor behind companies shifting to 'the cloud'. Even if the upfront costs are higher than maintaining bare metal servers, it takes little/no effort to ditch redundant infrastructure when it comes time to reduce costs.

Re: Software Development Has Diseconomies of Scale

#86

Normally EVERYTHING has both economies and diseconomies of scale. You model the price per unit as the sum of different curves. Complexity not only increases on Software, but if you design a thermal engine, or a plane, or a car. Working making something as simple as fiberglass, we had something like 100 components, like tensioactives. Most of them we had no idea what they were for, as they were added like decades ago…

Following your fiberglass manufacturing example. It sounds like you've identified technical debt as it applies to physical engineering.

In all likelihood, somebody on the 'business end' made some very bad decisions that were penny-wise/pound-foolish.

Long-term sustainability for mechanical systems require paying extra to maintain the design.

A lot of business decisions involving engineering are completely reactionary. Changes are usually required when some unforeseen event happens (ex something breaks, market changes, etc). Poor budgeting will lack a contingency plan to cover these costs.

Even with a full-time staff of engineers, their role is usually limited to daily operations. Future updates usually require contracting outside support, at a premium cost.

When it comes time to do the work, it's in the interest of the business decision makers to reduce the impact on the 'bottom line' as much as physically possible.

The work will need to be done on a short schedule to reduce the impact on production. As soon as the changes are complete, it's in the business' best interest to send the outside support home as soon as possible. As a result, design/documentation updates are left undone.

Technical Debt:

Over time, physical changes accumulate whereas the design stays the same. When the design/documentation is unreliable, the cost of future changes grows dramatically with each additional update.

Technical Bankruptcy:

At some point it gets so bad that the design/documentation isn't worth the paper it's printed on, nobody knows how the system works, the implementation becomes fragile. Nobody is willing to accept the risk/cost of a significant engineering redesign. The system runs until it fails and the business writes it off as a loss.

There are some parallels that can be drawn. Complexity, is inherently difficult to maintain in the long-term.

You could choose to favor:

- Traditional Design:

Monolithic architecture w/ deep-coupling in software vs monolithic engineering designed for production not maintainability.

Everything is delivered as one big ball of mud. Even with good test coverage, the sheer complexity makes it difficult to make changes.

- Black Boxes:

External libraries in software vs COTS components in hardware.

The internals are opaque, if it breaks the only option is to replace it. If it becomes obsolete and/or long-term support is cut, good luck.

- White Boxes:

Internal well-tested libraries/modules in software vs modular/pluggable well-tested designs in hardware.

White boxes solve the long-term support issue but require significant resources to maintain/develop.

- Gray Boxes:

Open source libraries/modules in software; open hardware designs.

May not be feature-complete or include sufficient test coverage. The upfront cost is dramatically reduced but there may be license/communication barriers that prevent changes from being made.

There are trade-offs and risks that come with each approach. Monolithic designs require better language-level architecture and internal separation of concerns. Modular design require better system-level architecture and external separation of concerns. The only constant is, the greater complexity the greater the cost to maintain.

Re: Software Development Has Diseconomies of Scale

#87

Bootloaders are small, but very important software. k/q is small but a very useful interpreter. There are so many examples, but it appears that to "the market" the most valued software development is large scale. The sentiment is create and contribute to large projects or go home. Stupid, but true. "Do one thing well" is more than just a UNIX philosophy. It is an essential truth. Most programs are lucky if they can d…

It's the intercommunication with the web that becomes so painful. Users have no way to connect a stream of web applications, and the web disincentivized solving the corporate distribution problem.

How so? The web is already standardized on REST as the 'universal' API.

Publicly accessible microservices are becoming more and more common over time and some of them are being used to create integrations.

Have you heard of Zapier? https://zapier.com/how-it-works/

Slack bots also interact with other services: https://slack.com/apps/category/At0EFT67V3-new-noteworthy

There's also the Huggin project: https://github.com/cantino/huginn

Re: Software Development Has Diseconomies of Scale

#88

Earlier quoted context omitted.

It's the intercommunication with the web that becomes so painful. Users have no way to connect a stream of web applications, and the web disincentivized solving the corporate distribution problem.

How so? The web is already standardized on REST as the 'universal' API. Publicly accessible microservices are becoming more and more common over time and some of them are being used to create integrations. Have you heard of Zapier? https://zapier.com/how-it-works/ Slack bots also interact with other services: https://slack.com/apps/category/At0EFT67V3-new-noteworthy There's also the Huggin project: https://github.com…

I am thinking more at the internal-software level.

Consider an enterprise that has a simple case management app -- essentially, a help desk app for one portion of the business.

There is another app within the system that was developed 5 years before to help choose when to start a case. It's maintained by a different team, so there's no natural coordination.

If these were two Unix tools built like programmers would build, someone could connect the output of one, to a filter in the middle, to the input of the next. However, in the real organization, there is an individual who spends 20 hours a month to manually copy the information from app 1 to app 2 because the work to create a proper hypermedia interface.

The combination of a corporate-aware Huginn (single sign on, etc) and a revolution in corporate apps that treated hypermedia as a first class citizen could make it happen. It's not happening now, however.

Re: Software Development Has Diseconomies of Scale

#89

Earlier quoted context omitted.

How so? The web is already standardized on REST as the 'universal' API. Publicly accessible microservices are becoming more and more common over time and some of them are being used to create integrations. Have you heard of Zapier? https://zapier.com/how-it-works/ Slack bots also interact with other services: https://slack.com/apps/category/At0EFT67V3-new-noteworthy There's also the Huggin project: https://github.com…

I am thinking more at the internal-software level. Consider an enterprise that has a simple case management app -- essentially, a help desk app for one portion of the business. There is another app within the system that was developed 5 years before to help choose when to start a case. It's maintained by a different team, so there's no natural coordination. If these were two Unix tools built like programmers would bu…

The problem with 'enterprisey' corporate applications has more to do with the culture of the developers who build those applications.

Instead of breaking applications apart into distinct services they devs of corporate apps prefer to use OOP to create complex inter-dependent monolithic architectures. APIs for such apps are usually exposed by using a library, so a lot of emphasis is placed on access privileges at the language level (ie private/internal/public) instead of at the interface level (ie pipes/REST).

The problem isn't that the capability doesn't exist, it's that corporations are either unaware or unwilling to experiment with alternative approaches.

For example, one employee could spend half their time collecting data in Excel files from multiple members of a team, organizing it in a presentation-ready format, and emailing it up the chain.

Alternatively, the team members could each have a Google Spreadsheet where they input the data and a timed trigger fires off a bot that processes the data, saves it to a new file, and emails the link up the chain.

Both accomplish the same result. The second is a lot less prone to error/bias, and is also a lot more cost effective. But, to get an organization to adopt the latter approach requires convincing the organization to give up Excel as their canonical tool for organizing data.

I have fought that battle and it's a lost cause. Even when adequate proof of cost/time savings are presented in a business-friendly format, inertia trumps reason.

A company could theoretically hire a developer to deploy a digital assistant to streamline/automate the mundane and repetitive business processes. Except, devs are expensive, in short supply, and it's a really hard sell business leaders on the idea of adopting processes/tools that they don't fully understand.

Post reply on HN