I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…
Programs should be Small
31–40 of 75 posts
Re: Programs should be Small
#32What are you supposed to do? Find the int main() and then make the program run in your head?
I can make an analogy with a car - I don't know every one piece of it but I can infer from the context. The scale is evident.
Re: Programs should be Small
#33This tension exists at any scale, from a single-developer hobby project up to massive enterprise projects and OSS giants, so I challenge the original premise of this blog post that having a large code base is the root cause of the problem. Going too far in either direction can result in absurdity, whether that’s “enterprise software” levels of boilerplate (too much tight coupling) or DLL hell and typical Linux package management (not enough cohesion).
Re: Programs should be Small
#34Yes yes yes yes yes yes yes yes yes. This is absolutely true. The program-to-programmer relationship deserves to be many-to-one. It's a rewarding way to do things. You solve a problem. You add value. It's Done. You may have to go back to a program later to add features, but you don't end up with massive codeballs. When the program-to-programmer relationship is inverted and becomes one-to-many, you get the enterprise…
So why do larger programs 'win' in the open source world as well[1]? Pop psychology about management doesn't seem sufficient to explain the phenomenon (although I'm sure it is a good way to sell a '101 habits of highly effective managers' book or get paid to give talks about management). Large systems are large, breaking them up into smaller pieces doesn't change that, but it makes navigating the code base harder (although I assume you don't care about that since judging from your blog posts you don't think tooling is important). It makes your interfaces less malleable (can be good can be bad), and moves a lot of communication to places where the compiler can't warn you about mistakes (again, if you don't care about tooling I guess this doesn't matter… but I would argue that this is bad).
It seems to me like systems of many small separate processes is basically dynamic OOP. Everything is late bound, dynamically typed and async. It's easy to make changes and also easy to break things. You can argue that this is better for certain problems, but I don't think it's universally better, and the community seems pretty divided on the issue too: look at the popularity of Go, statically typed and building concurrency into the language rather than using the OS like in the older C world.
Aa an aside; surely the web developer community is eventually going to grow tired of talking about how terrible Java is and how $idea_of_the_moment is good because it's 'not java'? As an outsider the obsession seems extremely unhealthy, and leads you to bizarre places like arguing against automated refactoring or interactive debugging or static type systems just because those things are associated with Java. I guess to maintain credibility I also need to point out that I don't and have never used Java…
[1] Firefox/Chromium vs uzbl, gcc/llvm/clang vs pcc, gdb vs printf debugging, sqlite/mysql vs directories and plain text files, perl vs sed/awk/grep shell scripts, emacs/vim vs ed/notepad etc etc.
Re: Programs should be Small
#35I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…
The problem with corporate, big-program development is that it's a premature abstraction. If the system-of-small-programs doesn't perform, then you're in a state where larger programs might make sense. If the problem is well-understood and the pieces have been built and refined by competent programmers, but it's impossible to go any further without some coupling and integration, then a large program isn't the worst t…
For example, the program I work on has to support a million row database that can be sorted and filtered both on the server and client with subsecond response time. The program is incredibly configurable based on data in the system, so many of the features depend on reading data and reacting to it.
The problem with "many small programs" is the cost of communication. I can pass a pointer to a list of 100,000 items to be sorted and filtered in a trivial amount of time. If I have to serialize that list to json to pass to a separate program that then has to deserialize that list and perform the function, then reserialize the sorted/filtered list, send it back, re-deserialize.... it'll take longer to do the communication than it does to do the sort.
However, that's not to say that the idea of separation of concerns still can't be applied to large program. And in fact, most enterprise devs do exactly that. That's what all these "services" are in the program. Except that instead of having to serialize data, I can just pass them a pointer.
Just because you can't see all the different programs, doesn't mean they're not there.
Re: Programs should be Small
#36Yes yes yes yes yes yes yes yes yes. This is absolutely true. The program-to-programmer relationship deserves to be many-to-one. It's a rewarding way to do things. You solve a problem. You add value. It's Done. You may have to go back to a program later to add features, but you don't end up with massive codeballs. When the program-to-programmer relationship is inverted and becomes one-to-many, you get the enterprise…
>So why is big-program development winning? There are a couple reasons for that. First, it gives managerial dinosaurs the illusion of control. So why do larger programs 'win' in the open source world as well[1]? Pop psychology about management doesn't seem sufficient to explain the phenomenon (although I'm sure it is a good way to sell a '101 habits of highly effective managers' book or get paid to give talks about m…
I'm actually a pretty big fan of static typing. You don't get static typing's main benefits in C++ or Java, though. You have to use a language like Haskell or Ocaml, or the right subset of Scala, to see the major benefits of that.
Open-source is a bit different because people choose whether they contribute to a project. The quality of code in the active open-source world is leagues above what you find in typical enterprise codeballs, because of survivor bias. No one has the authority to mandate that code be maintained by others, so the messes are cleaned up by people who actually care, not people slogging through it to keep a paycheck coming.
The big-program methodology of the corporate world is the evil. In FOSS, the major projects are an unusual set-- code-quality at a high level just not seen in the for-paycheck commodity-engineer world and large because of success-- rather than the reverse. There's a survivor bias that occurs because the best projects are the only ones people pay attention to.
The corporate world is screwy because projects become large or small based on political reasons that have nothing to do with code quality. In the FOSS world, code-quality problems related to growth will be self-limiting because no one has the authority to "force" the program to grow.
Re: Programs should be Small
#37Re: Programs should be Small
#38Re: Programs should be Small
#39I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…
When you have a big system with disjoint parts written in different languages, re-use and refactoring is a pain, and redundancy is almost certain to creep in (and with redundancy often comes inconsistency).