Live data from Hacker News

Programs should be Small

mkhadikov.com

61–70 of 75 posts

Re: Programs should be Small

#61
post #40

Beyond complexity management, breaking a design into many small programs opens it up to a rich set of well-known and proven OS services. Things like hardware memory protection, multi-processor support, queues, mutexes, monitoring, and cross language support. I'll take Unix over some language+standard library _any day_.

and it also opens it up to a rich set of well-known problems: sharing between processes and memory management, lock issues, system-call latencies, not to mention dealing with the monolithic environments that are almost definitely changing between each instance of your program you want to run (this 'environment' includes the shell, userspace services, kernel version and features...). Decomposing a program into multipl…

Absolutely. But it's better from a design perspective to start as pure as possible and then make concessions when you find your back against the wall.

Most of the "it's too hard and there's too much to do!" crowd doesn't understand the benefits of working clean.

Re: Programs should be Small

#62

Earlier quoted context omitted.

>So why is big-program development winning? There are a couple reasons for that. First, it gives managerial dinosaurs the illusion of control. So why do larger programs 'win' in the open source world as well[1]? Pop psychology about management doesn't seem sufficient to explain the phenomenon (although I'm sure it is a good way to sell a '101 habits of highly effective managers' book or get paid to give talks about m…

On the anti-Java bias, I think the issue is that there's more than one Java culture. There's the horrid commodity developer culture, but that's not the language's fault. I'm actually a pretty big fan of static typing. You don't get static typing's main benefits in C++ or Java, though. You have to use a language like Haskell or Ocaml, or the right subset of Scala, to see the major benefits of that. Open-source is a bi…

What do you mean by subset of scala, is it an absolute prohibition ("It will be flagged, you must be able to justify why you used this") on features used like the Google C++ style guide, or maybe the Levels in Cay Horstmann's book (which I think Odersky came up with), L1-3, A1-3, or something else?

Re: Programs should be Small

#63
post #35

Earlier quoted context omitted.

The problem with corporate, big-program development is that it's a premature abstraction. If the system-of-small-programs doesn't perform, then you're in a state where larger programs might make sense. If the problem is well-understood and the pieces have been built and refined by competent programmers, but it's impossible to go any further without some coupling and integration, then a large program isn't the worst t…

Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance. For example, the program I work on has to support a million row database that can be sorted and filtered both on the server and client with subsecond response time. The program is incredibly configurable based on data in the system, so many of the features depend on…

> Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance.

Not where I work.

And even then, this is no excuse to stick to a zeroth order heuristic, and make big programs every time. Some systems can be cleanly separated in simple components. Failing to see that is a waste.

Re: Programs should be Small

#64
post #20

I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…

It's disgusting that your comment - a nasty, nearly content-free spit of cynical contrarianism - is currently the most highly-ranked reply to what was actually a congenial and well thought out blog post.

Re: Programs should be Small

#65
post #59
post #44

Earlier quoted context omitted.

Yes heterogeneous systems are much easier to deal with, although from experience certain systems are a pig to deal with from end to end (anything Microsoft as a rule). Different languages are just different forms of integration and the mantra of integration is hell should be in the forefront of everyone's mind, always.

Depends on tools you use, I found Apache Thrift and Protobuf pretty to be sophisticated tools for integration between services.

Yet they are still entirely impractical for what we do. There is no one size fits all methodology which results in a heterogeneous communication layer. This means that you end up with technology fragmentation and therefore additional complexity.

Re: Programs should be Small

#66
post #38

My current theory is that a good program should compile in less than 1 second into an executable of less than 1 megabyte.

My current theory is that this is an arbitrary one. Its really, really, really easy to cross a megabyte with statically linked libraries.

I think the grandparent intended that the size measurement exclude statically linked libraries or assets, debug symbols, and compression technologies like UPX.

It can sometimes be beneficial from a distribution/deployment standpoint to have everything in one self-contained file. But you can't conclude much about the code quality of e.g. a computer game engine based on how many megabytes of graphics, music and sound effects a particular game based on that engine uses.

Re: Programs should be Small

#67

Yes yes yes yes yes yes yes yes yes. This is absolutely true. The program-to-programmer relationship deserves to be many-to-one. It's a rewarding way to do things. You solve a problem. You add value. It's Done. You may have to go back to a program later to add features, but you don't end up with massive codeballs. When the program-to-programmer relationship is inverted and becomes one-to-many, you get the enterprise…

>So why is big-program development winning? There are a couple reasons for that. First, it gives managerial dinosaurs the illusion of control. So why do larger programs 'win' in the open source world as well[1]? Pop psychology about management doesn't seem sufficient to explain the phenomenon (although I'm sure it is a good way to sell a '101 habits of highly effective managers' book or get paid to give talks about m…

I think async architectures are worthy but miss something important - the idea of "typed connections" between processes. Actors passing arbitrary messages doesn't make for an efficient "assembly line." Similarly, the Unix-style concurrency is tied to a single low-level protocol, which is not rich enough to adequately describe all data. It described "enough" of all data for 1970's era tasks, but our needs outgrew that.

When you set out to architect a customized, typed, async architecture, you end up with the "flow-based programming" style which has been captivating me recently. It tends to reduce to two events per component: "start" and "stop." The "integrated program" appears in a tiny top-level definition that sets up the components and connections. Components are relatively small and reminiscent of "pure" algorithmic code. Where synchronized behavior becomes essential, flows can be split into stages of processing and kicked off in a sequence.

This particular style has had a lengthy history of reappearing in numerous domains under various guises, and it has demonstrable effectiveness, but it can also feel alien and more "mathematical." The main issue is that has a lengthy design/prototyping time of weeks to months, and the initial complexity of the system looks high because you need a decent number of components to do anything substantial. This really, really goes against the "move fast and break things" mainstream, even within open source - everyone wants their project to just _instantly_ accelerate from 0 to 100 in terms of progress, and we've put most of our efforts into a toolchain that makes it easier and easier to do that.

The best remedy at present seems to be to embrace asynchronity, embrace static types, and maintain faith in both - i.e. to have a lot of discipline.

Re: Programs should be Small

#68
post #67

Earlier quoted context omitted.

>So why is big-program development winning? There are a couple reasons for that. First, it gives managerial dinosaurs the illusion of control. So why do larger programs 'win' in the open source world as well[1]? Pop psychology about management doesn't seem sufficient to explain the phenomenon (although I'm sure it is a good way to sell a '101 habits of highly effective managers' book or get paid to give talks about m…

I think async architectures are worthy but miss something important - the idea of "typed connections" between processes. Actors passing arbitrary messages doesn't make for an efficient "assembly line." Similarly, the Unix-style concurrency is tied to a single low-level protocol, which is not rich enough to adequately describe all data. It described "enough" of all data for 1970's era tasks, but our needs outgrew that…

I regret that I have only one upvote to give this post.

Re: Programs should be Small

#69
post #66

Earlier quoted context omitted.

My current theory is that this is an arbitrary one. Its really, really, really easy to cross a megabyte with statically linked libraries.

I think the grandparent intended that the size measurement exclude statically linked libraries or assets, debug symbols, and compression technologies like UPX. It can sometimes be beneficial from a distribution/deployment standpoint to have everything in one self-contained file. But you can't conclude much about the code quality of e.g. a computer game engine based on how many megabytes of graphics, music and sound e…

The rule is not meant to be universal, and I don't say other people should adopt it, but I think it's suitable for the work that I am doing right now.

Constraints like this can really shape a piece of software, for better or for worse. My inspiration is having work with a really powerful firmware system that had a hardware constraint to fit on a 1MB flash chip, everything included, and was done so well that it looked easy. Give yourself unlimited space and it's much easier to end up with UEFI...

I suspect quite a few programs out there would have turned out better if their authors had picked a semi-arbitrary maximum value for lines of code / bytes of RAM / bytes of disk / etc.

The actual rule I'm using for now is: - 1 second compile excluding dependencies. - 1 minute compile including dependencies (excl C compiler). - 1 MB executable including everything except libc and base OS.

Re: Programs should be Small

#70
post #66

Earlier quoted context omitted.

My current theory is that this is an arbitrary one. Its really, really, really easy to cross a megabyte with statically linked libraries.

I think the grandparent intended that the size measurement exclude statically linked libraries or assets, debug symbols, and compression technologies like UPX. It can sometimes be beneficial from a distribution/deployment standpoint to have everything in one self-contained file. But you can't conclude much about the code quality of e.g. a computer game engine based on how many megabytes of graphics, music and sound e…

[deleted]
Post reply on HN