Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Programs should be Small

mkhadikov.com

Post reply on HN
Print view View original

41–50 of 75 posts

Re: Programs should be Small

#41

Post by anon1385 »

michaelochurch wrote: post #36
anon1385 wrote: post #34

Earlier quoted context omitted.

>So why is big-program development winning? There are a couple reasons for that. First, it gives managerial dinosaurs the illusion of control. So why do larger programs 'win' in the open source world as well[1]? Pop psychology about management doesn't seem sufficient to explain the phenomenon (although I'm sure it is a good way to sell a '101 habits of highly effective managers' book or get paid to give talks about m…

On the anti-Java bias, I think the issue is that there's more than one Java culture. There's the horrid commodity developer culture, but that's not the language's fault. I'm actually a pretty big fan of static typing. You don't get static typing's main benefits in C++ or Java, though. You have to use a language like Haskell or Ocaml, or the right subset of Scala, to see the major benefits of that. Open-source is a bi…

I should mention that I've never worked in the corporate world, so my reaction is in that context. I can't talk about the corporate side of things since I've never experienced it.

Also yes static typing in Java does look pretty cumbersome. Personally I'm hoping that Rust takes off, I've enjoyed playing about with it over the last couple of weeks, although it has made me less happy using the more dynamic languages I normally use to do real work.

Re: Programs should be Small

#42

Post by michaelochurch »

NateDad wrote: post #35
michaelochurch wrote: post #30

Earlier quoted context omitted.

The problem with corporate, big-program development is that it's a premature abstraction. If the system-of-small-programs doesn't perform, then you're in a state where larger programs might make sense. If the problem is well-understood and the pieces have been built and refined by competent programmers, but it's impossible to go any further without some coupling and integration, then a large program isn't the worst t…

Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance. For example, the program I work on has to support a million row database that can be sorted and filtered both on the server and client with subsecond response time. The program is incredibly configurable based on data in the system, so many of the features depend on…

Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance.

Right, but there's a different process to it.

Databases solved a problem, and the requirements grew organically as people used them to solve harder problems. With product companies or with open-source software, the project owners can say, "We aren't doing that shit".

Enterprise projects accumulate requirements based on who has power within the organization. Each person who has the power to stop the project asks for a hand-out, and "We aren't doing that shit" isn't an option. It's like how businesses that want to operate in corrupt companies need to have a separate "bribe fund" for local officials. Over time, the result is an incoherent mess of requirements that make no sense together.

The requirement list for a typical enterprise project is the bribe trail.

However, that's not to say that the idea of separation of concerns still can't be applied to large program. And in fact, most enterprise devs do exactly that. That's what all these "services" are in the program. Except that instead of having to serialize data, I can just pass them a pointer.

Sure, but when you have a multi-developer project without an explicit API, what you end up is an undocumented and implicit API between peoples' code. This devolves into the software-as-spec situation where it's not clear what the rules are.

I think it's better to start with the inefficient service-oriented program, get that working, and then optimize with the merged, larger program if needed (and to document the API that has now become an implicit within-program beast).

Re: Programs should be Small

#43

Post by meaty »

michaelochurch wrote: post #42
NateDad wrote: post #35

Earlier quoted context omitted.

Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance. For example, the program I work on has to support a million row database that can be sorted and filtered both on the server and client with subsecond response time. The program is incredibly configurable based on data in the system, so many of the features depend on…

Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance. Right, but there's a different process to it. Databases solved a problem, and the requirements grew organically as people used them to solve harder problems. With product companies or with open-source software, the project owners can say, "We aren't doing that shit".…

> The requirement list for a typical enterprise project is the bribe trail.

I think this is purely a stereotype.

The behavior experienced is largely down to the fact that a large body of humans can't come up with a single consistent view of a large set of problems. You need singular control and ownership by someone with technical and business domain expertise. Some of this is politics (particularly from the MBA and psychotic corporate climber faction) but it's at least 80% standard human idiocy and ignorance.

I think from an architecture perspective (I'm an "enterprise architect" [whatever that is] by trade), clean service APIs are a good idea, but not necessarily the distribution model or fully decoupled integration path.

Re: Programs should be Small

#44

Post by meaty »

sorbits wrote: post #39
meaty wrote: post #20

I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…

Adding to that, his stated advantage of not having to limit yourself to one platform also seems opposite of my experience (i.e. keeping all on the same platform is an advantage). When you have a big system with disjoint parts written in different languages, re-use and refactoring is a pain, and redundancy is almost certain to creep in (and with redundancy often comes inconsistency).

Yes heterogeneous systems are much easier to deal with, although from experience certain systems are a pig to deal with from end to end (anything Microsoft as a rule).

Different languages are just different forms of integration and the mantra of integration is hell should be in the forefront of everyone's mind, always.

Re: Programs should be Small

#45

Post by bloaf »

NateDad wrote: post #35
michaelochurch wrote: post #30

Earlier quoted context omitted.

The problem with corporate, big-program development is that it's a premature abstraction. If the system-of-small-programs doesn't perform, then you're in a state where larger programs might make sense. If the problem is well-understood and the pieces have been built and refined by competent programmers, but it's impossible to go any further without some coupling and integration, then a large program isn't the worst t…

Most enterprise systems start off with requirements similar to those you think of with a database - a lot of data with high expectations of performance. For example, the program I work on has to support a million row database that can be sorted and filtered both on the server and client with subsecond response time. The program is incredibly configurable based on data in the system, so many of the features depend on…

I'm going to point to a development case which is outside the typical "database management" case everyone here seems to be thinking about: engineering modeling software like this: http://www.aspentech.com/fullproductlisting/

This company is ostensibly doing the right thing: they have developed a large number of "single purpose" programs. They also have some applications which attempt to integrate some of their technology into single packages. The problems, however, are exactly as you describe. From an end user perspective, having the various programs send data to one another is a crap shoot. Some applications are very tightly integrated while others seem to have been developed in a vacuum. The company has even developed an entire application that tries to fix this by allowing data to be automatically exported to and imported from excel. End users could try to use the COM interface to get and send data where they want it to go, but we have to remember that the target audience is engineers, not programmers.

Re: Programs should be Small

#46

Post by bitwize »

mimog wrote: post #37

Isn't this "the Unix way"? I have seen this style of small cohesive programs promoted a lot in Linux/Unix literature so the advise isn't really breaking any new ground. Look at git for instance. The advise is of course sound but it goes against the "enterprisy" way of doing things, in part because they tend to be using huge frameworks from the get go.

It used to be. It may still be if you hang out on #suckless. But most of the Linux folks these days love their huge monolithic PulseAudios and systemds.

Re: Programs should be Small

#47

Post by IvyMike »

mimog wrote: post #37

Isn't this "the Unix way"? I have seen this style of small cohesive programs promoted a lot in Linux/Unix literature so the advise isn't really breaking any new ground. Look at git for instance. The advise is of course sound but it goes against the "enterprisy" way of doing things, in part because they tend to be using huge frameworks from the get go.

On the other hand, think back to the famous macro- v. micro-kernel debate. The kernel itself is huge, and the "small cohesive service" philosophy microkernel advocates ended up never really taking over the world.

Re: Programs should be Small

#48

Post by gruseom »

meaty wrote: post #20

I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…

[Edit: deleted pointless bit here.]

If your comment consisted simply of its second and fourth paragraphs it would be better in every way and you would have contributed something of value.

Re: Programs should be Small

#49

Post by epscylonb »

meaty wrote: post #20

I love all these articles which are wholly ignorant of how complex software is in reality and how such advice isn't necessarily good. Sometimes decomposition results in problems at the other end of the scale such as communication performance, data duplication, extremely nested abstractions, messaging complexity, contract and API versioning hell etc. Getting the sweet spot between monolithic coupled blobs and fragment…

I'm pretty much convinced that the big problem today in software is deciding where your APIs should go.

My instincts are similar to the articles author, a preference for small discrete pieces software rather than a giant monolithic application. More Web Services!, if you will.

But you are correct, getting this sweet spot is hard. Truth be told, I am not even sure experience guarantees a successful design first time around.

There is a tendency with developers to want to keep everything nice and clean. For example app A is responsible for a data set, anytime other apps want to access it they have to talk to A, if they are asking for that data a lot you might be better off caching or periodically copying the data over to the parts of the system. I always try to decide whether to segment something by thinking about how many calls it is likely to receive as a web service, more than a couple in short succession and I start having doubts.

Ultimately what makes the entire system work best for users is the correct thing to do and sometimes it is very difficult to come up with something which does that and is pleasing to the discerning coders eye.

Re: Programs should be Small

#50

Post by owyn »

moe wrote: post #4

A good program is 500 lines or less.

I don't think that's true, even if you look at most open source software. On one extreme, Mediawiki is a pretty complicated piece of software, approximately 900k lines of "core" PHP/JS code plus another 1.5 million lines worth of extensions, many of which are pretty mandatory for basic functionality.

Is it "good" or "bad"? I'd argue that it's both. It's an important piece of software but a bad architecture, and it's grown in complexity over the years. I think you could write a piece of functional wiki software in about 500 lines of code, but complexity and features win in the marketplace, even if it's just for "mindshare".

Post reply on HN