Clojure 1.7 is now available

A language will often become used in one particular domain, often not the domain it was designed for. Take another JVM language Groovy -- it's used a lot for scripting throwaways used in testing, including build files for Gradle, and was extended with a meta-object protocol for Grails which has risen and fallen in sync with Rails. But then its backers retrofitted it with a static typing system and have tried to promote it for building systems, both on the JVM and Android but virtually no-one's biting.

Well, those are all tradeoffs, and the fact is that since the addition of annotations and later lambdas, all those "external XML-based configuration files" are receding, to the point they no longer exist in almost any of the newer libraries (or new versions of old libraries).

Java didn't start out with insufficient abstractions that were later added by other languages. Java started out as a reaction to languages with overly-powerful abstractions that hindered maintenance. You may not like the result and think it aesthetically unpleasing (although it's been getting better and better for quite a while now), but it is a fact that Java codebases are extremely maintainable. This is not a guess or a gut feeling. Those legacy Java codebases exist as a living proof of that. Other languages legacy code was either thrown away or frozen, unmaintained. And if you think "good, codebases shouldn't live for too long", well, that's a nice sentiment, but the fact is that long lived codebases save the industry a lot of time and money (even if young developers think they could have done it better if they'd just started from scratch). Again, we know that because we've seen the alternative.

BASIC and COBOL were never nearly as maintainable as Java (I know becas. I love Lisp (Scheme was among my first languages, and Clojure is my favorite application-programming language), but the point of a language designed for the industry shouldn't be expressiveness or power, nor readability, but usefulness and maintainability. Any other property should serve that. Professional software is written to serve a purpose -- they're not pieces of art (or not just pieces of art). You must remember that the average lifespan of a codebase is about a decade, and the cost of the project is spread -- unevenly -- over that decade. A language for the industry -- as well as other related tools -- is meant to reduce that cost.

I think Java does an excellent job of that, and I believe Clojure can do an excellent job -- we just don't have the data yet. But if you design a language for the industry, you must always look at the big picture -- at those ten years of the codebase as a goal -- that's the challenge. Coming up with a language that's powerful and expressive is easy. Doing that in a way that really serves the industry's need is much harder. Rich Hickey is a pragmatist, and he gets that. I think that the emphasis on transducers was a stumble, because he may have lost sight of the real goal.

But you are certainly right that Java was also meant to be familiar and unthreatening. It allowed itself to do that because it realized -- and was proven right -- that extra-linguistic features such as a GC and dynamic linking are essentially the features that contribute to productivity more than abstractions.

> Software companies can scale either horizontally (e.g. hiring more people) or vertically (hiring the best).

While this may be true, it has little to do with Java directly. For years I worked with the best algorithmeticians, physicists and mathematicians (physics simulation, real-time control). We were very much involved with the hard problems we were trying to solve with novel algorithms, and frankly couldn't be bothered with spending precious resources on elegant abstractions. Our problems was one of essential complexity -- not accidental complexity -- so we needed a simple language that can be easily shared and understood by everyone, one that was fast, and one that got the job done.

Believing that better developers opt for more powerful languages is what I call the "Blub developer paradox": the blub developer has only worked on simple, blub, problems (CRUD applications, web etc.), and hence believes his cool abstractions are the mark of the "smart developer". If he'd only look at developers working on seriously hard algorithms, he'd realize that abstractions are secondary, and it is developers above him on the essential complexity spectrum that opt for simpler languages with weaker abstractions.

That better developers choose the more powerfully-abstracting languages is completely and utterly false. The best developers I've ever known -- those who really came up with radical, game-changing solutions that truly advanced the state-of-the-art -- were all C and Java people, and I don't think it is a coincidence. Their mind is focused on the algorithm rather than on alluring, but ultimately distracting abstractions.

However, saying that only people who work on boring problems have the free mental resources to spend on nifty abstractions is as valid a generalization as the one you've made. There are many reasons to choose a particular programming language.

Personally, though, I believe that while the choice of the programming language matters, it matter so much less than design, choice of algorithms and availability of other tools. I no longer equate software with the code it's written in. A running program has many factors affecting its quality and cost, and the code used to write it is just one of them. Still, if I need to write a web app, I'd choose Clojure over Java every time. If I need to write a military command-and-control app, or a power-station control app? I'd probably go with Java.

This is wrong. You've missed the point.

    (map inc [1 2]) 

    (reduce ((map inc) conj) [] [1 2])

    (sequence (map inc) [1 2])

However, I am saying that the assumption that better developers invariably prefer more abstractions and more powerful languages is absolutely wrong.

Personally, I've had a terrible experience with Scala, found its abstractions distracting, its complexity hindering readability, its DSLism obstructing code sharing and staff migration among various teams, and the grapple with the tool stack a horrible waste of time -- all with little benefit and lots of harm. I know that my experience isn't universal, but it's far from unique, either.

Clojure is a completely different story, partly because its designers are not purists and not interested (at least not as a primary concern) in PL research, but even with Clojure I don't think we have enough data to conclude that its cleverness (which, thankfully, is much lower than Scala's[1]) is a net gain for large projects that require maintenance for a decade or more by large teams.

All in all, I've seen that a solid, fast runtime with a good compilation and linking story, excellent profilers and deep monitoring (all provided by the JVM) yield much bigger gains than most new languages. Given that those extra-linguistic features are free and switching a language is extremely costly, I'm always hesitant about whether it's worth it. Staying on the JVM reduces the switching costs, and good interoperation with Java -- like Clojure has -- reduces it further, so I find the risk to be worth it for some less-risky projects.

[1]: Then again, so is every other language in history.