Empirical Analysis of Programming Language Adoption [pdf] (2013)

Just to reiterate: I think the current situation is perfectly understandable. Obviously the ecosystem does make a huge difference to how productive someone can be in any given programming language, and the choices of language that projects are making are perfectly rational given their constraints and objectives.

It's just unfortunate that this creates momentum behind languages like C, PHP and JavaScript, which then concentrates the effort of library development within their ecosystems, creating a vicious cycle. In terms of robustness and productivity, the industry would be better served by moving almost all development work away from that sort of underpowered, poorly specified, error-prone language as soon as possible, but you can't do that without the libraries. Developing that ecosystem for any new language takes huge amounts of time and effort, so to have any realistic chance of catching on, you first need an excellent bridge to existing libraries. Sadly, the FFI infrastructure even in relatively good languages tends to be clunky at best, and that is a huge barrier to adoption for all the reasons you mentioned.

Starting off from first principles can be quite impractical these days, so stitching together others' work is becoming a common paradigm.

On the other hand, the trade-off for that reduction in NIH early development effort is that you now have a discovery problem up-front and a dependency management problem indefinitely. I don't think that as an industry we've really found a good balance yet between not reinventing the wheel and not spending forever choosing which of 57 pre-existing wheels to use.

What I'd love to see is a new generation of languages that accept the modern reality of combining packages from diverse sources and so place much greater emphasis on supporting that kind of software architecture. Interoperability between packages, each with their own related but probably slightly different concepts and conventions, needs to be much more of a priority IMHO. When we can compose data structures and algorithms across packages from different sources, and have all the marshalling and type safety and error handling mechanisms just work with minimal programmer intervention, then we'll be getting somewhere. But of course that's just one more substantial issue that any new language has to address on top of everything else, which is no small thing to ask.

I agree that the ridiculous dependency chains and brittle build tooling/bootstrapping are getting out of hand in modern development. Go actually gets this thing right, but is bare bones in every other regard (which I also appreciate, I enjoy Rob Pike's work).

I can't really think of any solution to this dilemma, though. Fundamentally, it's not just limitations of languages, but of operating systems, as well. Having a file server-centric OS design instead of a daemon-centric one for providing services alone will simplify a whole lot of things that we're used to abstracting in libraries.

The song sung by old engineers who refuse to learn new technology, so they try to make knowing few a positive (it isn't). The classic "It's not a bug, it's a feature!"

I don't think the core of a good developer has changed much since the early 90s. You still want people whose skill-set is T-shaped... You want them to go deep on at least a few domains, but you want them to have broad experience in lots of them. The broad experience makes for developers who are better than the sum of their parts, can adopt solutions from other domains and can communicate better.

I generally won't consider too seriously a "senior" engineer who has only a single "class" of knowledge. I realize that sometimes they have limited control over this (the jobs they happen to get, yada), but I just want the best developers... and in my experience they find a way to follow Norvig's advice and at least toy with some of the major flavors: DBC, functional, class, syntax, declarative, concurrent, parallel...

You don't necessarily have to reimplement every library, as long as you can use them easily from another language. Why would you want to do that, anyway? Because the other language is much better than what you already have.

Personally, I only work on one project right now that still uses Java, and it uses it as an applet on a web page, not in a server/enterprise role. The effort required just to keep basic functionality working in that environment is absurd these days, since it seems every major browser developer and Oracle themselves are doing their best to make the platform unviable as quickly as possible. But even if they weren't, this is a project that uses several very different languages for different components in the overall system, and Java is so underpowered compared to every other language we use that I won't be at all sorry to see it go.

The more I think about it, the more Java seems like a perfect example of what I was talking about before: a weak language that is now chosen primarily for its established supporting ecosystem and not because of any great technical merit.

I think Michael's point was that a senior developer might have a very different idea of what constitutes "deep" to a junior developer.

By the standards of a 20 year industry veteran, a six month project they worked on three years ago using language X might not even make X worth mentioning unless it was specifically asked for, next to the half-dozen languages they already show where they have 5+ years of experience including shipping multiple projects in each one.

By the standards of a junior 2 years out of school, a six month project working in language X last year might make it one of their top two languages and get presented as "expert-level" understanding. These are the people who tend to list fifteen programming languages, because they used each of those languages for about five minutes on a toy project during their CS course.