The Hard Thing About Software Development

Requirements gathering and scheduling and the ilk are but a part of computer science. They shouldn't be, it is a different discipline. This is what a systems analyst or a designer sighs be doing. Now maybe in a small company a programmer had to wear multiple hats, and it might be a good idea to learn some of that. But in specific it isn't a computer scientists job.

If you found the article interesting or insightful, I would encourage you strongly to take a look at the book "Peopleware" by Lister and DeMarco. The first edition was published in 1978; the edition I have is from the early 90s, and I think there are newer versions since (at the usual astronomical cost of books that get sold as required reading for college courses), but you can get whatever is cheap for a used copy. Most of it hasn't changed much.

That, ironically, is part of the authors' point: software engineering hasn't changed that much. They were saying that in the late 70s, looking back on the past two decades (all the way back to the late 50s, when software was typically written in assembler or machine language and had to be rewritten if you bought a new computer!), but it hasn't become much less true. The technology changes, sure, but the failure of software development projects has never been mostly the result of technology. It's always been on the human side -- failure to understand the requirements, failure to meet them, failure to estimate the effort appropriately, failure to work with the customer, failure of the customer to understand what they were paying for, etc. There's a long list of things that go wrong, and I suspect anyone who has been in software for a while has seen many of them.

It was eye-opening for me, the first time I read it, to realize that people had been dealing with the same issues I was dealing with, for longer than I'd been alive. (And a bit depressing, too, that we seemingly haven't gotten much better.) Languages and project-management methodologies come and go, and the tech skills and understanding are certainly necessary, but they are not sufficient. The business knowledge and human factors seem to be the difference, or at least the largest controllable variable that leads to a difference, in a successful or failed outcome.

That may be true, but if that is the case and you want to define a "computer scientist's" job that way, not many people are actually interested in hiring computer scientists.

What most firms are interested in are software engineers. The dearth of software engineers has led them to hire computer scientists (and, in fact, a rather large percentage of computer scientists that I know are quite happy for this).

But if you proceed down that road, I am not sure that what's left over, in terms of value-add, for the pure-play "computer scientist" who doesn't want to deal with requirements or schedule or that other messy stuff, really is. Once you have the problem suitably defined and the requirements nailed down... you can offshore the hell out of that job, or farm it out to some bidding-war site, and let people chisel each other out of a living wage while you laugh all the way to the bank.

I wouldn't want to hang my career on that. If you're willing to deal with the messy human-factors stuff, and sit in on the requirements-gathering meetings and deal with the client and work on setting the schedule and doing the estimation with the BD guys and all of that other shit ... you're never going to have to worry about some dude in India taking your job. That's not to say you don't need the technical skill. But if you're gunning for a job and it's between two people and one is a little better on the technical side but the other person can get involved in the process that much earlier, maybe iterate on the problem as part of requirements development (as many modern methodologies basically require)... I think they're going to do a lot better.

If you're a "throw it over the wall" coder, well, good luck with that. There are certainly jobs around for pure coders, but I tend to be suspicious of exactly how many (particularly if you want to make a top-end First World salary), and I suspect strongly that supply is going to outstrip demand for a generation or two.

FWIW, some of the most highly-compensated people I know are the ones who have some combination of really impressive technical skills but also have the business understanding and can participate in the process from kickoff, or close to it. It's also easier for them to negotiate since they have the option of going out on their own at pretty much any point, vs. a pure-play coder essentially depends on having someone around to feed them requirements and technical problems to solve.

The good thing is that I think a lot of people who think of themselves as pure coders probably have more business understanding (in some area) than they give themselves credit for, it's just a matter of developing that understanding, which can be hard in an organization that intentionally tries to keep its developers away from "the business".