Why Programming is Difficult (2014)

Yeah, I think the sheer manual friction of moving things around is a big impediment to change.

But also, I think there has to be some conceptual basis for us to be able to think about factoring in a clear way, and that kind of intellectual labor is hard on another level.

Eric Evans' book on domain-driven design has some examples of programming tasks that generate confusing and unclear code, until the coder together with the "domain expert" reaches some kind of epiphany—like "ohh, if we think of these three things as being parts of the same kind of thing, then this part of the program makes more sense."

A program is a kind of philosophical theory about a certain topic, and if the theory isn't very clear, then you'll end up with a lot of special cases and obscureness even if you have unit tests and refactoring tools. They say naming is the hardest thing in programming and that becomes more true when you start to introduce any level of abstraction.

"Design patterns" were an attempt to create more general small theoretical pieces, above the syntactic level. In the FP world, a similar vision is that algebraic and categorical abstractions can provide some of those pieces. We still haven't seen much of what algebraic abstractions can do for ordinary programming...

But these abstract pieces, I think, are pretty crucial if we want programs to be understandable as anything but arbitrary collections of procedures and types. So yes to refactoring tools, and yes to more discussion about design patterns—it didn't end with the Gang of Four! read Christopher Alexander and Richard P. Gabriel!—and yes to more inspiration from mathematics and engineering and philosophy and all kinds of stuff, to provide inspiration for factoring—yes to study of logic and language and ... I'm getting carried away.

Too many sentences like, "you where thing when designed this thing to the next guy/gal" make reading hard. Slow down dude, wtf?

There is another kind of "hard" though, which involves managing growing code bases, doing programming within a business setting, and so on. We mostly let the scientists work on well-defined problems in isolation, so they can really focus on those deeply intellectually challenging things, but even if those scientists are very clever I don't think they are necessarily good at larger-scale system development.

There are lots of jobs that are harder than mine in some sense, but if you look at the objective of building high-quality complex software in the minimum amount of time, keeping it running correctly, and then being able to change that software as effectively as possible, then that in itself is pretty much an infinite problem. Maybe there is lots of scientific work to be done to help us struggling coders, and maybe we need to get our shit together more as a profession... It's not all easy.

how big was the largest project you've ever worked on?

I am thankful that at my current job, we do get some time to experiment here and there.

Begin tangent:

I work for a university doing grant projects for a few state and federal agencies, though. Some might like it, some might consider it hell for an entrepreneur. It works for me :-)

We are in the process of re-writing our "flagship" app from 10+ year old java libraries. We were able to dodge the bullet of "upgrading" (???) from Struts to JSF. The new app is going to be a services based (REST + JSON) Java (mostly) back end, with an Angular based front end. We just did a demo of a subset of the new app at a trade show where a 3rd party app imports a partial record (via web service) into our app, then launches a nested browser window to run the rest of the relevant data entry. (the rest of the workflow will be handled by the legacy app until the complete new version is finished)

No, I couldn't persuade my coworkers and supervisor to discard Java entirely and go with Node.js :-) I'm not entirely sure I would be comfortable with that, either. We are doing some server side Javascript lately, though, by using the Java scripting engine interface.

We will likely soon be redoing another (30 or so year old) app from a sister university that the state uses, as well. Interesting times!

For almost 5 years before this, I worked at a financial services company. I got paid a bit more (bonuses), but I spent all my time, really, keeping 2 legacy applications going without ever getting to spend much time REALLY updating some sore spots.