Live data from Hacker News

Organizing complexity is the most important skill in software development

johndcook.com

101–110 of 286 posts

Re: Organizing complexity is the most important skill in software development

#101
post #85

There is a term for it. It's called managing scope. I'm surprised the article didn't mention this.

No, that's a totally different thing. The scope (requirements) of the software might be huge but the software could be designed and implemented cleanly and with maintainability in mind. Yet the scope might be tiny and the implementation a mess. This article discusses the skill of taking a mess and turning it into something clean.

Re: Organizing complexity is the most important skill in software development

#102
Mirror since the site is currently down:

The most important skill in software development

Posted on 18 June 2015 by John Here’s an insightful paragraph from James Hague’s blog post Organization skills beat algorithmic wizardry:

When it comes to writing code, the number one most important skill is how to keep a tangle of features from collapsing under the weight of its own complexity. I’ve worked on large telecommunications systems, console games, blogging software, a bunch of personal tools, and very rarely is there some tricky data structure or algorithm that casts a looming shadow over everything else. But there’s always lots of state to keep track of, rearranging of values, handling special cases, and carefully working out how all the pieces of a system interact. To a great extent the act of coding is one of organization. Refactoring. Simplifying. Figuring out how to remove extraneous manipulations here and there.

Algorithmic wizardry is easier to teach and easier to blog about than organizational skill, so we teach and blog about it instead. A one-hour class, or a blog post, can showcase a clever algorithm. But how do you present a clever bit of organization? If you jump to the solution, it’s unimpressive. “Here’s something simple I came up with. It may not look like much, but trust me, it was really hard to realize this was all I needed to do.” Or worse, “Here’s a moderately complicated pile of code, but you should have seen how much more complicated it was before. At least now someone stands a shot of understanding it.” Ho hum. I guess you had to be there.

You can’t appreciate a feat of organization until you experience the disorganization. But it’s hard to have the patience to wrap your head around a disorganized mess that you don’t care about. Only if the disorganized mess is your responsibility, something that means more to you than a case study, can you wrap your head around it and appreciate improvements. This means that while you can learn algorithmic wizardry through homework assignments, you’re unlikely to learn organization skills unless you work on a large project you care about, most likely because you’re paid to care about it.

Re: Organizing complexity is the most important skill in software development

#103
Or in other words: code should be as simple as possible, but no simpler. I guess you could call it organization or readability or just good design. It requires deeply understanding what you're trying to accomplish and structuring your code to reflect that. I don't think there's any rote, step-by-step procedure that will get you there. Often it is a flash of creative insight that breaks the log-jam and reveals the hidden inner structure of the problem. Once that is revealed the code writes itself. In other words, good code should always make the problem that was solved look easy.

Re: Organizing complexity is the most important skill in software development

#104
post #23

Earlier quoted context omitted.

I find diplomacy a highly important skill as a developer as well. Strangely I feel I've become worse at it as the years go on.

One of the problems I hit was that I would get really grumpy when particular things were asked for. Once I learnt to think about why I was getting grumpy, and vocalize the actual problem it was easier. e.g. A client said to me recently could clicking the blog link open a new window? When I started programming I'd have said "Yes! I can do that!". Then after a few years I'd have got grumpy about it and said something p…

I agree, diplomacy is a critical skill. Grumpiness is basically anger, and all anger comes from pain. Programming is fun, but getting yelled because we missed a deadline or got some feature slightly wrong is painful. When Suzy asks for a change to the link, she's not trying to cause pain, but it's easy for all that previous pain to come channeling through and end up all over her. Road rage works the same way.

Re: Organizing complexity is the most important skill in software development

#105
post #49

the number one most important skill is how to keep a tangle of features from collapsing under the weight of its own complexity Agreed. very rarely is there some tricky data structure or algorithm that casts a looming shadow over everything else Agreed, BUT... In order to organize, sooner or later, you will have to get clever (with tricky data structures or algorithms). How I have always built something big and/or com…

I find that the older I grow as a programmer the less I enjoy cleverness. There's never a good enough reason to use clever algorithms over boring ones. In fact, most of the refactors I did over the past year was to 'undo' cleverness (thankfully those programmers are no longer working here!) In terms of performance you sometimes need to do less conventional things, but that's precisely where documenting WHY you're doi…

> The small bits of performance you're going to get isn't worth the maintenance hell it's going to cause.

I've always said, that's the difference between programming and developing.

Re: Organizing complexity is the most important skill in software development

#106
post #97

Earlier quoted context omitted.

The abstractions provided by Java and C++ aren't very good compared to those provided by, say, Haskell and Clojure. For the most part, design patterns exist merely to fix the lack of a simpler lambda construct in the language. They more often than not add complexity rather than remove it. The biggest balls of spaghetti code I've ever seen we're always OOP with design patterns. I've seen far more clean C codebases tha…

C++ has lambdas and the standard library is, although baroque, heavily functional and well thought out.

My bad, I should've specified legacy C++. You're right that with lambdas now C++ has gotten much better.

However I would argue against the STL being well thought out. Every single large-scale C++ project I've worked on threw it out the window before even staring out. Our current project already takes an hour and a half to compile without parallel builds and barely uses templates at all.

I still prefer plain old C to C++ most of the time.

Re: Organizing complexity is the most important skill in software development

#107

Earlier quoted context omitted.

How'd you do it if the keyboard wasn't working?

Our main product was developed on a keyboard that was missing a "w". It's not a big deal, though - you can use a for(;;) loop instead of a while loop without hurting performance. And it's easy to just avoid "w" words in the view templates by being clever with word choice. Of course, that's not true at all. But giving someone a defective-but-not-entirely-broken tool can be a clever way of seeing how they work around p…

The W button? That's interesting. It wasn't related to the Clinton transition out of the White House[0] was it?

[0] http://www.truthorfiction.com/trashingthewhitehouse/

Re: Organizing complexity is the most important skill in software development

#108
post #49

the number one most important skill is how to keep a tangle of features from collapsing under the weight of its own complexity Agreed. very rarely is there some tricky data structure or algorithm that casts a looming shadow over everything else Agreed, BUT... In order to organize, sooner or later, you will have to get clever (with tricky data structures or algorithms). How I have always built something big and/or com…

Still too many lines of code!

Being organised has nothing to do with fewer lines of code. A 10 line script can be disorganised and a million LoC app can be well organised. What makes code organised is things not being tangled up together - having well encapsulated logical modules that do one thing (or a set of related things), with a well designed API to get each bit to work with the other bits.

I've found that the more I write testable code the better organised my code gets. Automated testing forces you to design maintainable, organised code - because your test suite really won't cope with disorganised code.

Re: Organizing complexity is the most important skill in software development

#109
post #39

there is an excellent book by jon lakos called 'large scale c++ design' which treats organizational or physical design of large scale c++ projects (> 1giga loc). highly recommended.

I've considered this book a few times, but the following review always stopped me cold: http://www.amazon.com/review/R1ICFUDTH8VXQ5/ref=cm_cr_dp_tit...

Any thoughts on it?

Re: Organizing complexity is the most important skill in software development

#110
post #41

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

> Sadly there's (seemingly) no way to interview people for this ability I think there is. Show the candidate some bad code. Not WTF code, but something you wouldn't be proud of but is realistic. Bonus points if you're comfortable enough to pull this out of your own production system. Ask the candidate about adding some functionality, but be clear that cleanup and refactoring changes are fine (even encouraged). The co…

We did something similar at previous job. We handed the candidate some code that was intentionally bad, explained that it was so (so they wouldn't think we wrote like that every day), and then asked what they'd do to improve it. This let us find out a few things - whether they could read & understand someone else's code, whether they could make improvements to existing code, what levels of abstraction they were comfortable working at, and whether they had a strong enough personality to mix with everyone else.

The key was in how we presented it - we didn't want to come across as elitist jerks (but we probably did to some people), so we tried to soft-sell it to them and work it in gradually during the interview.

Post reply on HN