Live data from Hacker News

Organizing complexity is the most important skill in software development

johndcook.com

221–230 of 286 posts

Re: Organizing complexity is the most important skill in software development

#221

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

You could give them some mild spaghetti, and ask how they'd try to improve the code.

I think one reason why what you did doesn't happen more often, though, is because they wouldn't have the freedom to reimplement in a language they felt was more suitable.

Re: Organizing complexity is the most important skill in software development

#222
post #43

Earlier quoted context omitted.

This is pretty much how I feel just now (after 12 years). I know Django pretty well after 4 years, and know where and how to add things to avoid breaking other parts of the code and make it work consistently with other parts. Yet nearly every job I apply for these days wants a 90 minute online test. (I just had to implement a sorting algorithm in psuedo code - I have never in 12 years had to implement a sorting algor…

>> Yet nearly every job I apply for these days wants a 90 minute online test. (I just had to implement a sorting algorithm in psuedo code - I have never in 12 years had to implement a sorting algorithm as I choose an appropriate library to do that for me). True, but if you can't implement a simple sorting algorithm how are you going to implement _____ ?

Since the process of implementing _____ is almost certainly completely orthogonal to regurgitating a memorized algorithm one would never directly write themselves (to a fantastically close approximation of never) I'm not sure how that question has any validity at all.

Re: Organizing complexity is the most important skill in software development

#223

On the topic of organization and related to another post about good Software Development books. What are some books that teach code organization as discussed in this post. One I can think of is "Refactoring" by Martin Fowler. What are some others?

Dealing effectively with legacy code:

http://www.amazon.com/Working-Effectively-Legacy-Michael-Fea...

Clean code:

http://www.amazon.com/Clean-Code-Handbook-Software-Craftsman...

Re: Organizing complexity is the most important skill in software development

#224

Earlier quoted context omitted.

How'd you do it if the keyboard wasn't working?

Our main product was developed on a keyboard that was missing a "w". It's not a big deal, though - you can use a for(;;) loop instead of a while loop without hurting performance. And it's easy to just avoid "w" words in the view templates by being clever with word choice. Of course, that's not true at all. But giving someone a defective-but-not-entirely-broken tool can be a clever way of seeing how they work around p…

I dunno. If you're not going to care enough about your employees to give them the best possible tools, why do I want to work there again?

Re: Organizing complexity is the most important skill in software development

#225

Important according to what metric? Making the software developer feel good, or making the company money? The former is almost certainly true, the latter is almost certainly not.

Avoiding rework can cut costs. Rapid Development by Steve McConnell explains this well.

Re: Organizing complexity is the most important skill in software development

#226
Out of all the things I know how to do in programming, reducing complexity is probably the one I'm best at. So how do I get a job doing this? Or a series of lucrative consulting gigs? :-)

I'm pretty sure I'm not as smart as I used to be, and I'm definitely not as smart or productive as some of the younger programmers I've worked with. (Sorry for the ageist remark!)

This may be my secret advantage: I have to keep my code simple enough that even I can understand it.

Here's a fun example that I've seen more than a few times in various forms: four-way navigation, either involving up/down/left/right or north/south/east/west, or both.

In one (somewhat disguised) project it worked like this: the code had several different modules to provide a keyboard interface for geographic navigation, while keeping the geo code separated from the low level details of key codes and events and such.

There was a keyboard manager that mapped keycodes to readable names that were defined in an enum:

  switch( keyCode ) {
      case 37:
          return KEY_LEFT;
      case 38:
          return KEY_UP;
      case 39:
          return KEY_RIGHT;
      case 40:
          return KEY_DOWN;
  }
Then an event manager broadcast navigation messages based on the KEY_xxxx codes:

  switch( keyEnum ) {
      case KEY_LEFT:
          BroadcastMessage( 'keyLeft' );
      case KEY_RIGHT:
          BroadcastMessage( 'keyRight' );
      case KEY_UP:
          BroadcastMessage( 'keyUp' );
      case KEY_DOWN:
          BroadcastMessage( 'keyDown' );
  }
A navigation manager received these messages and called individual navigation functions:

  // Don't forget to reverse the directions here
  events.on( 'keyLeft', function() {
      moveRight();
  });
  events.on( 'keyRight', function() {
      moveLeft();
  });
  events.on( 'keyUp', function() {
      moveDown();
  });
  events.on( 'keyDown', function() {
      moveUp();
  });
These navigation functions panned a map in one compass direction or another:

  function moveUp() {
      map.pan( maps.DIRECTION_NORTH );
  }
  function moveDown() {
      map.pan( maps.DIRECTION_SOUTH );
  }
  function moveLeft() {
      map.pan( maps.DIRECTION_WEST );
  }
  function moveRight() {
      map.pan( maps.DIRECTION_EAST );
  }
Of course most of you reading this can see the problem at a glance: Besides having so many layers of code, how many different names can we give to the same concept? We've got KEY_LEFT, keyLeft, moveLeft, and DIRECTION_WEST that all mean pretty much the same thing!

Imagine if math worked like this: You'd have to have two of every function, one for the positive numbers and another one for negative numbers. And probably four different functions if you are dealing with a complex number!

That of course suggests a solution: use numbers instead of names, +1 for up and -1 for down, ditto for right and left. And pass these numbers on through any of these layers of code so you only need half the functions. If you need to flip directions along the way (like the left arrow key navigating right), just multiply by -1 to reverse it instead of having to make special cases for each direction name.

You might even decide to combine the two axes, so instead of vertical and horizontal, you've got +1 and -1 there too (or 1 and 0, or something that lets you handle both axes with one piece of code). Now you could be down to a quarter of the original code.

Unfortunately, I was brought in on this project near the end to help wrap up a few other tricky problems, and all this navigation code was already set in stone. (And to be fair, working and tested, and who wants to go back and rewrite proven code, even if it is four times the code you need?)

But this would make a pretty good "how would you clean this code up" interview question!

Re: Organizing complexity is the most important skill in software development

#227
“Here’s something simple I came up with. It may not look like much, but trust me, it was really hard to realize this was all I needed to do.”

This reminds me of something Scott Shenker, my computer networking professor at Berkeley, drilled into us every chance he got: Don't manage complexity. Extract simplicity.

Finding complex solutions to complex problems is comparatively easy. Finding simple solutions to complex problems is hard.

Re: Organizing complexity is the most important skill in software development

#228
I'd say it's not really about "organizing complexity" - because that tends to just push it around somewhere else - but reducing complexity which is most important.

In my experience it has been that designs which look "locally simple" because they have such a high level of abstraction are actually the most complex overall. Such simplicity is deceptive. I think it's this deceptive simplicity which causes people to write a dozen classes with 2-3 methods of 2-3 lines each to do something that should only take a dozen lines of code (this is not that extreme - I've seen and rewritten such things before.)

Perhaps we should be focusing on teaching the techniques for reducing complexity more than hiding it, and that abstraction is more of a necessary evil than something to be applied liberally. From the beginning, programmers should be exposed to simple solutions so they can develop a good estimate of how much code it really takes to solve a problem, as seeing massively overcomplex solutions tends to distort their perspective on this; at the least, if more of them would be asking things like "why do I need to write all this code just to print 'Hello world', and the binary require over a million bytes of memory to run? Isn't that a bit too much?", that would be a good start.

Re: Organizing complexity is the most important skill in software development

#229

Earlier quoted context omitted.

> Do you really want to work in a company that can't conduct a proper interview and has broken/slow hardware? Slow is relative. The employee in question was trying to figure out why a remote server was experiencing extreme slowdown. He ssh-ed in, but he was able to type far faster than the beleaguered remote could echo his keystrokes. So he needed to just carefully type his commands, wait for them to appear, and then…

Yeah but poor guy - in an interview situation the pressure is different and public. He might have done fine at his own desk.

It wasn't during the interview. According to knodi123's first comment it was during the guy's first week on the job.

>Joking aside, we once let a guy go during his 1 week probationary period because he got a little too angry at a slow server.

Re: Organizing complexity is the most important skill in software development

#230

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

> Sadly there's (seemingly) no way to interview people for this ability I had one interview where the interviewer simply showed me a printed page of code, and asked me to talk about it. I spotted a few problems and debatable constructs, and we had a good 3 minutes discussion. Since at least I spend more time reading code than writing it, that felt like a truer test of my abilities than many of the coding tasks. PS. Y…

Agreed, being able to read and reconstruct models from OPC (other peoples code) is such a big piece of coding but hardly ever a part of the interview process.

(OPC also refers to one's own code that when you can't remember wtf you did and the comments aren't useful)

Post reply on HN