Live data from Hacker News

Organizing complexity is the most important skill in software development

johndcook.com

51–60 of 286 posts

Re: Organizing complexity is the most important skill in software development

#51
post #29

Organisational skill is actually an entrepreneurial skill. I'm learning the hard way that there are two diametrically opposing skills you need to master for both running a business and programming effectively: 1. Skills in Creating 2. Skills in Organising those creations The thing is, the Creating part is always exciting but it's disruptive in nature. The Organising part is boring because it's about taming or temperi…

This is something that I'm learning too, and I'm glad to hear it expressed as simply and clearly as you did, thanks.

We learn it when we're kids : "Clean your room (2) before playing with toys (1)".

Such a basic concept, so hard to scale.

Re: Organizing complexity is the most important skill in software development

#52

On the topic of organization and related to another post about good Software Development books. What are some books that teach code organization as discussed in this post. One I can think of is "Refactoring" by Martin Fowler. What are some others?

I would submit the main point is: You can't learn that from a book but only from experience. The best balance isn't cast in stone but must be found for each problem domain. As much as you can, do your own experiments and experience the consequences of your decisions yourself. Remember that for nontrivial problems, a usual part of the job is learning exactly what problem it is that you have to solve.

Re: Organizing complexity is the most important skill in software development

#53
post #49

the number one most important skill is how to keep a tangle of features from collapsing under the weight of its own complexity Agreed. very rarely is there some tricky data structure or algorithm that casts a looming shadow over everything else Agreed, BUT... In order to organize, sooner or later, you will have to get clever (with tricky data structures or algorithms). How I have always built something big and/or com…

From [1]:

> Nobody is really smart enough to program computers. Fully understanding an average program requires an almost limitless capacity to absorb details and an equal capacity to comprehend them all at the same time. The way you focus your intelligence is more important than how much intelligence you have.

> At the 1972 Turing Award lecture, Edsger Dijkstra delivered a paper titled "The Humble Programmer." He argued that most of programming is an attempt to compensate for the strictly limited size of our skulls. The people who are best at programming are the people who realize how small their brains are. They are humble. The people who are the worst at programming are the people who refuse to accept the fact that their brains aren't equal to the task. Their egos keep them from being great programmers. The more you learn to compensate for your small brain, the better a programmer you'll be. The more humble you are, the faster you'll improve.

[1] http://blog.codinghorror.com/why-im-the-best-programmer-in-t...

Re: Organizing complexity is the most important skill in software development

#54
post #31
post #18

Earlier quoted context omitted.

"Tell me about writing a compiler" There's no way to interview for what you're looking for, but compilers seem to require both great organizational skills and some algorithmic chops. there's enough complexity you can't hide lack of one side or the other. Anyway, asking about building and enhancing a compiler seems like one of the ways to get insight about how people grow code. Every program is a seed. Is this the kin…

In typical software today, much of the complexity arises from interactivity. There are user actions, network events and all sorts of contingencies that make the program flow non-deterministic. A compiler doesn't have that. It's blissfully old-fashioned, really: read files in, write files out -- no interruptions or real-time requirements. You could do compilers on punch cards. In a way, the compiler is the most comple…

Compilers are a nice example because they require a lot of code, and all the code has to be right. If you can't impose enough organization there, you don't have a chance at getting a distributed event driven system to work.

Re: Organizing complexity is the most important skill in software development

#55
post #41

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

> Sadly there's (seemingly) no way to interview people for this ability I think there is. Show the candidate some bad code. Not WTF code, but something you wouldn't be proud of but is realistic. Bonus points if you're comfortable enough to pull this out of your own production system. Ask the candidate about adding some functionality, but be clear that cleanup and refactoring changes are fine (even encouraged). The co…

This happened to me once. I went for an an interview at a huge real estate company. They had a chat with me for exactly five minutes, then gave me a computer (an old one, with keyboard not working properly) and some legacy code. I wasn't asked to fix it, but I was asked to make a small change and given half hour for it. That code had no documentation and no comments. Then 5 more mins to explain what I understood about the codebase - no questions, just me talking about the codebase as much as I understood in that half hour. The whole thing took like 45 mins or so in total. No whiteboard questions etc. And they made an offer.

Re: Organizing complexity is the most important skill in software development

#56

Arguably, this is what higher level languages like Java and C++ provide. Tight organizational language metaphors that help implement design patterns in a thoughtful, consistently structured manner.

The languages provide the mechanism.... The developer provides the skill needed to produce the organisation.

Re: Organizing complexity is the most important skill in software development

#57
post #29

Organisational skill is actually an entrepreneurial skill. I'm learning the hard way that there are two diametrically opposing skills you need to master for both running a business and programming effectively: 1. Skills in Creating 2. Skills in Organising those creations The thing is, the Creating part is always exciting but it's disruptive in nature. The Organising part is boring because it's about taming or temperi…

This is why most of my personal toy projects are coded like a 12 years old with giant functions, god objects everywhere and horrible, horrible, one liners. It's liberating.

This is also why I like game programming that much. A big chunk of the codebase is throwaway code that won't be transported to the next project. When the final deadline comes, you don't feel bad making hundreds of hacks everywhere, and it's fun.

Re: Organizing complexity is the most important skill in software development

#58

Design patterns by the Gang of Four was great for this.

Er, isn't design patterns (if you take the buzzwordery away) just a map from “conventional names” to “standard algorithms”?

This article isn't about the choice of an algorithm, it's about organisational aspects. That's something different altogether, and an even larger topic.

Re: Organizing complexity is the most important skill in software development

#59
post #34
post #18

Earlier quoted context omitted.

"Tell me about writing a compiler" There's no way to interview for what you're looking for, but compilers seem to require both great organizational skills and some algorithmic chops. there's enough complexity you can't hide lack of one side or the other. Anyway, asking about building and enhancing a compiler seems like one of the ways to get insight about how people grow code. Every program is a seed. Is this the kin…

Compilers are simple. They are organized as a pipeline. Frontend, middleend, backend. In more detail: lexer, parser, semantic analysis, optimzation 1 to n, instruction selection, scheduling, register allocation, assembly. Ok, you can do more complex. I work on a compiler, which does it lazily. Transformations are per file and depend on each other. If something goes wrong, you print out the graph to figure out the pro…

Yes, but there are a bunch of tradeoffs. do you try to lex the floating point format, or push that back to the parser? Do in introduce an extra pass for constant folding or do you try to shoehorn it into AST generation in the parser?

The nice thing about compilers, it requires a ton of code.

It's the kind of thing that any programmer can do (well good ones), and there are organizational tradeoffs to be made. There is no "right" answer, it's an opportunity to talk about organization. There are blind alleys that seem like a good idea, but turn out not to be. There are opportunities to merge and split passes. Some layers might want information from other layers, some layers might look very similar to other layers.

Re: Organizing complexity is the most important skill in software development

#60

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

I got fired from a job because I had done this and they couldn't figure out how to keep me billable after that. Well, I got fired because I would "show up late" and "leave early" because there was nothing else for me to do (even though I was actually pulling a full 40 hours, they were expecting 60 out of people). They could have given me more work, put me on new projects, etc., like I kept asking to be done, but inst…

I read this story and had a bit of a chuckle. And then I read your username and had a real laugh. Thanks for that!
Post reply on HN