Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Organizing complexity is the most important skill in software development

johndcook.com

Post reply on HN
Print view View original

61–70 of 286 posts

Re: Organizing complexity is the most important skill in software development

#61

Post by odonnellryan »

vonmoltke wrote: post #47
msandford wrote: post #35

Earlier quoted context omitted.

I was fortunate to be a valued player on a small team. But what you're talking about is basically trying to hire yourself in as a tech exec when you're just applying for a job as a programmer. I think that's because management (generally) doesn't really want to hear that they've got a blind spot even if they do. Ultimately all you can really do is try and work your way into a situation where you have some kind of mea…

> I wish there was a job title for "good all-arounder with knowledge of the state of the art in a role with substantial self-direction", but I think that's probably "founder" or something. Or "owner of a very small business" maybe? That's the way I am drifting. I just can't seem to get in gear on my own. :(

Why's that? I've been thinking about starting my own side company. It is a lot of work!

Re: Organizing complexity is the most important skill in software development

#62

Post by barrkel »

pavlov wrote: post #31
jfoutz wrote: post #18

Earlier quoted context omitted.

"Tell me about writing a compiler" There's no way to interview for what you're looking for, but compilers seem to require both great organizational skills and some algorithmic chops. there's enough complexity you can't hide lack of one side or the other. Anyway, asking about building and enhancing a compiler seems like one of the ways to get insight about how people grow code. Every program is a seed. Is this the kin…

In typical software today, much of the complexity arises from interactivity. There are user actions, network events and all sorts of contingencies that make the program flow non-deterministic. A compiler doesn't have that. It's blissfully old-fashioned, really: read files in, write files out -- no interruptions or real-time requirements. You could do compilers on punch cards. In a way, the compiler is the most comple…

I used to work on the Delphi compiler, for 6 years or so. That compiler was not a simple function of file input to file output. It lived in an IDE. It had to do incremental compilation, throwing away the compiled version of an in-memory file and recompiling it, but avoid recompiling the whole world of downstream dependencies if not necessary. It had to provide code completion, which meant parsing and type analysis of a file, but skipping codegen, with reasonably sophisticated error recovery. It had to provide the debug expression evaluator, which lived intermixed with the constant evaluator, and needed to know how to set up calls to symbols in the program being debugged. And it had to do all of these things in a long-lived process, with no memory leaks and high reliability.

The degree of asynchronicity is low, I agree. The most significant was just interrupting compilation on user request. But the complexity was not trivial.

These days I work at a startup on a full stack SaaS; C++, Java, Rails, and Coffeescript with as few blocking browser actions as possible. It is far less complex than compiler work, but it pays better. I've never had to debug OS code to diagnose runtime library issues in this job; nor write manual structured exception handling. I haven't had to try and build stack frames dynamically to implement reflection calls, nor unpick stack frames dynamically to implement virtual method interception. The relative complexity of potential races with SQL read-committed, or writing UI such that it can't become inconsistent, really doesn't compare.

Re: Organizing complexity is the most important skill in software development

#63

Post by jeremiep »

qznc wrote: post #40
ebiester wrote: post #8

Earlier quoted context omitted.

No. This is an orthogonal concern. This is the equivalent of knowing to use each pattern.

The promise of Design Patterns (tm) is that you can systematically learn to organize your code. The article argues that only experience does.

But design patterns are horrible at organizing code. They have a much higher chance of introducing unneeded complexity.

It should be no surprise that almost every single design pattern can be replaced with a simple lambda in languages supporting them.

Re: Organizing complexity is the most important skill in software development

#64

Post by bluedino »

collyw wrote: post #43
stinos wrote: post #11

Not only is it the most important skill (unless for small home projects maybe), it's also the one which takes the longest to learn and hence the one you really improve on during the years and which distinguishes the experienced ones from the lesser experienced ones. Coincidently, it's also the skill for which you won't find a ready-made answer on stackoverflow or any other site/book. thinking of it, I've also seen th…

This is pretty much how I feel just now (after 12 years). I know Django pretty well after 4 years, and know where and how to add things to avoid breaking other parts of the code and make it work consistently with other parts. Yet nearly every job I apply for these days wants a 90 minute online test. (I just had to implement a sorting algorithm in psuedo code - I have never in 12 years had to implement a sorting algor…

>> Yet nearly every job I apply for these days wants a 90 minute online test. (I just had to implement a sorting algorithm in psuedo code - I have never in 12 years had to implement a sorting algorithm as I choose an appropriate library to do that for me).

True, but if you can't implement a simple sorting algorithm how are you going to implement _____ ?

Re: Organizing complexity is the most important skill in software development

#65

Post by kazinator »

jfoutz wrote: post #18
msandford wrote: post #7

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

"Tell me about writing a compiler" There's no way to interview for what you're looking for, but compilers seem to require both great organizational skills and some algorithmic chops. there's enough complexity you can't hide lack of one side or the other. Anyway, asking about building and enhancing a compiler seems like one of the ways to get insight about how people grow code. Every program is a seed. Is this the kin…

Compilers aren't real-time, distributed, multi-user, ... basically anything that is hard from a software engineering perspective which includes deployment and testing. Compilers have reproducible test cases (if they don't, that is a bug in itself). You can catch regressions in a compiler with an automated test suite, and bisect their version history to see where it was introduced. Bug reports from the field tend to be easy to confirm. A compiler isn't going to work well when it has 10,000 users, but mysteriously spiral to a crash when there are 10,100.

Compilers have been developed by lone developers in isolation. Some compilers are only a few thousand lines of code.

Someone who can demonstrate knowledge of making a compiler could well be completely behind in their knowledge about language design. He or she shows how calculations are turned into a tree and then into code on a register machine (i.e. "for-mula tran-slation"). But no idea how to compile exception handling, closures, and doing advanced things with type (beyond just checks) and such.

That is to say, the bar for what can be legitimately called a compiler is quite low.

On the topic of languages, I'd rather interview someone who knows about a lot of modern language features and how they can contribute to the improvement of program organization, yet is foggy about the details of how some of them are might be compiled to machine code.

There is still "compiler worship". On a previous job, I fixed a code generation bug in gcc affecting MIPS targets. Everyone was talking about that: "like wow, he fixed gcc".

Re: Organizing complexity is the most important skill in software development

#66

Post by stinos »

collyw wrote: post #43
stinos wrote: post #11

Not only is it the most important skill (unless for small home projects maybe), it's also the one which takes the longest to learn and hence the one you really improve on during the years and which distinguishes the experienced ones from the lesser experienced ones. Coincidently, it's also the skill for which you won't find a ready-made answer on stackoverflow or any other site/book. thinking of it, I've also seen th…

This is pretty much how I feel just now (after 12 years). I know Django pretty well after 4 years, and know where and how to add things to avoid breaking other parts of the code and make it work consistently with other parts. Yet nearly every job I apply for these days wants a 90 minute online test. (I just had to implement a sorting algorithm in psuedo code - I have never in 12 years had to implement a sorting algor…

Yup, exactly this. Which is why when we hire, we try to at least get hold of code written by the interviewee and preferrably make him/her come over for a day or so and have him/her do some programming. Much more valueable than 90 minutes of putting someone under some stres-test which doesn't resemble an actual job at all. I read similar things on other comments here so hopefully there's a shift going on. Especially because I would fail terribly at standard interviews as well :]

Re: Organizing complexity is the most important skill in software development

#67

Post by mmcconnell1618 »

jondubois wrote: post #24

I agree that it is an important skill in engineering to recognize when the complexity got too high - At that point you need to take a step back (or several steps back) and find a different path to the solution - Sometimes that means throwing away large amounts of code. It's about acknowledging (and correcting) small mistakes to make sure that they don't pile up into a giant, disastrous one. Another thing I learned is…

I agree. I like to remember that I'm passing code into the future where I might not be working on it anymore. Clear, well named code helps the next developer far more than an obtuse algorithm that smashes all logic into a single line.

Re: Organizing complexity is the most important skill in software development

#68

Post by tempodox »

qznc wrote: post #40
ebiester wrote: post #8

Earlier quoted context omitted.

No. This is an orthogonal concern. This is the equivalent of knowing to use each pattern.

The promise of Design Patterns (tm) is that you can systematically learn to organize your code. The article argues that only experience does.

If it were that easy, you could write a compiler compiler that knows all the “design patterns” and generates perfect solutions for all problems.

Remember the 239th Rule of Acquisition, “Never be afraid to mislabel a product.” That goes for methodologies, too. If they could get away with it, they would probably say it cures baldness, too.

Re: Organizing complexity is the most important skill in software development

#69

Post by tome »

This is why I love Haskell. Haskell seems to increase my ability to manage complexity by one level.

Disclaimer: Haskell is not a silver bullet, not a panacea and I'm only claiming a modest increase, not miracles, but it helps me deal with complexity better than any other language I know.

Re: Organizing complexity is the most important skill in software development

#70

Post by machinshin_ »

vijayr wrote: post #55
ska wrote: post #41

Earlier quoted context omitted.

> Sadly there's (seemingly) no way to interview people for this ability I think there is. Show the candidate some bad code. Not WTF code, but something you wouldn't be proud of but is realistic. Bonus points if you're comfortable enough to pull this out of your own production system. Ask the candidate about adding some functionality, but be clear that cleanup and refactoring changes are fine (even encouraged). The co…

This happened to me once. I went for an an interview at a huge real estate company. They had a chat with me for exactly five minutes, then gave me a computer (an old one, with keyboard not working properly) and some legacy code. I wasn't asked to fix it, but I was asked to make a small change and given half hour for it. That code had no documentation and no comments. Then 5 more mins to explain what I understood abou…

Did you accept?
Post reply on HN