Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Organizing complexity is the most important skill in software development

johndcook.com

Post reply on HN
Print view View original

31–40 of 286 posts

Re: Organizing complexity is the most important skill in software development

#31

Post by pavlov »

jfoutz wrote: post #18
msandford wrote: post #7

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

"Tell me about writing a compiler" There's no way to interview for what you're looking for, but compilers seem to require both great organizational skills and some algorithmic chops. there's enough complexity you can't hide lack of one side or the other. Anyway, asking about building and enhancing a compiler seems like one of the ways to get insight about how people grow code. Every program is a seed. Is this the kin…

In typical software today, much of the complexity arises from interactivity. There are user actions, network events and all sorts of contingencies that make the program flow non-deterministic.

A compiler doesn't have that. It's blissfully old-fashioned, really: read files in, write files out -- no interruptions or real-time requirements. You could do compilers on punch cards.

In a way, the compiler is the most complex piece of software that you could imagine in 1960... But software has moved on and faces a different kind of complexity.

Hence I'm not sure if asking about compiler architecture is really representative of the organizational skills needed by today's software engineering.

Re: Organizing complexity is the most important skill in software development

#32

Post by DanielStraight »

aspirin wrote: post #2

Organization is the hardest part for me personally in getting better as a developer. How to build a structure that is easy to change and extend. Any tips where to find good books or online sources?

The classic essay Big Ball of Mud is relevant as well:

http://www.laputan.org/mud/mud.html

Re: Organizing complexity is the most important skill in software development

#33

Post by markbnj »

It's a very important skill for a programmer to have, especially in the modern environment where distributed systems built from many integrated components are the norm. That said, it's awfully difficult to disentangle the various skills needed for programming and assign an importance to each one, mush less to determine which of them is actually most important of all.

Re: Organizing complexity is the most important skill in software development

#34

Post by qznc »

jfoutz wrote: post #18
msandford wrote: post #7

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

"Tell me about writing a compiler" There's no way to interview for what you're looking for, but compilers seem to require both great organizational skills and some algorithmic chops. there's enough complexity you can't hide lack of one side or the other. Anyway, asking about building and enhancing a compiler seems like one of the ways to get insight about how people grow code. Every program is a seed. Is this the kin…

Compilers are simple. They are organized as a pipeline. Frontend, middleend, backend. In more detail: lexer, parser, semantic analysis, optimzation 1 to n, instruction selection, scheduling, register allocation, assembly.

Ok, you can do more complex. I work on a compiler, which does it lazily. Transformations are per file and depend on each other. If something goes wrong, you print out the graph to figure out the problem. No idea what the benefits could be. Might actually be a good example, where it could have been simpler.

Re: Organizing complexity is the most important skill in software development

#35

Post by msandford »

odonnellryan wrote: post #21
msandford wrote: post #7

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

I've done a few similar things at my previous employer. Basically, taken it upon myself to write internal reporting applications, to much success. I've also done some other small things (AHK, etc..) which worked out well. Point is, I've been modestly bringing these up during interviews when asked what I've been doing the past few years. It is always glossed over. I don't think companies want or recognize that they wa…

I was fortunate to be a valued player on a small team. But what you're talking about is basically trying to hire yourself in as a tech exec when you're just applying for a job as a programmer. I think that's because management (generally) doesn't really want to hear that they've got a blind spot even if they do.

Ultimately all you can really do is try and work your way into a situation where you have some kind of meaningful equity. Founder, co-founder, early hire, buy a piece of a company, whatever. That's the only way to really get rewarded for your skills; knowing enough about the state of the art to see what can be done better and also having the skills to see those projects through to completion and realizing the benefits thereof.

I wish there was a job title for "good all-arounder with knowledge of the state of the art in a role with substantial self-direction", but I think that's probably "founder" or something. Or "owner of a very small business" maybe?

Re: Organizing complexity is the most important skill in software development

#36

Post by moron4hire »

msandford wrote: post #7

This is incredibly true. I once turned 60kLoC of classic ASP in VBScript into about 20kLoC of python/django including templates. And added a bunch of features that would have been impossible on the old code-base. It turned the job from hellish (features were impossible to add) to very nearly boring (there wasn't much to do anymore). So with this newfound freedom I built some machines to automate the data entry and on…

I got fired from a job because I had done this and they couldn't figure out how to keep me billable after that. Well, I got fired because I would "show up late" and "leave early" because there was nothing else for me to do (even though I was actually pulling a full 40 hours, they were expecting 60 out of people).

They could have given me more work, put me on new projects, etc., like I kept asking to be done, but instead they just wanted me to figure out some way to bill the client for 60 hours a week, and not call it "bug fixes" or "new features", because the client wasn't paying for it. That left data entry, which I had automated from a whole day to 10 minutes. They wanted me to go back to entering data by hand.

I would have quit then, but I didn't want to give them the pleasure. A flood had recently destroyed all of my stuff and I used the insurance check to get out of debt rather than replace the lost stuff, so I suddenly didn't need money for a LONG time. It's funny how easily debt can make you a compliant little slave towards people who want you to perform bullshit tasks.

I've been freelancing ever since, making twice what they paid me for only half a week's worth of work. And the flexibility of the freelancing enabled me to pursue a long-distance relationship that eventually lead to marriage. So I say getting fired was the greatest thing that ever happened to me.

Re: Organizing complexity is the most important skill in software development

#37

Post by mattmanser »

donatj wrote: post #23
dsr_ wrote: post #10

The most important skill in operations: systematic debugging. If the developers did well at organizing, this is much easier. The most important skill in low-level technical support: diplomacy. The most important skill in high-level technical support: figuring out what people are actually complaining about. Note that many low-level technical support problems look like high-level problems, and vice versa.

I find diplomacy a highly important skill as a developer as well. Strangely I feel I've become worse at it as the years go on.

One of the problems I hit was that I would get really grumpy when particular things were asked for.

Once I learnt to think about why I was getting grumpy, and vocalize the actual problem it was easier.

e.g. A client said to me recently could clicking the blog link open a new window? When I started programming I'd have said "Yes! I can do that!". Then after a few years I'd have got grumpy about it and said something passive aggressive "I guess so. That's not normal though...". Now I think about it, realized I hate it when links on the web behave unexpectedly and that's what's making me grumpy. Then explain to the client that that's you shouldn't take control of other people's browsers on public websites, and the better way is to have a clear link to get back to the shop on the blog.

Re: Organizing complexity is the most important skill in software development

#38

Post by Untit1ed »

Omnipresent wrote: post #30

On the topic of organization and related to another post about good Software Development books. What are some books that teach code organization as discussed in this post. One I can think of is "Refactoring" by Martin Fowler. What are some others?

For a novice coder I couldn't recommend Head First Design Patterns enough - the gang of four design patterns are a bit controversial these days but the way it steps you through and makes you think twice about how to organise things was really eye-opening for me back in the day.

Re: Organizing complexity is the most important skill in software development

#40

Post by qznc »

ebiester wrote: post #8
blazespin wrote: post #5

Design patterns by the Gang of Four was great for this.

No. This is an orthogonal concern. This is the equivalent of knowing to use each pattern.

The promise of Design Patterns (tm) is that you can systematically learn to organize your code. The article argues that only experience does.
Post reply on HN