Live data from Hacker News

Lessons after five years of professional programming

siavoshb.tumblr.com

1–10 of 26 posts

Re: Lessons after five years of professional programming

#3
First advice is horrible. One shouldn't do in application things that are meant to be done in (relational) database, like ordering, grouping, joining... I guess 66% of my application performance tuning was removing iteration over large result sets in application code with a few lines of SQL. The other 30% being making sure the database is not hit with redundant queries.

I have rarely seen the case where minimizing the size of result set returned by database to the application is not the right choice.

EDIT: I'm speaking about relational databases.

Other advices are good, though.

Re: Lessons after five years of professional programming

#4

First advice is horrible. One shouldn't do in application things that are meant to be done in (relational) database, like ordering, grouping, joining... I guess 66% of my application performance tuning was removing iteration over large result sets in application code with a few lines of SQL. The other 30% being making sure the database is not hit with redundant queries. I have rarely seen the case where minimizing th…

I agree. Why would you sort in memory? RDBMS have been heavily optimised to do these things.

Re: Lessons after five years of professional programming

#5

First advice is horrible. One shouldn't do in application things that are meant to be done in (relational) database, like ordering, grouping, joining... I guess 66% of my application performance tuning was removing iteration over large result sets in application code with a few lines of SQL. The other 30% being making sure the database is not hit with redundant queries. I have rarely seen the case where minimizing th…

i think this is actually reasonable advice because in most systems, the application layer is pretty stateless and can easy be scaled out to more machines whereas the db is usually a central bottleneck.

Re: Lessons after five years of professional programming

#6
post #5

First advice is horrible. One shouldn't do in application things that are meant to be done in (relational) database, like ordering, grouping, joining... I guess 66% of my application performance tuning was removing iteration over large result sets in application code with a few lines of SQL. The other 30% being making sure the database is not hit with redundant queries. I have rarely seen the case where minimizing th…

i think this is actually reasonable advice because in most systems, the application layer is pretty stateless and can easy be scaled out to more machines whereas the db is usually a central bottleneck.

I think too that this was his point, it only might make sense in very specific scenarios though, for day-to-day web-dev this is terrible advice as others have said.

Re: Lessons after five years of professional programming

#7
Concurrency should not be avoided just because it's scary, especially in the modern world of multiple cores/processors everywhere. Threads can get you great performance gains. Yes, there are particular problems in this area, but they're not impossible to overcome or a guaranteed way to introduce horrible bugs.

Also I agree with the other comment on SQL layers.

I think the main lesson learned after 5 years ought to be that you still have a lot left to learn.

Re: Lessons after five years of professional programming

#8
1. When performance is an issue, if you can calculate or process it at the application layer, then take it out of the database layer. order by/group by are classic examples. It’s almost always easier to scale out your application layer than your database layer. As true for MySQL on your server as it is on the sqlite in your handheld.

I disagree. RDBMS are highly optimized for this kind of operations (including MS SQL in here as well) and you’d better off retrieve in the application layer only what you actually need to work on. Think about paging: why would you want to retrieve millions of records in the application layer when you just need a few hundreds to work on? You’re going to waste SELECT time, connection time, application layer processing time (as I believe you’re logic won’t be as optimized as the one that resides in a well optimized RDBMS)

2. Concurrency, avoid it if you can. If not, then remember that with great power comes great responsibility Avoid working directly with threads if you can. Work at a higher level of abstraction if possible. In iOS, for example: GCD, dispatch and operation queues are your friends. The human mind was not designed to reason about infinite temporal state—I get nauseous thinking about how I learned all this first hand.

Queue patterns are cool, but concurrency sometime gives you a great deal of performance improvement. Sure, it’s dangerous, fragile and can mess things up real quick. But I wouldn’t just exclude it because of this. From great developers _is expected_ great responsibility.

3. Minimize state as much as possible, and keep it as localized as possible. The functionalists were/are onto something.

Good point, without exaggeration.

4. Short composable methods are your friend.

Agree, as long as you don’t end up in a compulsing-composive behavior where you abstract and split every method into meaningless individual parts. I’ve seen pojects containing 50 folders, each of them with one file and one method, from people trying to follow this pattern.

5. Comments are dangerous since they can get out of date and mislead, but so is not having them. Don’t comment the trivial, but strategically write paragraphs if needed in specific sections. Your memory will fail you, as soon as tomorrow morning, even after coffee

Code should be self readable, and if you get to the point where it isn’t, then you may have to rewrite few bits. This is not always possible though, especially for temporary hotfixes or hacks. In this case, comments are a must.

6. If you feel one use-case scenario will “probably be ok”, that’s the one that’s going to lead to catastrophic failure a month in production. Trust your paranoid gut, test and verify.

I tend to be more concerned about the scenario you feel 100% confident about. In case of major failures, you’re going to be looking at the dubious parts of your system (scenarios, code, patterns) – making it more difficult to spot the issue, if it’s hidden inside a “safe” part.

7. When in doubt, over communicate all concerns with your team.

Communicate and discuss all concerns as well as proposed solutions.

8. Do the right thing—you usually know what that thing is.

Couldn’t agree more. This is very difficult to explain, but you may find yourself sometime writing code and you know that the right thing takes few hours more and requires more effort… don’t keep doing what you are doing just because “maybe you won’t need it”. Always go for the right thing. If you don’t, it is likely going to bit you back.

9. Your users aren’t stupid, they just don’t have the patience for your cut corners.

Or simply they don’t get your UX. User testing is key to success.

10. If an engineer is not tasked with the long term maintenance of the systems they build, view them with suspicion. 80% of the blood, sweat, and tears of software occurs after its been released—that’s when you become a world weary, but wiser “professional.”

I would expect every engineer in my team to write self readable, well planned, “long term” code. Even if it is for the stupidest internal tool that nobody else is ever going to upgrade. And most of the times this does not slow down development (short and long term). It just becomes automatic.

11. Checklists are your friends.

Kanban boards, checklists, basecamp, whatever helps your team to track stuff and get things done.

12. Take initiative to purposeful enjoy your work, sometimes this will take effort.

True, although sometime you have to do the boring parts as well – you’ll have to do things who sucks, to make sure your customers won’t.

13. Silent failures, I still have nightmares. Monitor, log, alert. But be wary of false positives and the inevitable desensitization it leads to. Keep your system senses clear and alert.

Spot on. Don’t end up with a notification system that sends you thousands of exceptions to your mailbox everyday. You’ll start to ignore them and it’ll be difficult to find out real issues. Keep your monitoring clean. Fix issues, or deprioritize them.

14. At the end of the day, we’re paid to manage complexity. Work accordingly.

…and make complexity simpler for our users.

Re: Lessons after five years of professional programming

#9

First advice is horrible. One shouldn't do in application things that are meant to be done in (relational) database, like ordering, grouping, joining... I guess 66% of my application performance tuning was removing iteration over large result sets in application code with a few lines of SQL. The other 30% being making sure the database is not hit with redundant queries. I have rarely seen the case where minimizing th…

I agree with you - I think this bit of advice addresses the effect, not the cause, of an ignored problem: design, design, design.

Design your database for your application - if you have some major hassle with your database after App 1.0 is developed, its because you missed this very important step.

Don't treat the .db like its an architectural sandbox, adding/tweaking/removing things 'to make it better' after the fact. Careful .db architecture means, once your app reaches a functional state, your .db should be already pretty much immutable.

Maybe the solution for those who can't escape this flaw is to simply build the App first, then the .db, so that there is little chance for the .db to be screwed up in the first place, who knows ..

Re: Lessons after five years of professional programming

#10
post #5

First advice is horrible. One shouldn't do in application things that are meant to be done in (relational) database, like ordering, grouping, joining... I guess 66% of my application performance tuning was removing iteration over large result sets in application code with a few lines of SQL. The other 30% being making sure the database is not hit with redundant queries. I have rarely seen the case where minimizing th…

i think this is actually reasonable advice because in most systems, the application layer is pretty stateless and can easy be scaled out to more machines whereas the db is usually a central bottleneck.

I agree - on the majority of entreprise system (ERP) I have worked this was the case. I can confirm that GROUP BY are huge performance killers and sometimes the simplest solution is to avoid them by slightly changing the business logic if possible.
Post reply on HN