Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rob Pike’s Rules of Programming (1989)

users.ece.utexas.edu

Post reply on HN
Print view View original

281–290 of 332 posts

Re: Rob Pike’s Rules of Programming (1989)

#281

Post by thdrdt »

Cthulhu_ wrote: post #265
thdrdt wrote: post #237

Earlier quoted context omitted.

It is all about lack of knowledge because checking where the bottlenecks are is one thing. Knowing why they are there is another. A retailer app was very slow. The developers proposed a newer faster server with a newer Oracle version. Then I took a look at one of the slowest queries. Changed it so it could use the indexes in a better way and the query went from +20 to 0.7 seconds. The developers measured the query wa…

Sounds like ops vs dev (or DBA) right there; it does make sense to a point that ops would grow into that kind of behaviour, given that they can't really allocate resources into performance enhancements for software (especially if it's from an external party or off-the-shelf). To them it's a black box. But if it's an in-house thing then there should be open communication lines. The SRE paradigm makes sense there, wher…

It was more like dev & dev.

I am not criticizing anyone because I was just lucky to notice the query could be rewritten. But it was also the fact that I had a little more knowledge about how the db engine handles queries.

So we all looked in the right direction, we all found the bottleneck but the solution was different based on different knowledge.

Re: Rob Pike’s Rules of Programming (1989)

#282

Post by bluGill »

zelly wrote: post #240
OliverJones wrote: post #81

In long-lived systems (systems that run for many years) it's almost impossible to choose the "right data structures" for the ages. The sources and uses of your data will not last nearly as long as the data itself. What to do about this? Two things: STORE YOUR TIMESTAMPS IN UTC. NOT US Pacific or any other local timezone. If you start out with the wrong timezone you'll never be able to fix it. And generations of progr…

> STORE YOUR TIMESTAMPS IN UTC. NOT US Pacific or any other local timezone. What difference does it make if the timestamp includes the timezone? The UTC value can be recovered. In some applications the timezone is useful e.g. when intraday times matter.

Odds are that you forgot to record the timezone at all, and it didn't matter until you already have users in different timezones who have saved data and they don't remember which timezone they saved everything in.

If you record the timezone you can convert. Even then, it is easier to use UTC just because everyone else does and so you can feed UTC into any third party library and it will work.

Re: Rob Pike’s Rules of Programming (1989)

#283

Post by bmaupin »

mathattack wrote: post #203
rumanator wrote: post #115

Earlier quoted context omitted.

Plenty of professional developers would benefit greatly if they read Domain-Driven Design.

Interesting. What’s the best resource beyond the Wikipedia page?

"The term was coined by Eric Evans in his book of the same title."

https://en.m.wikipedia.org/wiki/Domain-driven_design

Re: Rob Pike’s Rules of Programming (1989)

#284

Post by strgcmc »

thdrdt wrote: post #281
Cthulhu_ wrote: post #265

Earlier quoted context omitted.

Sounds like ops vs dev (or DBA) right there; it does make sense to a point that ops would grow into that kind of behaviour, given that they can't really allocate resources into performance enhancements for software (especially if it's from an external party or off-the-shelf). To them it's a black box. But if it's an in-house thing then there should be open communication lines. The SRE paradigm makes sense there, wher…

It was more like dev & dev. I am not criticizing anyone because I was just lucky to notice the query could be rewritten. But it was also the fact that I had a little more knowledge about how the db engine handles queries. So we all looked in the right direction, we all found the bottleneck but the solution was different based on different knowledge.

I would go so far as to say, their "solution" was not a solution at all. Granted, in the absence of being able to recognize alternatives, it may have been the best available option left, but it fundamentally missed the root cause of the query slowness.

Your experience exactly shows why having a diversity of opinions/backgrounds/expertise on a team is a very valuable trait. Had no one realized you could rewrite the query, would scaling vertically and upgrading Oracle been a fatal mistake for the team? Probably not, but damn if it wouldn't have been a big waste of time/money.

Re: Rob Pike’s Rules of Programming (1989)

#285

Post by dragonwriter »

wtetzner wrote: post #123
aserafini wrote: post #105

Earlier quoted context omitted.

Sometimes storing in UTC is simply not correct. For example a shop opening time. The shop opens 10am local time, whether DST or not. Their opening time is 10am local time all year but their UTC opening time actually changes depending on the time of year!

But a shop opening time is not a timestamp, so I think the original advice is still good. A timestamp is the time at which some event happened, which is different than a date/time used for specifying a schedule. For example, if you wanted to track the history of when the shop actually opened, it would make sense to store a UTC timestamp.

> A timestamp is the time at which some event happened,

It's important to the advice to make explicit that the use of “timestamp” in that sense is intended, because “timestamp” is also in many contexts “the data type that combines date and time of day and, perhaps optionally, time zone information”. The application of “timestamps” in the latter sense is not limited to when they represent “timestamps” in the former sense.

Re: Rob Pike’s Rules of Programming (1989)

#286

Post by karmakaze »

rurounijones wrote: post #239
karmakaze wrote: post #121

Earlier quoted context omitted.

The most interesting case of this I encountered was for photo 'timestamps' on a global sharing site. UTC was being used and I was proposing a change to local time. There was great debate as many drank the UTC juice and stopped thinking. It was when I showed them that we also have a 'shot at' location then proceeded to show Christmas eve photos showing the UTC time converted to the viewers local timezone (not always e…

Sounds like the problem was images being uploaded with a timestamp without a timzeone, in which case neither solution would work.

The timezone could be inferred from uploader's geoip as a fallback. The problem was that even if the timezone was known at time of upload it was converted to UTC and lost when stored.

Re: Rob Pike’s Rules of Programming (1989)

#287

Post by karmakaze »

Joker_vD wrote: post #259
karmakaze wrote: post #121

Earlier quoted context omitted.

The most interesting case of this I encountered was for photo 'timestamps' on a global sharing site. UTC was being used and I was proposing a change to local time. There was great debate as many drank the UTC juice and stopped thinking. It was when I showed them that we also have a 'shot at' location then proceeded to show Christmas eve photos showing the UTC time converted to the viewers local timezone (not always e…

For historical events, where the local time is important, the combination of "UTC timestamp" and "local time offset in effect at the moment the timestamp was taken" seems to be the choice. Allows you to easily learn what time the wall clock was showing at the moment.

Databases have support for a single type that encodes exactly like this. In postgresql a timestamptz shows as 'yyyy-mm-dd hh:mm:ss.123456+1234' but internally it's stored as UTC unixtime and tz offset.

Re: Rob Pike’s Rules of Programming (1989)

#288

Post by bpyne »

throwaway894345 wrote: post #184
dkarl wrote: post #126

> Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is. I wish people would follow this rule and just let stuff work. I recently encountered the most extreme version of this I've ever seen in my career: a design review where a guy proposed a Redis caching layer…

I ran into this early in my career when a senior engineer at a startup reviewed my Python PR for operating on a CSV dataset and insisted I rewrite it in Pandas for performance. It was a very simple program with naive Python (a handful of lines), but the Pandas version was far longer and more complex (I had to have the senior engineer help because it took some advanced Pandas-fu, and he spent nearly a full day on it),…

"I ran into this early in my career when a senior engineer at a startup reviewed my Python PR for operating on a CSV dataset and insisted I rewrite it in Pandas for performance."

Did a performance problem exist?

If not, then I would never promote that person to senior engineer. A working program should only be re-engineered for performance if it wasn't meeting the performance contract agreed upon when the program was written.

Re: Rob Pike’s Rules of Programming (1989)

#290

Post by bluGill »

dnautics wrote: post #122
sukilot wrote: post #34

Earlier quoted context omitted.

TDD is good for features (like web apps) but not so much for algorithms. The difference is that you only need to support a tiny fraction of possible features / use cases, but your algorithms need to be correct for a wide range of inputs.

I disagree. You code your algo for one input, come up with a corner case, write a failing test, refactor, repeat. For an algo, let's say it operates on a list, I'll start with test f([]) == 0, and implement f to output the constant 0. And then go from there.

That only works if the algorithm isn't already understood and so you don't know what optimal is. If you tdd sort there is no way to get divide the list to small chunks, insert sort each small chunk, then merge sort them. (20 years ago this is what gnu sort did, I assume that this algorithm has changed now that cache is a bigger factor in performance). Even knowing that the above is your end goal TDD to get there is wrong because you don't know how the best algorithm will change when something else comes up as important.

Tests are good, but they need to be universal for any implementation, which means you often cannot tests the internal details that prove you didn't use bogo sort (picking a pathological example to make the point)

Post reply on HN