Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How setting the TZ environment variable avoids thousands of system calls

blog.packagecloud.io

Post reply on HN
Print view View original

71–80 of 148 posts

Re: How setting the TZ environment variable avoids thousands of system calls

#71

Post by kelnos »

I really enjoy when people dig into things like this and report their findings. Having said that, I question the wisdom of "bothering" with this sort of thing. Everything you do that's non-standard or works against a system's default behavior incurs a cost. It's yet another thing you have to replicate when you migrate to a new version, change provisioning systems, etc.

And for what benefit? A few hundred syscalls per second? Linux syscalls are fast enough that something of that magnitude shouldn't matter much. Given that /etc/localtime will certainly be in cache with that frequency of access, a stat() should do little work in the kernel to return, so that won't be slow either.

It's good that they did some benchmarking to look at the differences, but this feels like a premature optimization to me. I can't imagine that this did anything but make their application a tiny fraction of a percent faster. Was it worth the time to dig into that for this increase? Was it worth the maintenance cost I mention in my first paragraph? I wouldn't think so.

I'm really trying not to take a crap on what they did; as I said, it's really cool to dig into these sorts of abstractions and find out where they're inefficient or leak (or just great as a learning exercise; we all depend on a mountain of code that most people don't understand at all). But, when looked at from a holistic systems approach, a grab bag of little "tweaks" like this can become harmful in the long run.

Re: How setting the TZ environment variable avoids thousands of system calls

#72

Post by vesinisa »

tytso wrote: post #18

System calls in Linux are really fast. So saving "thousands" of system calls when /etc/localtime is in cache doesn't actually save that much actual CPU time. I ran an experiment where I timed the runtime of the sample program provided in the OP, except I changed the number of calls to localtime() from ten times to a million. I then timed the difference with and without export TZ=:/etc/localhost. The net savings was .…

> TZ=:/etc/localhost

Hope this is just a typo in your comment, not the actual test ;)

Re: How setting the TZ environment variable avoids thousands of system calls

#74

Post by blunte »

This reminds me of a very similar behavior in Solaris over 20 years ago. Our C application was having odd performance problems on some client systems, and eventually we saw via truss that there were hundreds of fopen() calls every second to get the timezone. Setting the right environment variable solved the problem.

Re: How setting the TZ environment variable avoids thousands of system calls

#75

Post by lotyrin »

kelnos wrote: post #71

I really enjoy when people dig into things like this and report their findings. Having said that, I question the wisdom of "bothering" with this sort of thing. Everything you do that's non-standard or works against a system's default behavior incurs a cost. It's yet another thing you have to replicate when you migrate to a new version, change provisioning systems, etc. And for what benefit? A few hundred syscalls per…

You should be able to get maintenance for something like this pretty low. Add a line with a nice comment to the config template for running your apps that adds an extra env var. Any machine that runs any app now has the line...

I would definitely like to see a before/after real-world metric on impact here though.

Re: How setting the TZ environment variable avoids thousands of system calls

#77

Post by philsnow »

raverbashing wrote: post #53
tytso wrote: post #18

System calls in Linux are really fast. So saving "thousands" of system calls when /etc/localtime is in cache doesn't actually save that much actual CPU time. I ran an experiment where I timed the runtime of the sample program provided in the OP, except I changed the number of calls to localtime() from ten times to a million. I then timed the difference with and without export TZ=:/etc/localhost. The net savings was .…

System calls in x86 are fast. Other archs behave differently. And the syscall time is not the only thing that matters, but potentially yielding execution

I thought they were fast because x86 has multiple register files, enough for kernel space and user space to have their own, so that entry/exit to system calls doesn't require flushing registers to L1 (in the common case).

If that's true, then one test where you have a single process spinning into and out of a single syscall will have very different performance characteristics than a test where you have more processes than processor cores, because context switches flush the TLB.

Somebody who knows actual things about x86 and so forth please tell me if I'm spouting 90s-era comp sci architecture textbook stuff that no longer applies.

Re: How setting the TZ environment variable avoids thousands of system calls

#78

Post by kingosticks »

tytso wrote: post #45
rpcope1 wrote: post #29

Earlier quoted context omitted.

This might be true for your system and libc, where the system calls make use of things like vDSO for gettimeofday go fast, but in general this isn't guaranteed at all. Even on x64, for certain libc implementations, like musl, if I recall correctly, syscalls are made the old fashioned way by trapping 0x80, which would mean you would see a much bigger effect by reducing the number of syscalls.

There is no vDSO for calls to stat(2). The claim in the article was that by setting the TZ environment variable to ":/etc/localtime", one could save "thousands" of stat system calls. Even for old-fashioned system calls where you use trap 0x80, Linux is still amazingly fast. This can actually be a problem, since there are applications like git which assume stat is fast, and so it aggressively stat's all of the working…

It's also a disaster on NFS.

Re: How setting the TZ environment variable avoids thousands of system calls

#79

Post by jcapote »

ryanlol wrote: post #48
jlg23 wrote: post #10

Earlier quoted context omitted.

All the links are in the footer of the page.

For what it's worth, they disappear if your browser isn't wide enough.

Oops! I'll fix this shortly, thanks for pointing it out.
Post reply on HN