Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How setting the TZ environment variable avoids thousands of system calls

blog.packagecloud.io

Post reply on HN
Print view View original

121–130 of 148 posts

Re: How setting the TZ environment variable avoids thousands of system calls

#121

Post by bradfa »

kelnos wrote: post #71

I really enjoy when people dig into things like this and report their findings. Having said that, I question the wisdom of "bothering" with this sort of thing. Everything you do that's non-standard or works against a system's default behavior incurs a cost. It's yet another thing you have to replicate when you migrate to a new version, change provisioning systems, etc. And for what benefit? A few hundred syscalls per…

The embedded Linux system I'm working on right now takes about 17 us per stat() call due to this but time will always be kept internally as UTC, so taking advantage of this is worth considering for me.

Since most embedded systems can directly translate the amount of processing needed to achieve the product goal into a real dollar cost of the hardware, any savings, even a small one, is worth investigating. Since the hardware and system are generally well understood, implementing something like this is much more reasonable.

But I agree, on a general purpose OS doing general purpose thing, an optimization like what's proposed by the article may not be worth the other tradeoffs.

Re: How setting the TZ environment variable avoids thousands of system calls

#122

Post by lfowles »

deathanatos wrote: post #114
cbsmith wrote: post #56

Earlier quoted context omitted.

> System calls in Linux are really fast. So saving "thousands" of system calls when /etc/localtime is in cache doesn't actually save that much actual CPU time. "fast" is a relative term, and is somewhat orthogonal to "efficient". There's a reason why certain functions use a vDSO. If you're just going to use a syscall anyway, there's kind of no point.

You're assuming that all cases where the vDSO call is made gets paired with a real syscall; that's simply not the case. There are plenty of calls in a server that won't need localtime (basically, anything that just needs the current time in UTC: best-practice code should not be looking at the machine's TZ setting¹). Look at the examples the article's author offers: > formatting dates and times This shouldn't require…

>Breaking a seconds-since-epoch out into year/mo/day/etc. is "simple" math, and shouldn't require a filesystem access.

To do it simply yes, but not correctly. See the "Falsehoods programmers believe about time" series.

http://infiniteundo.com/post/25326999628/falsehoods-programm... http://infiniteundo.com/post/25509354022/more-falsehoods-pro...

Re: How setting the TZ environment variable avoids thousands of system calls

#123

Post by curlypaul924 »

geofft wrote: post #64
cbsmith wrote: post #55

This reads to me like a glibc bug. Glibc should just be watching "/etc/localtime" for changes, rather than calling out to hundreds of times a second.

It's hard to do that without any system calls, though. The way you'd do this is to open an inotify (or platform equivalent) file descriptor on the first call to localtime(), and on future calls, only bother statting /etc/localtime again if that file descriptor reports something has changed. But checking if an FD has data requires a system call; you'd do a non-blocking read on the FD (or a non-blocking select or poll,…

What about calling mmap to map /etc/localtime into memory? The file would still have to be parsed for every call to localtime(), but the system call is avoided.

(Better yet, if it were possible to memory-map the directory entry for /etc/localtime, parsing could be avoided as well).

I agree the best solution is to provide a more sensible API. Why should an application be limited to only _one_ timezone?

Re: How setting the TZ environment variable avoids thousands of system calls

#124

Post by mfukar »

kingosticks wrote: post #120
tytso wrote: post #18

System calls in Linux are really fast. So saving "thousands" of system calls when /etc/localtime is in cache doesn't actually save that much actual CPU time. I ran an experiment where I timed the runtime of the sample program provided in the OP, except I changed the number of calls to localtime() from ten times to a million. I then timed the difference with and without export TZ=:/etc/localhost. The net savings was .…

I did the same experiment on a Raspberry Pi 2. The net saving was 5.803 seconds, so 5.803 microseconds per call. Obviously if you care about performance then you wouldn't be running your program on a Raspberry Pi in the first place. But for everything else there's this free speed up.

I build a bunch of home automation stuff (as a hobby) using Pis and other microcontrollers. Performance in those things translates almost directly to power savings, and is very desirable.

OTOH, I've never encountered an issue like this on those systems.. (yet)

Re: How setting the TZ environment variable avoids thousands of system calls

#125

Post by zeveb »

kseistrup wrote: post #110
zeveb wrote: post #49

Earlier quoted context omitted.

Do all your work in UTC. Seriously, you'll be glad later. Convert to local time only on the edges, and only for end users.

All my VPSes run in UTC, no exception. What I'm talking about here is my desktop machine. I strongly prefer to run that in localtime.

If you set your machine to the DST timezone for your political unit, then the displayed time will automatically jump back and forth on the correct dates, as long as your time database is reasonably up-to-date; you don't need to manually change from non-DST to DST & back.

The non-DST timezones are for those who are lucky enough to live in political units which tell DST to get bent, and just stay on real time all year long.

DST delenda est.

Re: How setting the TZ environment variable avoids thousands of system calls

#126

Post by kseistrup »

zeveb wrote: post #125
kseistrup wrote: post #110

Earlier quoted context omitted.

All my VPSes run in UTC, no exception. What I'm talking about here is my desktop machine. I strongly prefer to run that in localtime.

If you set your machine to the DST timezone for your political unit, then the displayed time will automatically jump back and forth on the correct dates, as long as your time database is reasonably up-to-date; you don't need to manually change from non-DST to DST & back. The non-DST timezones are for those who are lucky enough to live in political units which tell DST to get bent, and just stay on real time all year…

Thanks, I didn't know that.

I believe I prefer the

    TZ=:Continent/City
notation in this case, though.

Re: How setting the TZ environment variable avoids thousands of system calls

#127

Post by scottlamb »

There's another easy way to avoid this: use localtime_r instead of localtime. From the glibc source:

    /* Update internal database according to current TZ setting.
       POSIX.1 8.3.7.2 says that localtime_r is not required to set tzname.
       This is a good idea since this allows at least a bit more parallelism.  */
    tzset_internal (tp == &_tmbuf && use_localtime, 1);
mktime also does the tzset call every time, though:

    time_t
    mktime (struct tm *tp)
    {
    #ifdef _LIBC
      /* POSIX.1 8.1.1 requires that whenever mktime() is called, the
         time zone names contained in the external variable 'tzname' shall
         be set as if the tzset() function had been called.  */
      __tzset ();
    #endif
and I don't see any way around that other than setting TZ=: or some such.

Re: How setting the TZ environment variable avoids thousands of system calls

#128

Post by deathanatos »

lfowles wrote: post #122
deathanatos wrote: post #114

Earlier quoted context omitted.

You're assuming that all cases where the vDSO call is made gets paired with a real syscall; that's simply not the case. There are plenty of calls in a server that won't need localtime (basically, anything that just needs the current time in UTC: best-practice code should not be looking at the machine's TZ setting¹). Look at the examples the article's author offers: > formatting dates and times This shouldn't require…

>Breaking a seconds-since-epoch out into year/mo/day/etc. is "simple" math, and shouldn't require a filesystem access. To do it simply yes, but not correctly. See the "Falsehoods programmers believe about time" series. http://infiniteundo.com/post/25326999628/falsehoods-programm... http://infiniteundo.com/post/25509354022/more-falsehoods-pro...

> To do it simply yes, but not correctly.

No, it do it correctly doesn't require filesystem access either. I've read both articles in the past: neither refutes the point I made above. If I were incorrect, linking to an article that enumerates tens of things (some of them arguably incorrect) isn't useful.

If you're trying to imply that you need to take timezones into account, yes, you do. Yes, typically those definitions are stored on disk, but the context here is requiring filesystem access each and every time; most libraries (including glibc) will load the timezone definitions once, and keep them in memory. Thus, you can break a seconds-since-epoch out into year/mo/day/etc. with "simple" math, and it doesn't require a filesystem access. (Beyond the amortized one time load, but given the point and purposes of the article, I'm not considering that.)

Re: How setting the TZ environment variable avoids thousands of system calls

#129

Post by andrewbinstock »

koverstreet wrote: post #70
andrewbinstock wrote: post #59

Earlier quoted context omitted.

Those who don't know George Santayana are condemned to repeat him. FTFY.

You seem to have missed the joke...

Totally got it. It's a refinement of the line. As the person below understands.

Re: How setting the TZ environment variable avoids thousands of system calls

#130

Post by cestith »

glandium wrote: post #85

What is missing in this post is: - Why does glibc check /etc/localtime every time localtime is called? Wild guess: so that new values of /etc/localtime are picked at runtime without restarting programs. - Corollary: why does glibc not check /etc/localtime every time localtime is called, when TZ is set to :/etc/localtime? Arguably the reason above should still apply when TZ is set to a file name, shouldn't it?

This is due to the multiuser nature of Unix-like systems.

/etc/localtime is set by the administrator. It may change without notice to the user.

TZ is part of the user's environment and the user sets it. All applications run by the user should honor the user's wishes if the user's not falling back to system defaults.

If you're setting TZ for yourself, your libc can update things when you update the variable and restart any applications you're running under the old value. It can therefore save cycles. If you're falling back to the system default that's not under the same user's control, then it must be ready to deal with unexpected changes.

Post reply on HN