Linux is very fast !!
How setting the TZ environment variable avoids thousands of system calls
101–110 of 148 posts
Re: How setting the TZ environment variable avoids thousands of system calls
#102What is missing in this post is: - Why does glibc check /etc/localtime every time localtime is called? Wild guess: so that new values of /etc/localtime are picked at runtime without restarting programs. - Corollary: why does glibc not check /etc/localtime every time localtime is called, when TZ is set to :/etc/localtime? Arguably the reason above should still apply when TZ is set to a file name, shouldn't it?
For the second question: There doesn't seem to be an explicit reason for the difference of treatment. The code that does it has been there since 1996, and hasn't changed since. The only reason given is "Caching happens based on the contents of the environment variable TZ.".
https://sourceware.org/git/?p=glibc.git;a=commit;h=68dbb3a69...
I'd argue it should cache the same when both old_tz and tz are NULL (but start with an old_tz that is not NULL).
I was about to file an upstream bug, but found https://sourceware.org/bugzilla/show_bug.cgi?id=5184 and https://sourceware.org/bugzilla/show_bug.cgi?id=5186
The latter actually implies the opposite should be happening: files given in TZ should be stat()ed just as much as /etc/localtime.
Re: How setting the TZ environment variable avoids thousands of system calls
#103System calls in Linux are really fast. So saving "thousands" of system calls when /etc/localtime is in cache doesn't actually save that much actual CPU time. I ran an experiment where I timed the runtime of the sample program provided in the OP, except I changed the number of calls to localtime() from ten times to a million. I then timed the difference with and without export TZ=:/etc/localhost. The net savings was .…
This might be true for your system and libc, where the system calls make use of things like vDSO for gettimeofday go fast, but in general this isn't guaranteed at all. Even on x64, for certain libc implementations, like musl, if I recall correctly, syscalls are made the old fashioned way by trapping 0x80, which would mean you would see a much bigger effect by reducing the number of syscalls.
Re: How setting the TZ environment variable avoids thousands of system calls
#104Earlier quoted context omitted.
On your base system, yes. Lots of things can hook random syscalls, or environments might have syscall monitoring. One example is the folks over at slack record every syscall for security auditing. https://slack.engineering/syscall-auditing-at-scale-e6a3ca8a...
Slack uses the Linux audit subsystem which is also certainly faster than you think it is. Consider how many system calls your typical application is issuing --- especially ones that are likely to be calling localtime() all the time, such as a web server. If system call auditing had that high of an overhead, everything would be horrifically slow --- but it isn't, because Linux audit sends its records out asynchronousl…
https://www.redhat.com/archives/linux-audit/2015-January/msg...
of course this is RHEL 2.6.32 and it's open/close but 200000 sc/s vs 3000 sc/s shows it has some overhead. Maybe someone can rerun that test code on git and see what the overhead is.
Re: How setting the TZ environment variable avoids thousands of system calls
#105> In other words: your system supports calling the time system call via the Linux kernel’s vDSO to avoid the cost of switching to the kernel. But, as soon as your program calls time, it calls localtime immediately after, which invokes a system call anyway. This reminds me of an article by Ted Unangst[1], in which he flattens the various libraries and abstractions to show how xterm (to cite one of many culprits) in on…
Xe just did mplayer as well. It calls non-blocking select(), then non-blocking poll(), then nanosleep(), in a loop.
Re: How setting the TZ environment variable avoids thousands of system calls
#106Interesting article but I can't reproduce the behavior on Ubuntu 16.01 LTS. I don't have TZ set (or anything locale-related for that matter). Here are the library dependencies: $ ldd test linux-vdso.so.1 => (0x00007ffd80baf000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8844bf7000) /lib64/ld-linux-x86-64.so.2 (0x00007f8844fbc000) Any thoughts why the behavior would be different?
There's also a surprising difference in behavior between tm = localtime() and localtime_r(..., &tm).
The former is the "traditional" function which returns a pointer to a statically allocated, global "struct_tm". The latter is the thread-safe version receiving a pointer to a use-supplied "struct tm" as it's second argument.
: do {
: t = time(NULL);
: localtime_r(&t, &tm);
: printf("The time is now %02d:%02d:%02d.\n",
: tm.tm_hour, tm.tm_min, tm.tm_sec);
: sleep(1);
: } while(--N);
write(1, "The time is now 07:23:33.\n", 26The time is now 07:23:33. ) = 26
nanosleep({tv_sec=1, tv_nsec=0}, 0x7ffd9e798470) = 0
write(1, "The time is now 07:23:34.\n", 26The time is now 07:23:34. ) = 26
nanosleep({tv_sec=1, tv_nsec=0}, 0x7ffd9e798470) = 0
write(1, "The time is now 07:23:35.\n", 26The time is now 07:23:35. ) = 26
nanosleep({tv_sec=1, tv_nsec=0}, 0x7ffd9e798470) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2335, ...}) = 0
write(1, "The time is now 07:30:56.\n", 26The time is now 07:30:56.) = 26
nanosleep({tv_sec=1, tv_nsec=0}, 0x7ffc868c3010) = 0
Note, this is using glibc 2.24 under Arch.
$ /lib/libc.so.6
GNU C Library (GNU libc) stable release version 2.24, by Roland McGrath et al.
(...)
Compiled by GNU CC version 6.1.1 20160802.Re: How setting the TZ environment variable avoids thousands of system calls
#107Overhead of localtime() is well-known, just RTFM. Anyway, this article provides very good explanation.
Re: How setting the TZ environment variable avoids thousands of system calls
#108BTW, packagecloud.io is the great hosting for RPM/DEB packages. We've been using it for the last couple years. GitHub + Travis CI + PackageCloud combination allows us build and publish packages for EVERY git commit in 30+ repositories targeting 15 different Linux distributions [1]. There is no more need to hire a special devops guy for that.
Re: How setting the TZ environment variable avoids thousands of system calls
#109Earlier quoted context omitted.
That's not how I read TZSET(3). According to TZSET(3) you can either use (example for Copenhagen shown) TZ=CET or TZ=:Europe/Copenhagen but not TZ=Europe/Copenhagen
Apologies, I didn't explain my response very well. You're totally right to say that in the context of TZSET, to load the timezone specification from a TZ-formatted file it needs to be prefixed with ':'. Rather than saying "Technically, you should set the value of TZ to 'Europe/London'", I was trying to say that my philosophical opinion of timezone recording - whether in a CMS, on an O/S level, in a compiled app, etc…
Ok, we agree then.
Re: How setting the TZ environment variable avoids thousands of system calls
#110It would be highly inconvenient to have to set this variable if you live in a country where you change the timezone twice a year due to summertime.
Do all your work in UTC. Seriously, you'll be glad later. Convert to local time only on the edges, and only for end users.
All my VPSes run in UTC, no exception. What I'm talking about here is my desktop machine. I strongly prefer to run that in localtime.