Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python datetime pitfalls, and what libraries are (not) doing about it

dev.arie.bovenberg.net

Post reply on HN
Print view View original

111–120 of 150 posts

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#111

Post by coldtea »

amluto wrote: post #34

> # The local system timezone > LocalDateTime, Here’s my feedback: one should essentially never use “local” time. I would even argue that the existence of systemwide local time is a historical mistake. Why would you ever want to use the “local” timezone of the system on which you are running? If the user is literally sitting in front of a monitor plugged in to the machine, it might be appropriate (but it should be a…

>But if the user is using X forwarding or ssh or a remote desktop, you have no idea what timezone the user is in.

Perhaps I don't care, and I want to present things from the point of view of the server's location?

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#112

Post by coldtea »

amluto wrote: post #87
joshribakoff wrote: post #80

Earlier quoted context omitted.

> don’t know what you mean by “4 pm local time” It means “whenever it is 4pm wherever you are”. As in, i want my alarm to go off at 4pm everyday regardless of whether i am traveling. Python calls it a “naive” time, it has no timezone or offset. Its just a data structure that says “4pm” rather than “4pm eastern time” An example ive used is when a user had to specify schedules for a digital menu board system. When the…

> As in, i want my alarm to go off at 4pm everyday regardless of whether i am traveling. I think this may be the exception that proves the rule. Sure, this is a valid, if specialized, semantic use case. But you only barely need a LocalDateTime type for this. If you stick that 4pm into a local database, you want to make sure that everything, including the database, knows you’re storing hours, minutes, and seconds rela…

>So IMO this is more of an UnspecifiedZoneTime than a LocalDateTime. (Also, it has no date portion.

It very well could also have a date portion.

E.g. "I want this alarm/cleanup/whatever to happen on 9am on the 1st Monday of each month" based on the timezone the user happens to be at that time - i.e. not tied to any specific timezone.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#113

Post by zokier »

ariebovenberg wrote: post #108
zokier wrote: post #85

Earlier quoted context omitted.

java.time doesn't handle leap seconds very well :( jshell> ChronoUnit.SECONDS.between(Instant.parse("1974-01-01T00:00:00Z"), Instant.parse("2024-01-01T00:00:00Z")) % 86400 $22 ==> 0 astropy gets you the correct answer: >>> from astropy.time import Time >>> (Time('2024-01-01T00:00:00Z') - Time('1974-01-01T00:00:00Z')).sec % 86400 24.0 of course astropy is bit specialized and not really something I'd use as general pur…

I wouldn't consider 'not handling leap seconds' as a pitfall. If you need to account for leap seconds, you're probably not going to use a general-purpose datetime library. Unix time ignores leap seconds, so does RFC 5545 (iCal), and most modern libraries (Chrono, Temporal, NodaTime).

You need to account for leap seconds literally any time you are mixing durations/timedeltas with instants/datetimes, otherwise you will get wrong answer. Also needed whenever you parse ISO8601/RFC3339 style strings. For example this (correct) timestamp fails to parse:

    >>> UTCDateTime.from_rfc3339('1998-12-31T23:59:60Z')
    Traceback (most recent call last):
      File "", line 1, in 
      File "/tmp/tmp.4kBsq6IYyF/venv/lib/python3.11/site-packages/whenever/__init__.py", line 797, in from_rfc3339
        return cls._from_py_unchecked(_parse_utc_rfc3339(s))
                                      ^^^^^^^^^^^^^^^^^^^^^
      File "/tmp/tmp.4kBsq6IYyF/venv/lib/python3.11/site-packages/whenever/__init__.py", line 2420, in _parse_utc_rfc3339
        return _fromisoformat(s.upper())
               ^^^^^^^^^^^^^^^^^^^^^^^^^
    ValueError: second must be in 0..59

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#114

Post by ahartmetz »

jncfhnb wrote: post #37
amluto wrote: post #34

> # The local system timezone > LocalDateTime, Here’s my feedback: one should essentially never use “local” time. I would even argue that the existence of systemwide local time is a historical mistake. Why would you ever want to use the “local” timezone of the system on which you are running? If the user is literally sitting in front of a monitor plugged in to the machine, it might be appropriate (but it should be a…

Imagine having the audacity to run software on the machine in front of you

You shouldn't do that, it's bad for some business models.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#115

Post by dwattttt »

ariebovenberg wrote: post #96
guitarnick wrote: post #31

The author provides this example to illustrate inconsistent handling of non-existent datetimes: # This time doesn't exist on this date d = datetime(2023, 3, 26, 2, 30, tzinfo=paris) # No timestamp exists, so it just makes one up t = d.timestamp() datetime.fromtimestamp(t) == d # False Criticisms: 1) The example would fail just as well for any datetime with given tzinfo. Because fromtimestamp returns native datetimes.…

Author of the blog post here— 1) You're absolutely right, I'll fix this mistake in the example. 2) "Made up" was perhaps stirring the pot too much ;), but the fact remains that a timestamp is created when one essentially doesn't exist. That this is meticulously documented in PEP495 doesn't change this fact. Note that PEP495 also explicitly describes some of the other pitfalls. Documenting a 'suprise' is not the same…

> Documenting a 'surprise' is not the same as eliminating it.

I'm going to have to remember that one

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#116

Post by CJefferson »

adrian_b wrote: post #56
CJefferson wrote: post #40

Earlier quoted context omitted.

What is your alternative? I suspect 99.999% of computer and phone users want the time of their local device. Do you want them all to have to explicitly set their time zone?

Since the appearance of computer networks, all operating systems require during installation an explicit setting of the time zone. This includes Windows and any operating system that chooses to keep the time as local time, because even those need to be able to convert the local time to other time zones. The users who buy a computer with a preinstalled operating system may not see this, but someone has explicitly set…

Windows 11 does not require setting timezone. I just did it in a VM to check I wasn't confused. Neither do android or iPhones.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#117

Post by Hackbraten »

lijok wrote: post #84
amluto wrote: post #57

Earlier quoted context omitted.

> Saying "local time" implies all of the above. It means that something is being specified according to the local convention of timekeeping. No, it really doesn’t. Saying “4pm at the coffee shop ” means that, and this is a big distinction. If I’m in a different time zone two days before the meeting, looking at my calendar, I sure hope my calendar understands the difference. If I move my entire home outside the timezo…

Unfortunately timezones have the nasty ability to change. See: https://codeblog.jonskeet.uk/2019/03/27/storing-utc-is-not-a...

That's why they wrote "state the actual timezone" and not "state the actual UTC offset."

Actual timezone would mean "2025-01-01 16:00:00 Europe/Berlin," not just "2025-01-01 16:00:00 +01:00."

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#118

Post by ariebovenberg »

zokier wrote: post #86
zokier wrote: post #85

Earlier quoted context omitted.

java.time doesn't handle leap seconds very well :( jshell> ChronoUnit.SECONDS.between(Instant.parse("1974-01-01T00:00:00Z"), Instant.parse("2024-01-01T00:00:00Z")) % 86400 $22 ==> 0 astropy gets you the correct answer: >>> from astropy.time import Time >>> (Time('2024-01-01T00:00:00Z') - Time('1974-01-01T00:00:00Z')).sec % 86400 24.0 of course astropy is bit specialized and not really something I'd use as general pur…

I tested this with the library from TFA, and it fails too :( >>> from whenever import UTCDateTime >>> (UTCDateTime(2024, 1, 1) - UTCDateTime(1974, 1, 1)).total_seconds() % 86400 0.0

Author here—

This is a conscious decision, I'll explicitly address this in the FAQ soon.

As I commented elsewhere, this is completely in line with industry standards (iCal, Unix time) and other modern libraries. I don't (yet) see any reason to deviate.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#119

Post by o11c »

MereInterest wrote: post #100
amluto wrote: post #81

Earlier quoted context omitted.

> Because the local time is the thing that you have immediate access to, with no additional conversions applied. Only in the sense that BIOS systems historically programmed non-DST-aware local time into the RTC. Other than that, you actually have no-conversion-needed access to UTC or something UTC-like. Linux mostly tracks the conversion from the hardware clock (TSC, for example) to UTC. NTP gives UTC, and GPS gives…

> Only in the sense that BIOS systems historically programmed non-DST-aware local time into the RTC I would love to hear proposals on how, at a hardware level, you would track Daylight Saving Time. Not just how a Daylight Saving Time correction would be applied, but also how the hardware would receive updates to dates at which DST is observed, within which geographic borders, and how to handle conflicting updates for…

>I would love to hear proposals on how, at a hardware level, you would track Daylight Saving Time.

Traditionally the BIOS ignores it, and Windows on reboot detects if the clock needs to be adjusted according to the latest rules that it knows about. This failed quite often in practice.

Linux systems using tzdata are much more reliable (Windows is still very buggy with historical data), but UTC time support on non-Linux systems was flaky for a long time, leading to bad API design in languages historically.

`timegm` despite being a critical and irreplaceable function is still not in POSIX, for example (though it is widely supported in practice)

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#120

Post by Hackbraten »

ariebovenberg wrote: post #96
guitarnick wrote: post #31

The author provides this example to illustrate inconsistent handling of non-existent datetimes: # This time doesn't exist on this date d = datetime(2023, 3, 26, 2, 30, tzinfo=paris) # No timestamp exists, so it just makes one up t = d.timestamp() datetime.fromtimestamp(t) == d # False Criticisms: 1) The example would fail just as well for any datetime with given tzinfo. Because fromtimestamp returns native datetimes.…

Author of the blog post here— 1) You're absolutely right, I'll fix this mistake in the example. 2) "Made up" was perhaps stirring the pot too much ;), but the fact remains that a timestamp is created when one essentially doesn't exist. That this is meticulously documented in PEP495 doesn't change this fact. Note that PEP495 also explicitly describes some of the other pitfalls. Documenting a 'suprise' is not the same…

> only time can tell

I see what you did there.

Post reply on HN