Earlier quoted context omitted.
Unfortunately timezones have the nasty ability to change. See: https://codeblog.jonskeet.uk/2019/03/27/storing-utc-is-not-a...
That's why they wrote "state the actual timezone" and not "state the actual UTC offset." Actual timezone would mean "2025-01-01 16:00:00 Europe/Berlin," not just "2025-01-01 16:00:00 +01:00."
Python datetime pitfalls, and what libraries are (not) doing about it
121–130 of 150 posts
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#122I will state this briefly as a tangent. This article mentions third party libraries; the top are Arrow and Pendulum. These are a wellspring of subtle bugs due to a decision they made that I suspect is based off the Javascript library Moment: They conflate Dates, Times, and Datetimes, as a single type. If you are using their Arrow etc type to represent a Date or Time, the resulting code will be fragile, and fail in su…
I actually did start out basically porting Chrono to Python, but decided against it for two reasons:
1. While I really liked handling local/UTC conversions with the "LocalResult" enum in Rust, it translated poorly to Python. Somewhere along the line you'd need to transition from Rust's "errors as values" to Python's exception-oriented world. I actually wish Python would go Rust's way, but I'm not going to swim against the current here...
2. Most modern libraries in other languages choose to disambiguate local datetimes. I felt it was safer to go the road more traveled here.
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#123Earlier quoted context omitted.
Author here. "Offset" is indeed relative to UTC. The problem with allowing adding timedeltas is that you give users the impression they are doing valid arithmetic, while they may not be. Example: You receive a timestamp of an event in Paris: 2024-03-31 01:00:00+01:00. If you allow addition, you may be tempted to think you can "just add three hours" and get 04:00:00+01:00. However, because Paris has a DST transition,…
> If you allow addition, you may be tempted to think you can "just add three hours" and get 04:00:00+01:00. However, because Paris has a DST transition, you'd actually want to have 05:00:00+02:00 Since you know the offset then adding three hours should be unambiguous no? Convert to UTC, add three hours, convert back to target timezone, which would net you the correct 05:00:00+02:00.
What if you don't even know the target offset?
You may happen to know it for an event three hours in the future, but what if you're adding three months? With that time frame, the relevant laws even may not have been passed yet.
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#124I don't think any of the DST complaints are legitimate. If doing work involved with timestamps, not keeping everything in GMT is questionable. GMT is the UTF-8 of timestamps: the low common-denominator rules. The datetime library is mechanism. DST is policy. Policy is wildly variable and your system library would be unmaintainable if it had to chase policy. Just say no.
DST and local time are relevant in calculations when humans are involved. A user in Japan shouldn't be told it's still Sunday just because it's still Sunday somewhere in western Europe. If I set up a recurring meeting every Monday at 10:00, it shouldn't suddenly shift to 09:00 or 11:00 just because of a DST switchover.
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#125Earlier quoted context omitted.
This is the only sane policy. Timezone conversion must be delayed to the very last moment before displaying time to users. If I needed to define a daily alarm in a local wall clock time, I'd happily deal with a zoneless dateless hour-minute tuple and determine the alarm time by combining a date and a time using a timezone. Right after I'd convert that and handle the rest in UTC.
> This is the only sane policy. Timezone conversion must be delayed to the very last moment before displaying time to users. No, this only works for datetimes in the _past_. For future datetimes you often must store the datetime in the user's (IANA) timezone. Otherwise your application could be off. And your app could be off by more than one hour. In 2011 Samoa moved across the international date line. See https://ww…
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#126Earlier quoted context omitted.
No. First, there's a fundamental difference between naïve datetimes and timezone-aware datetimes. Try to combine them in one representation, and you are guaranteed to end up with a broken datetime library that spawns endless blog posts attacking it. Second, there's a more subtle distinction between a datetime which has a known timezone and one which merely has a known instantaneous offset. A timezone is essentially a…
IMO, any program using naive timestamps is probably doing something wrong. Same with fixed offsets (except UTC obviously). For local time zone, how often does the local time zone change, and you want to modify all timestamps in the system? That actually seems like a huge pitfall
Here's an example where you'd legitimately want to process and store a naive (wall clock) time object:
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#127Earlier quoted context omitted.
DST and local time are relevant in calculations when humans are involved. A user in Japan shouldn't be told it's still Sunday just because it's still Sunday somewhere in western Europe. If I set up a recurring meeting every Monday at 10:00, it shouldn't suddenly shift to 09:00 or 11:00 just because of a DST switchover.
I didn't say "irrelevant". I questioned keeping that in the library.
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#128Earlier quoted context omitted.
> As in, i want my alarm to go off at 4pm everyday regardless of whether i am traveling. I think this may be the exception that proves the rule. Sure, this is a valid, if specialized, semantic use case. But you only barely need a LocalDateTime type for this. If you stick that 4pm into a local database, you want to make sure that everything, including the database, knows you’re storing hours, minutes, and seconds rela…
> So IMO this is more of an UnspecifiedZoneTime than a LocalDateTime. (Also, it has no date portion. It very well could also have a date portion. E.g. "I want this alarm/cleanup/whatever to happen on 9am on the 1st Monday of each month" based on the timezone the user happens to be at that time - i.e. not tied to any specific timezone.
And a recurrence library probably wants to avoid libc-style “localtime” here — plumbing the concept of users in places to it seems awkward. A genuinely naive time may be better — the interface could be “calculate the next recurrence of this recurring event in this timezone,” not “here’s a recurring event with associated ‘local’ time — calculate the next occurrence given my /etc/localtime and TZ environment variable”.
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#129Earlier quoted context omitted.
I wouldn't consider 'not handling leap seconds' as a pitfall. If you need to account for leap seconds, you're probably not going to use a general-purpose datetime library. Unix time ignores leap seconds, so does RFC 5545 (iCal), and most modern libraries (Chrono, Temporal, NodaTime).
You need to account for leap seconds literally any time you are mixing durations/timedeltas with instants/datetimes, otherwise you will get wrong answer. Also needed whenever you parse ISO8601/RFC3339 style strings. For example this (correct) timestamp fails to parse: >>> UTCDateTime.from_rfc3339('1998-12-31T23:59:60Z') Traceback (most recent call last): File " ", line 1, in File "/tmp/tmp.4kBsq6IYyF/venv/lib/python3…
Re: Python datetime pitfalls, and what libraries are (not) doing about it
#130Earlier quoted context omitted.
>That said, you should really use UTC. No, that "store UTC everywhere" is often repeated but it's incorrect advice when applied to future human-constructed datetimes such as appointments or social events. Future "human-interpreted wall-clock" datetimes cannot be unambiguously stored as future UTC values with perfect roundtrip fidelity. I've tried to explain the difference in previous comment: https://news.ycombinator…
That is a good point, but fortunately a rare case. The other problem is that when you are using human inputs such as appointments you may not know which is appropriate. If I arrange a meeting with someone in another timezone, and there is such a change, how can a system know whether I want the UTC time, my timezone, or the other person's timezone. I am not sure there is a good way to do this in many cases. As you say…
A decent system will allow the user to make that choice on a case-by-case basis but assume the common case as a default. I've seen apps pull that off successfully, e.g. iCal.