Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python datetime pitfalls, and what libraries are (not) doing about it

dev.arie.bovenberg.net

Post reply on HN
Print view View original

61–70 of 150 posts

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#61

Post by jcranmer »

assbuttbuttass wrote: post #43

from whenever import ( # For the "UTC everywhere" case UTCDateTime, # Simple localization sans DST OffsetDateTime, # Full-featured IANA timezones ZonedDateTime, # The local system timezone LocalDateTime, # Detached from any timezones NaiveDateTime, ) Why so many different types? Doesn't ZonedDateTime cover all of the other use cases?

No.

First, there's a fundamental difference between naïve datetimes and timezone-aware datetimes. Try to combine them in one representation, and you are guaranteed to end up with a broken datetime library that spawns endless blog posts attacking it.

Second, there's a more subtle distinction between a datetime which has a known timezone and one which merely has a known instantaneous offset. A timezone is essentially a naïve date time -> tzoffset, albeit one whose values for future dates tends to be surprisingly unrobust. Among the offset-only date times, UTC is a very common case that deserves its own alias.

Finally, separating the local-tz datetime from other tz-aware datetimes can be very useful, for if the user changes their local timezone, then a local-tz datetime usually wants to switch the timezone it is using as well.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#62

Post by rini17 »

TrackerFF wrote: post #5

ISO 8601 is the only way to go. With that said, it's hard to control what systems others use. I frequently work with datasets that have 4-5 different timestamp formats. One of the most annoying bugs I've worked on, was in a table where almost everything followed a DDMMYYYY format - sans a 2-3 month period where the dates were flipped to MMDDYYYY

Okay but it does not standardize location. To reliably manage local time conversions, including location together with date/time is the only way.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#63

Post by jcranmer »

amluto wrote: post #60
jcranmer wrote: post #50

Earlier quoted context omitted.

Schedule an event to occur at 4pm every day, say in crontab.

Works great until you have an office in a different time zone. The whole concept of a computer belonging to a timezone works to some extent when that computer has a single human tenant, but it falls apart pretty quickly when the scope broadens.

If I'm scheduling a meeting, then yes, I am going to want to schedule the meeting using a specific time zone, which may or may not be my local time zone at that moment.

But if I'm scheduling when I want my computer to reboot, I want that to be a fixed in time zone in my local time zone. And if I subsequently change my local time zone, I want it to remain at that time in the new local time zone, not the old one.

Yes, local time zones don't always make sense. But that is a far cry from saying they never make sense!

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#64

Post by targafarian »

Kwpolska wrote: post #45
frumiousirc wrote: post #19

Best code strategy here is twofold. 1. Maximize the code context where date/time is represented as a simple count of Unix epoch seconds. 2. Never pass calendar/clock representations through API methods. 3. Always pass a timezone to methods that require it. 4. Counting, like time, is hard. The only contexts I see where calendar/clock representation are needed are: - Accept calendar/clock info from the user or external…

I wouldn't use an int. Some things use milliseconds since 1970 instead of seconds to get sub-second precision without involving double, and you need to read the docs to understand what is expected. Or you can just pass a DateTime object standard for the language if running in a single binary, or something like ISO 8601 when there's a network in between. Databases can often store DateTime objects sanely.

64 bit unsigned integer nanoseconds gets you out to 584 years (that's the year 2554 if you're using the Unix epoch). That's good enough for me to use universally for passing times around in the internals of my code. User input and output are going to and from that representation.

Half as many, of course, if you use a signed integer. If you don't need nanoseconds, then use microseconds and you get 292 thousand years to work with.

Integers are just a bit easier than floats for timestamps in my experience (e.g., comparing floats to one another is fraught and you'll be fighting this at every turn in your code).

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#65

Post by demondemidi »

Excellent research. Clearly explains the problem, shows how the industry addresses it, and provides a solution. You’re hired!

And I never want to write time code ever again. I wonder how many DST bugs I have in the wild after 40 years of coding…

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#66

Post by bongodongobob »

amluto wrote: post #34

> # The local system timezone > LocalDateTime, Here’s my feedback: one should essentially never use “local” time. I would even argue that the existence of systemwide local time is a historical mistake. Why would you ever want to use the “local” timezone of the system on which you are running? If the user is literally sitting in front of a monitor plugged in to the machine, it might be appropriate (but it should be a…

Settle down. Most people aren't running or accessing things globally.

Local time is perfectly acceptable for a list of obvious reasons.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#67

Post by demondemidi »

Kwpolska wrote: post #39
kroolik wrote: post #21

Earlier quoted context omitted.

What if the meeting is between people in EU and USA, where DST transitions occur at different moments?

Calendars like Outlook let you specify a timezone for the event. If it's set to Europe/Warsaw, the US goes out of sync for a few weeks. If America/New_York, the EU does. This still isn't a problem for a meeting between LA and NY, for example (unless the meeting is in the middle of the night).

Yep! I hate those few weeks between PST and CET.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#68

Post by demondemidi »

airstrike wrote: post #35
kroolik wrote: post #21

Earlier quoted context omitted.

What if the meeting is between people in EU and USA, where DST transitions occur at different moments?

Tell those people not to meet at 2am on the night DST changes

All the meetings are hosed for about two weeks. This is a known issue for anyone working with time zones that don’t change together.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#69

Post by rhelz »

Really, there is no good technical solution for this, because in part, it is not even a technical problem. The laws about timezones, etc, very from country to country, from year to year etc. And some parts of the world are claimed by two or more different countries, each one of which has an opinion about what time it should be there.

One might say "well lets skip all this political b.s. and do everything in UTC" but really, viewing time as running according to a single universal clock is a relic of Newtonian Physics. There is no single, global answer to what time it is.

And we are getting to the point where special relativity is kicking in. High-frequency trading is already in the 50-microsecond range. Not so far off from the relativistic time dilations experienced even in flying in a jet airplane: flying around the world eastward results in about a 180 ns slowdown, flying around the world westward give about an 80 ns speedup.

Even concepts like simultaneity become relative at the speeds which we will soon be computing at. Imagine two corporate raiders flying in their private jets, biding on the same block of controlling stock: who bought it first would be different depending upon which frame of reference you were in.

If you have to write programs for when time really matters (as I did writing stock-trading software) my advice is to just accept the realities, and glory in the complexities. Being able to write software which can keep track of time, and obey all the rules and regulations of various exchanges and countries, is a competitive advantage for you.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#70

Post by make3 »

bongodongobob wrote: post #66
amluto wrote: post #34

> # The local system timezone > LocalDateTime, Here’s my feedback: one should essentially never use “local” time. I would even argue that the existence of systemwide local time is a historical mistake. Why would you ever want to use the “local” timezone of the system on which you are running? If the user is literally sitting in front of a monitor plugged in to the machine, it might be appropriate (but it should be a…

Settle down. Most people aren't running or accessing things globally. Local time is perfectly acceptable for a list of obvious reasons.

... a huge fraction of Python code runs on servers everywhere
Post reply on HN