Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python datetime pitfalls, and what libraries are (not) doing about it

dev.arie.bovenberg.net

Post reply on HN
Print view View original

91–100 of 150 posts

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#91

Post by Kwpolska »

zokier wrote: post #75
Kwpolska wrote: post #45

Earlier quoted context omitted.

I wouldn't use an int. Some things use milliseconds since 1970 instead of seconds to get sub-second precision without involving double, and you need to read the docs to understand what is expected. Or you can just pass a DateTime object standard for the language if running in a single binary, or something like ISO 8601 when there's a network in between. Databases can often store DateTime objects sanely.

> Some things use milliseconds since 1970 instead of seconds to get sub-second precision without involving double Or be like JS which uses both floats and milliseconds as its date format. That to me feels like taking worst parts of every option. But then, making fun of JS is like shooting fish in the barrel

(This is a side effect of JS not having a real integer type.)

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#92

Post by macintux »

autarch wrote: post #12
praash wrote: post #10

Earlier quoted context omitted.

A true observation, especially since many European countries (or just some citizens) wish to get rid of DST. However, ±1 hour is not much of an error when talking about a time delay of at least a year.

> However, ±1 hour is not much of an error when talking about a time delay of at least a year. It is if this a doctor's appointment or a meeting with a friend or ... you get my point.

I missed most of my last chance to see my cousin play college basketball because I trusted her college’s website, and it was off by an hour due to timezone confusion.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#93

Post by Kwpolska »

targafarian wrote: post #64
Kwpolska wrote: post #45

Earlier quoted context omitted.

I wouldn't use an int. Some things use milliseconds since 1970 instead of seconds to get sub-second precision without involving double, and you need to read the docs to understand what is expected. Or you can just pass a DateTime object standard for the language if running in a single binary, or something like ISO 8601 when there's a network in between. Databases can often store DateTime objects sanely.

64 bit unsigned integer nanoseconds gets you out to 584 years (that's the year 2554 if you're using the Unix epoch). That's good enough for me to use universally for passing times around in the internals of my code. User input and output are going to and from that representation. Half as many, of course, if you use a signed integer. If you don't need nanoseconds, then use microseconds and you get 292 thousand years t…

You have standardized on int64 = nanoseconds. Libraries you use might have standardized on int64 = milliseconds, int64 = seconds, double = seconds, or the preferred DateTime class/struct of your programming language — even the C standard library has `struct tm` [0].

If you’ve wrapped your int64 in some struct/class/type-alias-without-automatic-downcasting, it might be fine. But if you haven’t, you might end up mixing the different scales, or littering the code with pointless conversions to and from the standard DateTime class/struct.

[0] https://www.gnu.org/software/libc/manual/html_node/Broken_00...

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#94

Post by [deleted] »

targafarian wrote: post #64
Kwpolska wrote: post #45

Earlier quoted context omitted.

I wouldn't use an int. Some things use milliseconds since 1970 instead of seconds to get sub-second precision without involving double, and you need to read the docs to understand what is expected. Or you can just pass a DateTime object standard for the language if running in a single binary, or something like ISO 8601 when there's a network in between. Databases can often store DateTime objects sanely.

64 bit unsigned integer nanoseconds gets you out to 584 years (that's the year 2554 if you're using the Unix epoch). That's good enough for me to use universally for passing times around in the internals of my code. User input and output are going to and from that representation. Half as many, of course, if you use a signed integer. If you don't need nanoseconds, then use microseconds and you get 292 thousand years t…

[deleted]

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#95

Post by falserum »

amluto wrote: post #88
jcranmer wrote: post #63

Earlier quoted context omitted.

If I'm scheduling a meeting, then yes, I am going to want to schedule the meeting using a specific time zone, which may or may not be my local time zone at that moment. But if I'm scheduling when I want my computer to reboot, I want that to be a fixed in time zone in my local time zone. And if I subsequently change my local time zone, I want it to remain at that time in the new local time zone, not the old one. Yes,…

I don’t know about you, but I have computers that aren’t in the same time zone I’m it. I probably want them to reboot at night where I am, not where they are. But yes, “reboot a laptop at 2am” is in the general category of alarm clock use cases, where a time (but not a date!) with a separately determined time zone makes sense. The OP mentioned LocalDateTime, which doesn’t make sense for this use case.

Instant in history -> GMT+0/UTC

Future shedule for humans to coordinate (contracts, meetings) -> ZonedDateTime

Personal alarm that should be kept in sync with the sun (e.g. go to sleep) -> LocalDateTime

What’s good example/purpose for OffsetDateTime?

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#96

Post by ariebovenberg »

guitarnick wrote: post #31

The author provides this example to illustrate inconsistent handling of non-existent datetimes: # This time doesn't exist on this date d = datetime(2023, 3, 26, 2, 30, tzinfo=paris) # No timestamp exists, so it just makes one up t = d.timestamp() datetime.fromtimestamp(t) == d # False Criticisms: 1) The example would fail just as well for any datetime with given tzinfo. Because fromtimestamp returns native datetimes.…

Author of the blog post here—

1) You're absolutely right, I'll fix this mistake in the example.

2) "Made up" was perhaps stirring the pot too much ;), but the fact remains that a timestamp is created when one essentially doesn't exist. That this is meticulously documented in PEP495 doesn't change this fact.

Note that PEP495 also explicitly describes some of the other pitfalls. Documenting a 'suprise' is not the same as eliminating it.

> Finally if we disallow creating non-existent datetimes in the proposed library, how do we represent the 2am in "clock changes forwards at 2am"? Use 3am? There are tradeoffs.

Indeed, there are always tradeoffs. Often, only time can tell what was the 'least bad' decision. It is telling though that modern datetime libraries (Noda Time, Chrono, Temporal) in other languages choose to not represent these 'missing' local times.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#97

Post by DandyDev »

zokier wrote: post #86
zokier wrote: post #85

Earlier quoted context omitted.

java.time doesn't handle leap seconds very well :( jshell> ChronoUnit.SECONDS.between(Instant.parse("1974-01-01T00:00:00Z"), Instant.parse("2024-01-01T00:00:00Z")) % 86400 $22 ==> 0 astropy gets you the correct answer: >>> from astropy.time import Time >>> (Time('2024-01-01T00:00:00Z') - Time('1974-01-01T00:00:00Z')).sec % 86400 24.0 of course astropy is bit specialized and not really something I'd use as general pur…

I tested this with the library from TFA, and it fails too :( >>> from whenever import UTCDateTime >>> (UTCDateTime(2024, 1, 1) - UTCDateTime(1974, 1, 1)).total_seconds() % 86400 0.0

What does TFA mean here? Trifluoroacetic Acid?

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#98

Post by ariebovenberg »

svilen_dobrev wrote: post #71

what special font/characters is this article using? in that scorecard table i got all same-looking boxes. yes/no Y/N T/F x/o would have been much more readable. And also in all bullet points also see boxes..

Author of the post here—

They are emoji. I've added plain text to the table now, so it is readable without them.

edit: I've used a workaround to display emoji as images now

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#99

Post by aimor »

I don't really understand why OffsetDateTime doesn't allow adding time deltas (reason provided is that it might have it's offset change relative to UTC). Is the offset supposed to be relative to UTC or is it supposed to be relative to ANY TimeZone (including those with time changes)? I thought it was relative to UTC, but even the other way I just don't understand why that's a problem. I might be making some assumptions about the behavior I expect.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#100

Post by MereInterest »

amluto wrote: post #81
MereInterest wrote: post #51

Earlier quoted context omitted.

Because the local time is the thing that you have immediate access to, with no additional conversions applied. Every conversion requires an external source of information. Converting from local time to time zone requires measuring the system clock drift. Converting from time zone to UTC requires knowing which time zone you are in. Converting from UNIX timestamp to UTC requires measurements of the earth’s rotation to…

> Because the local time is the thing that you have immediate access to, with no additional conversions applied. Only in the sense that BIOS systems historically programmed non-DST-aware local time into the RTC. Other than that, you actually have no-conversion-needed access to UTC or something UTC-like. Linux mostly tracks the conversion from the hardware clock (TSC, for example) to UTC. NTP gives UTC, and GPS gives…

> Only in the sense that BIOS systems historically programmed non-DST-aware local time into the RTC

I would love to hear proposals on how, at a hardware level, you would track Daylight Saving Time. Not just how a Daylight Saving Time correction would be applied, but also how the hardware would receive updates to dates at which DST is observed, within which geographic borders, and how to handle conflicting updates for locations where the geographic borders are contested.

> you actually have no-conversion-needed access to UTC or something UTC-like

No, you don't. You may have access to an API that has already applied conversions when by the time a result it returned to you, but that is significantly different than having direct access to a universal clock. (And assumes that such a universal clock even exists.)

Your clock is a hardware-based oscillator, with a counter for the integer number of oscillations that has occurred. You do not have a UTC clock. You have an experimentally-determined and infrequently-updated conversion between your clock and the UTC clock.

> NTP gives UTC

No, it doesn't. NTP gives an approximation to UTC, and is only available when you have a network connection available. The accuracy of that approximation depends on quality of that network connection, and whether it has asymmetric delays.

> GPS gives something that is a lot closer to UTC than to local time

While true, this is only available for devices that have GPS capabilities, in a location without sharp elevation changes that would delay GPS signals.

Everything has a conversion factor. Everything. If you record the original measurement and the conversion factor used, you can recover from errors in the conversion. If you only record the measurement after conversion factors have been applied, you cannot.

Post reply on HN