Live data from Hacker News

Python datetime pitfalls, and what libraries are (not) doing about it

dev.arie.bovenberg.net

141–150 of 150 posts

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#141

Earlier quoted context omitted.

If the library isn't supposed to do all the heavy lifting, then who is?

Client code between the specific use-case and the general library is how this is typically done.

So you actually agree that keeping the policy data in a library is a good thing, you're just saying that it should be a separate library?

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#142

The author provides this example to illustrate inconsistent handling of non-existent datetimes: # This time doesn't exist on this date d = datetime(2023, 3, 26, 2, 30, tzinfo=paris) # No timestamp exists, so it just makes one up t = d.timestamp() datetime.fromtimestamp(t) == d # False Criticisms: 1) The example would fail just as well for any datetime with given tzinfo. Because fromtimestamp returns native datetimes.…

Author of the blog post here— 1) You're absolutely right, I'll fix this mistake in the example. 2) "Made up" was perhaps stirring the pot too much ;), but the fact remains that a timestamp is created when one essentially doesn't exist. That this is meticulously documented in PEP495 doesn't change this fact. Note that PEP495 also explicitly describes some of the other pitfalls. Documenting a 'suprise' is not the same…

I must add that I do appreciate you challenging the status quo. Excited to see where your library leads.

> modern datetime libraries … in other languages choose to not represent these 'missing' local times.

Another edge case with throwing errors over gap times is eventually a region’s dst policies may change. So what used to be valid can now be invalid (and vice versa). Updating the library must be done with an audit of existing data, or suffer loud unexpected failures.

Thanks for taking the time to address this critic. Easier to be a critic than a creator after all.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#143

Best code strategy here is twofold. 1. Maximize the code context where date/time is represented as a simple count of Unix epoch seconds. 2. Never pass calendar/clock representations through API methods. 3. Always pass a timezone to methods that require it. 4. Counting, like time, is hard. The only contexts I see where calendar/clock representation are needed are: - Accept calendar/clock info from the user or external…

The problem with Unix timestamps is that it doesn't take leap seconds into account, which makes it ambiguous during a leap second and makes it messy to compute the duration between two timestamps. I think you're on the right track but a timestamp should only count the actual number of seconds that have passed, not some half-measure that tries to align with the rotation of the earth.

Ah, thanks. I didn't know that Unix time is discontinuous over leap seconds! I had thought just the opposite. And now I understand some system failures I know about that occurred during leap seconds but which I've never understood.

I think to handle leap seconds, one must carry around both Unix timestamp and a "leap delta". The timestamp is needed to convert to calendar/clock representation and the "leap delta" is needed to compare two Unix timestamps for actual elapsed time.

What a headache!

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#144
post #27
post #8

Earlier quoted context omitted.

Does ISO 8601 fix any of the problems mentioned in the article? It doesn't really seem relevant to me.

> doesn't really seem relevant to me Isn't this overly 'you' centric? Wouldn't this require access to your inner monologue? Doesn't seem like a referentially invariant supporting argument, to _me_.

Sorry, I phrased it confusingly. I should have just said "it doesn't seem relevant".

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#145

Earlier quoted context omitted.

Charitable interpretation of previous comment would be “I think it does not seem relevant to problems specified in the article”. Iso 8601 is about date formatting, while article is about a bit orthogonal to that. (But I agree/hope that ISO 8601 is the future that is less ambiguous than current state)

> I think it does not seem relevant to problems specified in the article Is a low effort way to make someone form a rebuttal in response to no argument. It is pseudo intellectual way to sound smart while offering nothing in return. As if that persons opinion has weight because “they don’t see the relevance”, I know lots of people that “don’t see relevance”. Is our responsibility to educate their feigned confusion?

I was saying that TrackerFF's post seemed to me to be unrelated to the article, so I was confused why it was posted. I'm sure other HN readers were confused by the comment as well. I was asking for clarification. TrackerFF or someone else could then clarify the relevance, which I and other HN readers would find useful.

My confusion wasn't feigned. I'm actually confused how the comment was relevant to the article. Someone being confused and asking for clarification is a common way that conversations progress and learning happens. It's no one's responsibility to comment on HN. If someone wants to discuss this, that person can reply.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#146
post #102

Earlier quoted context omitted.

What does TFA mean here? Trifluoroacetic Acid?

The Fucking/Fine/... Article

For educationary purposes: TFA=The Fucking Article

(I’m voting for not being afraid of curse words; if they are relevant)

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#147

Earlier quoted context omitted.

Charitable interpretation of previous comment would be “I think it does not seem relevant to problems specified in the article”. Iso 8601 is about date formatting, while article is about a bit orthogonal to that. (But I agree/hope that ISO 8601 is the future that is less ambiguous than current state)

> I think it does not seem relevant to problems specified in the article Is a low effort way to make someone form a rebuttal in response to no argument. It is pseudo intellectual way to sound smart while offering nothing in return. As if that persons opinion has weight because “they don’t see the relevance”, I know lots of people that “don’t see relevance”. Is our responsibility to educate their feigned confusion?

Mist likely it was: It’s a low effort proxy for “this comment is most probably written only after only reading the title”

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#148

Earlier quoted context omitted.

Author of the blog post here— 1) You're absolutely right, I'll fix this mistake in the example. 2) "Made up" was perhaps stirring the pot too much ;), but the fact remains that a timestamp is created when one essentially doesn't exist. That this is meticulously documented in PEP495 doesn't change this fact. Note that PEP495 also explicitly describes some of the other pitfalls. Documenting a 'suprise' is not the same…

I must add that I do appreciate you challenging the status quo. Excited to see where your library leads. > modern datetime libraries … in other languages choose to not represent these 'missing' local times. Another edge case with throwing errors over gap times is eventually a region’s dst policies may change. So what used to be valid can now be invalid (and vice versa). Updating the library must be done with an audit…

Indeed, structural changes to timezones also need to be handled. Temporal has a good approach here https://tc39.es/proposal-temporal/docs/ambiguity.html#ambigu....

Basically:

- Store the offset and the tz ID

- When deserializing, allow explicit choice what to do: keep the offset (i.e. same moment in time) or keep the local time (i.e. same time on the 'wall clock')

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#149
post #129
post #113

Earlier quoted context omitted.

You need to account for leap seconds literally any time you are mixing durations/timedeltas with instants/datetimes, otherwise you will get wrong answer. Also needed whenever you parse ISO8601/RFC3339 style strings. For example this (correct) timestamp fails to parse: >>> UTCDateTime.from_rfc3339('1998-12-31T23:59:60Z') Traceback (most recent call last): File " ", line 1, in File "/tmp/tmp.4kBsq6IYyF/venv/lib/python3…

I do point out that if your internal representation would be simple naive epoch+duration(/timedelta) then you don't need any special accounting for leap seconds when doing time arithmetic like this. The problem arises only if you are trying to be somehow clever with your internal representation (and failing).

You are correct that there are improvements to be made. The current internal representation is—for the moment—simply the Python standard library.

Probably the approach of Chrono is best: allow leap seconds to be represented (i.e. 23:59:60), but not to account for them historically.

Re: Python datetime pitfalls, and what libraries are (not) doing about it

#150
post #46

Earlier quoted context omitted.

You start by figuring out what you’re even trying to do. For example, I read your comment three times, and I don’t know what you mean by “4 pm local time” . I don’t even have what I would consider to be a credible guess.

1. Contract says that goods must be delivered at 4pm. 2. Government suddenly changes the law and moves the offset of whole country. 3. Contract is still valid, goods have to be delivered still at 4pm, but with new offset (so at different utc time). (I guess timezoned time would still be better than local time, but UTC would not work here)

1. Contract says that goods must be delivered at 4pm UTC

Problem solved, unless the government changes the law too little before 4pm UTC. But handling this case in software is impractical.

Post reply on HN