Live data from Hacker News

Some DNS lookups causing 5xx errors due to leap second bug

cloudflarestatus.com

131–135 of 135 posts

Re: Some DNS lookups causing 5xx errors due to leap second bug

#131
post #95

Earlier quoted context omitted.

Sometimes it's best to store a scheduled events as "Event localtime" and "Timezone" (where timezone is a named description - e.g. "Europe/Madrid" - rather than an offset - e.g. "+1:00"). This allows the record to stay consistent, even if there are changes to the local time rules - e.g. leap seconds, daylight savings, timezone offset. Imagine a tech-camp had been planned in Cairo, Egypt, to start on 9am on July 10, 20…

Yup, I'm aware of this and should have mentioned it in my comment. Thanks for the follow up. As an aside, how often do the tz databases for each language get released? Are they usually responsive to notices 3 days out? Edit: I went looking into the pytz release for the Cairo example from parent. Olson Timezone Database: Release 2016f - 2016-07-05 16:26:51 +0200 https://github.com/stub42/pytz/commit/03a4e9b31dd90f3dac…

I just came across a note about Morocco, which entered daylight savings time in March 2016, but then left daylight-savings in June for 35 days, re-starting daylight-savings in July [1].

I've read that the explanation for this temporary suspension of daylight-savings is Ramadan [2], and Ramadan is dependent on the observed sighting of the new moon - so you can't necessarily predict the date in advance.

I ended up coming across that after looking for an explanation for something bizarre I experienced on a trip to Morocco in March 2016…with my iPhone set to use "Marrakesh, Morocco", the time on the phone displayed correctly, but the time on my sync'd Apple watch was an hour out. I think I ended up manually setting it to Paris time to get the correct time, but never did get an explanation for the difference.

So even across two devices from the same manufacturer, theoretically sharing the same date-time information, they can be inconsistent.

Conclusion: time is hard!

[1] https://www.timeanddate.com/time/change/morocco/tanger?year=...

[2] http://codeofmatt.com/2016/04/23/on-the-timing-of-time-zone-...

Re: Some DNS lookups causing 5xx errors due to leap second bug

#132

Earlier quoted context omitted.

Except people want to be able to talk about times years in the future despite not knowing the number of leap seconds that may happen in the intervening time. It is more useful in most fields to talk about an event happening every N years/months/days than an event happening every N seconds. Most people do not want a leap second to shift their scheduled event from 10:00:00 every Monday to 9:59:59 or 10:00:01 in the nam…

> Except people want to be able to talk about times years in the future despite not knowing the number of leap seconds that may happen in the intervening time. Doesn't the (TAI, leap second count) tuple solution work for this? Maybe I misunderstand the purpose, but you could use the leap second count to figure out how many seconds the TAI is off by. But that doesn't matter, because date intervals shouldn't be represe…

…I forgot to mention this in my original comment, but real-world wall-clock time is in any case discontinuous due to daylight savings time and other timezone changes. This means that it's not only months and years that change in length, but days and weeks too.

Re: Some DNS lookups causing 5xx errors due to leap second bug

#133
post #115
post #90

Earlier quoted context omitted.

I'm confused on why a DNS server would need to rely on a monotonic clock for its use cases. Is there a part of DNS that relies on the assumption of synchronized, monotonic time? (Perhaps TTL/expiry of records? But I still don't see why having a non monotonic clock source would harm if CF is using Go timers for expiry)

One example might be rate limiting. Count requests over elapsed time. If elapsed time is a negative number, the math might trigger a bug that causes CF to block requests...too many requests over time period X.

Cloudflare posted a post-mortem [1]. They were measuring round trip time, and supplying the result of that into the golang rand.Int63n() function, which panics the process when given a negative number.

[1] https://blog.cloudflare.com/how-and-why-the-leap-second-affe...

Re: Some DNS lookups causing 5xx errors due to leap second bug

#134
post #67
post #7

My CDMA phone dropped service for a few minutes after the leap second. It's absurd that we continue to keep subjecting ourselves to these disruptions and the considerable amount of work that goes into handling leap seconds for the systems that aren't disrupted by them. Leap seconds serve no useful purpose. Applications that care about solar time care usually care about the local solar time, while UT1 is a 'mean solar…

CDMA system time is already defined as free of leap seconds. --- 3GPP2 C.S0002-A section 1.3 "CDMA System Time": All base station digital transmissions are referenced to a common CDMA system-wide time scale that uses the Global Positioning System (GPS) time scale, which is traceable to, and synchronous with, Universal Coordinated Time (UTC). GPS and UTC differ by an integer number of seconds, specifically the number…

Some GPSDO that are used to time CDMA base stations are known to misbehave around leap seconds (though often when the GPS signal sends the leapsecond warning, not at the leapsecond itself).

Re: Some DNS lookups causing 5xx errors due to leap second bug

#135
post #67
post #7

My CDMA phone dropped service for a few minutes after the leap second. It's absurd that we continue to keep subjecting ourselves to these disruptions and the considerable amount of work that goes into handling leap seconds for the systems that aren't disrupted by them. Leap seconds serve no useful purpose. Applications that care about solar time care usually care about the local solar time, while UT1 is a 'mean solar…

CDMA system time is already defined as free of leap seconds. --- 3GPP2 C.S0002-A section 1.3 "CDMA System Time": All base station digital transmissions are referenced to a common CDMA system-wide time scale that uses the Global Positioning System (GPS) time scale, which is traceable to, and synchronous with, Universal Coordinated Time (UTC). GPS and UTC differ by an integer number of seconds, specifically the number…

Some GPSDO that are used to time CDMA base stations are known to misbehave around leap seconds (though often when the GPS signal sends the leapsecond warning, not at the leapsecond itself).
Post reply on HN