Live data from Hacker News

Owning my own data, part 1: Integrating a self-hosted calendar solution

emilygorcenski.com

131–140 of 157 posts

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#131
post #97

Earlier quoted context omitted.

Calendar apps expose all of the weird edge cases in dates and times. For a specific instance of a specific event that never needs to move, TAI will work. However, suppose that you schedule a meeting for 28 March at 13:00. You then move that meeting forward a week, crossing a DST switch. If you simply add 604800 seconds to the TAI, that meeting will be at 14:00, which is surprising. OK, easy enough to solve; convert t…

I'll preface this by stating that I fully realize that localized datetime is absolutely byzantine and fraught with difficulties. That's the rationale for leaving it to an external library as close to 100% of the time as possible. I also realize that there might well be complexities involved in calendar software that I'm unaware of. Hence my comment - I'm genuinely interested to learn. That said, I think your examples…

IMHO, based on zero evidence, for scheduling, very few people are scheduling events beyond a minute of precision. TAI is a better timescale than UTC, but UTC is better supported and leap seconds might be going away, so pretending they don't exist is probably a better choice.

But timezones are super byzantine. Especially when the rules update. More especially when the rules update on short notice.

If you scheduled an event at 5 pm local time at some place on some day and the rules change, the event's UTC (and TAI) would change. But you may want to confirm with the user, because maybe they're calling into an event somewhere else and they entered their local time but the event is really scheduled in the other time.

You might also have fun when multiple clients are using the same event and the clients and the server don't all agree on timezone definitions. iCalendar suggests that the timezone definition be included in the files, but my experience is definitions are often missing and most things ignore the definitions if present; you've got to include a timezone name the software is going to process as you want or you won't have a good time (or use UTC and hope)

If the event was scheduled in the DST window, the time may no longer exist, or may be ambiguous. iCalendar has rules for that, but they might not actually meet your user's needs (otoh, who is scheduleding things at 2-3am on a Sunday ... probably not a lot of people)

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#132
post #27

I honestly prefer a pen and paper planner and just manually migrating my contacts over. Meanwhile, my personal gmail is 99.9% contacts I have no intention to ever speaking to again.

Begging the question why not delete the contacts? I've been thinking about it: certainly most of the numbers in my contacts list are probably invalid by now. But there's definitely a delta of finality to going "oh there's like 10 people I will ever call and that number is only going to go down... "

Last time I tried I couldn't find a way to bulk delete contacts. I had to delete them all one by one. I couldn't stand how tedious it was and gave up. I don't feel any kind of sentimentality towards these contacts, given most of these people I haven't spoken to in nearly 15 years

I don't connect google to my phone anymore, so at least they don't pollute any data elsewhere

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#133
post #63

Earlier quoted context omitted.

The JMAP for Calendars RFC [1] is soon to get published by IETF. Disclaimer: I work for Fastmail and am the co-author of RFC 8984 [2] which defines the calendar event data model used in JMAP. [1] https://www.ietf.org/archive/id/draft-ietf-jmap-calendars-22... [2] https://www.rfc-editor.org/rfc/rfc8984.html

Are the protocols compatible enough such that someone could proxy JMAP to CalDAV?

Yes, but I am not aware of an existing proxy implementation that would accept JMAP requests and forward these to a CalDAV server, or the other way round.

What I do know for sure is that it‘s feasible for a calendar server to service both JMAP for Calendars and CalDAV. All our work in that regard is open-source in the Cyrus server [1] and is used at Fastmail.

Apart from the exchange protocol one also needs to convert between iCalendar and the data structures used in JMAP. For this, I am currently working on an IETF RFC [2]. Anyone also interested in implementing conversion between iCalendar and JSCalendar, please contact me! We have a test suite for interoperation tests.

[1] https://github.com/cyrusimap/cyrus-imapd [2] https://datatracker.ietf.org/doc/draft-ietf-calext-jscalenda...

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#134

I know I will probably get pushback over this, but CalDAV sucks . People say that it's great and that it's easy; yet if that were true there would be way more self-hosted CalDAV solutions out there and they would be far more ergonomic than what's available. I was not impressed with Radicale. Because there's really only one calendar file I wanted to serve to myself, I tried implementing my own CalDAV server just for t…

My CalDAV provider is ZOHO.

To sync with Android I use DAVX5 which runs a background sync service (open source and non-root). I had a lot of issues getting it working, could be because of different CalDAV implementations. DAVX5 to ZOHO calendar sync takes forever.

Evolution on Ubuntu has pretty good CalDAV support, but it would refresh the entire calendar each time upon switchin between the mail view and calendar view. Refreshing that ZOHO calendar takes forever and ZOHO has pretty frugal rate limits, so it ends up soft-banning me.

Latest Thunderbird has a CaldDAV incompatibility where it expects a different HTTP success code. ZOHO gives a 200, but Thunderbird expects something else. (PS Thunderbird CalDAV code is written in JS). PS This only happens on the second calendar. Whenever you add a second one, it starts failing.

So it is quite a pain, at least with ZOHO.

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#135
post #121

Earlier quoted context omitted.

I'll preface this by stating that I fully realize that localized datetime is absolutely byzantine and fraught with difficulties. That's the rationale for leaving it to an external library as close to 100% of the time as possible. I also realize that there might well be complexities involved in calendar software that I'm unaware of. Hence my comment - I'm genuinely interested to learn. That said, I think your examples…

The problem is TAI is not the source of truth in real life. In real life, Arizona can change its laws and suddenly that event is going to happen at a different TAI timestamp. That’s why the unambiguous storage format has to be a date time with time zone, and not just a timestamp.

Good point. Seems I also made a reasoning error by failing to consider that timezones, being arbitrary legal constructs, can be changed at the drop of a hat.

However it doesn't seem particularly difficult to fix. My error was suggesting using TAI for storage. I guess that works only for events in the past. So the timezone that the event belongs to is what should have been used for storage, you should forget leap seconds exist because this is a human centric calendar so who cares, and you should trust the datetime library to just do the right thing when converting timestamps.

Is there some other issue I'm missing? Because so long as all the timezone complexity is stuffed into the datetime library (and thus NotMyProblem™) it seems like the really difficult part is already solved for you.

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#136

Earlier quoted context omitted.

I'll preface this by stating that I fully realize that localized datetime is absolutely byzantine and fraught with difficulties. That's the rationale for leaving it to an external library as close to 100% of the time as possible. I also realize that there might well be complexities involved in calendar software that I'm unaware of. Hence my comment - I'm genuinely interested to learn. That said, I think your examples…

> You never move events around in TAI (the unambiguous storage format) just as you don't go manually flipping bits in an SQLite database. You actually do. From the point of view of future scheduled events, a event scheduled for 13h00 in a specific timezone is still "13h00 in a specific timezone" even when crazy people on that timezone suddenly declare a new Daylight savings time to start before the event happens. All…

It's a good point. I responded to the adjacent comment.

After glancing at that RFC I'm unclear about the purpose of the UTC offset. Is it simply an error check to catch the case of a datetime library with outdated timezone information? Otherwise it seems like the library will have all the historical information on hand so it shouldn't have any use for it.

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#137
post #131

Earlier quoted context omitted.

I'll preface this by stating that I fully realize that localized datetime is absolutely byzantine and fraught with difficulties. That's the rationale for leaving it to an external library as close to 100% of the time as possible. I also realize that there might well be complexities involved in calendar software that I'm unaware of. Hence my comment - I'm genuinely interested to learn. That said, I think your examples…

IMHO, based on zero evidence, for scheduling, very few people are scheduling events beyond a minute of precision. TAI is a better timescale than UTC, but UTC is better supported and leap seconds might be going away, so pretending they don't exist is probably a better choice. But timezones are super byzantine. Especially when the rules update. More especially when the rules update on short notice. If you scheduled an…

> the clients and the server don't all agree on timezone definitions

Isn't that scenario unsalvageable regardless? It presumably means one or more of the devices are displaying the wrong time to the user. Or at least will be upon arrival in the timezone of the event.

For example (intentionally extreme) suppose the library used by my calendar thinks EST is -4, the OS on my device thinks EST is -6, and everyone else thinks EST is -5. What possible program logic could prevent disaster in this scenario?

It's a really interesting edge case to point out, I'm just not clear what a sensible algorithm for handling it would look like. I strongly suspect that assuming that can never happen is the right answer. Everyone agrees the event is scheduled for 9 am EST. If outdated client software translates that to PST incorrectly or displays the incorrect time on the clock then the affected user is going to have issues and I don't see what the programmer can realistically do about it.

> ... DST window, the time may no longer exist, or may be ambiguous.

Assuming the timestamp disambiguates the daylight timezone from the standard timezone this should be handled gracefully by the datetime library since the stale timestamp is merely "out of bounds" for that timezone now that the laws changed. I'm failing to think of a scenario where it becomes ambiguous (although reasoning through this is once again reinforcing my view that we should have ditched DST long ago).

However this did cause me to realize that there's an additional edge case for far future events where the timezone is modified multiple times. If the event timestamp was never updated in the interim you land in an ambiguous situation and have no way to detect it.

I'm not sure that particular edge case is worth worrying about, but it seems like tracking a second timestamp indicating the last time that the event was processed by the datetime library would resolve it. The library should hopefully have all the necessary historical information on hand to update a long since stale timestamp (I haven't tested this though). Anyway if it doesn't I don't see that there's anything you can do about it.

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#138

Earlier quoted context omitted.

I just did this a few months ago. I used PhotoPrism https://github.com/photoprism/photoprism

I used Photoprism for over a year, even paid the "server license" and I'm now using Immich which is much more powerful, sleek, gets new features very quickly (sometimes it's even too fast paced) and in general has got much more traction with the community.

Oh and I forgot to mention the mobile app! When I did the switch the difference was like night and day. Actually Photoprism didn't even have an official mobile app, you could just sync photos in the background with SyncThing for example and keep using your phone's photo galley app, although there was a Photoprism app as well. Immich is just a backend service like a mobile app, it's only current big defect is that it's useless if it cannot contact the server on start-up

Re: Owning my own data, part 1: Integrating a self-hosted calendar solution

#139
post #65

I too have been on a very aggressive mission to decloudify myself lately(as in, since comrade musk and co took over everything and I have 0 reason to trust him or anyone around him). And to be fair, for all the hundreds of gigabytes of data I have, the migration took no longer than two weeks, on and off, and it works infinitely better than I was hoping. The only service I've outsourced is email since I hate dealing w…

> It's biggest strength is that it's both accessible and has a slot for an M.2 NVME and a SATA drive so you can have a safe backup in one spot. If you want to increase the resiliency of your data, look up the 3-2-1 backup strategy (and maybe also RAID).

It is next in the pipeline but I need to clean up my home lab first cause it's become a bit of a mess.
Post reply on HN