Live data from Hacker News

FastMail's Email Search Architecture

blog.fastmail.com

31–32 of 32 posts

Re: FastMail's Email Search Architecture

#31
post #17

Seems like they independently invented a sort of log structured merge tree.

LSM-type storage is frequently used in IR; it's not a new technique. I accidentally "invented" it a few years ago before I realized it had a name. Even Lucene 1.x used a variation of this for its "segment" files (which it still does, at least up to 3.x, afaik).

The reason is that you want to keep the inverted indexes sorted on disk, but you don't want to sort the entire index every time you update. So you create one mini-index per update and merge them lazily when you get too many of them.

Re: FastMail's Email Search Architecture

#32

Earlier quoted context omitted.

Sorry, I don't understand what you mean. iCloud is a service, and FastMail's web client doesn't talk to iCloud, it only talks to FastMail's servers - at least for now.

I was contrasting the iCloud interface to FastMail, an implication they were connected wasn't intended.

Ahh, ok. I misread your post then, sorry.
Post reply on HN