With ElasticSearch you wouldn't have that problem. It uses push instead pull replication ... which is also (near) real time aware. Even for peta whatever byte.
Turbocharging Solr Index Replication with BitTorrent
11–14 of 14 posts
Re: Turbocharging Solr Index Replication with BitTorrent
#12With ElasticSearch you wouldn't have that problem. It uses push instead pull replication ... which is also (near) real time aware. Even for peta whatever byte.
Here is the video where Shay explained the difference http://vimeo.com/26710663 (from 18min on)
Re: Turbocharging Solr Index Replication with BitTorrent
#13Great article and insight.
FYI mrsync is confusingly named since it doesn’t actually use the rsync algorithm it performs multicast file transfer. (The whole file is sent not just the deltas).
I’ve always thought it would be cool if someone added rsync capabilities on top of bittorrent. I’m not sure Etsy would see any benefit but I bet Twitter with their murder server deploy would.
Re: Turbocharging Solr Index Replication with BitTorrent
#14With ElasticSearch you wouldn't have that problem. It uses push instead pull replication ... which is also (near) real time aware. Even for peta whatever byte.
They are heavily using ExternalFileField - I don't think there's anything similar in ElasticSearch.