Is LevelDB linked in this article related to leveldb developed at Google ( https://github.com/google/leveldb ) ?
Firefly – Full Text Search Engine for Dropbox
11–20 of 22 posts
Re: Firefly – Full Text Search Engine for Dropbox
#12Can it do substring search? Unfortunately, Firefly is only available for business customers.
Re: Firefly – Full Text Search Engine for Dropbox
#13Aren't documents on dropbox supposed to be encrypted?
Only if you're the one encrypting them.
Re: Firefly – Full Text Search Engine for Dropbox
#14Re: Firefly – Full Text Search Engine for Dropbox
#15Is LevelDB linked in this article related to leveldb developed at Google ( https://github.com/google/leveldb ) ?
The "C++ Library" link on http://leveldb.org/ points at Google's LevelDB, so yes.
Re: Firefly – Full Text Search Engine for Dropbox
#16Today. We have indexed all of your documents, you can search easily inside your documents, even though you have created good directory structure and named your files accordingly. Tomorrow. Hmm we have your data, lots of data, we wanted to know what is interesting to our users, so we decided to analyse them and find people with common interest. Day after tomorrow. Miss Rice challenged us "can you find terrorist users…
You are of course welcome to not put your content on Dropbox, a private service run by a for-profit company.
They're adding a feature that they think the majority of their users will find useful, and betting that most of their users realize this doesn't increase Dropbox's access to their content: they could have written this indexing years ago and never disclosed it, using it solely for whatever tinfoil-hat conspiracy theory style acts they'd like.
If they lose the bet, they either pull the feature or go out of business. I'm betting their side, though, given that I expect most Dropbox users either don't care (which is their right) or care and made a willful decision to store their data on somebody else's systems (which is also their right).
Re: Firefly – Full Text Search Engine for Dropbox
#17I can't help but I just don't want Dropbox or anybody else to read all my files.
If you really didn't want Dropbox having access to your files, why would you upload it in the first place?
Most likely the average HN reader is well aware that dropbox must have access to read your files. But possibly an average joe who doesn't read the TOS and who doesn't think everything through would assume that private storage is private to joe only and that the advertised encryption extends beyond transport to storage.
Re: Firefly – Full Text Search Engine for Dropbox
#18Today. We have indexed all of your documents, you can search easily inside your documents, even though you have created good directory structure and named your files accordingly. Tomorrow. Hmm we have your data, lots of data, we wanted to know what is interesting to our users, so we decided to analyse them and find people with common interest. Day after tomorrow. Miss Rice challenged us "can you find terrorist users…
You are of course welcome to not put your content on Dropbox, a private service run by a for-profit company. They're adding a feature that they think the majority of their users will find useful, and betting that most of their users realize this doesn't increase Dropbox's access to their content: they could have written this indexing years ago and never disclosed it, using it solely for whatever tinfoil-hat conspirac…
Part of the concern is your average joe who doesn't read the ToS or understand computers might assume that their private Dropbox.com folder is exactly a computer analogue of a physical private folder where you drop your physical files from your physical desk, which then does some computer magic to sync them between your work and home desk.
Re: Firefly – Full Text Search Engine for Dropbox
#19"Firstly, we expect some users to have a large number of documents in their Dropbox, making it non-trivial to update their corresponding index “instantly”."
If the alternative is maintaining a single index, won't the time it takes to update it at least be the time it takes to update per-user indexes? The former naively sounds like updating a single, gigantic binary search tree, the latter seems like updating a hashmap of UserId/BST pairs.
"Secondly, this approach requires the system to maintain as many indices as there are users with each stored in a separate file. With over 300 million users, keeping track of so many indices in production would be an operational nightmare."
..Why?
Anyway the stuff about shared documents is enough to make per-user indexing probably a bad idea, but I don't understand the reasons they provided above.
Re: Firefly – Full Text Search Engine for Dropbox
#20RabbitMQ, interesting. I was just reading up on NSQ and it seems like a good alternate.
Any details on the tech Firefly was coded in? Go, C++, Java?