Firefly – Full Text Search Engine for Dropbox

I would've loved to hear more details about why you built your own. For example, you mention that Elastic Search wasn't deployed at your scale and there was some talk about machine footprint, but it doesn't explain how your solution compares to something like ES.

Did ES just didn't scale when you tried it? Is your solution better/faster? If so, by how much and on what workloads?

Contrast this with something like RocksDB. They just show you the numbers - http://rocksdb.org/

They could index it before encrypting it. Still counts, right? :)

No, not really. They lied.

https://en.wikipedia.org/wiki/Dropbox_%28service%29#Privacy_...

If you really didn't want Dropbox having access to your files, why would you upload it in the first place?

Today. We have indexed all of your documents, you can search easily inside your documents, even though you have created good directory structure and named your files accordingly.

Tomorrow. Hmm we have your data, lots of data, we wanted to know what is interesting to our users, so we decided to analyse them and find people with common interest.

Day after tomorrow. Miss Rice challenged us "can you find terrorist users using all of the documents you have indexed and analysed?"

Future. Hey user your first name is strange, your documents contain some strange characters, you are uploading data from country where our political leaders have problems, are you terrorist?