Earlier quoted context omitted.
Wow, does 240 people even count as a sample. At Yahoo and Google log sizes its probably the error from cosmic rays in the data center.
If they selected them in a properly random way and had an effect close to 40% then yes, that probably does count as a sample.
SeenBefore: A search engine for what you have seen before
81–90 of 97 posts
Re: SeenBefore: A search engine for what you have seen before
#82Earlier quoted context omitted.
Storing a list of 1.7 million strings for us takes 70mb stored in memory. Testing for membership is an O(1) op. Very happy with it. We use mongo as a dumb data store as well as a bunch of other infrastructure tools, like http://circleci.com we could have only dreamt of years ago.
Why not use a bloom filter?
Re: SeenBefore: A search engine for what you have seen before
#83Earlier quoted context omitted.
Why not use a bloom filter?
Maybe because of this fact (according to Wikipedia)?: "The more elements that are added to the set, the larger the probability of false positives."
Re: SeenBefore: A search engine for what you have seen before
#84Earlier quoted context omitted.
Why not use a bloom filter?
The main benefit of Bloom filters is that they can be made small . Given that his database takes only 70MB or so and he's not trying to ship this to devices that might have much in terms of space limitations, there would appear to be little point.
Re: SeenBefore: A search engine for what you have seen before
#85Earlier quoted context omitted.
Porn sites are not recorded
How does your system define "porn sites"? What about if it was some porn site no one has ever heard of with an innocent-sounding name/domain?
Re: SeenBefore: A search engine for what you have seen before
#86Earlier quoted context omitted.
And would it be possible to configure it to use my own "cloud"?
Definitely something we are looking into. Major barrier is the cost for someone keeping a server running 24*7 in cloud(Micro instance on AWS is 175 dollars a year).
Re: SeenBefore: A search engine for what you have seen before
#87Interesting idea. Some quick questions: - How much data do you store per user? - How do I delete certain results? (preferably after the search comes back) - Another thing to consider is - After how much time does this just become as painful as finding that page through a search engine? - What version of the page gets stored? The latest or the one that I saw? I guess its one step better than Evernoting a page and addi…
Main issue with Evernoting and Bookmarking is that it requires an effort to say that today, this page is useful and I want to store it. Most pages I want to find are very things I did not think was useful at the time. Each unique page(unique as per the content) is stored per user. Our goal is to build the tools needed to find the information quickly, similiar to what hipmunk.com did for airline search. We have the ad…
I'm a paying Xmarks user, but if you were to add a way of tagging sites or adding a note, I'd happily pay for this instead. Just a freeform text field that I could add some keywords into that gets treated as part of the search would actually be sufficient for me.
Re: SeenBefore: A search engine for what you have seen before
#88Co-founder here. This took us by surprise, we were planning to have Firefox and Safari support done by launch. At this stage, it is priceless to know if we are solving a real problem people have. Also, is this something people would pay for (loops back to if this is enough of a pain point). From the moment we start charging, is the moment we start learning.
Re: SeenBefore: A search engine for what you have seen before
#89Hmm, this is similar to http://historio.us , which I built. However, this doesn't require any user interaction, which might work well. Do you store just the URL and depend on Google returning the results? How does it work exactly?
But Seen Before requires less effort on my part as a user -> I am more likely to use it. I just continue to google as per normal and now I have an extra option on the right to filter results.
Re: SeenBefore: A search engine for what you have seen before
#90Hmm, this is similar to http://historio.us , which I built. However, this doesn't require any user interaction, which might work well. Do you store just the URL and depend on Google returning the results? How does it work exactly?
I remember thinking that http://historio.us was a neat idea. But Seen Before requires less effort on my part as a user -> I am more likely to use it. I just continue to google as per normal and now I have an extra option on the right to filter results.