Full Text Search in the Cloud
blurpr.com
Full Text Search in the Cloud
1–10 of 11 posts
Re: Full Text Search in the Cloud
#2Re: Full Text Search in the Cloud
#3Hey! I'm one of the websolr guys. Obviously, we're working on this problem, and it's a hard (but fun and rewarding) one for a number of reasons!
Search is fundamentally hard to put into the cloud, because it requires so many IO operations. In addition, delivering really high quality results requires machine learning and linguistics.
We have a few tricks up our sleeves to handle these issues, and I'm excited not only to shed the beta-ish feel, but to roll out some truly exiting features :) There's a "Review my startup" post coming pretty soon.
Re: Full Text Search in the Cloud
#4Hey! I'm one of the websolr guys. Obviously, we're working on this problem, and it's a hard (but fun and rewarding) one for a number of reasons! Search is fundamentally hard to put into the cloud, because it requires so many IO operations. In addition, delivering really high quality results requires machine learning and linguistics. We have a few tricks up our sleeves to handle these issues, and I'm excited not only…
Hey! I'm one of the Loggly guys. Wanna grab a beer and chat about how we're doing it?
Re: Full Text Search in the Cloud
#5I have Solr mostly working on AppEngine.
Obviously (for those of you who know the Solr codebase), there are some pretty extreme hacks to get around the lack of file system access, but nothing that couldn't be cleaned up.
I was a little surprised about the lack of interest in it when I emailed the solr-dev list.
Re: Full Text Search in the Cloud
#6I feel dumb, but I don't totally understand what is being wanted. Basically a google search with "site:example.com query" but you have full control of the results?
Re: Full Text Search in the Cloud
#7I have Solr mostly working on AppEngine. Obviously (for those of you who know the Solr codebase), there are some pretty extreme hacks to get around the lack of file system access, but nothing that couldn't be cleaned up. I was a little surprised about the lack of interest in it when I emailed the solr-dev list.
The Compass GAE walkthrough (http://www.kimchy.org/searchable-google-appengine-with-compa...) was also able to get a rudimentary Lucene indexer up and running. However many people eventually ran into problems with App Engine's 30-second request processing limit. For your Solr instance did you utilize the Task Queue or have to do anything special to work around the 30-second limitation?
Re: Full Text Search in the Cloud
#8I feel dumb, but I don't totally understand what is being wanted. Basically a google search with "site:example.com query" but you have full control of the results?
While Google/Bing/etc are great for searching public sites (when you can tolerate the latency between when content is posted and when google indexes it), they doesn't work at all for sites which are private/intranetted/password-protected or otherwise inaccessible to web crawlers.
Re: Full Text Search in the Cloud
#9I feel dumb, but I don't totally understand what is being wanted. Basically a google search with "site:example.com query" but you have full control of the results?
While Google/Bing/etc are great for searching public sites (when you can tolerate the latency between when content is posted and when google indexes it), they doesn't work at all for sites which are private/intranetted/password-protected or otherwise inaccessible to web crawlers.
Makes sense, thank you.
Re: Full Text Search in the Cloud
#10I am a co-founder in a startup (Fablo, http://fablo.pl/) that does exactly that. In order not to get wiped out by Google the day they decide to give that functionality away for free, we decided to specialize on E-commerce search, particularly in inflected languages, which is a much harder task than searching English.