Ask HN: Are Lucene/Solr/ES Still Used for Search? • Hacker News Board

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#221

Post by bogomipz » Wed, Jul 24, 2019, 6:21 AM UTC

crawdog wrote: post #68

bogomipz wrote: post #64

Earlier quoted context omitted.

>"For retail/catalog search SOLR dominates." Interesting, could you elaborate on why SOLR is dominant in that space over say Elasticsearch?

Definitely. Product catalog information as a whole doesn't change often. Price and availability does. With retail catalogs you often do a full reindex of the data in your master catalog and then run partials to account for price/availability if you don't do that in realtime using filters. Since the system of record is not the search index, Elastic is often not a good solution here. Also, relevancy in retail is often…

Thanks for the explanation and insight. I hadn't ever considered how different the catalog use case is from my own use cases. Makes good sense. Cheers.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#222

Post by hiram112 » Fri, Jul 26, 2019, 5:05 PM UTC

manigandham wrote: post #169

hiram112 wrote: post #166

Earlier quoted context omitted.

Can you explain a bit why ES isn't a good solution for storing data itself? I inherited a legacy Mongo solution, and all the data is duplicated and indexed in ES, so I've always wondered why we're using both. Mongo has none of the SQL capabilities that would make my life easier, and the types of queries allowed by Mongo could be done with ES. What are the negatives of ES alone?

It's not reliable: https://www.quora.com/Why-shouldnt-I-use-ElasticSearch-as-my... The v7 upgrade to a new cluster protocol (zen2) has improved things but overall the system has a long history of losing or destroying data. It's better to have a primary OLTP system that's ACID and reliable while using ES as the secondary search source. You can also remove the _source field if you just need matches without the original…

Thank you very much. It seemed like we were just duplicating a bunch of scheme-less json for no reason, but if ES can lose data, that's probably not a good idea.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#223

Post by fulmicoton » Mon, Jul 29, 2019, 10:19 AM UTC

bpicolo wrote: post #189

fulmicoton wrote: post #182

Earlier quoted context omitted.

Tantivy main dev here. Thanks for the free marketing :) It is not exactly a port but yeah. tantivy is strongly inspired from Lucene. > Lucene has spent a ton of time getting the details right, and it's unlikely we'll see an order of magnitude of innovation in that particular space. Have you checked out the perf gain in Lucene 8.0 ? Block-WAND proved you wrong.

I suppose I could have phrased that better. I appreciate the correction. I mostly mean to say that the functionality is really impressive today, and serves its use case very well for the intended target of lower level search primitives. Tantivy is a cool project, but I have to say the part I love most about it is your blog posts on it. They're a great introduction for people who are unfamiliar with the underlying tec…

> Tantivy is a cool project, but I have to say the part I love most about it is your blog posts on it. They're a great introduction for people who are unfamiliar with the underlying tech of search engines.

Thanks a lot! I am not a native speaker, and I often feel very bad at conveying engineering concepts. The positive feedback is actually very helpful :)