My experience is mostly Japan-centric nowadays but SOLR is very widely used here and there is demand for people with that background. A lot of work has been done with SOLR to better support the intricacies of dealing with Japanese text which differs substantially from other languages. Most of the search and NLP jobs I've seen recently outside of Google and Amazon expect some SOLR experience.
Ask HN: Are Lucene/Solr/ES Still Used for Search?
181–190 of 223 posts
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#182Earlier quoted context omitted.
There's folks working on Bleve (written in Go) and developers that I work with want to use it (we use Elasticsearch heavily), but as I've told everyone like you just did, Lucene has a 20 year head start. Thing is, there's heavy demand for something more performant than Elasticsearch, so eventually the market will provide. Meanwhile, Redis Enterprise is trying to grab some 'share with RediSearch, which has some severe…
Tantivy is an interesting project I'd point to in this space: https://github.com/tantivy-search/tantivy That said - it's effectively Lucene rewritten in rust, so the main win is some performance gains. Lucene has spent a ton of time getting the details right, and it's unlikely we'll see an order of magnitude of innovation in that particular space. At the higher level for querying / query understanding it feels like t…
Tantivy main dev here. Thanks for the free marketing :)
It is not exactly a port but yeah. tantivy is strongly inspired from Lucene.
> Lucene has spent a ton of time getting the details right, and it's unlikely we'll see an order of magnitude of innovation in that particular space.
Have you checked out the perf gain in Lucene 8.0 ? Block-WAND proved you wrong.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#183We have been using https://www.algolia.com/ completely as a replacement for ES. Pros: - Managed search engine - Great API / Developer experience Cons: - Cloud only makes it hard for local development - Expensive (I guess it depends on the usage)
I have been working on an open source alternative: https://github.com/typesense/typesense
Would love to hear your feedback :)
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#184This has been a great thread, and there's some heavyweight indexes here. But what about at the other end of the scale? Say when you've got 10k-50k contact details (name, email, phone) and you want to provide a quick, autocomplete lookup. I've used basic SQL string matching for this, but it doesn't catch mis-spellings and the rest. Running SOLR or ES is overkill for this. Is there a tool that fits this niche?
Yes! Please take a look at an open source search engine I am working on. You will definitely like it:
https://github.com/typesense/typesense
Would love to hear your feedback :)
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#185yeah, there aren't many alternatives unfortunately. I've used Sphinx a lot, but am now stuck with ES and it is horrible to operate, probably because we don't need a cluster solution so it is total overkill. Yeeeah for technical debt. For small projects (for some 1000s of documents), I'd probably go with Postgresql FTS if possible. Sphinx/Solr for anything with indices smaller than a couple 100GBs After that, ES seems…
This was exactly my pain point as well. For smaller projects, ES is a overkill. So I decided to do something about it! I started working on an open source, really, developer friendly search engine that just works. It's pretty stable now and quite a few people use it and like it: https://github.com/typesense/typesense
Would love to hear your feedback.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#186Earlier quoted context omitted.
I do not have a completely fair comparison. But a migration from Elasticsearch 5 (2016) to the Vespa 7 (2019) we reduced half of our nodes, and cut in half the average response time. Another amazing feature during the migration, is that Vespa allows you to reduce or increase the number of nodes dynamically. And it take full care about the data distribution. In ES we (used to) had to follow the limits of the pre-confi…
I have been wondering why Vespa isn't getting much traction. Everyone still defaults to ES, even in new project.
It should be marketed better, I feel. Some SEO for "Solr/lucene vs X" queries might help. I have been spending the last 3-4 months studying open source and commercial search systems, but it's only in this thread that I discovered Vespa.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#187Are people still using Sphinx Search ( http://sphinxsearch.com ) at all? It doesn't seem like it gets many releases anymore...since they unpublished the source code, it's hard to see how much activity there is.
We use it on https://dealbert.net
I see you don't have autocomplete in the search box. You might be interested in this interactive course https://play.manticoresearch.com/simpleautocomplete/
It's about Manticore, but may be used with Sphinx too.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#188Are people still using Sphinx Search ( http://sphinxsearch.com ) at all? It doesn't seem like it gets many releases anymore...since they unpublished the source code, it's hard to see how much activity there is.
We just migrated from Sphinx to using the full text search indexes in PostgreSQL, we had to deal with some changes in how special characters are handled, but it's worked well enough.
As far as I remember few years ago they didn't have BM25 and even TF-IDF support. Have they added that? Are you experiencing any issues with full-text search quality after migrating from Sphinx (you probably used BM15(+F) which is BM25 w/o doc length).
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#189Earlier quoted context omitted.
Tantivy is an interesting project I'd point to in this space: https://github.com/tantivy-search/tantivy That said - it's effectively Lucene rewritten in rust, so the main win is some performance gains. Lucene has spent a ton of time getting the details right, and it's unlikely we'll see an order of magnitude of innovation in that particular space. At the higher level for querying / query understanding it feels like t…
Tantivy main dev here. Thanks for the free marketing :) It is not exactly a port but yeah. tantivy is strongly inspired from Lucene. > Lucene has spent a ton of time getting the details right, and it's unlikely we'll see an order of magnitude of innovation in that particular space. Have you checked out the perf gain in Lucene 8.0 ? Block-WAND proved you wrong.
I suppose I could have phrased that better. I appreciate the correction. I mostly mean to say that the functionality is really impressive today, and serves its use case very well for the intended target of lower level search primitives.
Tantivy is a cool project, but I have to say the part I love most about it is your blog posts on it. They're a great introduction for people who are unfamiliar with the underlying tech of search engines.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#190We use ES for our searching data within our application. We store about 20,000,000 rows of data in the primary table, with plenty of dependent and secondary tables. ES takes the load off our MySQL cluster for generating reports and fulfilling searches.