Live data from Hacker News

Search Benchmarking: RediSearch vs. Elasticsearch

redislabs.com

61–70 of 81 posts

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#61
post #59
post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

What would be your go-to solution for a basic word search - lets say you only have a few MBs of data - not GBs...

Full-text search from MySQL or other similar database. When that gets overloaded, then consider something like Elastic Search. That is my rule.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#62
A problem with RediSearch, at least for me is:

Note: clustering is only available in RediSearch’s Enterprise version

https://redislabs.com/redis-enterprise/technology/redis-sear...

At least with ES i can build and play with the clustering of the nodes. This is probably why they only made a 1 node ES, because they would have to push their Enterprise software to do make a cluster of RediSearch. Maybe i am wrong.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#63
post #19

Earlier quoted context omitted.

Not to mention spelling correction, synonyms, nested taxonomies, etc. Search is incredibly complex, and I always snort when I see someone trying to create one from scratch.

I was just going to ask: Will SQL work with spelling corrections? I was under the impression that if you wanted to do auto-complete, you need to handle mis-spellings, and that ElasticSearch is one of the best options for this.

SQL doesn't. Some relational databases have full-text search extensions like Postgres and SQL Server but they offer the basic stemming and trigram stuff, no spelling or synonyms. You can get an autocomplete working using wildcard matches but you won't be able to recognize that a word is misspelled without maintaining your own dictionary.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#65
post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

I've seen ppl use it as key value store, time series database( i think they have some apm support too) , nosql datastore.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#66

How silly to emphasize things like "built as a C extension" and "uses modern data-structures" as if these were useful criteria for choosing a search engine. It's about minimizing the effort needed to find what you're looking for. Speed of index construction time, unless we're talking orders of magnitude, isn't really meaningful. I don't know if this is just a really clumsy attempt at "marketing" or what, but I can't…

Also complaining that "Lucene is 20 years old" is about the same as saying "Linux is ~30 years old" Lucene is a pretty rock-solid open source project that has been battle tested over those 20 years and had some of the best engineers in the world improve over a long time frame. That's an asset for Lucene!

> Lucene is 20 years old

Damn I feel old. I remember when Lucene was hot new kid in the block.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#67
post #4

I'm curious if this scales down well. The test was done on "One AWS c4.8xlarge with 36vCPU and 60GiB Memory". But could I run this on a tiny vps to index, search, catalog my million-odd documents?

You can check out Sonic[1] and Tantivy[2] both are lightweight search engine

[1] https://github.com/valeriansaliou/sonic

[2] https://github.com/tantivy-search/tantivy

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#68
post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

To counter some of the comments here, and after looking at the sources, the RediSearch module is pretty capable and matches a lot of the Elasticsearch features: https://github.com/RedisLabsModules/RediSearch

Agree with the general claim that this benchmark is poor though. A real study of complex searches with faceting, ranking and ordering against both databases in a distributed setup would be much more interesting.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#70
post #12
post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

This. The best benchmarking for search engines is: 1. Does it return relevant results? 2. Can it handle complex queries? 2) is only required in specific use-cases, but when it's needed it's _really needed_. 1) is the main measure users care about, and in my experience is best evaluated by building a search in each system with the same corpus and giving to subject-matter experts.

The classic metrics here are recall and precision. Does it return all of the results that it should and does it list the best results first.

Without a good search engine you might have the results you needed plus lots of other results. You'd have to scroll to page 20 of your results to actually see the result that you wanted, which means it wasn't very precise.

Think of internet search engines pre-google. With e.g. alta vista you had great recall but extremely poor precision. You'd often be scrolling multiple pages of results. Google turned that around by having great precision and similar recall. They made it so good that they implemented the "i feel lucky" button.

The trick with search is to have great precision and still good enough recall. That's super hard because what is precise is very subjective and highly dependent on your usecases, data, languages, etc.

This is why Elasticsearch is such a hugely complicated product: it includes a lot of solutions for essentially any use case you can imagine around search.

I have no experience with Redisearch; so I'll reserve my judgment. But this article is not doing it any favors.

There are competing things out there for Elasticsearch. Most of the serious ones also use Apache Lucene (e.g. Solr). Some of the upcoming ones are attempting to rebuild what Lucene does and may or may not be good enough depending on your use case. There have been some lucene ports over the years, including a C port. Most of those have fallen behind or are no longer maintained. The Java implementation is actually pretty good as is and has had a lot of performance and optimization work done to it over the years. You'd be hard pressed to build something as good and as fast without essentially using the same algorithms and reinventing a lot of the same wheels.

IMHO the current effort to build a search engine in Rust makes a lot of sense. The language is uniquely suited to doing the kinds of things Lucene does and they seem to be pretty serious about doing things properly.

Post reply on HN