Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Search Benchmarking: RediSearch vs. Elasticsearch

redislabs.com

Post reply on HN
Print view View original

51–60 of 81 posts

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#51

Post by jstarfish »

gt565k wrote: post #39
jstarfish wrote: post #35

Earlier quoted context omitted.

Isn't that exactly what they're trying to demonstrate though? That all this arcana you have to invoke to get a stable ES cluster barely breaks a sweat on Redisearch? The specific test deployment was multitenant anyway-- you can't account or optimize for what tenants are going to index.

I'm not familiar with RediSearch, but I'm just trying to point out that you can't misconfigure ES and then benchmark against a misconfigured cluster. This is comparing apples to oranges. Not to mention I'm not sure of the feature difference between the 2 search engines, but I'd bet ES is much more feature rich, thus its use cases are vastly different. If you are just comparing text search, sure, maybe redis is faster…

I'm not familiar with Redisearch either but I agree, there's definitely a bent to this in that they made something that isn't ES, then compare it to ES via a a benchmark that shows how poorly ES performs at...being something other than ES.

The impression I got was that they were trying to demonstrate for two specific workloads, how much more a single node of RS can do than a single node of ES, and that we should extrapolate the savings and performance if scaled out from there.

A properly deployed ES cluster versus a single RS node isn't a fair comparison either. It's a strained comparison in any case.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#52

Post by Xylakant »

AznHisoka wrote: post #27
wyldfire wrote: post #16

Earlier quoted context omitted.

> Where elasticsearch shines is in complex queries ... If the "Multi-tenant indexing benchmark" is accurate it seems like it might be a robustness concern for ES. "Elasticsearch crashed after 921 indices and just couldn’t cope with this load." -- does that mean memory exhaustion or some other crash? If it's the latter, it seems like a quality problem more than a performance one.

Very very few customers actually have 921 indices in production. That is an insane amount.. by a large factor.

Judging from what I see on irc and when I get called for “our ES cluster is on fire, can you put it out?”, 921 indices is not much. I sometimes joke that I could replace myself with a bot that answers “less indices, less shards” to each and every question about performance and that bot could solve 90% of the problems at a fraction of my cost. But alas, nobody wants to pay for a visit from my bot.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#54

Post by Xylakant »

groestl wrote: post #48
alkz wrote: post #26

Earlier quoted context omitted.

each ES shard is actually a lucene index, and it uses memory... why would anyone need thousand of indices on a single node?

What's the difference, memory-wise, between a single shard and two shards holding half the data each?

I can’t quantify in bytes, but a shard comes with quite a bit of baggage, it has a mapping and other data stores in the cluster state, some other bookkeeping data attached to it (where, is it primary or replica, in sync or not, ...) and each shard allocates a chunk of the HEAP for index operations. That chunks size depends on whether the shard received writes or not and ranges from 5MB to 256(?)MB. The exact maximum varies from version to version and I don’t think it’s in the ES docs.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#55

Post by manigandham »

RedisLabs has done great work in developing Redis but these extensions to retrofit Redis into a multi-model database have issues.

Raw latency is usually not the primary concern most of the time and having everything in RAM can be a major cost problem, further compounded by the lack of compression available as with other persistent stores. The RESP protocol is also overloaded and hard to work with when dealing with json and search queries.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#56

Post by yazaddaruvala »

groestl wrote: post #48
alkz wrote: post #26

Earlier quoted context omitted.

each ES shard is actually a lucene index, and it uses memory... why would anyone need thousand of indices on a single node?

What's the difference, memory-wise, between a single shard and two shards holding half the data each?

I'll give a very, very basic example of why two shards with "half" the data is less optimal. More complicated optimizations can be left as an exercise to the reader.

Lets pretend the only data structure within a Lucene Shard is a Trie.

Given 4 strings, ["Hello", "World", "Help", "Thanks"]; A total of 20 chars.

With one shard, Lucene can utilize prefixing to find overlap between "Hello" and "Help". Meanwhile, "World" and "Thanks" are always fully stored. Resulting in a Trie of only 17 chars, i.e. a whopping (1 - 17/20) = 15% storage optimization!

With two shards Lucene potentially looses that optimization.

If the split is: ["Hello", "Help"], ["World", "Thanks"] then Lucene needs to store two Tries with 6 chars and 11 chars. Totaling: 17 chars and we still get a 15% optimization.

However, if the split is: ["Hello", "World"], ["Help", "Thanks"] then Lucene needs to store two Tries with 10 chars and 10 chars. Totaling: 20 chars for a 0% optimization :(

Now lets get back to reality, and remember that Lucene not only uses a LOT of optimizations (for both storage, and query performance), but also (for many reasons) pre-processing the data to find optimal shard placement is generally not an option, and the amount of data being indexed is generally so large that these optimizations are extremely powerful.

Just to make sure this comment is never used out of context: Sharding is still extremely important, and using a single shard is only recommended if you have insignificant amounts of data.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#57

Post by jimbokun »

jstarfish wrote: post #35
gt565k wrote: post #24

WOW. Hahahaha. This is a massive misconfiguration of an elastic search cluster. 50k indices? 500 documents per index? 500 records per index at 5shards/index is 100 records per shard. Yeah, let's shard our data so much that we introduce tremendous amounts of disk i/o overhead!!! Author should learn how to properly configure an ES cluster before posting ridiculous benchmarks like this. What an utter pile of garbage ben…

Isn't that exactly what they're trying to demonstrate though? That all this arcana you have to invoke to get a stable ES cluster barely breaks a sweat on Redisearch? The specific test deployment was multitenant anyway-- you can't account or optimize for what tenants are going to index.

"The specific test deployment was multitenant anyway-- you can't account or optimize for what tenants are going to index."

So in other words:

"If your specific use case is supporting 50,000 customers each having around 500 documents and only needing basic text search queries and relevance is not a major concern, RedisLabs Search might give you better performance than ElasticSearch!"

(This is assuming there isn't a different way to configure ElasticSearch to work for this scenario, that gives similar performance.)

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#58

Post by atombender »

showerst wrote: post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

Not to mention that Elasticsearch is excellent for non-text search.

One application I worked on indexes a Postgres database into Elasticsearch for live front-end queries. We index every single field, sometimes hundreds of fields in a single index. ES does this easily. Thanks to Lucene's quasi-columnar/quasi-LSM tree storage, new indexed fields aren't very expensive, and searches -- even fairly complicated ones -- are very fast.

ES is also extremely fast at aggregations. Even complex multi-level aggregations (e.g. group by date, then multiple nested buckets by different fields with "top k" results for each) take just a few hundred milliseconds for latge million-document datasets.

Where ES has problems are areas like replication, consistency and memory usage. It's very hard to tune ES; due to JVM GC and caches, it's basically impossible to predict how much RAM ES will need, and OOMs are common. There's also still no way to ask for a consistent index on query; the best you can do is use "waitfor=refresh" on indexing, which is the wrong time for it. I'd love a consistent Raft-based ES.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#59

Post by anonu »

showerst wrote: post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

What would be your go-to solution for a basic word search - lets say you only have a few MBs of data - not GBs...

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#60

Post by entangld »

atombender wrote: post #58
showerst wrote: post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

Not to mention that Elasticsearch is excellent for non-text search. One application I worked on indexes a Postgres database into Elasticsearch for live front-end queries. We index every single field, sometimes hundreds of fields in a single index. ES does this easily. Thanks to Lucene's quasi-columnar/quasi-LSM tree storage, new indexed fields aren't very expensive, and searches -- even fairly complicated ones -- are…

https://www.elastic.co/blog/a-new-era-for-cluster-coordinati...
Post reply on HN