Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Search Benchmarking: RediSearch vs. Elasticsearch

redislabs.com

Post reply on HN
Print view View original

21–30 of 81 posts

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#21

Post by coleifer »

How silly to emphasize things like "built as a C extension" and "uses modern data-structures" as if these were useful criteria for choosing a search engine.

It's about minimizing the effort needed to find what you're looking for. Speed of index construction time, unless we're talking orders of magnitude, isn't really meaningful. I don't know if this is just a really clumsy attempt at "marketing" or what, but I can't imagine this is going to convince anyone to drop es for this thing.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#22

Post by softwaredoug »

coleifer wrote: post #21

How silly to emphasize things like "built as a C extension" and "uses modern data-structures" as if these were useful criteria for choosing a search engine. It's about minimizing the effort needed to find what you're looking for. Speed of index construction time, unless we're talking orders of magnitude, isn't really meaningful. I don't know if this is just a really clumsy attempt at "marketing" or what, but I can't…

Also complaining that "Lucene is 20 years old" is about the same as saying "Linux is ~30 years old"

Lucene is a pretty rock-solid open source project that has been battle tested over those 20 years and had some of the best engineers in the world improve over a long time frame. That's an asset for Lucene!

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#23

Post by sidi »

"The more advanced multi-tenant use case – where RediSearch was able to complete 25 million indices in just 201 seconds or ~125K indices/sec, while Elasticsearch crashed after it indexed 921 documents, showing that it was not designed to cope with this level of load." previously stated that "Elasticsearch crashed after 921 indices and just couldn’t cope with this load."

It's hard to mistake documents for indices. Both original and the currently edited statement sound strongly suspect and make me question the benchmarking methodology used. What caused the ES to crash after indexing 921 documents? Why is comparing indexing speeds on a 1-node setup even a legit benchmarking test?

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#24

Post by gt565k »

WOW. Hahahaha.

This is a massive misconfiguration of an elastic search cluster. 50k indices? 500 documents per index?

500 records per index at 5shards/index is 100 records per shard.

Yeah, let's shard our data so much that we introduce tremendous amounts of disk i/o overhead!!!

Author should learn how to properly configure an ES cluster before posting ridiculous benchmarks like this.

What an utter pile of garbage benchmark this is.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#25

Post by papito »

softwaredoug wrote: post #22
coleifer wrote: post #21

How silly to emphasize things like "built as a C extension" and "uses modern data-structures" as if these were useful criteria for choosing a search engine. It's about minimizing the effort needed to find what you're looking for. Speed of index construction time, unless we're talking orders of magnitude, isn't really meaningful. I don't know if this is just a really clumsy attempt at "marketing" or what, but I can't…

Also complaining that "Lucene is 20 years old" is about the same as saying "Linux is ~30 years old" Lucene is a pretty rock-solid open source project that has been battle tested over those 20 years and had some of the best engineers in the world improve over a long time frame. That's an asset for Lucene!

That's a testament to how hard search engines are. Armies of engineers spent years getting to this point. This project just started as a POC for Redis modules, but this is really forced and futile. Trying to do this is like attaching wings to a motorcycle - it's not how search works. This falls apart beyond a primitive word lookup, which you can do with any SQL database. And it has been done, MANY times.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#26

Post by alkz »

wyldfire wrote: post #16
showerst wrote: post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

> Where elasticsearch shines is in complex queries ... If the "Multi-tenant indexing benchmark" is accurate it seems like it might be a robustness concern for ES. "Elasticsearch crashed after 921 indices and just couldn’t cope with this load." -- does that mean memory exhaustion or some other crash? If it's the latter, it seems like a quality problem more than a performance one.

each ES shard is actually a lucene index, and it uses memory... why would anyone need thousand of indices on a single node?

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#27

Post by AznHisoka »

wyldfire wrote: post #16
showerst wrote: post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

> Where elasticsearch shines is in complex queries ... If the "Multi-tenant indexing benchmark" is accurate it seems like it might be a robustness concern for ES. "Elasticsearch crashed after 921 indices and just couldn’t cope with this load." -- does that mean memory exhaustion or some other crash? If it's the latter, it seems like a quality problem more than a performance one.

Very very few customers actually have 921 indices in production. That is an insane amount.. by a large factor.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#28

Post by Scaevolus »

ademup wrote: post #4

I'm curious if this scales down well. The test was done on "One AWS c4.8xlarge with 36vCPU and 60GiB Memory". But could I run this on a tiny vps to index, search, catalog my million-odd documents?

SQLite's FTS5 module works surprisingly well on that number of documents, and stores the search index on disk instead of RAM.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#29

Post by AznHisoka »

showerst wrote: post #6

I've seen a lot of ES competitor posts pop up on HN lately, and I think they're missing the point of Elastic. If you only need very basic word search, ES is probably not worth the complexity in your stack, especially if you're already running a SQL database with decent plaintext search. Where elasticsearch shines is in complex queries: "Show me every match where this field contains 'extinction' within 10 words of 'im…

This X 2. I feel all these people who feel anything else is a viable alternative to Elasticsearch have a dumb, simple, small-scale use case, where even full-text search over Postgres would suffice.

Re: Search Benchmarking: RediSearch vs. Elasticsearch

#30

Post by gt565k »

gt565k wrote: post #24

WOW. Hahahaha. This is a massive misconfiguration of an elastic search cluster. 50k indices? 500 documents per index? 500 records per index at 5shards/index is 100 records per shard. Yeah, let's shard our data so much that we introduce tremendous amounts of disk i/o overhead!!! Author should learn how to properly configure an ES cluster before posting ridiculous benchmarks like this. What an utter pile of garbage ben…

Oh fuck me, I didn't even realize they used a single instance (node).

To expand a little bit, the whole point of using multiple shards per index in an ES cluster is so that the shards spread across multiple nodes (servers) and distribute the load (disk i/o) and handle redundancy. ES automatically scales and reshuffles its shards across multiple nodes in the cluster to handle fault-tolerance as well. If one or more nodes go down, the cluster still has all of the data through replica shards etc...

Either way, in this particular case, the data is so small, having 5 shards per index with 50k indices results in 250k shards for 5GBs of data.

5GB / 250k shards = 20kb per shard.

You have shards of size ~ 20kb ... total cluster misconfiguration.

Post reply on HN