Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

github.com

Post reply on HN
Print view View original

61–70 of 116 posts

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#61

Post by comex »

burntsushi wrote: post #58
comex wrote: post #57

Earlier quoted context omitted.

> Rust's safety story covers neither race conditions nor leaks. It covers a type of race condition, namely unsynchronized concurrent access to memory.

Data races and race conditions are orthogonal, according to some: https://blog.regehr.org/archives/490 I think you've disagreed with this in the past, and I don't know how to resolve that. But certainly, I think we can agree that saying that Rust's safety story prevents race conditions is, at minimum, very imprecise.

Oops, I wasn't trying to reopen an argument. I didn't recall us having discussed it before (still don't, but I forget things easily).

And yeah, I'd agree with your last sentence.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#62

Post by yazaddaruvala »

_8j50 wrote: post #60
pqdbr wrote: post #47

I'm impressed. I have a database with 15k documents, each with around 70 pages of text, HTML formatted. I'm using ElasticSearch currently, with the Searchkick gem. 30 min playing with MeiliSearch. So far: - Blazing fast to index, like 10x more performant than using ElasticSearch / Searchkick; - Blazing fast to search, at least 3x faster in all my random tests so far; - Literally zero config; - Uses 140MB of RAM curre…

You can add more nodes to scale search speed with ES, can you do the same with this?

More nodes is more throughput, not lower latency.

You’re always bounded by max single shard latency AND by coordination latency.

Ignoring how expensive it would be, over-sharding and over scaling (I.e. low volumes of data per shard and low shards per host) could reduce max single shard/host latency, however it’ll increase coordination latency but also memory (which directly or indirectly will cause more coordination latency).

Perfect data per shard and perfect shard per host numbers are currently an unsolved problem. They heavily depend on the domain, I.e. data types, data volume, data ingest, mappings, query types, query load.

:) if anyone has found a way to consistently add hosts to reduce latency, please let me know!

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#63

Post by natefox »

pqdbr wrote: post #47

I'm impressed. I have a database with 15k documents, each with around 70 pages of text, HTML formatted. I'm using ElasticSearch currently, with the Searchkick gem. 30 min playing with MeiliSearch. So far: - Blazing fast to index, like 10x more performant than using ElasticSearch / Searchkick; - Blazing fast to search, at least 3x faster in all my random tests so far; - Literally zero config; - Uses 140MB of RAM curre…

> in production.

I went looking, but found nothing regarding any operations management.

* How does this scale?

* How is it monitored? Where do I get the metrics for it? (indexing performance, search performance, etc.. Stuff not found in the OS)

* Are there any kind of throttling or queueing capabilities?

* What's the redundancy/HA approach?

* I'll ask about backups, though its the least of my worries as indexing databases like this and ES should be able to be rehydrated from source. However, snapshots may be faster to restore than reindexing.

This might be a nice local dev tool for something, but I'm not sure how you run a business critical application with it? I'm wondering if I'm missing something.

Edit: formatting

Edit2: also wondering about security too

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#64

Post by cormacrelf »

jniedrauer wrote: post #40
heipei wrote: post #20

I know the project doesn't claim it, but the title somewhat implies this: I honestly don't understand people claiming ElasticSearch is hard to operate, especially not at small scales. If anything, ElasticSearch for me has been one of the easiest pieces of infrastructure to operate, for me pretty much "zero-config". Let me elaborate: You can run ElasticSearch via Docker command-line, if you want a cluster you just sup…

I've been bitten by elasticsearch twice in my career, and I've seen others bitten by it as well. Once you put it in production, you can't just run it from docker on your workstation. You have to set up a cluster with enough capacity for whatever load you're going to throw at it, gracefully handle failures, updates, scaling up as load increases, etc. There are so many switches and dials to tune, and unless you really…

I once wasted a whole day trying to get two instances up on GKE. Permissions problems, about ten configs for the JVM alone, many more for ElasticSearch. You would fix one, restart, wait ten minutes, browse 50 pages of logs, google for half an hour, add a config, and goto 1. Never got it going in the end.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#65

Post by natefox »

jniedrauer wrote: post #40
heipei wrote: post #20

I know the project doesn't claim it, but the title somewhat implies this: I honestly don't understand people claiming ElasticSearch is hard to operate, especially not at small scales. If anything, ElasticSearch for me has been one of the easiest pieces of infrastructure to operate, for me pretty much "zero-config". Let me elaborate: You can run ElasticSearch via Docker command-line, if you want a cluster you just sup…

I've been bitten by elasticsearch twice in my career, and I've seen others bitten by it as well. Once you put it in production, you can't just run it from docker on your workstation. You have to set up a cluster with enough capacity for whatever load you're going to throw at it, gracefully handle failures, updates, scaling up as load increases, etc. There are so many switches and dials to tune, and unless you really…

> Once you put it in production, you can't just run it from docker on your workstation.

But that's true for any data store. This isn't any different. Nor is an RDBMS. They all need HA/replication. And that is rarely trivial.

Honestly, I think this is why managed/hosted solutions (AWS RDS for example) are so popular - they remove a large part of the complexity for you.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#66

Post by wolco »

the_arun wrote: post #59
eliseumds wrote: post #17

Pretty heavy user of ES here, and one cannot compare the two products.

What is the rationale for not comparing the two?

Only one filter. Very fast limited search. Not great for anything remotely complex like searching with two conditions.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#68

Post by atombender »

heipei wrote: post #20

I know the project doesn't claim it, but the title somewhat implies this: I honestly don't understand people claiming ElasticSearch is hard to operate, especially not at small scales. If anything, ElasticSearch for me has been one of the easiest pieces of infrastructure to operate, for me pretty much "zero-config". Let me elaborate: You can run ElasticSearch via Docker command-line, if you want a cluster you just sup…

Elasticsearch is very memory-intensive, and it's difficult to know exactly how much memory it will actually use, so you just have to throw a lot of RAM at it to avoid OOMing, then monitor it carefully, and hope your query concurrency won't accidentally blow the limits. Understanding why Elasticsearch is caving unpredictably under load is difficult, and GC pausing can be a significant performance sink.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#69

Post by hashhar »

rubyn00bie wrote: post #49
pqdbr wrote: post #47

I'm impressed. I have a database with 15k documents, each with around 70 pages of text, HTML formatted. I'm using ElasticSearch currently, with the Searchkick gem. 30 min playing with MeiliSearch. So far: - Blazing fast to index, like 10x more performant than using ElasticSearch / Searchkick; - Blazing fast to search, at least 3x faster in all my random tests so far; - Literally zero config; - Uses 140MB of RAM curre…

But... what happens if I need more than one instance? I'm genuinely curious. I hope this doesn't come off as an asshole comment. Isn't the whole point of ES versus just plain ol' lucene or solr the horizontal scalability of it?

I agree with all your points but a minor nit pick that Solr has been horizontally scalable for quite a while now.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#70

Post by tmzt »

manigandham wrote: post #52

Awesome, glad to see all the competition in the search space now. There are other projects like Sonic, Tantivy, Toshi and more that have more functionality if you need alternatives. Here's a public list of search projects (in rust, c, go): https://gist.github.com/manigandham/58320ddb24fed654b57b4ba2...

Are there any that fit the log searching use case, apart from loki which doesn't do full text searching?
Post reply on HN