Live data from Hacker News

Meilisearch 1.0 – Open-source search engine built in Rust

blog.meilisearch.com

171–180 of 186 posts

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#171
post #164

Earlier quoted context omitted.

50000 people per day is ‘large customer’ territory? That’s less than a request per second.

Would you mind sharing your thought process to get from `daily_users` to `reqs_per_sec`? I'm playing with some estimations of `concurrent_users` for a basic website, and I'd be quite interested in the breakdown.

let's think step by step:

24hrs * 60mins * 60secs = 86400

86400 QED

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#172

Great news! Been following along for a while and it's a great project. ElasticSearch needs some competition. For us, there are two things missing for us before we could make the switch: 1. Multi-index search; Standard use-case is searching across e.g. users and companies. Common in many SaaS-applications, where you want a single search field with type-ahead for e.g. contacts/organisations/tasks/events. 2. Decay funct…

Elastic already has a great competitor called solr, which I prefer on multiple aspects over elastic by the way.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#173

The most specific criticism I have read of Meilisearch is https://news.ycombinator.com/item?id=32940683 . It has four points: (1) words beyond 65535 are silently ignored (this is documented in https://docs.meilisearch.com/learn/advanced/known_limitation... ); (2) the position of a matching word in a document non-optionally affects ranking; (3) to get the match information you must retrieve the entire attribute; (4) t…

What’s funny is that (1) doesn’t look like a real limit when you know that the first Harry Potter book is nearly 77000 words. The recommended way is to split your documents by paragraph to increase relevancy, this way you can see the exact part that match. About (2) we will work on exposing two new ranking rules to be able to control that. For (3) I thought it was fixed. We decided to implement (4) the PUT and POST t…

Thanks for your reply. I agree about (1). I have checked the datasets I have set up search for, and they either have no or under 1% of documents with more than 65535 words. (This is without any processing to break up the documents into sections.)

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#174
post #164

Earlier quoted context omitted.

Your usage seems to be in the "large" customer category where provisioned capacity is a better deal. Algolia does have volume discounts if you talk to them, but yes the other alternatives might be a better fit.

50000 people per day is ‘large customer’ territory? That’s less than a request per second.

You're missing the point. Their scale (in number of records and queries) is a better fit for provisioned capacity than pay-as-you-go in the context of billing.

The exact small/large label doesn't matter, nor does the requests-per-second.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#175
post #21

Earlier quoted context omitted.

Do you have any numeric definitions for few and lots?

Our index was aimed at handling 20 000 documents at total of 35MB of CSV, this would balloon into 0.7GB to 1GB of RAM and we expected at least 1000 of these indexes, which would require dedicated servers with 1TB of RAM. This was when Meili was at version 0.27. With manticore, we've tried to run into these issues in benchmarks, but the only problem we got was temporary high IO load when indexes need to be re-indexed…

BTW if you are using the default row-wise Manticore storage, you may try out the Manticore columnar storage [1]. It can decrease the RAM consumption further.

[1] https://manual.manticoresearch.com/Introduction#Storage-opti...

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#176
post #126

Earlier quoted context omitted.

Thanks. Would I be correct in assuming that there should be less of a burden on me as an end user with Rust though as I only need to update the one binary that I installed?

How did you install the binary? If you installed it from source, you'll need to `cargo update && cargo build`. If you downloaded a binary or installed it from your distro's repo, generally you just need to update that one binary, yes.

Thanks for clarifying. Yes, I'd typically install a Rust program from the distro's repos.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#178
post #150

Earlier quoted context omitted.

>May I know which part of the comparison table makes it sound like that? Well a few things. Normally I'd try to coach these a bit kinder and all that, but I hope you don't mind if I just come out and talk about the issues. Keep in mind that these are just my interpretations after a quick read through. # Bias > Instant Search-as-you-type Experiences for up to a few hundred thousand records, that don't require a produc…

I really appreciate that you took the time to write this detailed comment! Thank you! > but for me one example of production looks like a raspberry pi in a school house in rural africa (internet in a box project) This is an interesting perspective, one that I hadn't considered before. You're saying that software can be run in a variety of different environments and that the definition of what a "production" environme…

Like I say, it's just my subjective gut reaction. I guess if there was one main take away it's "describe your competitors with the same language you'd use to describe yourself", where possible.

None, completely stand-alone with built in http(s) server | None, recommends a reverse proxy

As an example.

>So when I say Meilisearch is not "production-ready", it's in this specific context - it can only be run on a single node, and it cannot handle infrastructure failures natively. So it could become single point of failure.

And I don't really disagree with that, but it really is up to a judgement call on whoever is setting it up. If search isn't a critical feature than whoever is setting up might prefer meilisearch for it's memory model. For example I once worked on "the great canadian encyclopedia", which ran on a single VPS and needed search capability. It already had a single point of failure, so running search on the same VPS wasn't a big concern. There are also different roll-over policies, different uptime guarantees, different architectures, etc. If "production grade" was some kind of industry standard that would be one thing, but it really really does depend on the client.

I think that the single point of failure thing is a very important consideration, and should probably be in your overview along side the memory/data model, but I do honestly think typesense's memory-only disqualifies it from a lot of production systems I've worked on, and that meilisearch's single point of failure hasn't. Fault-tolerance and single point of failure deserves it's own row in your overview, it shouldn't be thrown into the use-case column.

Honestly it's only really an issue when taken all together, fix a few of those and you'll be in much better shape I think.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#179
post #105

Earlier quoted context omitted.

For context (and I say that as a Rust developer), please note that Rust has the same strength/weakness as Node in terms of dependencies. There is ongoing work to strengthen this. I do not know the status.

Thanks. Would I be correct in assuming that there should be less of a burden on me as an end user with Rust though as I only need to update the one binary that I installed?

Usually you would only need to update a single binary, as the dependencies are compiled in.

You may get some libc-issues if you try to run a binary built for a newer Linux on an older Linux, unless it is built to target musl - don’t remember the details 100%

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#180
post #7

We’ve been using a Meilisearch for the last six months or so and have been delighted with its performance and usability. It uses a fraction of the resources as Elasticsearch, and the language support is extensive and very active. That being said, our cluster is much smaller than other ones I’ve worked with in the past, so I can’t comment on its reliability at massive scale. I’ve also been very impressed with how acti…

This is the thing I find when people post "ElasticSearch Alternative". 80% of ElasticSearch's value add (wrt search anyway) is all the clustering and frame work that allows you to span the search over tens or hundreds of machines "easily". I think the same is true here. Probably the comparison should be with the underlying search libraries that ES sits on. I suppose this comparison makes sense in a world where most p…

This was the sense I got as well, though I have only started playing w/ Meilisearch. Clustering was one of the top 3 features that let Elasticsearch take over the market so quickly. In the playing around w/ Meilisearch I've done, it seems more like a replacement for something like Sphinx so far.
Post reply on HN