Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meilisearch 1.0 – Open-source search engine built in Rust

blog.meilisearch.com

Post reply on HN
Print view View original

161–170 of 186 posts

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#161

Post by luhn »

naiv wrote: post #58
manigandham wrote: post #8

Congrats to the team, it's been interesting to watch the development of Meilisearch (and it's close competitor Typesense). Algolia has really paved the way here but it's nice to see the open-source options with more configurations and better default UX. There's also many search libraries if you want to embed search more deeply into your app. I have a list of modern search systems and libraries here: https://manigandh…

I will never understand who the target group of Algolia is besides a website where the number of records coincidentally is in the range of the number of queries. At least they got rid of the pricing per indexing transaction which made it even more absurd. If Algolia would offer an instance based pricing on cpu, ram and storage they would be the clear winner imho.

A while back I priced out what Algolia would cost us and it ended up being thousands of dollars per month for something that was currently running a t3.micro Elasticsearch instance. Our usage was just worst-case when it came to their pricing dimensions: A large number of very small documents with a low search volume.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#162

Post by adamnemecek »

dividedbyzero wrote: post #137
adamnemecek wrote: post #69

Earlier quoted context omitted.

Anything. Legit no language comes even close in terms of how easy it is to git clone something and get it to build.

But contributing takes a lot more than just a working build environment.

Correct but getting a build environment easily is such a quality of life improvement.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#163

Post by nezirus »

Kerollmops wrote: post #128
schappim wrote: post #124

We’ve used Meilisearch in production and it is the closest thing to self hosted Algolia you can get, which in itself is pretty amazing. Unfortunately the performance of indexing (constantly changing records) wasn’t great and Meilisearch would fall behind on indexing records for hours. Meilisearch has been amazingly great for projects where records don’t change all that much (eg docs, or even a customer database), but…

Thank you very much for this amazing feedback, really appreciated. We did a lot of improvement to the indexing part of the engine and now can auto-batch updates which gaves incredible improvements. We will continue to work on this in 2023. Can I know the version you were using?

My experience with indexing is similar. Up to lets say 1M docs it works fine, but after that it goes south. Even with auto-batch I had to manually prepare large bulk updates and wait for completion during inserts to not overload MS. (I am using Rust client).

Other than that, it is simply great. Ranking stuff is great, simple, I only need custom weights there, some additional functions (not just asc/desc) and it would be perfect.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#164

Post by Aeolun »

manigandham wrote: post #157
naiv wrote: post #106

Earlier quoted context omitted.

If you eg index geonames, you have 4 mio. records but you might only have 50.000 queries a month. you pay $4,000 for minimal compute resources, 4GB of RAM and 3 gigabytes of storage space. Would be less but algolia requires you to create a replica for each sort option separately. With 4 mio. records and 4 mio. queries I would pay the same. But then at least have 4 mio. queries. The other way around, if we would just…

Your usage seems to be in the "large" customer category where provisioned capacity is a better deal. Algolia does have volume discounts if you talk to them, but yes the other alternatives might be a better fit.

50000 people per day is ‘large customer’ territory? That’s less than a request per second.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#165

Post by message »

schappim wrote: post #124

We’ve used Meilisearch in production and it is the closest thing to self hosted Algolia you can get, which in itself is pretty amazing. Unfortunately the performance of indexing (constantly changing records) wasn’t great and Meilisearch would fall behind on indexing records for hours. Meilisearch has been amazingly great for projects where records don’t change all that much (eg docs, or even a customer database), but…

Same ran into same version inconsistencies with Java library.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#167

Post by amateurdev0_07 »

Thank you for making MeiliSearch. I use it for a personal project that gets a few hits a day, mostly from me and my friends.

https://pulpflakes.com/fmisearch/

It's a search over an index of fiction in the English language, first published in periodicals. Searchable by author, artist, magazine name and specific issue. Biggest index has about 200K documents, doc sizes are tiny.

Integrated with my WordPress site by handwritten PHP. Which was fun.

Performance is great. I didn't run into too many issues, and those I did i could resolve. What i remember:

1. The rules for text searches are too strict by default and if the order of words is different, will result in no matches. A, B will not return a result if B A is in the database.

2. Creating an index, uploading documents and changing settings required quite a bit of work. A week's worth of coding, almost. Would have loved to have a reasonably robust shell script that could take a JSON file with metadata on index and do the grunt work.

3. I have multiple types of documents, would have liked search to cover all of them so I don't have to change search type manually each time.

4. The default number of documents and max uploaded file size is too low. 200K and 200 MB or something. But it fails even on smaller file size.

The above sound like complaints. They're problems I ran into and others might. I love how productive Meilisearch made me. Thank you.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#168

Post by maxFlow »

Aeolun wrote: post #164
manigandham wrote: post #157

Earlier quoted context omitted.

Your usage seems to be in the "large" customer category where provisioned capacity is a better deal. Algolia does have volume discounts if you talk to them, but yes the other alternatives might be a better fit.

50000 people per day is ‘large customer’ territory? That’s less than a request per second.

Would you mind sharing your thought process to get from `daily_users` to `reqs_per_sec`? I'm playing with some estimations of `concurrent_users` for a basic website, and I'd be quite interested in the breakdown.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#169

Post by nacs »

nezirus wrote: post #163
Kerollmops wrote: post #128

Earlier quoted context omitted.

Thank you very much for this amazing feedback, really appreciated. We did a lot of improvement to the indexing part of the engine and now can auto-batch updates which gaves incredible improvements. We will continue to work on this in 2023. Can I know the version you were using?

My experience with indexing is similar. Up to lets say 1M docs it works fine, but after that it goes south. Even with auto-batch I had to manually prepare large bulk updates and wait for completion during inserts to not overload MS. (I am using Rust client). Other than that, it is simply great. Ranking stuff is great, simple, I only need custom weights there, some additional functions (not just asc/desc) and it would…

I had the same experience.

Pro: Meilisearch search speed and memory use was great compared to others (at the cost of large storage requirements but that's the cheapest thing to upgrade).

Con: Indexing documents (even with recommended batch sizes) was extremely intensive on the system as the document count increased (upwards of 20 million docs to index).

I had to modify the indexing script to completely pause indexing when system load average went too high to prevent the whole server crashing.

Also, this 1.0 upgrade apparently requires a full export and import of data if you're upgrading from the previous release? I hope this isn't the case for >= v1.0 releases because I'm not looking forward to exporting/reimporting 200+GB of Meilisearch data files over and over again.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#170

Post by 1248 »

pdimitar wrote: post #25
chimen wrote: post #10

Is Rust that important that you have to place "built in Rust" in the title? Is this like a cult following that we only bet on traffic and interest coming from other evangelists where Rust is the only feature that matter? 4 months ago: " Meilisearch, open-source alternative to Algolia in Rust lands a $15M Series A" It's not the first time I see, there are at least 2-3 daily submissions reaching the FP in this manner s…

It's important, yes. If it was written in C/C++ I'll fully expect to have my servers pwned due to a memory safety bug that these (and many other) languages don't protect against. There's a number of technical people with decision-making powers that pay attention. And a part of them prioritize Rust-written projects. It's a sound (literally) and safe investment. I don't get the people getting ticked off by the "written…

[dead]
Post reply on HN