Meilisearch 1.0 – Open-source search engine built in Rust • Hacker News Board

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#181

Post by survirtual » Thu, Feb 09, 2023, 5:37 PM UTC

This looks like an effective piece for a project I have. It would be significantly more effective if it was published on crates.io and could be instantiated within Rust, and was able to operate in memory (or have a filesystem passed to it, so that can be simulated)

I found this issue which tracks crates.io publication: https://github.com/meilisearch/meilisearch/issues/3367

Would be nice to see that made a priority. Having a powerful search engine that can be embedded in a larger application and made portable (like being able to deploy to WASM) would be extremely novel and valuable. Given Rust is already in use, I think it may not necessarily demand too much effort. When search becomes a focus for what I’m working on, perhaps I will make that happen if not already done yet.

Thanks for making this available to people.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#182

Post by wtetzner » Thu, Feb 09, 2023, 8:56 PM UTC

chimen wrote: post #10

Is Rust that important that you have to place "built in Rust" in the title? Is this like a cult following that we only bet on traffic and interest coming from other evangelists where Rust is the only feature that matter? 4 months ago: " Meilisearch, open-source alternative to Algolia in Rust lands a $15M Series A" It's not the first time I see, there are at least 2-3 daily submissions reaching the FP in this manner s…

This is Hacker News, where people are ostensibly interested in the technology used to build projects.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#183

Post by xarope » Fri, Feb 10, 2023, 1:40 AM UTC

I tried RTFM'ing, but can Meilisearch handle restricted documents (P&C) and integrated with LDAP/AD to pull security groups?

P.S. great to see your documentation search is powered by your own product (!)

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#184

Post by tmikaeld » Fri, Feb 10, 2023, 5:27 AM UTC

snikolaev wrote: post #175

tmikaeld wrote: post #121

Earlier quoted context omitted.

Our index was aimed at handling 20 000 documents at total of 35MB of CSV, this would balloon into 0.7GB to 1GB of RAM and we expected at least 1000 of these indexes, which would require dedicated servers with 1TB of RAM. This was when Meili was at version 0.27. With manticore, we've tried to run into these issues in benchmarks, but the only problem we got was temporary high IO load when indexes need to be re-indexed…

BTW if you are using the default row-wise Manticore storage, you may try out the Manticore columnar storage [1]. It can decrease the RAM consumption further. [1] https://manual.manticoresearch.com/Introduction#Storage-opti...

Thank you for that, I'll give it a go today!

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#185

Post by tmikaeld » Sat, Feb 11, 2023, 2:27 PM UTC

tmikaeld wrote: post #184

snikolaev wrote: post #175

Earlier quoted context omitted.

BTW if you are using the default row-wise Manticore storage, you may try out the Manticore columnar storage [1]. It can decrease the RAM consumption further. [1] https://manual.manticoresearch.com/Introduction#Storage-opti...

Thank you for that, I'll give it a go today!

It seems to be too slow for our amount of updates, updates would need to rewrite the whole column.

Re: Meilisearch 1.0 – Open-source search engine built in Rust

#186

Post by fyzix » Sat, Feb 11, 2023, 3:23 PM UTC

They need to structure their pricing page better. A quick glance had me thinking that $1200 was the minimum for production use. But the free tier is actually pay as you go.