BM25 in PostgreSQL • Hacker News Board

Re: BM25 in PostgreSQL

#31

Post by rpcope1 » Tue, Mar 04, 2025, 7:30 AM UTC

gaocegege wrote: post #14

rpcope1 wrote: post #7

I'd be interested to see how this stacks up against Manticore or Meilisearch.

Thanks for sharing your feedback! Could you let me know the reason behind it? Are you currently using Manticore or Meilisearch?

As someone who has used Sphinx search for a long time, and wound up using Manticore after it got forked off of Sphinx, I found that it does much better than the native PostgreSQL search, while not being as resource hungry as Elastic. Especially because it's flexible and can pull from PostgreSQL (and usually the next step after the built in PostgreSQL FTS, for me), it's definitely something I'd be weighing any new search tooling in Postgres (such as yours or ParadeDB's) against.

Re: BM25 in PostgreSQL

#32

Post by snikolaev » Fri, Mar 07, 2025, 7:37 AM UTC

Jysix wrote: post #15

gaocegege wrote: post #14

Earlier quoted context omitted.

Thanks for sharing your feedback! Could you let me know the reason behind it? Are you currently using Manticore or Meilisearch?

Personally, I also compare it to using Meilisearch. What I find there is a superb doc, ease of use, full support for other European languages, possibility of sorting by field in addition to BM25, custom stop words (even better if the stop words for each language are already defined), custom ordering. If I could find all this in a Postgresql extension, it would be a dream. Congrats for the progress made, I'll give it…

Meilisearch doesn't support BM25, does it?

Re: BM25 in PostgreSQL

#33

Post by Kerollmops » Fri, Mar 07, 2025, 11:21 AM UTC

snikolaev wrote: post #32

Jysix wrote: post #15

Earlier quoted context omitted.

Personally, I also compare it to using Meilisearch. What I find there is a superb doc, ease of use, full support for other European languages, possibility of sorting by field in addition to BM25, custom stop words (even better if the stop words for each language are already defined), custom ordering. If I could find all this in a Postgresql extension, it would be a dream. Congrats for the progress made, I'll give it…

Meilisearch doesn't support BM25, does it?

Nope, it doesn't. It's based on Cascade Ranking, also called [bucket sorting][1]. We released our new Hybrid search ranking system, combining the best full-text search results (our Cascade Ranking) with semantic results (with arroy, our full-Rust Vector Store). You can try that at https://wheretowatch.meilisearch.com.

[1]: https://en.wikipedia.org/wiki/Bucket_sort

Re: BM25 in PostgreSQL

#34

Post by Kerollmops » Fri, Mar 07, 2025, 11:24 AM UTC

Jysix wrote: post #15

gaocegege wrote: post #14

Earlier quoted context omitted.

Thanks for sharing your feedback! Could you let me know the reason behind it? Are you currently using Manticore or Meilisearch?

Personally, I also compare it to using Meilisearch. What I find there is a superb doc, ease of use, full support for other European languages, possibility of sorting by field in addition to BM25, custom stop words (even better if the stop words for each language are already defined), custom ordering. If I could find all this in a Postgresql extension, it would be a dream. Congrats for the progress made, I'll give it…

Thank you very much! We put a lot of effort into our documentation and be ready for the next version of our documentation coming soon. The experience will be even better and faster. We also put much effort recently into simplifying how people can migrate to the next engine version with the [dumpless upgrade feature][1]. We also stabilized our full Rust Vector Store and Hybrid search (AI-powered search) feature in v1.13.

[1]: https://github.com/meilisearch/meilisearch/releases/tag/v1.1...