Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

github.com

Post reply on HN
Print view View original

101–110 of 116 posts

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#101

Post by Kerollmops »

niyazpk wrote: post #100

Does anyone know if this supports bulk indexing? My team has a lot of data in S3 in parquet format. (We could change the format to something else if that helps). It would be really nice to be able to point tools like MeilliSearch or ElasticSearch to a data location and have it index all the data without me writing code to send individual records to the API.

This is not something that MeiliSearch supports currently but I am working on making the engine be able to index other formats than JSON, I saw great performance improvements when indexing simple CSVs.

We will probably make MeiliSearch accept different indexable formats (i.e. CSV, JSON, JSON-lines) in a future version.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#104

Post by Kerollmops »

dhruvkar wrote: post #103

I've never used elasticsearch and only had a brief toy project with Algolia. The demo on the github repo looks awesome. Can this run on top of my postgres database?

To make MeiliSearch expose the documents that are stored in your PostgreSQL (or any other database) you must extract them and store them in our engine using the HTTP API we provide to you. https://docs.meilisearch.com/references/documents.html#add-o...

For that you will need to also define the different attributes your document is composed of.

We thought about providing a simple tool to extract the documents from an SQL table into the MeiliSearch directly.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#105

Post by mleonhard »

MuffinFlavored wrote: post #4

The real power of Elasticsearch for me is the ability to filter logs by: 1. exact match this nested JSON field (with support for lists of values) 2. negative match this nested JSON field (with support for lists of values) coupled with the ability to filter by "timeframe", then pump it through to visualizations (tables/graphs) in Kibana MeiliSearch would be cool if it spoke the API Kibana expects from Elasticsearch

If only one could set up Elasticsearch and Kibana using infrastructure-as-code (IaC). I spent several days trying and still haven't succeeded. Elasticsearch config is full of foot-guns.

There are tons of easy setup examples but they lack access control and encryption. All of my servers must write logs. When one of them gets cracked, the attacker must not be able to read all the other servers' logs and steal all the PII. An attacker can use an ARP attack to MITM server connections to Elasticsearch. Without encryption, that attack yields all the PII.

I hope Meilisearch can someday help fill this gap in the free DevOps toolset.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#106

Post by mleonhard »

> MeiliSearch can serve multiple indexes, with different kinds of documents, therefore, it is required to create the index before sending documents to it.

https://github.com/meilisearch/MeiliSearch#create-an-index-a...

Indexes are config. This is not really zero-config if you require API calls before it can receive data.

Also, there's nothing about TLS or access control. These will be required for any production deployment. At the minimum, let us specify a TLS key.pem and cert.pem file and create write-only and read-only access tokens.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#108

Post by manigandham »

qdequelen wrote: post #76
manigandham wrote: post #52

Awesome, glad to see all the competition in the search space now. There are other projects like Sonic, Tantivy, Toshi and more that have more functionality if you need alternatives. Here's a public list of search projects (in rust, c, go): https://gist.github.com/manigandham/58320ddb24fed654b57b4ba2...

Take a look at our comparison page: https://docs.meilisearch.com/resources/comparison_to_alterna...

That's a good overview of the alternatives. Nice work on this.

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#109

Post by dijit »

heipei wrote: post #86
dijit wrote: post #38

Earlier quoted context omitted.

Elasticsearch is easier than mongo in some ways and harder in others. I run a few 10TiB ES clusters (which, is not much to be fair) but infrequently find that I have to reindex or reshard the cluster because I can’t just add another node. There’s something to be said for understanding the index rotation too, and access patterns. It’s easy to make an ES cluster, it’s difficult to maintain one, it’s nearly impossible t…

Would be interested to hear why you can't add another node in some cases.

If you have 5 shards on an index, adding a 6th node isn’t going to improve performance.

(Contrived example for the sake of illustrating the point)

Re: MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

#110

Post by _8j50 »

abofh wrote: post #97
fnord123 wrote: post #79

Earlier quoted context omitted.

That's marked for Q3: https://github.com/meilisearch/MeiliSearch/issues?q=is%3Aiss... But probably 99% of users using ES don't need sharding.

More accurately, 99% of the use cases ES is appropriate for don't need sharding. Every time I've needed to shard ES has been a nightmare bad enough that ES was abandoned.

I had a typical case of ingesting a ton of logs into ES. I needed sharding to keep up with multi-threaded writes while something else is doing intensive search queries. I think sharding was very useful in processing a lot of data efficiently.
Post reply on HN