Viewing profile — francoismassot

it's tantivy :)

Co-founder of Quickwit here. Seeing our acquisition by Datadog on the HN front page feels like a truly full-circle moment. HN has been interwoven with Quickwit's journey from the v…

Latest HN thread on quickwit (Binance built a 100PB log service with Quickwit): https://news.ycombinator.com/item?id=40935701 I also wrote a benchmark on Loki vs. Quickwit: https:/…

Indeed. They benefit from a discount, but we don't know the discount figure. To further reduce the storage costs, you can use S3 Storage Classes or cheaper object storage like Alib…

They have 181 trillion logs

Good question. Let's estimate the costs of compute. For indexing, they need 2800 vCPUs[1], and they are using c6g instances; on-demand hourly price is $0.034/h per vCPU. So indexin…

But you don’t have fast search on those files stored on object storage.

If you don't need vector search and have very large Elasticsearch deployment, you can have a look at Quickwit, it's a search engine on object storage, it's OSS and works for append…

One workaround is to use the JSON field, see doc https://github.com/quickwit-oss/tantivy/blob/main/doc/src/js...

Well, MongoDB was under AGPL v3.0 :)

Quickwit is an alternative with a strong focus on scalability (max we have seen is 40PB) with a decoupled compute and storage architecture. But we do only logs and traces for now. …

This is awesome; we need this kind of alternative to overpriced software like Splunk. We built and open-sourced Quickwit to see this kind of tool built on top of it. We will follow…

tantivy, not tantivity!!!!!

Thanks! Quickwit is the distributed engine built on top of tantivy, we basically separated compute and storage for search, I wrote this blog post to introduce the architecture: htt…

Some companies are using it with AWS Lambda to scale to 0.

Building the inverted index is quite CPU-intensive, and we are also merging index files called "splits".

BigQuery is just too costly... Do you know if the dataset is public? We should just offer a cheap alternative and ditch BigQuery.

Oh I forget to add stract is using tantivy too, I really hope this project will take off. https://stract.com/ https://github.com/StractOrg/stract https://news.ycombinator.com/item?…

> "Do store the Sonic database on SSD-backed file systems only." From the README, it works only on SSD. All those projects serve different purposes, and several are not actively ma…

The geocoder is built on top of tantivy which is fast and uses low resources too ( https://github.com/quickwit-oss/tantivy ). I'm curious about the comparison between those two.

I used it with my 10 years old boy for spelling. I like the method. I found the app is still rough on the edges, and now I want to code a small one dedicated to science fields for …