Live data from Hacker News

Sonic: Fast, lightweight and schemaless search back end in Rust

github.com

11–20 of 40 posts

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#11
post #9

Wow. I’ve spent the last three weeks building a custom search solution in Kotlin - in my case I’m using tokenizers from Lucene and using a radix trie as an index. I actually looked at using Bleve (another rust search lib) initially but it didn’t have the right language support Glancing over this it looks like a nearly perfect fit for my use case I just wish I had seen this a couple of weeks earlier!

And that's why I believe that the search/discovery problem is not solved yet by google

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#12
post #9

Wow. I’ve spent the last three weeks building a custom search solution in Kotlin - in my case I’m using tokenizers from Lucene and using a radix trie as an index. I actually looked at using Bleve (another rust search lib) initially but it didn’t have the right language support Glancing over this it looks like a nearly perfect fit for my use case I just wish I had seen this a couple of weeks earlier!

Bleve, from Couchbase, is in Go :-)

https://github.com/blevesearch/bleve

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#13

Talk about perfect timing! I was looking for something just like this for a project in my team. We had been using this setup where a huge chunk of the data was being stored in triplicate: some of it in ES, some more of it in another database and finally the whole dataset in our data warehouse. Hopefully I can use this to only provide the index + full text capability and just use the warehouse itself as the main db be…

What's preventing you from using ES for everything? Slow writes?

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#14
post #13

Talk about perfect timing! I was looking for something just like this for a project in my team. We had been using this setup where a huge chunk of the data was being stored in triplicate: some of it in ES, some more of it in another database and finally the whole dataset in our data warehouse. Hopefully I can use this to only provide the index + full text capability and just use the warehouse itself as the main db be…

What's preventing you from using ES for everything? Slow writes?

Write speed is fine, it’s more the fact that the dataset is reasonably large, and to run an instance with enough capacity and nodes (even with spill to disk), is silly expensive.

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#15
post #13

Earlier quoted context omitted.

What's preventing you from using ES for everything? Slow writes?

Write speed is fine, it’s more the fact that the dataset is reasonably large, and to run an instance with enough capacity and nodes (even with spill to disk), is silly expensive.

Being in a simular situation what do you consider "reasonably large"?

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#16
post #4

This looks pretty cool! I'd like to try it, but it looks like I'll have to wait for the Python client library first.

The protocol description makes it look fairly trivial to script,

https://github.com/valeriansaliou/sonic/blob/master/PROTOCOL...

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#18
post #11
post #9

Wow. I’ve spent the last three weeks building a custom search solution in Kotlin - in my case I’m using tokenizers from Lucene and using a radix trie as an index. I actually looked at using Bleve (another rust search lib) initially but it didn’t have the right language support Glancing over this it looks like a nearly perfect fit for my use case I just wish I had seen this a couple of weeks earlier!

And that's why I believe that the search/discovery problem is not solved yet by google

Far from it, search sucks nowadays. Just think about how hard it is to find stuff from Twitter, reddit etc, even though that's where most of the content is.

And if you don't know what you want exactly, you can't find anything.

Re: Sonic: Fast, lightweight and schemaless search back end in Rust

#19
post #4

This looks pretty cool! I'd like to try it, but it looks like I'll have to wait for the Python client library first.

Wanna build it with me?

Normally I would, but I don't have an immediate use for this, which kills my motivation
Post reply on HN