Using this for our new Svelte community website. Very pleased with it so far.
What's the url for the Svelte community website?
SQLite FTS5 Extension
31–40 of 61 posts
Re: SQLite FTS5 Extension
#32Using this for our new Svelte community website. Very pleased with it so far.
What's the url for the Svelte community website?
The old one lives at https://sveltesociety.dev - be aware, lots of very old content, but components and packages pages are up-to-date.
The new one lives here but is under heavy construction: https://v2.sveltesociety.dev
Ironically, the Search doesn't work on the deployed version at the moment.
Edit: Looks like the search actually works if you're logged in.
Re: SQLite FTS5 Extension
#33I use it in one of my macOS apps (ShellHistory) for storing and searching history from shell (zsh, bash, fish). Works very well.
Re: SQLite FTS5 Extension
#34This is hugely underrated in my opinion: it’s a very competent search engine. It also ships as part of the Python standard library, so if your machine has Python installed you have a high quality search engine ready to use without installing anything else. I have a CLI tool (and Python library) for working with it here: https://sqlite-utils.datasette.io/en/stable/cli.html#configu...
On a lark I used this to build a local-first code search tool for my organisation. Of course I used datasette for running the queries. Other than getting all my git access revoked and receiving a very concerned email from our security team, it was a great project.
Wait, why they revoked git access?
Re: SQLite FTS5 Extension
#35Does anyone have successfully worked with Non-English text with FTS5 in Sqlite? I could not find any reference for German, e.g. and the default stemming does not seem to work properly (given some short tests).
> Does anyone have successfully worked with Non-English text with FTS5 in Sqlite? I could not find any reference for German, e.g.
We use it in the Fossil SCM project and users have reported success with Chinese and Russian, so it presumably works fine with any European/Germanic language.
> and the default stemming does not seem to work properly (given some short tests).
The Porter Stemmer is documented as only being useful for English.
Re: SQLite FTS5 Extension
#36Earlier quoted context omitted.
On a lark I used this to build a local-first code search tool for my organisation. Of course I used datasette for running the queries. Other than getting all my git access revoked and receiving a very concerned email from our security team, it was a great project.
Wait, why they revoked git access?
Someone unexpectedly ingesting all git repos at once would look a lot like a compromised workstation being used to harvest company data.
I assume everything was fine once they explained what was going on.
Re: SQLite FTS5 Extension
#37Not sure if I'm holding it wrong, but when I played with it I nowhere got results that are anything remotely like a proper search engine (thinking of Algolia, Elasticsearch, etc.).
It was good at finding the text I was searching for but in terms of ranking them, it felt like it needed a lot of post processing.
Re: SQLite FTS5 Extension
#38Earlier quoted context omitted.
I don't think that will work for me, since I needed something that can handle mistakes in the words, like Du'ha to duha etc, Rahman to rehman, basically whatever looks closest.
One thing you could do: FTS5 has the `fts5vocab` virtual table [1] that has all the terms. You could provide a user-defined function that computes the levenshtein distance between your query terms and the terms in that table, obtain candidate terms that way and build a big query that searches for all those lexically close terms. [1] https://www.sqlite.org/fts5.html#the_fts5vocab_virtual_table...
I can confirm an approach like this works in practice.
Although instead of levenshtein I use spellfix (maybe it uses that under the covers? not sure). If there is no match from the first search, I use the sqlite spellfix extension [0] to find matches. Then feed those candidates into the terms.
Re: SQLite FTS5 Extension
#39Not sure if I'm holding it wrong, but when I played with it I nowhere got results that are anything remotely like a proper search engine (thinking of Algolia, Elasticsearch, etc.). It was good at finding the text I was searching for but in terms of ranking them, it felt like it needed a lot of post processing.
Have you used ORDER BY bm25(...)?
Re: SQLite FTS5 Extension
#40[deleted]