61–70 of 82 posts
Earlier quoted context omitted.
Agree. But this also reminds me fondly of the days where the sounds of my computer so intimately indicated what’s going on.
Amiga floppy disk sounds are the deepest of sense memories.
Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD. It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared…
Awesome. We have a dictionary of words to OpenStreetMap tags here : https://codeberg.org/cartes/web/src/branch/master/components... Do you think your work could help us let users type "pancake" and get "crêpe" without writing an explicit "pancake = crêpe" dictionary entry ? In practice : if I understand well, your lib would first need to download 5 Mb, once and for all, and would then be used as we use Fuse.js right…
Ideally you would have real query data (e.g. from cartes.app telemetry), then you could get a LLM to write a bespoke Overpass query for each one and use that as the ground truth. Alternatively, start from the list of OSM tag values used in the wild and ask an LLM to list possible reasons to visit that POI.
You could then use that data to finetune an embedding model for your use case. But, you know, somewhere in that model there's going to be a token vocabulary that the model knows about and at the other end you get a similarity score for each tag value. If you don't need to support complex queries where interactions between words matter ("any restaurant that is NOT Korean"), you could get away with a simple list of words and tags that they match to. Which is right where you started, except it could be more exhaustive. Why limit yourself to two Korean dishes when you can have a LLM list many more for you?
This way on the frontend you can lazily load this. Maybe you could even store the HNSW in chunks and just load the pieces you need for your specific search query.
i.e. like https://pagefind.app/ but to get fully static vector search.
This would be nice as an Astro (or generic meta-framework plugin) that automatically parses all generated html files and generates a small db of embeddings. This way on the frontend you can lazily load this. Maybe you could even store the HNSW in chunks and just load the pieces you need for your specific search query. i.e. like https://pagefind.app/ but to get fully static vector search.
If anybody knows of a good solution in this space, or if I’m wrong about SQLite-vec, please let me know. For our own SSG we’ve basically decided that we’ll give it a couple months while we work on other infra we want, then if they’re still not done we’ll just do it ourselves.
Earlier quoted context omitted.
Awesome. We have a dictionary of words to OpenStreetMap tags here : https://codeberg.org/cartes/web/src/branch/master/components... Do you think your work could help us let users type "pancake" and get "crêpe" without writing an explicit "pancake = crêpe" dictionary entry ? In practice : if I understand well, your lib would first need to download 5 Mb, once and for all, and would then be used as we use Fuse.js right…
The OSM tag wiki would probably not make for good training data because it only has a single short description for each tag (and even on the French wiki many descriptions seem to be in English?) whereas you want to map multiple descriptions to the same tag. Ideally you would have real query data (e.g. from cartes.app telemetry), then you could get a LLM to write a bespoke Overpass query for each one and use that as t…
It’s advertised 7MB, but also comes with a 5MB mini version.
Looks like mini saves space by using 256 element vectors internally instead of 384, but then projects it up to 384 at the end for compatibility.
It’s a third smaller, but the loss is not linear, looks like you give up less than 1/3 of information with the smaller data path.