[flagged]
Ternlight – 7 MB embedding model that runs in browser (WASM)
51–60 of 82 posts
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#52cool stuff
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#53Cool project!
I tried something similar a while ago [1] - I wanted to load up an embedding model and semantically order texts, all in the browser.
So I pull ONNX weights from HuggingFace (MPNet, MiniLM), use Transformers.js to embed, and use a clusterer from scikit-learn (running on pyiodide - it was a surprise to me that this worked flawlessly) on the page - all client-side.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#54This would be a pretty cool addition to the duckdb HNSW search project I found on here some time ago: https://github.com/jasonjmcghee/portable-hnsw What I think is really cool is that the search happens using http range queries across statically hosted parquet files. I think things like this could bloom into a relatively open and distributed search ecosystem that isn’t controlled by major corporations.
Cool idea. I love range requests and other static-hosted client-navigable formats!
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#55Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD. It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared…
What is the process for adding different text? What are the limitations on that process? The demo is very cool by the way.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#56Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD. It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared…
Awesome. We have a dictionary of words to OpenStreetMap tags here : https://codeberg.org/cartes/web/src/branch/master/components...
Do you think your work could help us let users type "pancake" and get "crêpe" without writing an explicit "pancake = crêpe" dictionary entry ?
In practice : if I understand well, your lib would first need to download 5 Mb, once and for all, and would then be used as we use Fuse.js right now ?
How well does it handle languages other than English ?
Could it be "trained" on the OpenStreetMap tag wiki ?
Thanks a lot for your work.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#57This is cool! but also maybe you could put a button on the landing page to trigger the demo because it's a bit startling to hear my fans go crazy when opening a webpage.
Same here, when the fans started up I got startled. However my bread toaster often scares me too
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#58This would be a pretty cool addition to the duckdb HNSW search project I found on here some time ago: https://github.com/jasonjmcghee/portable-hnsw What I think is really cool is that the search happens using http range queries across statically hosted parquet files. I think things like this could bloom into a relatively open and distributed search ecosystem that isn’t controlled by major corporations.
Similar idea here that may be of interest: SQLite DB on static host via HTTP range + WASM.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#59Prime example of wasm supremacy over JavaScript.
Stack machines for the win hehe
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#60Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD. It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared…
Thank you for this tool!
We've just used it to embed the entire django doc + our private knowledge base, allowing us to search in the 2 sources instantly!