Live data from Hacker News

Ternlight – 7 MB embedding model that runs in browser (WASM)

ternlight-demo.vercel.app

81–82 of 82 posts

Re: Ternlight – 7 MB embedding model that runs in browser (WASM)

#81
post #79

Apart from maximizing compatibility, is there a technical reason why GPU is not used if available?

mainly this is a portability and compatibility solution.. I think with GPU available deployments, it wouldn't benefit from ternary distillation. It would be a different problem domain.

Re: Ternlight – 7 MB embedding model that runs in browser (WASM)

#82
post #24

I added an offline search engine to app.wazzup.im/search (no login or payment required). First search downloads the model from the internet and subsequent runs are from the cache. The model is very small so it's not the best for everything but it's good for basic math and coding. Give it a try.

In Safari, stuck on: Loading model... + Loading search results... Or sometimes "Service Worker API is available and in use." + "Loading search results...".

Also does not work in Firefox 152.0.6 (aarch64). Due to WebGPU issue.
Post reply on HN