[flagged]
Ternlight – 7 MB embedding model that runs in browser (WASM)
21–30 of 82 posts
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#22Why do these things download into the browser automatically? This could be used to distribute malware and also or hog excessive browser memory.
That's... how the web works? You download things on demand.
There are JS files larger than 7MB in the wild. They run on JIT engines that displayed severe CVEs over the years. PDFs, video running directly on special hardware encoders. That's the web now.
A WASM model is not that offensive.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#23That's really impressive, congratulations. It's nice to see novel applications of browser models.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#24I added an offline search engine to app.wazzup.im/search (no login or payment required). First search downloads the model from the internet and subsequent runs are from the cache. The model is very small so it's not the best for everything but it's good for basic math and coding. Give it a try.
In Safari, stuck on:
Loading model... + Loading search results...
Or sometimes "Service Worker API is available and in use." + "Loading search results...".
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#25Great, now my websites are gonna push entire LLMs onto my browser in order to use my CPU to make inferences about my shopping habits or whatever.
Ha, I was literally thinking this but from the other side. "Hmm, 7MB would barely make a dent in the size of the app and allow us to do some of our basic ML without calling the backend" Probably a lot more practical to use this though: https://developer.apple.com/apple-intelligence/
[deleted]
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#26Why do these things download into the browser automatically? This could be used to distribute malware and also or hog excessive browser memory.
This doesn't add any malware risks beyond what a JavaScript-enabled browser already allows.
Re excessive browser memory use: Yes, it adds non-negligible weight, but again, you could already achieve excessive browser memory usage before this. For comparison, a true color 1080p image, uncompressed (which is needed for actual display on screen) is only slightly smaller at 6.22Mb.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#27I added an offline search engine to app.wazzup.im/search (no login or payment required). First search downloads the model from the internet and subsequent runs are from the cache. The model is very small so it's not the best for everything but it's good for basic math and coding. Give it a try.
In Safari, stuck on: Loading model... + Loading search results... Or sometimes "Service Worker API is available and in use." + "Loading search results...".
This is a known issue and I am actively trying to find why this is happening. So far it's pretty good on Brave/Chrome.
Tested on Macbook Pro M1 8gb RAM and Macbook Air M1 8gb RAM. Mostly likely because of M series of chips. All tests were done on Brave/Chrome.
Does not work on iPhone 11 Pro Max and iPhone 16 Pro. Mostly likely because of A series of chips. Tests were done on Safari and Chrome and it crashes on both.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#28Very cool! I'd love to point it at my own corpus to index/embed. Would be cool if you could give it a link to a markdown file or even a website to crawl.
love the idea! Will think of a way to host it probably on huggingface
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#29I added an offline search engine to app.wazzup.im/search (no login or payment required). First search downloads the model from the internet and subsequent runs are from the cache. The model is very small so it's not the best for everything but it's good for basic math and coding. Give it a try.
In Safari, stuck on: Loading model... + Loading search results... Or sometimes "Service Worker API is available and in use." + "Loading search results...".
ohh thanks for the report.. probably has to do with wasm runtime.. Will note this as a known issue
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#30That's really impressive, congratulations. It's nice to see novel applications of browser models.
thank you! hopefully it can unlock some novel applications, that would be cool