What we need is a W3C LLM API like the one Chrome already offers: https://developer.chrome.com/docs/ai/built-in
I think standardizing the runtime is pretty effective, it then open up portability
Ternlight – 7 MB embedding model that runs in browser (WASM)
11–20 of 82 posts
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#12Great, now my websites are gonna push entire LLMs onto my browser in order to use my CPU to make inferences about my shopping habits or whatever.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#13[dead]
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#14This is cool! but also maybe you could put a button on the landing page to trigger the demo because it's a bit startling to hear my fans go crazy when opening a webpage.
Agree. But this also reminds me fondly of the days where the sounds of my computer so intimately indicated what’s going on.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#15Very cool! I'd love to point it at my own corpus to index/embed. Would be cool if you could give it a link to a markdown file or even a website to crawl.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#16Why do these things download into the browser automatically? This could be used to distribute malware and also or hog excessive browser memory.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#17Great, now my websites are gonna push entire LLMs onto my browser in order to use my CPU to make inferences about my shopping habits or whatever.
Disabling WASM is the new disable JavaScript
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#18This is cool! but also maybe you could put a button on the landing page to trigger the demo because it's a bit startling to hear my fans go crazy when opening a webpage.
Agree. But this also reminds me fondly of the days where the sounds of my computer so intimately indicated what’s going on.
Amiga floppy disk sounds are the deepest of sense memories.
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#19Why do these things download into the browser automatically? This could be used to distribute malware and also or hog excessive browser memory.
[deleted]
Re: Ternlight – 7 MB embedding model that runs in browser (WASM)
#20Great, now my websites are gonna push entire LLMs onto my browser in order to use my CPU to make inferences about my shopping habits or whatever.
Ha, I was literally thinking this but from the other side.
"Hmm, 7MB would barely make a dent in the size of the app and allow us to do some of our basic ML without calling the backend"
Probably a lot more practical to use this though: https://developer.apple.com/apple-intelligence/