Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformers.js – Run Transformers directly in the browser

github.com

Post reply on HN
Print view View original

31–40 of 54 posts

Re: Transformers.js – Run Transformers directly in the browser

#31

Post by sroussey »

grey8 wrote: post #25

transformers.js is such a cool library. I made a small web app with it that uses it to remove backgrounds from images (with BRIA AI's RMBG1.4 model) at https://aether.nco.dev The fact you don't need to send your data to an API and this runs even on smartphones is really cool. I foresee lots of projects using this in the future, be it small vision, language or other utility models (depth estimation, background removal…

I’m using it for a simple project: https://github.com/sroussey/ellmers

My plan is to test embedding and retrieval strategies for different RAG strategies to be used by a server or electron app.

Re: Transformers.js – Run Transformers directly in the browser

#32

Post by pjmlp »

jsheard wrote: post #3
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Basically the same problem that's plagued games on the web ever since the first Unreal/Unity asmjs demos a decade ago, and pretty much no progress has been made towards a solution in that time. You just can't practically make a web app which needs gigs of data on the client because there's no reliable way to make sure it stays cached for as long as the user wants it to, and as you say, even if you could reliably cach…

Actually more like the first Java Applets, Flash, and initially Unreal targeted Flash on the Web, and PNaCL, before asmjs came to be.

The Unreal 3 demo using Flash is still on YouTube.

And this is why most game studios are playing wait-and-see with streaming instead, proper native 3D APIs, with easier to debug tooling (Web still hasn't anything better than SpectorJS), and big size assets.

Re: Transformers.js – Run Transformers directly in the browser

#33

Post by fauigerzigerk »

jsheard wrote: post #26
fauigerzigerk wrote: post #24

Earlier quoted context omitted.

>Any new API which lets sites save stuff on the client is going to have the same issue. I don't think it would have the same issues, because the files could be stored in a user specified location outside the browser's own storage area. Browser vendors can't just delete stuff that may be used by other software on a user's system. And they cannot put a cap on it either, because users can store whatever they like in tho…

If that's how it works then it would avoid the problem I mentioned, but the UX around using that to cache data internal to the site implementation sounds pretty terrible. You click on "Listen to this article" on a webpage and it opens a file chooser expecting you to open t2s-hq-fast-en+pl.model if you already have it? Users won't be able to make any sense of that.

The API (or at least the Chrome implementation) appears to be unfinished, but the plan seems to be to eventually support persistent directory permissions.

So the web app could ask the user to pick a directory for model storage and henceforth store and load models from there without further interaction.

Re: Transformers.js – Run Transformers directly in the browser

#35

Post by fauigerzigerk »

jfoster wrote: post #27
fauigerzigerk wrote: post #10

Earlier quoted context omitted.

It's an inherent problem with on-device AI processing, not just in the browser. I think this will only get better when operating systems start to preinstall models and provide an API that browser vendors can use as well. Even then I think cloud hosted models will probably always be far better for most tasks.

> I think cloud hosted models will probably always be far better for most tasks It might depend on just how good you need it to be. There are lots of use-cases where an LLM like GPT 3.5 might be "good enough" such that a better model won't be so noticeable. Cloud models will likely have the advantage of being more cutting-edge, but running "good enough" models locally will probably be more economical.

I agree. The economic advantages of a hybrid approach could be very significant.

Re: Transformers.js – Run Transformers directly in the browser

#36

Post by xenova »

jfoster wrote: post #30
xenova wrote: post #19

Earlier quoted context omitted.

We’ve put out a ton of demos that use much smaller models (10-60 MB), including: - (44MB) In-browser background removal: https://huggingface.co/spaces/Xenova/remove-background-web . (We also put out a WebGPU version: https://huggingface.co/spaces/Xenova/remove-background-webgp... ). - (51MB) Whisper Web for automatic speech recognition: https://huggingface.co/spaces/Xenova/whisper-web (just select the quantized versi…

Thank you for the reply. Seems like all of the links are down at the moment, but it does sound a bit more feasible for some applications than I had assumed. Really glad to hear the last part. Some of the new capabilities seem fundamental enough that they ought to be in browsers, in my opinion.

Odd, the links seem to work for me. What error do you see? Can you try on a different network (e.g., mobile)?

Re: Transformers.js – Run Transformers directly in the browser

#37

Post by jfoster »

xenova wrote: post #36
jfoster wrote: post #30

Earlier quoted context omitted.

Thank you for the reply. Seems like all of the links are down at the moment, but it does sound a bit more feasible for some applications than I had assumed. Really glad to hear the last part. Some of the new capabilities seem fundamental enough that they ought to be in browsers, in my opinion.

Odd, the links seem to work for me. What error do you see? Can you try on a different network (e.g., mobile)?

Error is "xenova-segment-anything-web.static.hf.space unexpectedly closed the connection."

Works on mobile network, though, so might just be my internet connection.

Re: Transformers.js – Run Transformers directly in the browser

#39

Post by karaterobot »

Solvency wrote: post #29

can someone explain what this means I can do with it if I know vanilla JavaScript? for example, I used the image upscaling playground on hugging face all the time. But I do it manually here: https://huggingface.co/spaces/bookbot/Image-Upscaling-Playgr... would transformers.js allow me to somehow executive that in my own local or online app programmatically?

I think running locally is exactly what it's meant to do. I see an example using image upscaling (https://huggingface.co/docs/transformers.js/main/en/api/pipe...) but I have not used it.

Re: Transformers.js – Run Transformers directly in the browser

#40

Post by dajas »

I’m using this library to generate embeddings with gte-small (~0.07gb) and using Upstash Vector for storage.

It’s only 384 dimensions but it works surprisingly well with a paragraph of text! It also ranks better than text-embedding-ada-002 on the leaderboard

https://huggingface.co/spaces/mteb/leaderboard

Post reply on HN