Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformers.js – Run Transformers directly in the browser

github.com

Post reply on HN
Print view View original

21–30 of 54 posts

Re: Transformers.js – Run Transformers directly in the browser

#21

Post by jampekka »

elpocko wrote: post #17
jampekka wrote: post #13

Earlier quoted context omitted.

There is such API. https://developer.mozilla.org/en-US/docs/Web/API/File_System...

Still requires drag&drop on most browsers because the File/DirectoryPicker API isn't universally supported.

Origin private file system is supported in all modern browsers. That does make sharing the models between origins difficult at best, but for one origin works fine.

And in any case it's easier to direct users to install Chrome (or preferably Chromium) or instruct drag&drop than doing the brittle and error and bitrot prone virtualenv-pip-docker-git song and dance.

Re: Transformers.js – Run Transformers directly in the browser

#22

Post by yeldarb »

jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Not using transformers, but we do object detection in the browser with small quantized yolo models that are about 7mb and run at 30+ fps on modern laptops via tensorflow.js and onnxruntime-web.

Lots of cool demos and real world applications you can build with it. Eg we powered an AR card ID feature for Magic: The Gathering, built a scavenger hunt for SXSW, a test proctoring assistant (to warn you if you’re likely to get DQ’d for eg wearing headphones), and a pill counter for pharmacists. Really powerful for distribution to not make users install an app or need anything other than their smartphone.

Re: Transformers.js – Run Transformers directly in the browser

#23

Post by binarymax »

jsheard wrote: post #3
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Basically the same problem that's plagued games on the web ever since the first Unreal/Unity asmjs demos a decade ago, and pretty much no progress has been made towards a solution in that time. You just can't practically make a web app which needs gigs of data on the client because there's no reliable way to make sure it stays cached for as long as the user wants it to, and as you say, even if you could reliably cach…

Been beating this drum for years. Wrote this 9 years ago! https://max.io/articles/the-state-of-state-in-the-browser/

Re: Transformers.js – Run Transformers directly in the browser

#24

Post by fauigerzigerk »

jsheard wrote: post #18
fauigerzigerk wrote: post #16

Earlier quoted context omitted.

The File System Access API seems promising.

I'm not sure more APIs are the solution, LocalStorage could already theoretically fill the role of a persistent large data store if browsers didn't cap the storage at 5-10MB for UX reasons. Removing that cap would require user facing changes to allow them to manage the storage used by sites and clean it up it manually when it inevitably gets bloated. Any new API which lets sites save stuff on the client is going to h…

>Any new API which lets sites save stuff on the client is going to have the same issue.

I don't think it would have the same issues, because the files could be stored in a user specified location outside the browser's own storage area.

Browser vendors can't just delete stuff that may be used by other software on a user's system. And they cannot put a cap on it either, because users can store whatever they like in those directories, bypassing the browser entirely.

But I have never used this API, so maybe I misunderstand how it's supposed to work.

Re: Transformers.js – Run Transformers directly in the browser

#25

Post by grey8 »

transformers.js is such a cool library.

I made a small web app with it that uses it to remove backgrounds from images (with BRIA AI's RMBG1.4 model) at https://aether.nco.dev

The fact you don't need to send your data to an API and this runs even on smartphones is really cool. I foresee lots of projects using this in the future, be it small vision, language or other utility models (depth estimation, background removal, etc), looks like a bright future for the web!

I'm already working on my next project, and it'll definitely use transformers.js again!

Re: Transformers.js – Run Transformers directly in the browser

#26

Post by jsheard »

fauigerzigerk wrote: post #24
jsheard wrote: post #18

Earlier quoted context omitted.

I'm not sure more APIs are the solution, LocalStorage could already theoretically fill the role of a persistent large data store if browsers didn't cap the storage at 5-10MB for UX reasons. Removing that cap would require user facing changes to allow them to manage the storage used by sites and clean it up it manually when it inevitably gets bloated. Any new API which lets sites save stuff on the client is going to h…

>Any new API which lets sites save stuff on the client is going to have the same issue. I don't think it would have the same issues, because the files could be stored in a user specified location outside the browser's own storage area. Browser vendors can't just delete stuff that may be used by other software on a user's system. And they cannot put a cap on it either, because users can store whatever they like in tho…

If that's how it works then it would avoid the problem I mentioned, but the UX around using that to cache data internal to the site implementation sounds pretty terrible. You click on "Listen to this article" on a webpage and it opens a file chooser expecting you to open t2s-hq-fast-en+pl.model if you already have it? Users won't be able to make any sense of that.

Re: Transformers.js – Run Transformers directly in the browser

#27

Post by jfoster »

fauigerzigerk wrote: post #10
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

It's an inherent problem with on-device AI processing, not just in the browser. I think this will only get better when operating systems start to preinstall models and provide an API that browser vendors can use as well. Even then I think cloud hosted models will probably always be far better for most tasks.

> I think cloud hosted models will probably always be far better for most tasks

It might depend on just how good you need it to be. There are lots of use-cases where an LLM like GPT 3.5 might be "good enough" such that a better model won't be so noticeable.

Cloud models will likely have the advantage of being more cutting-edge, but running "good enough" models locally will probably be more economical.

Re: Transformers.js – Run Transformers directly in the browser

#28

Post by sroussey »

simonw wrote: post #11

This library is so cool. It makes spinning up a quick demo incredibly easy - I've used it in Observable notebooks a few times: - CLIP in a browser: https://observablehq.com/@simonw/openai-clip-in-a-browser - Image object detection with detra-resnet-50: https://observablehq.com/@simonw/detect-objects-in-images The size of the models feels limiting at first, but for quite a few applications telling a user with a good l…

Binary embeddings will require additional re-rankings, but will be fun to test.

I’ve made an npm package of transformers.js v3, which I should update (not sure if I include this yet).

Mostly, I’ve had to have a fork so it runs on bun. V3 when released will support bun just fine. Although the webgpu won’t work, but that’s optional.

[edit: dm if you want to use it, I don’t want to promote a fork]

Re: Transformers.js – Run Transformers directly in the browser

#29

Post by Solvency »

can someone explain what this means I can do with it if I know vanilla JavaScript?

for example, I used the image upscaling playground on hugging face all the time. But I do it manually here: https://huggingface.co/spaces/bookbot/Image-Upscaling-Playgr...

would transformers.js allow me to somehow executive that in my own local or online app programmatically?

Re: Transformers.js – Run Transformers directly in the browser

#30

Post by jfoster »

xenova wrote: post #19
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

We’ve put out a ton of demos that use much smaller models (10-60 MB), including: - (44MB) In-browser background removal: https://huggingface.co/spaces/Xenova/remove-background-web . (We also put out a WebGPU version: https://huggingface.co/spaces/Xenova/remove-background-webgp... ). - (51MB) Whisper Web for automatic speech recognition: https://huggingface.co/spaces/Xenova/whisper-web (just select the quantized versi…

Thank you for the reply. Seems like all of the links are down at the moment, but it does sound a bit more feasible for some applications than I had assumed.

Really glad to hear the last part. Some of the new capabilities seem fundamental enough that they ought to be in browsers, in my opinion.

Post reply on HN