Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformers.js – Run Transformers directly in the browser

github.com

Post reply on HN
Print view View original

11–20 of 54 posts

Re: Transformers.js – Run Transformers directly in the browser

#11

Post by simonw »

This library is so cool. It makes spinning up a quick demo incredibly easy - I've used it in Observable notebooks a few times:

- CLIP in a browser: https://observablehq.com/@simonw/openai-clip-in-a-browser

- Image object detection with detra-resnet-50: https://observablehq.com/@simonw/detect-objects-in-images

The size of the models feels limiting at first, but for quite a few applications telling a user with a good laptop and connection that they have to wait 30s for it to load isn't unthinkable.

The latest releases adds binary embedding quantization support which I'm really looking forward to trying out: https://github.com/xenova/transformers.js/releases/tag/2.17....

Re: Transformers.js – Run Transformers directly in the browser

#12

Post by echelon »

jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Apple's future is predicated on local machine learning instead of cloud machine learning. They're betting big on it, and you can see the chess pieces being moved into place. They desperately do not want to become a thin client for magical cloud AI.

I'd look to see Apple doing some stuff here.

Re: Transformers.js – Run Transformers directly in the browser

#13

Post by jampekka »

CapsAdmin wrote: post #7
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

If they are single files or directories they could be drag dropped on use. Not very convenient though. Maybe just some sort of api to give the website fine grained access to the filesystem might be enough. You'd specify a directory or single file the website can read from at any time. However at some point you will have to download large files. I feel when done implicitly it's bad user experience. On top of that the…

There is such API.

https://developer.mozilla.org/en-US/docs/Web/API/File_System...

Re: Transformers.js – Run Transformers directly in the browser

#14

Post by breck »

jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

This is why I was hoping the startup MightyApp would succeed. Then it would be practical to build web apps that operated on GB/TB of data in a single tab. Most of the time people would use their normal browser, but for big data jobs you would use your Mighty browser with persistence and unlimited RAM streamed from the cloud. A path to get the best of web apps with the power of native apps. Glad they gave it a shot. Definitely was an idea worth trying.

Re: Transformers.js – Run Transformers directly in the browser

#15

Post by jampekka »

jsheard wrote: post #3
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Basically the same problem that's plagued games on the web ever since the first Unreal/Unity asmjs demos a decade ago, and pretty much no progress has been made towards a solution in that time. You just can't practically make a web app which needs gigs of data on the client because there's no reliable way to make sure it stays cached for as long as the user wants it to, and as you say, even if you could reliably cach…

There are ways to do it. E.g. https://developer.mozilla.org/en-US/docs/Web/API/File_System...

Re: Transformers.js – Run Transformers directly in the browser

#16

Post by fauigerzigerk »

jsheard wrote: post #3
jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Basically the same problem that's plagued games on the web ever since the first Unreal/Unity asmjs demos a decade ago, and pretty much no progress has been made towards a solution in that time. You just can't practically make a web app which needs gigs of data on the client because there's no reliable way to make sure it stays cached for as long as the user wants it to, and as you say, even if you could reliably cach…

The File System Access API seems promising.

Re: Transformers.js – Run Transformers directly in the browser

#17

Post by elpocko »

jampekka wrote: post #13
CapsAdmin wrote: post #7

Earlier quoted context omitted.

If they are single files or directories they could be drag dropped on use. Not very convenient though. Maybe just some sort of api to give the website fine grained access to the filesystem might be enough. You'd specify a directory or single file the website can read from at any time. However at some point you will have to download large files. I feel when done implicitly it's bad user experience. On top of that the…

There is such API. https://developer.mozilla.org/en-US/docs/Web/API/File_System...

Still requires drag&drop on most browsers because the File/DirectoryPicker API isn't universally supported.

Re: Transformers.js – Run Transformers directly in the browser

#18

Post by jsheard »

fauigerzigerk wrote: post #16
jsheard wrote: post #3

Earlier quoted context omitted.

Basically the same problem that's plagued games on the web ever since the first Unreal/Unity asmjs demos a decade ago, and pretty much no progress has been made towards a solution in that time. You just can't practically make a web app which needs gigs of data on the client because there's no reliable way to make sure it stays cached for as long as the user wants it to, and as you say, even if you could reliably cach…

The File System Access API seems promising.

I'm not sure more APIs are the solution, LocalStorage could already theoretically fill the role of a persistent large data store if browsers didn't cap the storage at 5-10MB for UX reasons. Removing that cap would require user facing changes to allow them to manage the storage used by sites and clean it up it manually when it inevitably gets bloated. Any new API which lets sites save stuff on the client is going to have the same issue.

Re: Transformers.js – Run Transformers directly in the browser

#19

Post by xenova »

jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

We’ve put out a ton of demos that use much smaller models (10-60 MB), including:

- (44MB) In-browser background removal: https://huggingface.co/spaces/Xenova/remove-background-web. (We also put out a WebGPU version: https://huggingface.co/spaces/Xenova/remove-background-webgp...).

- (51MB) Whisper Web for automatic speech recognition: https://huggingface.co/spaces/Xenova/whisper-web (just select the quantized version in settings).

- (28MB) Depth Anything Web for monocular depth estimation: https://huggingface.co/spaces/Xenova/depth-anything-web

- (14MB) Segment Anything Web for image segmentation: https://huggingface.co/spaces/Xenova/segment-anything-web

- (20MB) Doodle Dash, an ML-powered sketch detection game: https://huggingface.co/spaces/Xenova/doodle-dash

… and many many more! Check out the Transformers.js demos collection for some others: https://huggingface.co/collections/Xenova/transformersjs-dem....

Models are cached on a per-domain basis (using the Web Cache API), meaning you don’t need to re-download the model on every page load. If you would like to persist the model across domains, you can create browser extensions with the library! :)

As for your last point, there are efforts underway, but nothing I can speak about yet!

Re: Transformers.js – Run Transformers directly in the browser

#20

Post by ranulo »

jfoster wrote: post #2

This is super cool, but unfortunately it also seems super impractical. Models tend to be quite large, so even if a browser can run them, getting them to the browser involves either: 1. Large downloads on every visit to a website. 2. Large downloads and high storage consumption for each website using large models. (150 websites x 800 MB models => 120 GB of storage used) Both of those options seem terrible. I think it…

Might make more sense for web apps and electron applications.
Post reply on HN