Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformers.js

xenova.github.io

Post reply on HN
Print view View original

61–70 of 80 posts

Re: Transformers.js

#62

Post by xenova »

junrushao1994 wrote: post #61

Curious if this library can be integrated with WebGPU - there is a recent post on ( https://news.ycombinator.com/item?id=35191687 ) announced that WebGPU can now be used for large models

Once ONNX runtime releases their WebGPU backend, we will add support for it! :)

It should also be noted that browser support for it isn’t very high at the moment… so, unfortunately, we are stuck with WASM (CPU) for now.

Re: Transformers.js

#63

Post by rkagerer »

xenova wrote: post #35

Hi everyone! Creator of Transformers.js here :) ... Thanks so much to everyone for sharing! It's awesome to see the positive feedback from the community. As you'll see from the demo, everything runs inside the browser! As of 2023/03/16, the library supports BERT, ALBERT, DistilBERT, T5, T5v1.1, FLAN-T5, GPT2, BART, CodeGen, Whisper, CLIP, Vision Transformer, and VisionEncoderDecoder models, for a variety of tasks inc…

Is there an Optimus model yet for Prime number encoding?

Re: Transformers.js

#64

Post by bredren »

refulgentis wrote: post #37
illiarian wrote: post #30

Earlier quoted context omitted.

> I'm personally wondering in how far ML compute will be done on consumer devices, rather than on servers. Running ML on the device has been one of Apple's value propositions for a long time. They are currently silent on everything that's unfolding, but I expect them to at least mention something and WWDC (and trying to run that something on the device)

They’re there, released Core ML Stable Diffusion a couple months ago.

I am not a Swift dev but it seemed like the speed of this release was very fast by Apple standards.

Can anyone in the know confirm that?

Re: Transformers.js

#65

Post by nl »

rektide wrote: post #60
nl wrote: post #51

Earlier quoted context omitted.

> Making each webapp target & optimize ML for every possible device target sounds terrible. Yes it does. Did something I said imply that? OpenXLA is an intermediate layer that frameworks like PyTorch or JAX can use. It has pluggable backends, and so if there was a web-compatible backend (WebGL or WASM) then everyone could use it and all models that were built using something that used OpenXLA[1] would be compatible.…

I support having web targets. It'd be a good offering. But it feels upside down to me from what we really all should want, which is a safe way to let the web target any backend you have. WebGPU or WebGL or wasm are going to be OK targets, but with limited hardware support & tons of constraints that mean they won't perform as well as openxla. Also how will these targets get profiled? Do we ship the same WebGL to a 600…

> WebGPU or WebGL or wasm are going to be OK targets, but with limited hardware support & tons of constraints that mean they won't perform as well as wasm.

I don't understand. "WebGPU or WebGL or wasm".. "won't perform as well as wasm".

Re: Transformers.js

#66

Post by bredren »

illiarian wrote: post #30
sva_ wrote: post #2

That's pretty neat. I'm personally wondering in how far ML compute will be done on consumer devices, rather than on servers. We're currently seeing a lot of models that are so large that it doesn't seem feasible to run them locally. But I think there is reason to believe that these models carry a lot of redundancy. Redundancy that could lead to order of magnitude less memory/compute needed. Or perhaps hardware will c…

> I'm personally wondering in how far ML compute will be done on consumer devices, rather than on servers. Running ML on the device has been one of Apple's value propositions for a long time. They are currently silent on everything that's unfolding, but I expect them to at least mention something and WWDC (and trying to run that something on the device)

If I understand correctly, there was an all-company invited annual AI day which was silent on recent developments.

But then ~two weeks later there was what seemed like an on-background / press leak about the XDG group that specifically mentioned AI as a current discipline. (Gurman / Bloomberg)

It seems to me that the release of Core ML stable diffusion (mentions itt) is something if a comment in of itself. At least in the read between the lines / hiding in plain sight style of Apple.

The company is unveiling a new and presumably next major computing platform at a quality level only they could possibly deliver.

So the relative quiet / lack of comment may be in deference to the gravity of that work.

That said, these changes are too big to ignore—-we should at least hear language that acknowledges the major developments in AI of late at WWDC and some idea for how Apple is thinking about them.

Re: Transformers.js

#67

Post by rektide »

nl wrote: post #65
rektide wrote: post #60

Earlier quoted context omitted.

I support having web targets. It'd be a good offering. But it feels upside down to me from what we really all should want, which is a safe way to let the web target any backend you have. WebGPU or WebGL or wasm are going to be OK targets, but with limited hardware support & tons of constraints that mean they won't perform as well as openxla. Also how will these targets get profiled? Do we ship the same WebGL to a 600…

> WebGPU or WebGL or wasm are going to be OK targets, but with limited hardware support & tons of constraints that mean they won't perform as well as wasm. I don't understand. "WebGPU or WebGL or wasm".. "won't perform as well as wasm".

*OpenXLA, edited

Re: Transformers.js

#69

Post by thebruce87m »

nwoli wrote: post #5
sva_ wrote: post #2

That's pretty neat. I'm personally wondering in how far ML compute will be done on consumer devices, rather than on servers. We're currently seeing a lot of models that are so large that it doesn't seem feasible to run them locally. But I think there is reason to believe that these models carry a lot of redundancy. Redundancy that could lead to order of magnitude less memory/compute needed. Or perhaps hardware will c…

The trick here will be using large models as data generators to distill some sub task into a web computable model. (I’ve done it a few times for vision rather than text and it’s amazing how potent it is.)

Can you describe the vision bit? I have a general idea but would like to know the details, e.g. which models you used.
Post reply on HN