Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WebGPU GPT Model Demo

kmeans.org

Post reply on HN
Print view View original

21–30 of 32 posts

Re: WebGPU GPT Model Demo

#21

Post by ianpurton »

Question. I can see in the code the WGSL that's needed to implement inference on the GPU. https://github.com/0hq/WebGPT/blob/main/kernels.js

Could this code also be used to train models or only for inference?

What I'm getting at, is could I take the WGSL and using rust wgpu create a mini ChatGPT that runs on all GPU's?

Re: WebGPU GPT Model Demo

#22

Post by HopenHeyHi »

ianpurton wrote: post #21

Question. I can see in the code the WGSL that's needed to implement inference on the GPU. https://github.com/0hq/WebGPT/blob/main/kernels.js Could this code also be used to train models or only for inference? What I'm getting at, is could I take the WGSL and using rust wgpu create a mini ChatGPT that runs on all GPU's?

No, that is not how that works, sadly you cannot.

Re: WebGPU GPT Model Demo

#24

Post by b_mc2 »

tormeh wrote: post #23

Any way to run this kind of thing outside the browser? Chrome hasn't enabled WebGPU on Linux yet.

I've got it running on Chrome v113 beta on Ubuntu with an older AMD RX 580. The feature flags don't seem to be taking for me in chrome GUI but if you start chrome from terminal like this it works.

google-chrome --enable-unsafe-webgpu --enable-features=Vulkan,UseSkiaRenderer --enable-dawn-features=disable_robustness

GPU doesn't work in --headless though.

Re: WebGPU GPT Model Demo

#25

Post by FL33TW00D »

HopenHeyHi wrote: post #22
ianpurton wrote: post #21

Question. I can see in the code the WGSL that's needed to implement inference on the GPU. https://github.com/0hq/WebGPT/blob/main/kernels.js Could this code also be used to train models or only for inference? What I'm getting at, is could I take the WGSL and using rust wgpu create a mini ChatGPT that runs on all GPU's?

No, that is not how that works, sadly you cannot.

Why not? You could absolutely write the training code in WGSL.

Re: WebGPU GPT Model Demo

#26

Post by VierScar »

FL33TW00D wrote: post #20
lxe wrote: post #19

Earlier quoted context omitted.

That's pretty impressive. What model are you using?

FLAN-T5 Base currently, 780M parameter variant coming shortly!

What kind of model? Question-Answering? I imagine it must be quite specialised at <1b Params when many are 7b, 13b, or more?

Re: WebGPU GPT Model Demo

#28

Post by spencerchubb »

ianpurton wrote: post #21

Question. I can see in the code the WGSL that's needed to implement inference on the GPU. https://github.com/0hq/WebGPT/blob/main/kernels.js Could this code also be used to train models or only for inference? What I'm getting at, is could I take the WGSL and using rust wgpu create a mini ChatGPT that runs on all GPU's?

This repo only does inference but it should be possible to write training code that runs on WebGPU.

Re: WebGPU GPT Model Demo

#29

Post by luizfelberti »

It's really a shame that there is no 8-bit float support in the WebGPU spec. Even though few cards support it natively, it'd still massively benefit ML workloads.

Another annoying constraint but specific to wgpu (Rust's implementation of WebGPU) is that it does not support f16 yet (which IS in the spec), only through SPIR-V passthrough...

Post reply on HN