WebGPU GPT Model Demo

#1

Post by b_mc2 » Fri, Apr 21, 2023, 12:54 AM UTC

Tweet by creator: https://twitter.com/willdepue/status/1649147091573432321

WebGPU GPT Model Demo
kmeans.org

Re: WebGPU GPT Model Demo

#2

Post by [deleted] » Fri, Apr 21, 2023, 12:54 AM UTC

[deleted]

Re: WebGPU GPT Model Demo

#3

Post by doodlesdev » Fri, Apr 21, 2023, 2:08 AM UTC

   > WebGPU is supported in your browser!

   > Uncaught (in promise) DOMException: WebGPU is not yet available in Release or Beta builds.

Anyone using Chromium care to chime in?

If no one chimes in I might set up a Chromium browser up just to take a look at this, seems pretty cool.

Re: WebGPU GPT Model Demo

#4

Post by DustinBrett » Fri, Apr 21, 2023, 2:26 AM UTC

It indeed works and loads quick. I am more interested currently in the Vicuna 7B example from https://mlc.ai/web-llm/

Also instead of just "Update Chrome to v113" the domain owner could sign up for an origin trial https://developer.chrome.com/origintrials/#/view_trial/11821...

Re: WebGPU GPT Model Demo

#5

Post by EMM_386 » Fri, Apr 21, 2023, 2:28 AM UTC

doodlesdev wrote: post #3

> WebGPU is supported in your browser! > Uncaught (in promise) DOMException: WebGPU is not yet available in Release or Beta builds. Anyone using Chromium care to chime in? If no one chimes in I might set up a Chromium browser up just to take a look at this, seems pretty cool.

I'm on the latest Brave (Chromium 112.0.5615.165) and it tells me it is not supported.

Re: WebGPU GPT Model Demo

#6

Post by adamkochanowicz » Fri, Apr 21, 2023, 2:36 AM UTC

EMM_386 wrote: post #5

doodlesdev wrote: post #3

> WebGPU is supported in your browser! > Uncaught (in promise) DOMException: WebGPU is not yet available in Release or Beta builds. Anyone using Chromium care to chime in? If no one chimes in I might set up a Chromium browser up just to take a look at this, seems pretty cool.

I'm on the latest Brave (Chromium 112.0.5615.165) and it tells me it is not supported.

Try chrome canary

Re: WebGPU GPT Model Demo

#7

Post by doodlesdev » Fri, Apr 21, 2023, 2:57 AM UTC

EMM_386 wrote: post #5

doodlesdev wrote: post #3

> WebGPU is supported in your browser! > Uncaught (in promise) DOMException: WebGPU is not yet available in Release or Beta builds. Anyone using Chromium care to chime in? If no one chimes in I might set up a Chromium browser up just to take a look at this, seems pretty cool.

I'm on the latest Brave (Chromium 112.0.5615.165) and it tells me it is not supported.

It needs to be Chromium 113+, otherwise you'll have to enable webGPU in the browser flags like I have for Firefox.

Re: WebGPU GPT Model Demo

#8

Post by fredrickd » Fri, Apr 21, 2023, 3:00 AM UTC

DustinBrett wrote: post #4

It indeed works and loads quick. I am more interested currently in the Vicuna 7B example from https://mlc.ai/web-llm/ Also instead of just "Update Chrome to v113" the domain owner could sign up for an origin trial https://developer.chrome.com/origintrials/#/view_trial/11821...

Thanks for the tip!

Re: WebGPU GPT Model Demo

#9

Post by MuffinFlavored » Fri, Apr 21, 2023, 3:35 AM UTC

> At the moment, WebGPT averages ~300ms per token on GPT-2 124M running on a 2020 M1 Mac with Chrome Canary.

How do ChatGPT on GPT-3.5 / GPT-4 compare?

Re: WebGPU GPT Model Demo

#10

Post by zamadatix » Fri, Apr 21, 2023, 3:42 AM UTC

adamkochanowicz wrote: post #6

EMM_386 wrote: post #5

Earlier quoted context omitted.

I'm on the latest Brave (Chromium 112.0.5615.165) and it tells me it is not supported.

Try chrome canary

Chrome Beta works as well.