Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

andreinwald.github.io

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#41

Post by apitman »

fooker wrote: post #33
apitman wrote: post #28

Does anyone know why all of these WebGPU LLM demos have you download the models to browser storage rather than letting you open a gguf already on your local drive? I have several models downloaded already that I would be interested in trying.

Browsers are sandboxed away from user storage. You can change this by changing settings, command line arguments, build flags, etc. But can’t really expect people to do this just to use your website.

You can open a file for performant access in all major browsers. It's the same API used for uploading files (), but you can also just load them into memory and do stuff.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#43

Post by ethan_smith »

apitman wrote: post #28

Does anyone know why all of these WebGPU LLM demos have you download the models to browser storage rather than letting you open a gguf already on your local drive? I have several models downloaded already that I would be interested in trying.

WebGPU's security model restricts direct file system access, requiring models to be loaded through fetch/cache APIs rather than local file paths.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#44

Post by amelius »

AndrewDucker wrote: post #38

I asked "Why is the sky blue?" and got back a response of "coppia RR TalentDevExpressincer+'.//////////////////////////////////////////////////////////////////////// cha ولا.AutoSizesaving proleงคicate Like"/> infos эти za cornerback economical (%]\ enumligne.execRELEASEPropagation_station Bucks проHEME seas GASPOST[Unit(suffix Gloves" (and so on, for a few more paragraphs). Am I missing something?

Well, at least it is more informative than an answer like "42".

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#46

Post by fooker »

apitman wrote: post #41
fooker wrote: post #33

Earlier quoted context omitted.

Browsers are sandboxed away from user storage. You can change this by changing settings, command line arguments, build flags, etc. But can’t really expect people to do this just to use your website.

You can open a file for performant access in all major browsers. It's the same API used for uploading files ( ), but you can also just load them into memory and do stuff.

Sure, with the caveat that the file is specifically selected by the user.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#47

Post by swores »

refulgentis wrote: post #26
swores wrote: post #14

Earlier quoted context omitted.

Is this correct? It doesn't seem so to me, either from the way it works or from what little of the code I've looked at... But I don't have time to do more than the quick glance I just did at a few of the files of each and need to run, so hopefully someone cleverer than me who won't need as much time as me to answer the question could confirm while I'm afk

Entirely correct, c.f. LLM.ts. The bit to Show HN here, is about ~40 lines of code. Simple Typescript MVP of calling the library. (source: maintains an LLM client that works across MLC/llama.cpp/3P providers; author of sibling comment that misunderstood initially)

Thanks! (Have to admit your sibling comment confused the hell out of me.)

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#48

Post by 0points »

AndrewDucker wrote: post #38

I asked "Why is the sky blue?" and got back a response of "coppia RR TalentDevExpressincer+'.//////////////////////////////////////////////////////////////////////// cha ولا.AutoSizesaving proleงคicate Like"/> infos эти za cornerback economical (%]\ enumligne.execRELEASEPropagation_station Bucks проHEME seas GASPOST[Unit(suffix Gloves" (and so on, for a few more paragraphs). Am I missing something?

> Am I missing something?

Realistic expectations.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#49

Post by AndrewDucker »

0points wrote: post #48
AndrewDucker wrote: post #38

I asked "Why is the sky blue?" and got back a response of "coppia RR TalentDevExpressincer+'.//////////////////////////////////////////////////////////////////////// cha ولا.AutoSizesaving proleงคicate Like"/> infos эти za cornerback economical (%]\ enumligne.execRELEASEPropagation_station Bucks проHEME seas GASPOST[Unit(suffix Gloves" (and so on, for a few more paragraphs). Am I missing something?

> Am I missing something? Realistic expectations.

It seems to be working for some people. I'm just curious whether there was something I could change to make it work.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#50

Post by khalic »

echoangle wrote: post #20

Fun demo but the model that's used seems to be pretty stupid: > What's the best way to get to space? >> Unfortunately, it's not currently possible for humans to travel to space in the same way that astronauts do. While there have been several manned missions to space, such as those to the International Space Station, the technology and resources required to make interstellar travel feasible are still in the early sta…

Are you kidding me? 500MB? And it’s that coherent?
Post reply on HN