Live data from Hacker News

Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

andreinwald.github.io

Post reply on HN

Print view View original

1–10 of 60 posts

Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#1

Post by andreinwald » Sat, Aug 02, 2025, 2:09 PM UTC

Browser LLM demo working on JavaScript and WebGPU. WebGPU is already supported in Chrome, Safari, Firefox, iOS (v26) and Android.

Demo, similar to ChatGPT https://andreinwald.github.io/browser-llm/

Code https://github.com/andreinwald/browser-llm

- No need to use your OPENAI_API_KEY - its local model that runs on your device

- No network requests to any API

- No need to install any program

- No need to download files on your device (model is cached in browser)

- Site will ask before downloading large files (llm model) to browser cache

- Hosted on Github Pages from this repo - secure, because you see what you are running

Show HN: WebGPU enables local LLM in the browser – demo site with AI chat
andreinwald.github.io

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#2

Post by [deleted] » Sat, Aug 02, 2025, 2:09 PM UTC

[deleted]

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#3

Post by andsoitis » Sat, Aug 02, 2025, 2:29 PM UTC

very cool. improvement would be if the input text box is always on screen, rather than having to manually scroll down as the screen fills.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#4

Post by maxmcd » Sat, Aug 02, 2025, 2:41 PM UTC

Looks like this is a wrapper around: https://github.com/mlc-ai/web-llm

Which has a full web demo: https://chat.webllm.ai/

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#5

Post by pjmlp » Sat, Aug 02, 2025, 3:20 PM UTC

Beware of opening this on mobile Internet.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#6

Post by andreinwald » Sat, Aug 02, 2025, 3:22 PM UTC

pjmlp wrote: post #5

Beware of opening this on mobile Internet.

Demo site is asking before download

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#7

Post by refulgentis » Sat, Aug 02, 2025, 3:42 PM UTC

maxmcd wrote: post #4

Looks like this is a wrapper around: https://github.com/mlc-ai/web-llm Which has a full web demo: https://chat.webllm.ai/

I am to see it regardless - projects been very low activity for months. Just last night I was thinking about ripping it out before launch. No observable future.

EDIT: Ah, wrapper ~= using it as a dependency, no more

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#8

Post by asim » Sat, Aug 02, 2025, 3:44 PM UTC

What's the performance of a model like vs an OpenAI API? What's the comparable here? Edit: I see it's same models locally that you'd run using Ollama or something else. So basically just constrained by the size of the model, GPU and perf of the machine.

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#9

Post by lukan » Sat, Aug 02, 2025, 3:48 PM UTC

pjmlp wrote: post #5

Beware of opening this on mobile Internet.

Well, I am on a mobile right now, can someone maybe share anything about the performance?

Re: Show HN: WebGPU enables local LLM in the browser – demo site with AI chat

#10

Post by petermcneeley » Sat, Aug 02, 2025, 3:49 PM UTC

This demo only works if you have the webgpu feature "f16". You can find out if you have this by checking for the feature list in https://webgpureport.org/ . The page itself can of course check for this but since f16 support is common they probably just didnt bother.

Post reply on HN