How does it compare to WebLLM (https://github.com/mlc-ai/web-llm)?
Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
11–20 of 25 posts
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#12I don’t see any encoders (BERT family) available yet. How will you do RAG, BM25/tf-idf?
Oh yes, because the library was so large, we decided to start by removing some things and porting, to be honest, one of the bad decisions of my life trying to Port JS to TS but luckily it only took 3 days and a few headaches!
Will add the encoders as needed, should be easy now, but a great point.
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#13How does it compare to WebLLM ( https://github.com/mlc-ai/web-llm )?
We use Webllm under the hood and for text-to-text generation, the model compression is awesome and RAM usage is also less. But we are conducting more experiments, One thing we noticed is some quantized models using MLC sometimes start throwing gibberish, so will get back to you after more experiments on which is better.
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#14When I read the title, I thought the project would be an LLM browser plugin (or something of the sort) that would automatically use the current page as context. However, after viewing the GitHub project, it seems like a browser interface for local LLMs. Is my understanding correct? This is not my domain of expertise.
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#15[flagged]
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#16Deepseek R1 just got ported to WebGPU as well! Exciting future for local web AI: Thread - https://news.ycombinator.com/item?id=42795782
[deleted]
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#17Would it be good for language translation?
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#18[deleted]
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#19How's the performance and features compared to pinokio?
Re: Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
#20When I read the title, I thought the project would be an LLM browser plugin (or something of the sort) that would automatically use the current page as context. However, after viewing the GitHub project, it seems like a browser interface for local LLMs. Is my understanding correct? This is not my domain of expertise.
Yes, it's currently a framework for running LLMs locally in the browser. Browser extension for page context is in our roadmap, but right now we're focused on optimizing multimodal LLMs to work efficiently in the browser environment, so that we can use them for a variety of use cases.