This is amazing! I always wanted something like this, thank you so much!
Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
51–60 of 150 posts
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#52Phi 1.5 isn't quite there is it? Caught it having some inner dialog. What does that mean? Instruct: I'm sorry, but I'm not sure what you mean by "wutt". Output: Oh, I'm sorry. I meant "what does that mean?" Instruct: No problem, "wutt" means "that doesn't make any sense". Output: Ah, I see. Thank you for clarifying! Instruct: You're welcome! If you have any more questions, feel free to ask. Instruct: I'm glad I could…
That boy needs therapy.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#53Earlier quoted context omitted.
This need wasn’t super prevalent in the pre LLM days. It’s rare to have a multi-GB blob that should be commonly used across sites.
It was a real need given how almost all sites use large JavaScript deps. However, any hopes of sharing those were destroyed by adtech people timing resource downloads to track people.
Lots and lots of websites still use Google and other CDNs for JS deps, fonts, etc.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#54This is awesome. I have been using ChatGPT4 for almost a year and haven't really experimented with locally running LLMs because I assumed that the processing time would take too long per token. This demo has shown me that my RTX 2080 running Llama 3 can compete with ChatGPT4 for a lot of my prompts. This has sparked a curiosity in me to play with more LLms locally, thank you!
Uh oh, I had that same moment a bit over a year ago with MLC's old WebLLM. Take a deep breath before you jump into this rabbit hole because once you're in there's no escape :)
New models just keep rolling in day after day on r/locallama, tunes for this or that, new prompt formats, new quantization types, people doing all kinds of tests and analyses, new arxiv papers on some breakthrough and llama.cpp implementing it 3 days later. Every few weeks a new base model drops from somebody. So many things to try that nobody has tried before. It's genuinely like crack.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#55Earlier quoted context omitted.
It was a real need given how almost all sites use large JavaScript deps. However, any hopes of sharing those were destroyed by adtech people timing resource downloads to track people.
Lots and lots of websites still use Google and other CDNs for JS deps, fonts, etc.
They are cached independently these days to avoid privacy issues. So if websites A and B both use the same JavaScript dependency from a public CDN and you visit them both, you will download the JavaScript dependency twice, even if you have it cached from your visit to the first website.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#56Very interesting! I would be quite interested to see this implemented as some sort of API for browser chatbots or possibly even local AI powered web games?
If you don't know what Ollama is I suggest checking it out. Also I think adding the phi3 model to this would be a good idea.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#57Earlier quoted context omitted.
Last I checked ff explicitly does not support webgpu, webhid, webusb, etc. Apparently nightly is supposed to support it: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Exp...
So here's a howler to the new Mozilla CEO and FF teams who're looking for ways to save their org: - release WebGPU support everywhere, also embed llama.cpp or something similar for non GPU users - add UI for easy model downloading and sharing among sites - write the LLM browser API that enables easy access and sets the standard - add security: "this website wants to use local LLM. Allow?"
Hmm but what about another mobile phone OS instead? Or a vpn service? Surely people don't care about browser features.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#58This is awesome. I have been using ChatGPT4 for almost a year and haven't really experimented with locally running LLMs because I assumed that the processing time would take too long per token. This demo has shown me that my RTX 2080 running Llama 3 can compete with ChatGPT4 for a lot of my prompts. This has sparked a curiosity in me to play with more LLms locally, thank you!
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#59Question - Do I compromise on quality on answers if I use models using WebLLM (like this) compare to using them on system console.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#60What therapy prompts have you found useful?