Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

github.com

Post reply on HN
Print view View original

111–120 of 150 posts

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#111

Post by KennyBlanken »

pseudosavant wrote: post #103
wg0 wrote: post #87

How do people use something like this as coach or therapist? This is genuine question. Side note, impressive project. Future of AI is offline mostly with few APIs in the cloud maybe.

I tried using Claude and ChatGPT like this: I would just write a free form journal entry. The feedback it gave was typically very useful and made journaling more rewarding.

Given user data is folded back into the models, there is a snowball's chance in hell that I would input stuff I'd talk to a therapist about.

When are people going to realize that their interactions with AIs are likely being analyzed/characterized, and that at some point, that analysis will be monetized?

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#112

Post by edgineer »

stevenicr wrote: post #107
afavour wrote: post #93

Earlier quoted context omitted.

But surely, expulsion is not the answer!

Where in the heck did you all discover this song? It's never been on the radio I assume. I only found it because it cropped up in yahoo messenger/launchcast at some point - never seen it anywhere else.

It was big on YTMND back in the day

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#113

Post by gordinmitya »

KennyBlanken wrote: post #110
_akhe wrote: post #104

Amazing work, feels like a step forward for LLM usability. Would be interesting if there was a web browser that managed the download/install of models so you could go to a site like this, or any other LLM site/app and it detects whether or not you have models, similar to detecting if you have a webcam or mic for a video call. The user can click "Allow" to allow use of GPU and allow running of models in the background…

Mozilla won't even allow WebSerial to be implemented because it was deemed "too dangerous" - with all sorts of absurd whinging about the devastation that could be unleashed by unsuspecting users allowing a malicious site to access USB serial devices. When someone pointed out that Chrome has had this functionality for years and the world has not imploded...and has enabled many open source projects and web-based microc…

already allow by default in nightly builds

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#114

Post by whartung »

notarealllama wrote: post #68
littlestymaar wrote: post #37

This is very cool, it's something I wish existed since Llama came out, having to install Ollama + Cuda to get locally working LLM didn't felt right to me when there's all what's needed in the browser. Llamafile solves the first half of the problem, but you still need to install Cuda/ROCm for it to work with GPU acceleration. WebGPU is the way to go if we want to put AI on consumer hardware and break the oligopoly, I…

Tested on Ubuntu 22.04 with Chrome, sure enough, "Could not load the model because Error: Cannot find adapter that matches the request". It really is too bad WebGPU isn't supported on Linux, I mean, that's a no-brainer right there.

I get the same thing on Chrome and my last generation Intel iMac.

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#115

Post by squigz »

KennyBlanken wrote: post #110
_akhe wrote: post #104

Amazing work, feels like a step forward for LLM usability. Would be interesting if there was a web browser that managed the download/install of models so you could go to a site like this, or any other LLM site/app and it detects whether or not you have models, similar to detecting if you have a webcam or mic for a video call. The user can click "Allow" to allow use of GPU and allow running of models in the background…

Mozilla won't even allow WebSerial to be implemented because it was deemed "too dangerous" - with all sorts of absurd whinging about the devastation that could be unleashed by unsuspecting users allowing a malicious site to access USB serial devices. When someone pointed out that Chrome has had this functionality for years and the world has not imploded...and has enabled many open source projects and web-based microc…

> (If Chrome is such a user privacy dumpsterfire, why not implement WebSerial so that people don't have to run Chrome in order to communicate with and program microcontrollers?)

This doesn't seem like a logical comparison. Is there no other way to program microcontrollers outside of Chrome?

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#117

Post by Cheer2171 »

_akhe wrote: post #104

Amazing work, feels like a step forward for LLM usability. Would be interesting if there was a web browser that managed the download/install of models so you could go to a site like this, or any other LLM site/app and it detects whether or not you have models, similar to detecting if you have a webcam or mic for a video call. The user can click "Allow" to allow use of GPU and allow running of models in the background…

Sounds like ollama with open webui

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#118

Post by mcculley »

neurostimulant wrote: post #100
mcculley wrote: post #74

Earlier quoted context omitted.

They are not “trained on all publicly available human knowledge”. Go look at the training data sets used. Most human knowledge that has been digitized is not publicly available (e.g., Google Books). These models are not able to get to data sets behind paywalls (e.g., scientific journals). It will be a huge step forward for humanity when we can run algorithms across all human knowledge. We are far from that.

There is a rumor that OpenAI might've used libgen in their training data.

Someone will. The potential gains are too high to ignore it.

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#119

Post by andai »

dosinga wrote: post #25

Phi 1.5 isn't quite there is it? Caught it having some inner dialog. What does that mean? Instruct: I'm sorry, but I'm not sure what you mean by "wutt". Output: Oh, I'm sorry. I meant "what does that mean?" Instruct: No problem, "wutt" means "that doesn't make any sense". Output: Ah, I see. Thank you for clarifying! Instruct: You're welcome! If you have any more questions, feel free to ask. Instruct: I'm glad I could…

I wasn't able to get much use from Phi 1.5 (except for leaking Microsoft's proprietary training data).

Phi 3 is great though.

Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU

#120

Post by intended »

wg0 wrote: post #87

How do people use something like this as coach or therapist? This is genuine question. Side note, impressive project. Future of AI is offline mostly with few APIs in the cloud maybe.

Ever had a day where your bandwidth was constrained and you just knew something was wrong with a situation, but your brain lacked the juice or dexterity to connect/articulate the issue?

If I have the presence of mind, I offload the work here. At the same time I have a strong understanding of how coaching works, as does my brain.

I suspect that with all things LLM, some amount of proficiency is needed to truly get the prompts to work.

The simplest option is to ask it to be a coach for you. This is going to be hit and miss.

The better version is to specify the kind of coaching you want, or provide a rough outline of the issues on your mind and then ask for what kind of coach or therapist would make sense.

I use either of these for example - 1) over designed - https://chat.openai.com/g/g-KD6jm0l4c-thought-council 2) base ver - https://chat.openai.com/g/g-Cdq3drl87-two-guides

Sadly OpenAI doesnt let you share active chats anymore, so it’s going to need a plus subscription.

Post reply on HN