I'm just seeing ERR_SSL_VERSION_OR_CIPHER_MISMATCH at https://secretllama.com/ and at http://secretllama.com/ I see "secretllama.com has been registered at Porkbun but the owner has not put up a site yet. Visit again soon to see what amazing website they decide to build."
Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
11–20 of 150 posts
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#12It's truly amazing how quickly my browser loads 0.6GB of data. I remember when downloading a 1MB file involved phoning up a sysop in advance and leaving the modem on all night. We've come so far.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#13This is awesome. I have been using ChatGPT4 for almost a year and haven't really experimented with locally running LLMs because I assumed that the processing time would take too long per token. This demo has shown me that my RTX 2080 running Llama 3 can compete with ChatGPT4 for a lot of my prompts.
This has sparked a curiosity in me to play with more LLms locally, thank you!
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#14Amazing! It's surprisingly fast to load and run given the size of the downloaded models.
Do you think it would be feasible to extend it to support web browsing?
I'd like to help if you could give some pointers on how to extend it.
When asked about web browsing, the bot said it could fetch web pages but then obviously didn't work when asked to summarize a web page.
[EDIT] The Llama 3 model was able to summarize web pages!
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#15Amazing! It's surprisingly fast to load and run given the size of the downloaded models. Do you think it would be feasible to extend it to support web browsing? I'd like to help if you could give some pointers on how to extend it. When asked about web browsing, the bot said it could fetch web pages but then obviously didn't work when asked to summarize a web page. [EDIT] The Llama 3 model was able to summarize web pa…
I commented too soon. The TinyLlama model didn't seem to be able to summarize web pages but Llama 3 worked perfectly! Very cool.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#16Earlier quoted context omitted.
Yes, it only starts the download after you send the first message so visiting the site won’t use up any space. Approx sizes are listed in the GitHub README. Models are stored in indexeddb and will be managed by the browser. Might get evicted.
I see you have Phi1.5-q4f16_1-1k - any chance you could add Phi-3?
Would love to. It uses MLC AIs webllm so just need to convert it to that format.
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#17I'm just seeing ERR_SSL_VERSION_OR_CIPHER_MISMATCH at https://secretllama.com/ and at http://secretllama.com/ I see "secretllama.com has been registered at Porkbun but the owner has not put up a site yet. Visit again soon to see what amazing website they decide to build."
Just bought the domain a couple of hours ago so DNS might not have propagated. Try back tomorrow or download and install it from GitHub (it’s just 2 steps)
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#18Earlier quoted context omitted.
Yes, it only starts the download after you send the first message so visiting the site won’t use up any space. Approx sizes are listed in the GitHub README. Models are stored in indexeddb and will be managed by the browser. Might get evicted.
I thought browser tabs only had access to ~400mb How do you have access to 5gb?
A lot of the more modern options allow for many gigabytes for typical user setups https://developer.mozilla.org/en-US/docs/Web/API/Storage_API....
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#19Earlier quoted context omitted.
I thought browser tabs only had access to ~400mb How do you have access to 5gb?
A lot of the more modern options allow for many gigabytes for typical user setups https://developer.mozilla.org/en-US/docs/Web/API/Storage_API... .
Very interesting! Thank you so much
I was always under the impression that the max blob size was 400mb and so you couldn't store files any bigger than that. Google gives so many different answers to these questions.
Do you know any other resources I can go more in depth on browser storage limits?
Re: Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
#20Looks like all the heavy lifting is being done by webllm [0]. What we have here is basically one of the demos from that.