71–80 of 150 posts
It's sadly stuck on "Loading model from cache[24/24]: 0MB loaded. 0% completed, 0 secs elapsed." on my iPhone 13 Pro Max :(
https://github.com/abi/secret-llama?tab=readme-ov-file#syste...
Very cool! I wish there was chat history. Also if you click the "New Chat" button while an answer is generating I think some of the output gets fed back into the model, it causes some weird output [0] but was kind of cool/fun. Here is a video of it as well [1], I almost think this should be some kind of special mode you can run. I'd be interested to know what the bug causes, is it just the existing output sent as inp…
Nice personal hosted image service!
Yasssssss! Thank you. This is the future. I am predicting Apple will make progress on groq like chipsets built in to their newer devices for hyper fast inference. LLMs leave a lot to be desired but since they are trained on all publicly available human knowledge they know something no about everything. My life has been better since I’ve been able to ask all sorts of adhoc questions about “is this healthy? Why healthy…
It will be a huge step forward for humanity when we can run algorithms across all human knowledge. We are far from that.
I tried this on my M1 and ran LLama3, I think it's the quantized 7B version. It ran with around 4-5 tokens per second which was way faster than I expected on my browser.
IMO eventually users should be able to advertise what embedding models they have so we don't redundantly redownload.
That's not possible with current web tech, is it? Different webapps can't share common dependencies stored in localstorage afaik.
Amazing! It's surprisingly fast to load and run given the size of the downloaded models. Do you think it would be feasible to extend it to support web browsing? I'd like to help if you could give some pointers on how to extend it. When asked about web browsing, the bot said it could fetch web pages but then obviously didn't work when asked to summarize a web page. [EDIT] The Llama 3 model was able to summarize web pa…
I commented too soon. The TinyLlama model didn't seem to be able to summarize web pages but Llama 3 worked perfectly! Very cool.
edit: typo
Could not load the model because Error: Cannot find WebGPU in the environment
> Should work on Desktop with Chrome or Edge.