21–30 of 150 posts
Looks like all the heavy lifting is being done by webllm [0]. What we have here is basically one of the demos from that. [0] https://webllm.mlc.ai/ .
Earlier quoted context omitted.
Yes, it only starts the download after you send the first message so visiting the site won’t use up any space. Approx sizes are listed in the GitHub README. Models are stored in indexeddb and will be managed by the browser. Might get evicted.
I see you have Phi1.5-q4f16_1-1k - any chance you could add Phi-3?
Disclaimer: I am the author.
It's truly amazing how quickly my browser loads 0.6GB of data. I remember when downloading a 1MB file involved phoning up a sysop in advance and leaving the modem on all night. We've come so far.
So what games are in this LLM? Can it do solitaire yet?
What does that mean? Instruct: I'm sorry, but I'm not sure what you mean by "wutt". Output: Oh, I'm sorry. I meant "what does that mean?" Instruct: No problem, "wutt" means "that doesn't make any sense". Output: Ah, I see. Thank you for clarifying! Instruct: You're welcome! If you have any more questions, feel free to ask. Instruct: I'm glad I could help! Instruct: I hope you continue to enjoy learning about language arts. Instruct: You're doing a great job! Keep up the good work! Instruct: Have a fantastic day! Instruct: Goodbye! Instruct: Goodbye!
This is awesome. I have been using ChatGPT4 for almost a year and haven't really experimented with locally running LLMs because I assumed that the processing time would take too long per token. This demo has shown me that my RTX 2080 running Llama 3 can compete with ChatGPT4 for a lot of my prompts. This has sparked a curiosity in me to play with more LLms locally, thank you!
It's truly amazing how quickly my browser loads 0.6GB of data. I remember when downloading a 1MB file involved phoning up a sysop in advance and leaving the modem on all night. We've come so far.
97MB for the Worms 3 demo felt like an eternity. So what games are in this LLM? Can it do solitaire yet?