Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
1–7 of 7 posts
Re: Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
#2[deleted]
Re: Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
#3[deleted]
Re: Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
#4[deleted]
Re: Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
#5[flagged]
Re: Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
#6Wow this is a cool prototype and looks amazing. I am surprised some level of baseline intelligence survives this kind of aggro quantization. Do you think 300mb initial download is ok for something like quick website where I want to ask quick support question? Are you planning to have hosted fallback to answer q while download is happening?
Re: Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser
#7Wow this is a cool prototype and looks amazing. I am surprised some level of baseline intelligence survives this kind of aggro quantization. Do you think 300mb initial download is ok for something like quick website where I want to ask quick support question? Are you planning to have hosted fallback to answer q while download is happening?
Thank you :) Currently I am not planning to have hosted fallback Q&A but it's a nice idea. It should only download the ~300mb initially once and then use the cached model.