Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1-Bit LLM in the Browser

huggingface.co

Post reply on HN
Print view View original

1–10 of 58 posts

Re: 1-Bit LLM in the Browser

#4

Post by mgoetzke »

stfurkan wrote: post #2

I'm currently working on an open-source engine [1] exactly for this purpose. If anyone wants to try or has any suggestions, I'm happy to listen :) 1. https://github.com/stfurkan/bitgpu 2. https://aidekin.com --> this is one of my projects that's currently using bitgpu engine

Interesting. But trying the demo with 27G model and simple 8+5 did not work.

Guessing there is some work to do still

``` calculate({"expression":"g"}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages. What is 8+5? Use the calculator get_time({}) → Mon Jul 20 2026 10:36:28 GMT+0200 (Central European Summer Time) error: No user query found in messages. hi Hello! 2 tokens · 6.2 tok/s · prefill 4563 ms · confidence 87% which tools do you know about ? I'myepic know about the following are two tools I am aware of course, I currently i know about I know about me know I am aware of course I know I know I am aware of the following

calculate({"expression":" I know I am aware of know I am aware of I am aware I know I know know "}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages.```

Re: 1-Bit LLM in the Browser

#5

Post by alex7o »

stfurkan wrote: post #2

I'm currently working on an open-source engine [1] exactly for this purpose. If anyone wants to try or has any suggestions, I'm happy to listen :) 1. https://github.com/stfurkan/bitgpu 2. https://aidekin.com --> this is one of my projects that's currently using bitgpu engine

Hay man, I really want a bonsai style embeddings or re-ranking model for you think sth like that is feasible?

Re: 1-Bit LLM in the Browser

#6

Post by stfurkan »

mgoetzke wrote: post #4
stfurkan wrote: post #2

I'm currently working on an open-source engine [1] exactly for this purpose. If anyone wants to try or has any suggestions, I'm happy to listen :) 1. https://github.com/stfurkan/bitgpu 2. https://aidekin.com --> this is one of my projects that's currently using bitgpu engine

Interesting. But trying the demo with 27G model and simple 8+5 did not work. Guessing there is some work to do still ``` calculate({"expression":"g"}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages. What is 8+5? Use the calculator get_time({}) → Mon Jul 20 2026 10:36:28 GMT+0200 (Central European Summer Time) error: No user query found in messages. hi Hello! 2 tokens…

Thank you for trying it :) I just added the 27B support 1 hour ago. Let me check and try to fix it. All other models should work as expected but if you have issues with them too, I can take a look.

Re: 1-Bit LLM in the Browser

#7

Post by stfurkan »

alex7o wrote: post #5
stfurkan wrote: post #2

I'm currently working on an open-source engine [1] exactly for this purpose. If anyone wants to try or has any suggestions, I'm happy to listen :) 1. https://github.com/stfurkan/bitgpu 2. https://aidekin.com --> this is one of my projects that's currently using bitgpu engine

Hay man, I really want a bonsai style embeddings or re-ranking model for you think sth like that is feasible?

Hi :) There are some smaller embedding and re-ranking models (I am currently using some of them on my aidekin project) but having it like bonsai models will be great to have.
Post reply on HN