Fails "car wash" test miserably. Ask it:
> I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
1-Bit LLM in the Browser
11–20 of 58 posts
Re: 1-Bit LLM in the Browser
#12Fails "car wash" test miserably. Ask it: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ref: https://opper.ai/blog/car-wash-test
[dead]
Re: 1-Bit LLM in the Browser
#13Fails "car wash" test miserably. Ask it: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ref: https://opper.ai/blog/car-wash-test
Even very large models failed tests like this up until a year or two ago...this is a very, very, small model.
Re: 1-Bit LLM in the Browser
#14Could not get any usable output from it for any real usecase.
Can't see any real value in using 1 bit llms except and apart for the sake of running it for demo or feel good factor of running a higher param model locally and in limited hardware!
Since use as it is very limited, could someone throw some light on any actual use case?
Re: 1-Bit LLM in the Browser
#15Fails "car wash" test miserably. Ask it: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ref: https://opper.ai/blog/car-wash-test
Works with the Bonsai-27B-1bit-mlx model on my mac.
Edit: this is running with MLX, and not with WebGPU on the browser, also it is a fail.
"""
You should *walk*.
Here’s why:
- *Distance*: 50 meters is very short — about 0.3 miles or 0.5 kilometers.
- *Time*: Walking takes roughly 2–3 minutes. Driving would take longer due to parking, starting the engine, and maneuvering.
- *Effort*: Walking is light exercise and avoids parking hassle.
- *Safety*: Less risk of misjudging parking space or damaging the car.
Unless you have a disability that makes walking difficult, walking is the most practical and efficient choice.
"""
Re: 1-Bit LLM in the Browser
#16Fails "car wash" test miserably. Ask it: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ref: https://opper.ai/blog/car-wash-test
Works with the Bonsai-27B-1bit-mlx model on my mac. Edit: this is running with MLX, and not with WebGPU on the browser, also it is a fail. """ You should *walk*. Here’s why: - *Distance*: 50 meters is very short — about 0.3 miles or 0.5 kilometers. - *Time*: Walking takes roughly 2–3 minutes. Driving would take longer due to parking, starting the engine, and maneuvering. - *Effort*: Walking is light exercise and avoi…
How is that not a fail?
Re: 1-Bit LLM in the Browser
#17Could not get any usable output from it for any real usecase. Can't see any real value in using 1 bit llms except and apart for the sake of running it for demo or feel good factor of running a higher param model locally and in limited hardware! Since use as it is very limited, could someone throw some light on any actual use case?
[dead]
Re: 1-Bit LLM in the Browser
#18Earlier quoted context omitted.
Works with the Bonsai-27B-1bit-mlx model on my mac. Edit: this is running with MLX, and not with WebGPU on the browser, also it is a fail. """ You should *walk*. Here’s why: - *Distance*: 50 meters is very short — about 0.3 miles or 0.5 kilometers. - *Time*: Walking takes roughly 2–3 minutes. Driving would take longer due to parking, starting the engine, and maneuvering. - *Effort*: Walking is light exercise and avoi…
How is that not a fail?
[flagged]
Re: 1-Bit LLM in the Browser
#19Fails "car wash" test miserably. Ask it: > I want to wash my car. The car wash is 50 meters away. Should I walk or drive? Ref: https://opper.ai/blog/car-wash-test
Thats absolutly not the point of these models.
They are here to route things, do tool calls, ask experts.
Re: 1-Bit LLM in the Browser
#20Earlier quoted context omitted.
I get 9-10 tps on my MacBook Pro. Though the small model seem really limited
Interestingly, I get 44 tps on Safari, MacBook Pro base model.
60tps in Chrome here - 14" MBP M4. Safari loads 3% of the model and then stops - repeatedly.
Basically zero using a 7700S in Chrome on Ubuntu though. Must be not working at all, as the performance difference can't be that large.