21–30 of 47 posts
Curious about one thing though, how does it handle switching between languages? I work with both Greek and English daily and local models usually struggle with that.
Great work, bookmarking this.
If I run this with internet connected it works flawlessly. Even if I disconnect my internet afterwards it still goes on working fine.
Why there has to be an internet connection established at the time I open the localhost site when all of this should be working purely on device?
Despite of this, I am really impressed that this actually works so fast with video input on my M4 Pro 48 GB.
I’ve been looking for a good video summarizing / understanding model!
If I run this without internet connection it says "loading..." at the bottom of the localhost site and won't work. If I run this with internet connected it works flawlessly. Even if I disconnect my internet afterwards it still goes on working fine. Why there has to be an internet connection established at the time I open the localhost site when all of this should be working purely on device? Despite of this, I am rea…
Is there anything unique here happening for the video aspect or is it just taking snapshots over and over? I’ve been looking for a good video summarizing / understanding model!
In regards to the video capability, I haven't tested it myself, but here's a benchmark/comparison from Google [0]
This is really impressive for running locally on an M3 Pro. The latency looks surprisingly good for real-time audio and video input. Curious about one thing though, how does it handle switching between languages? I work with both Greek and English daily and local models usually struggle with that. Great work, bookmarking this.
Is there anything unique here happening for the video aspect or is it just taking snapshots over and over? I’ve been looking for a good video summarizing / understanding model!
Nothing unique, it's just taking a snapshot when it's processing the input. Even processing a single image will increase the TTFT by ~0.5s on my machine, so for now, it seems to be impossible for feeding a live video and expecting a real-time response. In regards to the video capability, I haven't tested it myself, but here's a benchmark/comparison from Google [0] [0] https://huggingface.co/blog/gemma4#video-understa…
sure, maybe it's still frame-by-frame but so fast and so often that the model retains a rolling context of what's going on and can answer cleanly temporal questions.
"how packages were delivered over the last hour", etc.
If I run this without internet connection it says "loading..." at the bottom of the localhost site and won't work. If I run this with internet connected it works flawlessly. Even if I disconnect my internet afterwards it still goes on working fine. Why there has to be an internet connection established at the time I open the localhost site when all of this should be working purely on device? Despite of this, I am rea…
Huh that's weird. I just tried it and it works on my machine. Could you perhaps create a GitHub issue and share the reproduction steps and any relevant logs?
For now what I did was:
- Tested in Chrome/Safari/Firefox on Tahoe.
- Followed the quick start install instructions from github repo
- Everything worked
- Closed terminal
- Disconnected internet (Wifi off)
- Opened terminal
- Started server again (uv run server.py)
- Opened localhost in browser, it asked for camera/mic normally, granted access, saw camera live feed but "loading..." at bottom center of the site and AI did not listen/respond
- Reproduced this about 3 times with switching between wifi on/off before starting the server, always the same (working with internet; not working without)
- Figured it also works fine if I start the server with internet connected and disconnect it afterwards
That is very, very interesting. I've been hoping to have an assistant in the workshop (hands-free!) that I could talk to and have it help me with simple tasks: timers, calculating, digging up notes, etc. — basically, what the phone assistants were supposed to be, but aren't. "You will have to unlock your iphone first" is kind of a deal-breaker when you are in the middle of mixing polyurethane resin and have gloves an…
That is very, very interesting. I've been hoping to have an assistant in the workshop (hands-free!) that I could talk to and have it help me with simple tasks: timers, calculating, digging up notes, etc. — basically, what the phone assistants were supposed to be, but aren't. "You will have to unlock your iphone first" is kind of a deal-breaker when you are in the middle of mixing polyurethane resin and have gloves an…
You might be interested in the open-source https://www.home-assistant.io/voice-pe/ .
It's truly absurd how the Google voice assistant USED to work properly for setting timers, playing music, etc, and then they had to break it 15 times and finally replace it with much slower AI that only kinda does what you want. I'm done.
Selfhosted is the way to go if you want to keep your sanity. My wife has basically given up on any Google/Apple voice assistants being able to do anything useful above "set a 10 minute timer".