Live data from Hacker News

Show HN: Real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B

github.com

41–47 of 47 posts

Re: Show HN: Real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B

#42
post #40
post #35

I've been trying to do this, but I can't get voice recognition to work fast enough (meaning live) with Gemma E2B, on either an M1 max (64GB), a 5060 Ti (16Gb) or a SnapDragon 8 Gen2. Any pointers?

What's your average response time with M1 max and what's the target?

I'm only at about 650msec and, well, ideally 100 would be great.

Re: Show HN: Real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B

#43
post #42
post #40

Earlier quoted context omitted.

What's your average response time with M1 max and what's the target?

I'm only at about 650msec and, well, ideally 100 would be great.

Well, on my demo it's around 2.5s and I already consider it as a "real-time". One way to improve it is to disable the image input.

Re: Show HN: Real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B

#44

It might interest people to know you can also easily fine-tune the text portion of this specific model (E2B) to behave however you want! I fine-tuned it to talk like a pirate but you can get it to do anything you have (or can generate) training data for. (This wouldn't make it to the text to speech portion though.) So you can easily train it to act a certain way or give certain types of responses. Video: https://www.…

We're getting closer now to black mirror level of technology.

Re: Show HN: Real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B

#46
post #38

If I run this without internet connection it says "loading..." at the bottom of the localhost site and won't work. If I run this with internet connected it works flawlessly. Even if I disconnect my internet afterwards it still goes on working fine. Why there has to be an internet connection established at the time I open the localhost site when all of this should be working purely on device? Despite of this, I am rea…

the index.html is loading remote js files: https://github.com/fikrikarim/parlor/blob/main/src/index.htm... I saved them locally and changed the reference, and it worked perfectly.

This also fixed it for me. Thx.

Re: Show HN: Real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B

#47
post #39

Earlier quoted context omitted.

Don't have the time right now but will play around with it next weekend for sure and will give you more feedback with logs when I see that I can reproduce it. For now what I did was: - Tested in Chrome/Safari/Firefox on Tahoe. - Followed the quick start install instructions from github repo - Everything worked - Closed terminal - Disconnected internet (Wifi off) - Opened terminal - Started server again (uv run server…

Is this the problem? https://news.ycombinator.com/item?id=47669954

Yes. This also fixed it for me. Thx @matula.
Post reply on HN