Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

blog.ivan.digital

Post reply on HN
Print view View original

121–130 of 139 posts

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#121

Post by nowittyusername »

pettyjohn wrote: post #101
nowittyusername wrote: post #31

Earlier quoted context omitted.

I've been working on building my own voice agent as well for a while and would love to talk to you and swap notes if you have the time. I have many things id like to discuss, but mainly right now im trying to figure out how a full duplex pipeline like this could fit in to an agentic framework. Ive had no issues with the traditional route of stt > llm > tts pipeline as that naturally lends itself with any agentic beha…

+1 what's your use case and what specific LLMs are you using? I'm using stt > post-trained models > tts for the education tool I'm building, but full STS would be the end-game. e-mail and discord username are in my profile if you want to connect!

sent!

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#122

Post by taf2 »

ttul wrote: post #89
taf2 wrote: post #55

Earlier quoted context omitted.

I forked and added tool calling by running another llm in parallel to infer when to call tools it works well for me to toggle lights on and off. Code updates here https://github.com/taf2/personaplex

Cool approach. So basically the part that needs to be realtime - the voice that speaks back to you - can be a bit dumb so long as the slower-moving genius behind the curtain is making the right things happen.

Yes exactly- one part I did not like is we have to also separately transcribe because it does not also provide what the person said only what the ai said

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#123

Post by 4dregress »

lxgr wrote: post #66
mentalgear wrote: post #13

Earlier quoted context omitted.

Your article does a great job of summerizing the dangers (no idea what those people are that downvote you for it): > Before long, Gavalas and Gemini were having conversations as if they were a romantic couple. The chatbot called him “my love” and “my king” and Gavalas quickly fell into an alternate world, according to his chat logs. > kill himself, something the chatbot called “transference” and “the real final step”…

There are tons of safety concerns of this shape around LLMs, but do they have anything to do with the particular one presented in this article? Unless I'm missing something, what's being presented is a small speech on-device model, not an explicit use case like a "virtual friend".

In the article the change of interface lead to the person killing themselves.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#126

Post by villgax »

ricardobeat wrote: post #56

No mention of tool use. If the model cannot emit both text and audio at the same time, to enable tools, it’s not really useful at all for voice agents.

it's clearly labeled a research project, feel free to DIY

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#127

Post by AlexeyBelov »

andreadev wrote: post #117
armcat wrote: post #26

I really like this, and have actually tried (unsuccessfully) to get PersonaPlex to run on my blackwell device - I will try this on Mac now as well. There are a few caveats here, for those of you venturing in this, since I've spent considerable time looking at these voice agents. First is that a VAD->ASR->LLM->TTS pipeline can still feel real-time with sub-second RTT. For example, see my project https://github.com/aca…

The framing in this thread is full-duplex vs composable pipeline, but I think the real architecture is both running simultaneously — and this library is already halfway there. The fact that qwen3-asr-swift bundles ASR, TTS, and PersonaPlex in one Swift package means you already have all the pieces. PersonaPlex handles the "mouth" — low-latency backchanneling, natural turn-taking, filler responses at RTF 0.87. Meanwhi…

LLM slop.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#128

Post by andreadev »

AlexeyBelov wrote: post #127
andreadev wrote: post #117

Earlier quoted context omitted.

The framing in this thread is full-duplex vs composable pipeline, but I think the real architecture is both running simultaneously — and this library is already halfway there. The fact that qwen3-asr-swift bundles ASR, TTS, and PersonaPlex in one Swift package means you already have all the pieces. PersonaPlex handles the "mouth" — low-latency backchanneling, natural turn-taking, filler responses at RTF 0.87. Meanwhi…

LLM slop.

Which part specifically?

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#129

Post by lxgr »

hedora wrote: post #98
4dregress wrote: post #7

This sounds quite dangerous https://www.theguardian.com/technology/2026/mar/04/gemini-ch...

I really think a small amount of education on what LLMs actually are (document completers) and how context works (like present it as a top-level UI element, complete with fork and rollback) would solve most of these issues. Given how they work, it's really not surprising that if it sees the first half of a lovers' suicide pact, it'll successfully fill in the second half. A small amount of understanding of the underly…

> a small amount of education on what LLMs actually are (document completers)

At this point in capabilities, this seems like the wrong layer of reasoning about LLM.

In particular, I don't think this framing will be very effective in preventing possible harm, similarly to how knowing that depression is "probably just some chemical imbalance between neurotransmitters in your brain" is not a good way to help people suffering from it in getting better.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#130

Post by jwr »

atiorh wrote: post #120
jwr wrote: post #10

As a heavy user of MacWhisper (for dictation), I'm looking forward to better speech-to-text models. MacWhisper with Whisper Large v3 Turbo model works fine, but latency adds up quickly, especially if you use online LLMs for post-processing (and it really improves things a lot).

MacWhisper supports 10x faster models with the same accuracy like Parakeet v2 (they were the first to do it 6-9 months ago). Have you tried those?

Yes. I found Parakeet V3 (1.24GB) to be worse than Whisper (Large v3 Turbo, 20240930). Subtly worse, but still. I went back to Whisper because with dictation if you can't trust the tool, you won't bother to dictate.
Post reply on HN