Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift • Hacker News Board

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#101

Post by pettyjohn » Thu, Mar 05, 2026, 4:48 PM UTC

nowittyusername wrote: post #31

armcat wrote: post #26

I really like this, and have actually tried (unsuccessfully) to get PersonaPlex to run on my blackwell device - I will try this on Mac now as well. There are a few caveats here, for those of you venturing in this, since I've spent considerable time looking at these voice agents. First is that a VAD->ASR->LLM->TTS pipeline can still feel real-time with sub-second RTT. For example, see my project https://github.com/aca…

I've been working on building my own voice agent as well for a while and would love to talk to you and swap notes if you have the time. I have many things id like to discuss, but mainly right now im trying to figure out how a full duplex pipeline like this could fit in to an agentic framework. Ive had no issues with the traditional route of stt > llm > tts pipeline as that naturally lends itself with any agentic beha…

+1

what's your use case and what specific LLMs are you using?

I'm using stt > post-trained models > tts for the education tool I'm building, but full STS would be the end-game. e-mail and discord username are in my profile if you want to connect!

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#102

Post by robotswantdata » Thu, Mar 05, 2026, 5:03 PM UTC

armcat wrote: post #26

I really like this, and have actually tried (unsuccessfully) to get PersonaPlex to run on my blackwell device - I will try this on Mac now as well. There are a few caveats here, for those of you venturing in this, since I've spent considerable time looking at these voice agents. First is that a VAD->ASR->LLM->TTS pipeline can still feel real-time with sub-second RTT. For example, see my project https://github.com/aca…

+ 1 , agree still prefer composable pipeline architecture for voice agents. The flexibility on switching LLM for cost optimisation or quality is great for scaled use cases.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#103

Post by biomcgary » Thu, Mar 05, 2026, 5:20 PM UTC

armcat wrote: post #26

I really like this, and have actually tried (unsuccessfully) to get PersonaPlex to run on my blackwell device - I will try this on Mac now as well. There are a few caveats here, for those of you venturing in this, since I've spent considerable time looking at these voice agents. First is that a VAD->ASR->LLM->TTS pipeline can still feel real-time with sub-second RTT. For example, see my project https://github.com/aca…

Do you know if any of these multi-stage approaches can run on an 8gb M1 Air?

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#104

Post by sarmike31 » Thu, Mar 05, 2026, 5:24 PM UTC

If you're interested in Demos without installing the thing, he has a site here: https://research.nvidia.com/labs/adlr/personaplex/

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#105

Post by 0xbadcafebee » Thu, Mar 05, 2026, 5:26 PM UTC

KaiserPister wrote: post #63

I am strongly put off by the LLM writing in this piece. It makes me question quality of the project before even attempting a download. Who would put effort into building this only to compose a low effort puff piece?

I wasn't put off by it. I read the article, got all the information I needed, it was interesting and informative. (In fact, I find the human-written ones more often annoying; most people are not good at writing, and are apt to create huge walls of text, whereas the AI is biased towards making the information easy to consume)

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#106

Post by reactordev » Thu, Mar 05, 2026, 5:36 PM UTC

Anonbrit wrote: post #65

reactordev wrote: post #48

Earlier quoted context omitted.

I beg to differ. My agent just one-shotted a MicrophoneBufferManager in swift when asked. Complete with AVFoundation and a tap for the audio buffer. It really is trivial.

Any chance of pushing it to GitHub? My swift knowledge could be written out on an oversized beer coaster currently, so I'm still collecting useful snippets

https://gist.github.com/gabereiser/cd8c67262717afd2539dc9c3d...

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#107

Post by hirvi74 » Thu, Mar 05, 2026, 5:44 PM UTC

reactordev wrote: post #48

isodev wrote: post #35

Earlier quoted context omitted.

> You just need an audio buffer That alone to do right on macOS using Swift is an exercise in pain that even coding bots aren't able to solve first time right :)

I beg to differ. My agent just one-shotted a MicrophoneBufferManager in swift when asked. Complete with AVFoundation and a tap for the audio buffer. It really is trivial.

I've also had great results with using LLMs to pry into Apple's private and undocumented APIs. I've been impressed with the lack of hallucinations for C/C++ and Obj-C functions.

I can attest that the quality in this domain has greatly improved over the years too. I am not always fan of the quality of the Swift code that my LLM produces, but I am impressed that what is often produced works in one shot, as well. The quality also is not that important to me because I can just refactor the logic myself, and often prefer to do it anyway. I cannot hold an LLM to any idiosyncrasies that I do not share with it.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#108

Post by tadfisher » Thu, Mar 05, 2026, 6:26 PM UTC

hedora wrote: post #98

4dregress wrote: post #7

This sounds quite dangerous https://www.theguardian.com/technology/2026/mar/04/gemini-ch...

I really think a small amount of education on what LLMs actually are (document completers) and how context works (like present it as a top-level UI element, complete with fork and rollback) would solve most of these issues. Given how they work, it's really not surprising that if it sees the first half of a lovers' suicide pact, it'll successfully fill in the second half. A small amount of understanding of the underly…

"Fraudulent" is more apt. They have weaponized trust in these things to sell their services, and now ads.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#109

Post by _magiic_kards » Thu, Mar 05, 2026, 6:28 PM UTC

armcat wrote: post #26

I really like this, and have actually tried (unsuccessfully) to get PersonaPlex to run on my blackwell device - I will try this on Mac now as well. There are a few caveats here, for those of you venturing in this, since I've spent considerable time looking at these voice agents. First is that a VAD->ASR->LLM->TTS pipeline can still feel real-time with sub-second RTT. For example, see my project https://github.com/aca…

+1 on this pipeline! You can use a super small model to perform an immediate response and a structured output that pipes into a tool call (which may be a call to a "more intelligent" model) or initiates skill execution. Having this async function with a fast response (TTS) to the user + tool call simultaneously is awesome.

Re: Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speech-to-Speech in Swift

#110

Post by tadfisher » Thu, Mar 05, 2026, 6:43 PM UTC

allreduce wrote: post #70

zozbot234 wrote: post #33

Earlier quoted context omitted.

> The chatbot took on a persona that Gavalas hadn’t prompted That's an interesting claim, how can we be sure of it? If Gavalas didn't have to do anything special to elicit the bizarre conspiracy-adjacent content from Gemini Pro, why aren't we all getting such content in our voice chats? Mind you, the case is still extremely concerning and a severe failure of AI safety. Mass-marketed audio models should clearly includ…

There are many explanations why these incidents could be rare but not impossible. These models are still stochastic and very good at picking up nuances in human speech. It may be simply unlikely to go off the rails like that or (more terrifyingly) it might pick up on some character trait or affectation. Honestly I'm appalled by the lack of safety culture here. "My plane killed only 1% of pilots" and variations thereo…

The default mode that untrained people enter when thinking about mental illness is denial, as in, "thank that will never happen to me". Appallingly, that is ingrained in AI product safety; why would we sacrifice double-digit effectiveness/performance/whatever to prevent negative interactions with the single-digit population who are susceptible to mental illness in the first place?

We just aren't comfortable with the idea that all of us are fragile, and when we think we could endure a situation that would induce self-harm in others, we are likely wrong.