Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built an AI voice agent for Gmail

pocket.computer

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Show HN: I built an AI voice agent for Gmail

#21

Post by chrisnolet »

vidyesh wrote: post #6

I am not a gmail(web interface) user, so haven't used it but congrats on the launch! I like how your landing page is so simple and small. And the domain is amazing! I just wanted to point out, I love my Firefox but the gradient animation is so bad on Firefox! At first I thought oh cool bg animation , checked the dev tools to know what it is only to realize this shouldn't be a color band animation as I see it! Chromiu…

Oooh, thanks for the report! It looks like Firefox doesn't have dithering on gradients. (There's a bug report, but it's been opened for 14 years!)

The gradient animation is super-subtle :) Do you think I should disable it for Firefox users, or do you still think 'cool background animation' in spite of the banding?

Re: Show HN: I built an AI voice agent for Gmail

#22

Post by moralestapia »

chrisnolet wrote: post #20
moralestapia wrote: post #19

Nice, is this using an offline model? (For the AI)

It's using OpenAI's API at the moment, actually. An offline model could _probably_ handle the conversation and tool calling, but it just needs to be really fast to keep up with conversational speeds. (And really, GPT-4o is a bit too slow for my liking in this current iteration. I'm hoping that GPT-4.5 will be faster.) I'm writing up a full accounting of the stack for the post above, so check back for that and let me…

Interesting, congrats on shipping!

Re: Show HN: I built an AI voice agent for Gmail

#24

Post by chrisnolet »

upwardbound2 wrote: post #8

This looks incredibly cool and I really want to try it with my real email account (rather than a throwaway test account). In order to enable people to consider taking that leap, can you please provide more information about where the data will be sent and stored, and your legal liability, if any? Everyone's real email accounts contain extremely sensitive financial and medical secrets that allow identity theft or coul…

Thank you so much for this question, and for your thoughtful post below. It's really easy to put privacy and security to one side when you're launching a startup. And lots of users don't mind privacy when they're signing up for products. But it's something that's personally very close to my heart, and I put a tremendous effort into privacy and security because I knew I wouldn't be able to sleep at night if I cut any corners.

I worked at Apple for many years and their approach to privacy really left a mark on me. I strongly believe that preserving privacy is a moral obligation. (Especially when you're handling people's emails.)

Now, while the beta is running, when you log in to Pocket, there is a big blue switch above the fold under the title 'Privacy.' It says: 'Share recordings with our team.' If you leave it on, that's really helpful for me! But it does exactly what it says, and if you have anything sensitive you don't want to share with me, turn it off.

For your questions:

- The voice data is routed through Retell and the transcripts are passed to OpenAI's API.

- Sensitive data is retained by Retell for 10 minutes (when sharing is off).

- Sensitive data is retained by OpenAI for 30 days 'to identify abuse.'

I'm working with OpenAI to get Zero Data Retention. As it stands, their commitment has been that they will not use API input or output to train models. (I personally trust that commitment, but I understand the skepticism and if that's a deal-breaker for you.)

Retell is HIPAA-compliant and SOC 2 Type II certified. They've been great to work with.

- Regarding the privacy policy: 'User data obtained through third-party APIs (will not be used) to develop, improve, or train generalized AI and/or ML models.' This language was actually required by Google. The use of the word 'generalized' here is actually less specific; it's not AGI, but includes any kind of foundation model. There might be a point in the future where we can fine-tune one model per user with a LoRA, but I agree that the risk of PII leaking from a shared model is far too great.

- The company is a Delaware C-corp and subject to U.S. and California laws.

I really appreciate the opportunity to discuss this. I want to put privacy and security first always, and make sure that's baked into the company culture. Thanks for advocating!

Re: Show HN: I built an AI voice agent for Gmail

#25

Post by genewitch »

oxcabe wrote: post #11
upwardbound2 wrote: post #8

This looks incredibly cool and I really want to try it with my real email account (rather than a throwaway test account). In order to enable people to consider taking that leap, can you please provide more information about where the data will be sent and stored, and your legal liability, if any? Everyone's real email accounts contain extremely sensitive financial and medical secrets that allow identity theft or coul…

These concerns, IMO, are at least as important as the actual value proposition. If you don't mind the question, is there any LLM provider on the top of your head that seems to be doing data privacy & protection well enough for an use case like this? Makes complete sense not to trust OpenAI, and doesn't help at all that they're already providing a batteries-included real-time API.

Yeah, the services I provide. If someone wants to use say stable diffusion I can link a new folder to the outputs folder and start stable diffusion up. Then just unlink the folder from outputs

Did I mention the linked folder resides in tmpfs?

This stuff is not hard, but user data is so delectable.

Re: Show HN: I built an AI voice agent for Gmail

#26

Post by genewitch »

oxcabe wrote: post #13
upwardbound2 wrote: post #12

Earlier quoted context omitted.

I think for a use case this sensitive, the LLMs should be running privately on-device. I use DeepSeek-R1 in ollama, and Llama3.3 also in ollama, and both work well for simple agentic use cases for me. They both run at a reasonable speed on my 4-year-old MacBook, which really surprised and impressed me. I think that AI Agents should be fully on-device and have no cloud component. For example, on the immigrants' rights…

Thanks for such a complete reply. I've locally tried ollama with the models and sizes you mention on a MacBook with M3 Pro chip. It often hallucinated, used a lot of battery and increased the hardware temperature substantially as well. (Still, I'd argue I didn't put much time into configuring it, which could've solve the hallucinations) Ideally, we should all have accesss to local, offline, private LLM usage, but har…

I hate to be this person but the system prompt matters. The model size matters.

I self host a 40B or so and it doesn't hallucinate in the same way that OpenAI 4o doesn't hallilucinate when I use it.

Small models are incredibly impressive but require a lot more attention to how you interact with it. There are tools like aider that can take advantage of the speed of smaller models and have a larger model check for obvious BS.

I think this idea got spread because at least deepseek qwen distilled and llama support this now you can use a 20GB llama and pair it with a 1.5B parameter model and it screams. The small model usually manages 30-50% of the total output tokens, with the rest corrected by the large model.

This results in a ~30-50% speedup, ostensibly. I haven't literally compared but it is a lot faster than it was for barely any more memory commit.

Re: Show HN: I built an AI voice agent for Gmail

#27

Post by upwardbound2 »

chrisnolet wrote: post #24
upwardbound2 wrote: post #8

This looks incredibly cool and I really want to try it with my real email account (rather than a throwaway test account). In order to enable people to consider taking that leap, can you please provide more information about where the data will be sent and stored, and your legal liability, if any? Everyone's real email accounts contain extremely sensitive financial and medical secrets that allow identity theft or coul…

Thank you so much for this question, and for your thoughtful post below. It's really easy to put privacy and security to one side when you're launching a startup. And lots of users don't mind privacy when they're signing up for products. But it's something that's personally very close to my heart, and I put a tremendous effort into privacy and security because I knew I wouldn't be able to sleep at night if I cut any…

Thank you for these details! Would you consider putting these answers on a page on the site, and also allowing send a notification email to users anytime any of this is going to change, so users have a chance to stop using the product if there will be a change they do not agree with?

Would you consider allowing the user to select between OpenAI vs Anthropic for the foundation model? I'd recommend making Anthropic the default, as does the Perplexity team: https://www.anthropic.com/customers/perplexity

In the Privacy Policy, maybe you can keep the Google-required sentence, and also add another sentence that makes it explicit that user data will only be used to train user-specific models. This would go a long way towards reassuring many people.

I'd love to try your DSL if you are accepting dev partners. You could reach me at strangecompanyventure@gmail.com if so, I'd love to try it out and it seems very powerful if you also used it for the D&D game project.

Is the game still available somewhere? The old link doesn't seem to still point to it but I'm a big fan of the interactive fiction genre and would love to test the game too, and any other examples you have of the DSL you're designing.

Cheers and thank you for your commitment to principles. You have my respect and probably a number of other readers too.

Re: Show HN: I built an AI voice agent for Gmail

#28

Post by vidyesh »

chrisnolet wrote: post #21
vidyesh wrote: post #6

I am not a gmail(web interface) user, so haven't used it but congrats on the launch! I like how your landing page is so simple and small. And the domain is amazing! I just wanted to point out, I love my Firefox but the gradient animation is so bad on Firefox! At first I thought oh cool bg animation , checked the dev tools to know what it is only to realize this shouldn't be a color band animation as I see it! Chromiu…

Oooh, thanks for the report! It looks like Firefox doesn't have dithering on gradients. (There's a bug report, but it's been opened for 14 years!) The gradient animation is super-subtle :) Do you think I should disable it for Firefox users, or do you still think 'cool background animation' in spite of the banding?

You are welcome.

I think better to disable it, imo its more distracting and glitchy than being cool.

14 years!

Re: Show HN: I built an AI voice agent for Gmail

#30

Post by protocolture »

chrisnolet wrote: post #14
protocolture wrote: post #4

>I previously did a Show HN for ‘D&D meets Siri’: I have been messing around with something similar for roleplaying. If you have sourcecode or something to release I would be interested.

Nice! I’d love to check it out when you’re ready to share it. I’ll likely release the source code for mine if/when I publish the DSL. In the meantime, since the original link has changed, feel free to try it out at: https://pocket.computer/dungeons . Happy to chat more if you want to know how parts of it are done!

Mine will be pretty bespoke. I am trying to build an NPC that can be interacted with in the context of Alien, so likely a malfunctioning AI.

I was using Home Assistant as a base, but I found its ability to interact with hooks very hit and miss, so this year I am just building it from scratch in python.

Post reply on HN