Live Voice Translation with Twilio and OpenAI's Realtime API

#1

Post by jeiden » Tue, Oct 01, 2024, 10:42 PM UTC

Live Voice Translation with Twilio and OpenAI's Realtime API
twilio.com

Re: Live Voice Translation with Twilio and OpenAI's Realtime API

#2

Post by jeiden » Tue, Oct 01, 2024, 10:42 PM UTC

Speech-to-Speech model from OpenAI integrated with Twilio's Voice channel demonstrating AI powered live call translation between 2 parties speaking different languages

Re: Live Voice Translation with Twilio and OpenAI's Realtime API

#3

Post by gautam14 » Tue, Oct 01, 2024, 10:46 PM UTC

In my experience, having built the infra for this, Twilio's MediaStreams and PSTN had higher latency than a simple SIP that interfaces with WebRTC, which you could connect to your backend. Curious how this changes with the OpenAI realtime, since my approach was stringing gpt-4o with Elevenlabs for the backend.

Re: Live Voice Translation with Twilio and OpenAI's Realtime API

#4

Post by jeiden » Tue, Oct 01, 2024, 10:51 PM UTC

gautam14 wrote: post #3

In my experience, having built the infra for this, Twilio's MediaStreams and PSTN had higher latency than a simple SIP that interfaces with WebRTC, which you could connect to your backend. Curious how this changes with the OpenAI realtime, since my approach was stringing gpt-4o with Elevenlabs for the backend.

Only one way to find out :) Give the open source starter app a whirl and let us know what you think! https://github.com/twilio-samples/live-translation-openai-re...