This is the first startup here where I think the tech should essentially be illegal. It's cool tech, yes I'm impressed at the achievement. Nuclear weapons are impressive too. OTOH this kind of thing is getting easier and easier to do, so what's a realistic way forward?
We have these measures in place and are working on others to make sure the technology is used towards the betterment of humanity.
1/ Auto moderation on text to block harmful/malicious speech. 2/ As someone pointed out in the comments, we had a manual review process in place where the user is required to read out a consent and a member from Play.ht would review it before approving the voice. We're working on improving and adding this back. 3/ The user facing service is paywalled so we don't allow everyone in. 4/ Users trying to create malicious content are flagged and reviewed. 5/ A classifier to detect AI generated speech