Earlier quoted context omitted.
How would granny prove the scammer used play.ht?
If law enforcement ever busts a scammer and discovers a tool like this was essential to the scam, that would generate lawsuits.
True
Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
191–200 of 471 posts
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#192This is the first startup here where I think the tech should essentially be illegal. It's cool tech, yes I'm impressed at the achievement. Nuclear weapons are impressive too. OTOH this kind of thing is getting easier and easier to do, so what's a realistic way forward?
You may get your wish. The FTC posted an article about this a week ago. [1] > The FTC Act’s prohibition on deceptive or unfair conduct can apply if you make, sell, or use a tool that is effectively designed to deceive – even if that’s not its intended or sole purpose. It seems like an awfully broad rule? But they probably could go after this startup if they noticed it. There are some kinds of businesses where making…
Thanks for sharing this ^
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#193Earlier quoted context omitted.
> Making then the illegal would accomplish nothing since it's already out in the wild. Not true. Making it illegal wouldn't make it nonexistent -- that's true. But making it illegal would provide at least some method of mitigating some of the harm. That's more than what we have right now. > We have to accept that this tech exists and there will be both positive and negative outcomes from it. Of course. But that doesn…
> Not true. Making it illegal wouldn't make it nonexistent -- that's true. But making it illegal would provide at least some method of mitigating some of the harm. It's already illegal to impersonate someone to steal money or scam them, and those laws were on the books before computers existed. > Of course. But that doesn't mean it's futile to try to reduce the negative outcomes. You can run something on a consumer G…
> It's already illegal to impersonate someone to steal money or scam them, and those laws were on the books before computers existed.
There are two hurdles a criminal has to get past:
1. decide to break the law 2. figure out how to pull off their scam
It sounds like you're saying that since hurdle #1 already exists, hurdle #2 is irrelevant? No, of course it isn't. That's like saying that gun control can't possible help because it's already illegal to shoot someone.
Adding difficulty to a crime reduces (but does not eliminate) the prevalence of the crime.
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#194This is already being used for scams. https://playground.play.ht/listen/1079 ( https://archive.ph/HKjue ) How exactly do you expect to combat this type of content?
This is not a full solution, just spitballing, but I wonder how effective it would be to have a flagging system built with GPT4 where the prompt was some form of "This is text submitted to a text-to-voice model. Determine the probability that this is being used maliciously." Then manually review anything that returns >X%.
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#195Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#196Looks great. I've waiting for a service like that ever since Microsoft released their paper on speech synthesis using voice samples. Feature requests: - make the voice generation available via API so devs can embed that in their app - expose a streaming API like Polly so we can feed it text in real time and get the voice as an audio stream - make it Hipaa compliant and have a plans that offer signing a BAA I'll be yo…
We have an API - https://docs.play.ht/reference/api-getting-started
We have a beta streaming endpoint but the latency is not real time yet (something we're working on) and are adding an endpoint to create voices.
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#197I recommend you immediately add identity verification (state-issued identification verification), set up appropriate secrets store for PII, and audit trail EVERYTHING your users are doing, storing the contents in a secure location. Yesterday. This service will be used to harm others, shortly. I do think that there are exciting, honest things that can be done with this service but you need to set up some friction for…
Couldn't agree more with your comment. We are working on counter measures like manual verification of voice, a classifier to detect cloned speech, etc. As of now we have auto moderation in place that detects and blocks hate/harmful speech.
The cat's out of the bag, I'd say you guys should just go full steam ahead and make sure it's your names in the headlines
No need for a bunch of onerous kyc or anything IMO
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#198Tried it out and it made me sound British (I'm Australian, but I only have a mild accent). It seems to have gotten my tone of voice close but not my accent.
And then my pacing seems really off. Even a simple "Hey this is afro88. How's it going?" sounded inhuman.
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#199So you could grab leaked info from a YouTuber and fully impersonate a celebrity in any service that doesn't support 2FA(?), this is also very bad for any podcaster.
Voices aren't unique at all. Nobody should have been using a voice pattern as authentication at any point
Re: Launch HN: Play.ht (YC W23) – Generate and clone voices from 20 seconds of audio
#200This is the first startup here where I think the tech should essentially be illegal. It's cool tech, yes I'm impressed at the achievement. Nuclear weapons are impressive too. OTOH this kind of thing is getting easier and easier to do, so what's a realistic way forward?
Cofounder here,
What you see in the above demo is a very rate-limited demo of our upcoming model.
We realize how dangerous this technology can be and have built a lot of mitigations on our main product (Play.ht) to reduce possible abuse:
- We strictly moderate the generated text of any sexual, offensive, racist, or threatening content. It automatically gets detected and blocked.
- We built and are offering for free a tool that can identify AI generated vs human-generated audio (https://play.ht/voice-classifier-detect-ai-voices/), we will continue to invest in this tool, and we hope it helps with deploying this technology safely.
- If we get any reports of a cloned voice without consent, we block the user and remove the voice instantly.
- The price of high-fidelity voice cloning is too high for scammers to use at scale; we have been live with it for four months and haven't had any cases of abuse so far.
Like any technology, it has the potential to be abused, and we are working hard to mitigate that and deploy it safely. We will continue to observe the use cases and user feedback and improve the safety of the service accordingly.
Since we launched voice cloning 4 months ago, we have seen enough genuine use cases which motivated us to keep moving forward and figure out safe ways to make the technology useful for all.