In the future this stuff will get so good that the public will beg to be surveilled at all times because it will be the only way to prove what you didn't do. You will learn to love Total Information Awareness. Consent status: manufactured :)
AI-Generated Voice Evidence Poses Dangers in Court
21–30 of 171 posts
Re: AI-Generated Voice Evidence Poses Dangers in Court
#22Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…
I agree. The article didn't touch on this aspect, but we're now at the point where even authentic recordings could be plausibly denied and claimed to be fake. So the entire usage of recordings as evidence will suffer a hit. We may essentially be knocked back to an 18th century level of reliance on eyewitness testimony. One wonders what the consequences for justice will be.
There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#23I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the people that contributed to this. You cannot create something and pretend its use has nothing to do with you.
The tools aren't perfect yet, so it's not too late to stop. Stop the ridiculous image and audio generation tools before it's too late. Nothing of value is lost when these models are made private again, and research is simply halted.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#24How is AI voice faking any different than any other type of faking? How is it different than a manipulated recording, or a recording where someone is imitating another? It is just as easy to fake many paper documents, and we have accepted documents as evidence for centuries. Photos can be faked, video can be edited or faked, witnesses lie or misremember. Is this just about telling lawyers that unvetted audio recordin…
Nuclear weapons require nation-state levels of resources. Now imagine you can build one in your basement for $10k. Does that change things at all?
Re: AI-Generated Voice Evidence Poses Dangers in Court
#25I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the peop…
It actually is too late for that. For anyone unaware, the open source models are already more than sufficient enough for imitations and deepfakes. For better or worse there's no going back.
Personally, I'd rather we all know this tech is out there and develop defense mechanisms rather than thinking hiding it away will prevent harm.
The cyber security industry exists because of all the privacy and security issues posed by all the tech we already have had for the past several decades.
I'm confident the same will happen for AI simply because it is a business opportunity and other businesses and institutions are already talking about these issues.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#26I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the peop…
You cannot create something and pretend its use has nothing to do with you.
has always worked with guns so it'll always work with anything else :) and guns will kill more people that AI-generated shit for foreseeable future
Re: AI-Generated Voice Evidence Poses Dangers in Court
#27How is AI voice faking any different than any other type of faking? How is it different than a manipulated recording, or a recording where someone is imitating another? It is just as easy to fake many paper documents, and we have accepted documents as evidence for centuries. Photos can be faked, video can be edited or faked, witnesses lie or misremember. Is this just about telling lawyers that unvetted audio recordin…
Nuclear weapons require nation-state levels of resources. Now imagine you can build one in your basement for $10k. Does that change things at all?
I understand what you are saying, but my point is that I could make plausible sounding recordings that did not reflect reality by, for example, cutting recordings up with freeware like Audacity, or even using a consumer level double tapedeck before that. It wasn't Manhattan project levels of effort before this.
This seems more like people losing their minds over 3d printed guns, when hobbyists with a drill press have been making guns in the garage for decades.
Yeah, its easier now to fake voice, but its not as if what this article warns against wasn't possible before the latest AI hype cycle. And it is also worth noting that voice cloning/changing technology is not particularly new either (I've been able to sound like Morgan Freeman using a phone app for at least half a decade).
I agree that courts should be cautious around accepting voice recording evidence, I just don't think that the ability to do this is new.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#28Here's my plan.
1. Cryptographically hash each piece of media when it's recorded.
2. Submit the hash to a "trusted" authority.
3. It will add a timestamp and sign the result.
4. Now, as long as you keep the original, without re-compressing, and you trust the authority, you have some evidence that the media existed at a timestamp. On or before.
This doesn't prove authenticity, but in many cases, establishing a timestamp would be enough. Forgeries probably wouldn't be created until later, after the shit hit the fan.
Or maybe this doesn't work at all.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#29I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…
Excluding fake evidence is very much a responsibility of the judge. In the age of Fox News, letting the jury decide for themselves whether or not made-up bullshit is actual evidence seems like a recipe for disaster, and not necessarily one that errs on the side of caution.
To the contrary, in US courts the jury determines whether evidence (documentary, testimonial) is credible or not, and what weight (if any) to assign it. (Experts, à la Daubert etc., are a different matter because they give expert opinions, not factual evidence based on personal witnessing of the events in the case, so the judge does perform a gatekeeping function, essentially to ensure the underlying field/science is reliable.)
While certainly Fox News headlines would not reach the jury in most instances, that is on account of hearsay, lack of qualification, materiality, relevance, and similar rules. It is not a prior credibility or weight determination by the judge, as I understand TFA to be advocating. So: did the witness hear a voice that he believed to be the one in question? If so, jury gets to decide (unless unfairly prejudicial or some other overriding rule comes into play).
Re: AI-Generated Voice Evidence Poses Dangers in Court
#30Why can't objecting party ask to voir dire the witness and find out if they can distinguish between AI and real recordings?