Earlier quoted context omitted.
Excluding fake evidence is very much a responsibility of the judge. In the age of Fox News, letting the jury decide for themselves whether or not made-up bullshit is actual evidence seems like a recipe for disaster, and not necessarily one that errs on the side of caution.
To the contrary, in US courts the jury determines whether evidence (documentary, testimonial) is credible or not, and what weight (if any) to assign it. (Experts, à la Daubert etc., are a different matter because they give expert opinions, not factual evidence based on personal witnessing of the events in the case, so the judge does perform a gatekeeping function, essentially to ensure the underlying field/science is…
AI-Generated Voice Evidence Poses Dangers in Court
41–50 of 171 posts
Re: AI-Generated Voice Evidence Poses Dangers in Court
#42I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…
I don't know what the latest is, but often judges are supposed to not allow "expert testimony" without checking that the person is an expert. However this is a really complex area. Judges don't want to be the one deciding a case, but not allowing some "expert" is in a way deciding the case.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#43Earlier quoted context omitted.
I don't know what the latest is, but often judges are supposed to not allow "expert testimony" without checking that the person is an expert. However this is a really complex area. Judges don't want to be the one deciding a case, but not allowing some "expert" is in a way deciding the case.
Isn't it up to the opposing party, and not the judge, to impeach an expert?
Re: AI-Generated Voice Evidence Poses Dangers in Court
#44I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the peop…
Re: AI-Generated Voice Evidence Poses Dangers in Court
#45Earlier quoted context omitted.
I don't know what the latest is, but often judges are supposed to not allow "expert testimony" without checking that the person is an expert. However this is a really complex area. Judges don't want to be the one deciding a case, but not allowing some "expert" is in a way deciding the case.
Isn't it up to the opposing party, and not the judge, to impeach an expert?
Re: AI-Generated Voice Evidence Poses Dangers in Court
#46Earlier quoted context omitted.
There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.
This is unironically a usecase for blockchain.
Imagine if you will, that the NVR (recording system) has a unique private key flashed in during manufacturing, with the corresponding public key printed on it's nameplate. The device can sign a video clip and its related meta-data before exporting. Now, any decent hacker could see possible holes in this system, but it could be made tamper-resistant enough that any non-expert wouldn't be able to fabricate a signed video. Then the evidence becomes the signed video and the NVR's serial number and public key. Not perfect, but probably good enough.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#47Earlier quoted context omitted.
Nuclear weapons require nation-state levels of resources. Now imagine you can build one in your basement for $10k. Does that change things at all?
I understand what you are saying, but my point is that I could make plausible sounding recordings that did not reflect reality by, for example, cutting recordings up with freeware like Audacity, or even using a consumer level double tapedeck before that. It wasn't Manhattan project levels of effort before this. This seems more like people losing their minds over 3d printed guns, when hobbyists with a drill press have…
Sure, and https://en.wikipedia.org/wiki/David_Hahn managed to get quite a bit of nuclear material.
Being able to do it at scale, convincingly, in real-time, for any arbitrary text, with just 30s or so of someone's voice as a sample, changes the calculus a lot.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#48I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…
Re: AI-Generated Voice Evidence Poses Dangers in Court
#49Earlier quoted context omitted.
Excluding fake evidence is very much a responsibility of the judge. In the age of Fox News, letting the jury decide for themselves whether or not made-up bullshit is actual evidence seems like a recipe for disaster, and not necessarily one that errs on the side of caution.
To the contrary, in US courts the jury determines whether evidence (documentary, testimonial) is credible or not, and what weight (if any) to assign it. (Experts, à la Daubert etc., are a different matter because they give expert opinions, not factual evidence based on personal witnessing of the events in the case, so the judge does perform a gatekeeping function, essentially to ensure the underlying field/science is…
Nor should obviously fake evidence reach the jury. They can judge for themselves whether testimony is credible, but this is far different than admitting faked evidence. And if you can't see the difference, I'm not sure I'm qualified to explain it to you.
Re: AI-Generated Voice Evidence Poses Dangers in Court
#50Earlier quoted context omitted.
Nuclear weapons require nation-state levels of resources. Now imagine you can build one in your basement for $10k. Does that change things at all?
I understand what you are saying, but my point is that I could make plausible sounding recordings that did not reflect reality by, for example, cutting recordings up with freeware like Audacity, or even using a consumer level double tapedeck before that. It wasn't Manhattan project levels of effort before this. This seems more like people losing their minds over 3d printed guns, when hobbyists with a drill press have…
Cutting up what recordings, exactly? If you're mixing-and-matching, you need a pretty broad corpus of source material with fairly consistent recording quality (background noise, etc), and even still you're limited to reproducing words that are already present. You can't cut-and-splice together a recording of me saying 'Yes, dghlsakjg, I embezzled $1 million and blew it on the ponies' because there will be no recording of me saying your username, 'embezzled', or 'ponies.'
The problem comes with the voice-cloning technology that can construct entirely new sentences based on relatively short voice profile samples.
> I've been able to sound like Morgan Freeman using a phone app for at least half a decade
You've been able to sound like Morgan Freeman because of specific, hard tuning work put into the voice changer. Now, you can sound like your boss, or your neighbour, or your ex.