Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI-Generated Voice Evidence Poses Dangers in Court

lawfaremedia.org

Post reply on HN
Print view View original

11–20 of 171 posts

Re: AI-Generated Voice Evidence Poses Dangers in Court

#11

Post by [deleted] »

rout39574 wrote: post #10
dghlsakjg wrote: post #5

How is AI voice faking any different than any other type of faking? How is it different than a manipulated recording, or a recording where someone is imitating another? It is just as easy to fake many paper documents, and we have accepted documents as evidence for centuries. Photos can be faked, video can be edited or faked, witnesses lie or misremember. Is this just about telling lawyers that unvetted audio recordin…

Presuming good faith here, faking recordings has been harder to do, easier to detect, and less equivocal in the past than it is now. If it takes an FX house to generate a plausible recording of me saying something I didn't say, that's a risky enterprise with a lot of witnesses. If my enemy can do it in their basement with an hour of research, the exposure risk goes way down, and consequently the expectation you'll se…

[deleted]

Re: AI-Generated Voice Evidence Poses Dangers in Court

#13

Post by datadrivenangel »

mitthrowaway2 wrote: post #3
fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

I agree. The article didn't touch on this aspect, but we're now at the point where even authentic recordings could be plausibly denied and claimed to be fake. So the entire usage of recordings as evidence will suffer a hit. We may essentially be knocked back to an 18th century level of reliance on eyewitness testimony. One wonders what the consequences for justice will be.

Digital forensics will continue to be an in-demand skill!

Re: AI-Generated Voice Evidence Poses Dangers in Court

#14

Post by nine_k »

mitthrowaway2 wrote: post #3
fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

I agree. The article didn't touch on this aspect, but we're now at the point where even authentic recordings could be plausibly denied and claimed to be fake. So the entire usage of recordings as evidence will suffer a hit. We may essentially be knocked back to an 18th century level of reliance on eyewitness testimony. One wonders what the consequences for justice will be.

I can easily imagine a future where video evidence is only acceptable in the form of chemically developed analog film, at resolutions that are prohibitively expensive to model, and audio recordings of any kind are not admissible as evidence at all. Signatures on paper, faxes, etc are, of course, inadmissible, too.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#15

Post by treetalker »

I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness).

Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the witness's credibility or ability to discern, when that's what juries are for?

All that said, emulated voices do pose big problems for litigation.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#16

Post by paulnpace »

carra wrote: post #7

Wait some more time and photos or even video recordings will be deemed just as dangerous. And then what? Even if there is real evidence, it will have to be discarded unless it can't be sufficiently validated. It will get very hard to prove anything.

If it goes that direction, one silver lining could be that they also realize video conference tools used for court proceedings should also go away.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#17

Post by bluGill »

treetalker wrote: post #15

I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…

I don't know what the latest is, but often judges are supposed to not allow "expert testimony" without checking that the person is an expert. However this is a really complex area. Judges don't want to be the one deciding a case, but not allowing some "expert" is in a way deciding the case.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#18

Post by NoMoreNicksLeft »

treetalker wrote: post #15

I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…

Excluding fake evidence is very much a responsibility of the judge. In the age of Fox News, letting the jury decide for themselves whether or not made-up bullshit is actual evidence seems like a recipe for disaster, and not necessarily one that errs on the side of caution.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#20

Post by bluGill »

carra wrote: post #7

Wait some more time and photos or even video recordings will be deemed just as dangerous. And then what? Even if there is real evidence, it will have to be discarded unless it can't be sufficiently validated. It will get very hard to prove anything.

Or perhaps camera manufactures will start putting traces in. Anyone who makes surveillance systems (like stores use) should put some end to end things in so they can say "this camera took that recording and we can see that it wasn't tampered with by...". (if anyone works for a surveillance company please run with this!) With encryption we can verify a lot of things, but it sets the bar higher than someone took a picture.

Of course in a darkroom someone skilled could always make a fake photo - but the bar is a lot lower with AI.

Post reply on HN