Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI-Generated Voice Evidence Poses Dangers in Court

lawfaremedia.org

Post reply on HN
Print view View original

31–40 of 171 posts

Re: AI-Generated Voice Evidence Poses Dangers in Court

#31

Post by mrandish »

thereddaikon wrote: post #22
mitthrowaway2 wrote: post #3

Earlier quoted context omitted.

I agree. The article didn't touch on this aspect, but we're now at the point where even authentic recordings could be plausibly denied and claimed to be fake. So the entire usage of recordings as evidence will suffer a hit. We may essentially be knocked back to an 18th century level of reliance on eyewitness testimony. One wonders what the consequences for justice will be.

There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.

Yep, "chain of custody." Came here hoping to see that concept discussed since it's how the system already deals with cases of potential evidence tampering. If the evidence is of material importance and there's no sufficiently credible chain of custody, then its validity can be questioned. The concept started around purely physical evidence but applies to image, audio and video. The good thing about the ubiquity of deepfake memes on social media is that it familiarizes judges and juries with how easy it now is to create plausible fake media.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#32

Post by qwertox »

tgv wrote: post #23

I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the peop…

I do want a computer which talks to me like a person, with that agility. I am used to listening to voice.

I am tired of reading so much stuff which could well be spoken to me. Like this comment here, which you now need to read.

It's only very recently that living beings on this world have learned to read and write, it's not normal. It's normal to communicate through sound.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#33

Post by AnthonyMouse »

thereddaikon wrote: post #22
mitthrowaway2 wrote: post #3

Earlier quoted context omitted.

I agree. The article didn't touch on this aspect, but we're now at the point where even authentic recordings could be plausibly denied and claimed to be fake. So the entire usage of recordings as evidence will suffer a hit. We may essentially be knocked back to an 18th century level of reliance on eyewitness testimony. One wonders what the consequences for justice will be.

There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.

The usual chain of custody goes something like: The store has a video surveillance system which the police collect the footage from, so the chain of custody goes through the store and the police which implies that nobody other than those two have tampered with it.

But then you have an inside job where the perpetrators work for the store and have doctored the footage before the police come to pick it up, or a corrupt cop who wants to convict someone without proving their case or is accepting bribes to convict the wrong person and now has easy access to forgeries. Chain of custody can't help you in either of these cases, and both of those things definitely happen in real life, so how do you determine when they happen or don't?

Re: AI-Generated Voice Evidence Poses Dangers in Court

#34

Post by nostrademons »

Lammy wrote: post #21

In the future this stuff will get so good that the public will beg to be surveilled at all times because it will be the only way to prove what you didn't do. You will learn to love Total Information Awareness. Consent status: manufactured :)

It could easily go the other way, where the public doesn't care what people think they did or didn't do and just does whatever they want, because they don't respect the state and believe the social contract has been broken. "Fuck justice, talk to my AR-15."

There's ample evidence that this is already happening, eg. recent headlines about kids being radicalized at increasingly younger ages, groups like No Lives Matter that embrace violent nihilism, increased domestic terrorism, record high gun ownership across both sides of the political spectrum, authority figures that just do whatever they want and ignore any form of law or accountability, etc.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#35

Post by krainboltgreene »

qwertox wrote: post #32
tgv wrote: post #23

I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the peop…

I do want a computer which talks to me like a person, with that agility. I am used to listening to voice. I am tired of reading so much stuff which could well be spoken to me. Like this comment here, which you now need to read. It's only very recently that living beings on this world have learned to read and write, it's not normal. It's normal to communicate through sound.

What an incredible line to end on. “It’s not normal to read/write”.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#36

Post by AnthonyMouse »

bluGill wrote: post #20
carra wrote: post #7

Wait some more time and photos or even video recordings will be deemed just as dangerous. And then what? Even if there is real evidence, it will have to be discarded unless it can't be sufficiently validated. It will get very hard to prove anything.

Or perhaps camera manufactures will start putting traces in. Anyone who makes surveillance systems (like stores use) should put some end to end things in so they can say "this camera took that recording and we can see that it wasn't tampered with by...". (if anyone works for a surveillance company please run with this!) With encryption we can verify a lot of things, but it sets the bar higher than someone took a pict…

Cryptography doesn't really fix it. There are a zillion camera makers and all it takes is one of them to have poor security and leak the keys. Then the forger uses any of the cameras with extractable keys to sign their forgery.

Or they just point a camera at a high resolution playback of a forged video.

This also assumes you can trust all the camera makers, because by doing this you're implicitly giving them each authorization to produce forged videos. Recall that many of these companies are based in China.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#37

Post by Lanolderen »

carra wrote: post #7

Wait some more time and photos or even video recordings will be deemed just as dangerous. And then what? Even if there is real evidence, it will have to be discarded unless it can't be sufficiently validated. It will get very hard to prove anything.

The smart encryption people will fix it. At some point it had to be fixed anyway since LLMs have only made it less effort.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#38

Post by AnthonyMouse »

recursive wrote: post #28

Here's my plan. 1. Cryptographically hash each piece of media when it's recorded. 2. Submit the hash to a "trusted" authority. 3. It will add a timestamp and sign the result. 4. Now, as long as you keep the original, without re-compressing, and you trust the authority, you have some evidence that the media existed at a timestamp. On or before. This doesn't prove authenticity, but in many cases, establishing a timesta…

Thieves are planning an inside job. They forge the surveillance video ahead of time, do the theft and submit the forgery to be timestamped while it's happening.

Also, a lot of surveillance systems are purposely kept offline to prevent them from being compromised, but your system doesn't allow that because they would need external connectivity to get signatures.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#39

Post by LPisGood »

thereddaikon wrote: post #22
mitthrowaway2 wrote: post #3

Earlier quoted context omitted.

I agree. The article didn't touch on this aspect, but we're now at the point where even authentic recordings could be plausibly denied and claimed to be fake. So the entire usage of recordings as evidence will suffer a hit. We may essentially be knocked back to an 18th century level of reliance on eyewitness testimony. One wonders what the consequences for justice will be.

There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.

This is unironically a usecase for blockchain.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#40

Post by iamacyborg »

AnthonyMouse wrote: post #33
thereddaikon wrote: post #22

Earlier quoted context omitted.

There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.

The usual chain of custody goes something like: The store has a video surveillance system which the police collect the footage from, so the chain of custody goes through the store and the police which implies that nobody other than those two have tampered with it. But then you have an inside job where the perpetrators work for the store and have doctored the footage before the police come to pick it up, or a corrupt…

Surely chain of custody applies if the accused has access to the evidence? Perhaps I’m missing your point or I’m overly optimistic about the legal system.
Post reply on HN