Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI-Generated Voice Evidence Poses Dangers in Court

lawfaremedia.org

Post reply on HN
Print view View original

61–70 of 171 posts

Re: AI-Generated Voice Evidence Poses Dangers in Court

#61

Post by lolc »

AnthonyMouse wrote: post #38
recursive wrote: post #28

Here's my plan. 1. Cryptographically hash each piece of media when it's recorded. 2. Submit the hash to a "trusted" authority. 3. It will add a timestamp and sign the result. 4. Now, as long as you keep the original, without re-compressing, and you trust the authority, you have some evidence that the media existed at a timestamp. On or before. This doesn't prove authenticity, but in many cases, establishing a timesta…

Thieves are planning an inside job. They forge the surveillance video ahead of time, do the theft and submit the forgery to be timestamped while it's happening. Also, a lot of surveillance systems are purposely kept offline to prevent them from being compromised, but your system doesn't allow that because they would need external connectivity to get signatures.

Sure when you're controlling the source, you can fake it. But requiring the fakes be prepared ahead of time locks the faker into one story that may be contradicted by other evidence.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#62

Post by nostrademons »

bilbo0s wrote: post #54
nostrademons wrote: post #34

Earlier quoted context omitted.

It could easily go the other way, where the public doesn't care what people think they did or didn't do and just does whatever they want, because they don't respect the state and believe the social contract has been broken. "Fuck justice, talk to my AR-15." There's ample evidence that this is already happening, eg. recent headlines about kids being radicalized at increasingly younger ages, groups like No Lives Matter…

I don't know man? We're already at a place where most people don't care what other people think of them. Issue is, as long as the government has the big guns, what the government thinks of you will still matter in a major way. In such an environment, most people are going to choose to have some kind of way to prove to the government what they did and what they didn't do. Not because they care what other people think…

Government is based on the threat of the use of force, not the actual use of force. If you're a government that is regularly using force against a significant proportion of your citizens, you have problems, and probably will not remain the government for long.

I suspect that we're saying the same thing but with reversed causality. Both of us agree that non-deterministic enforcement breaks down the incentives needed for pro-social behavior. You're saying that this will cause people to demand ways to improve the governments enforcement abilities. I'm saying that this will cause people to adapt their behavior to the new, lessened enforcement abilities. In defense of my point, I'd point out that changes to government are a coordination problem while adaptation of behavior is an individual-only response, and it is much easier to effect changes to your own behavior than it is to convince 300 million people to agree on a solution and implement it, particularly when the root problem is a lack of enforcement ability.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#64

Post by AnthonyMouse »

iamacyborg wrote: post #40
AnthonyMouse wrote: post #33

Earlier quoted context omitted.

The usual chain of custody goes something like: The store has a video surveillance system which the police collect the footage from, so the chain of custody goes through the store and the police which implies that nobody other than those two have tampered with it. But then you have an inside job where the perpetrators work for the store and have doctored the footage before the police come to pick it up, or a corrupt…

Surely chain of custody applies if the accused has access to the evidence? Perhaps I’m missing your point or I’m overly optimistic about the legal system.

Suppose the store manager is having a dispute with a kid who keeps skateboarding in the parking lot, so the store manager decides to commit insurance fraud by robbing the store herself and then submits forged video of the kid doing it to the police.

The store manager is in the chain of custody but isn't a suspect, the accused is the kid. The kid doesn't even know who actually committed the crime. How is the kid supposed to prove this?

Re: AI-Generated Voice Evidence Poses Dangers in Court

#65

Post by AnthonyMouse »

lolc wrote: post #61
AnthonyMouse wrote: post #38

Earlier quoted context omitted.

Thieves are planning an inside job. They forge the surveillance video ahead of time, do the theft and submit the forgery to be timestamped while it's happening. Also, a lot of surveillance systems are purposely kept offline to prevent them from being compromised, but your system doesn't allow that because they would need external connectivity to get signatures.

Sure when you're controlling the source, you can fake it. But requiring the fakes be prepared ahead of time locks the faker into one story that may be contradicted by other evidence.

That's pretty much what happens anyway. The police are going to come collect the footage as soon as the crime is reported and you can't change it after they do.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#66

Post by klipklop »

_DeadFred_ wrote: post #57
treetalker wrote: post #15

I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…

You mean like expert witness polygraphers that were treated as fact by the courts for years and still used to re-incarcerate people on parole/probation? Or gun matching (ballistics) that are no longer considered conclusive but subjective? https://www.mdcourts.gov/data/opinions/coa/2023/10a22.pdf Hair and Fiber expert analysis that was wrong? https://innocenceproject.org/fbi-agents-gave-erroneous-testi... Or do you me…

What always worried me is that nobody ever challenged these methods. We just accepted it as fact. Endless crime tv shows reenforce that these methods are infallible. People that watch these shows then become jurors. Scary.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#67

Post by a2128 »

tgv wrote: post #23

I'll say it again, even though it is rather unpopular here: there has never been a need to develop these tools, nor one to make them easy to deploy, nor one to make them easy to use. Yet all this has happened, and now it may occur that someone is acquitted because AI generated media is so good, the evidence might be artificial. If that happens, and the suspect commits another crime, it's on the conscience of the peop…

There is some use to bringing deepfakes to mass adoption. The thing is that since the tech exists, powerful actors with lots of resources will develop these tools for their own use either way. The question is whether they'll be able to fool the masses who are unaware that such realistic deepfakes can exist, or whether they will have no effect as everyone and their mom have already seen similar AI slop on their Facebook feed

Re: AI-Generated Voice Evidence Poses Dangers in Court

#68

Post by zusammen »

bluGill wrote: post #17
treetalker wrote: post #15

I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…

I don't know what the latest is, but often judges are supposed to not allow "expert testimony" without checking that the person is an expert. However this is a really complex area. Judges don't want to be the one deciding a case, but not allowing some "expert" is in a way deciding the case.

There’s a built-in design paradox. How does a judge assess an expert in a field where he is not one? There’s probably some improvement that comes from experience but it’s not perfect.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#69

Post by bilbo0s »

nostrademons wrote: post #62
bilbo0s wrote: post #54

Earlier quoted context omitted.

I don't know man? We're already at a place where most people don't care what other people think of them. Issue is, as long as the government has the big guns, what the government thinks of you will still matter in a major way. In such an environment, most people are going to choose to have some kind of way to prove to the government what they did and what they didn't do. Not because they care what other people think…

Government is based on the threat of the use of force, not the actual use of force. If you're a government that is regularly using force against a significant proportion of your citizens, you have problems, and probably will not remain the government for long. I suspect that we're saying the same thing but with reversed causality. Both of us agree that non-deterministic enforcement breaks down the incentives needed f…

Keep in mind that being able to demonstrate your innocence to the government is also just an individual change in behavior. A person might not necessarily care if you use the tracking technologies or submit yourself to all the cameras in society. But they'll choose to do it themselves just so that there's that record out there.

The government coming up with one way to track everyone is not really necessary to get people to submit to tracking. In fact, most of the law enforcement "watcher" types wouldn't want that anyway. Not only is having a myriad number of ways to track and surveil people is far preferable to law enforcement, but it also allows people to individually choose to set up all the Ring doorbells and security cameras and GPS trackers and smart glasses based body cams etc etc all on their own.

And they'll happily choose to buy all that stuff voluntarily and without regard to what everyone else is doing.

Post reply on HN