Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI-Generated Voice Evidence Poses Dangers in Court

lawfaremedia.org

Post reply on HN
Print view View original

91–100 of 171 posts

Re: AI-Generated Voice Evidence Poses Dangers in Court

#91

Post by EGreg »

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

I kind of want to have an LLM which takes absolutely any criticism of AI or news of it doing something bad and then generates a plausible HN comment that basically goes "I don't think it's a new problem, X has always been possible, there's nothing really new"... because that comment always appears like clockwork beneath it :)

Re: AI-Generated Voice Evidence Poses Dangers in Court

#92

Post by dragonwriter »

EGreg wrote: post #91
fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

I kind of want to have an LLM which takes absolutely any criticism of AI or news of it doing something bad and then generates a plausible HN comment that basically goes "I don't think it's a new problem, X has always been possible, there's nothing really new"... because that comment always appears like clockwork beneath it :)

I mean, you are already getting the result for free, so why do you need an LLM for it?

In any case, you can probably do this with a fairly simple prompt with any LLM.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#93

Post by dragonwriter »

mrandish wrote: post #31
thereddaikon wrote: post #22

Earlier quoted context omitted.

There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.

Yep, "chain of custody." Came here hoping to see that concept discussed since it's how the system already deals with cases of potential evidence tampering. If the evidence is of material importance and there's no sufficiently credible chain of custody, then its validity can be questioned. The concept started around purely physical evidence but applies to image, audio and video. The good thing about the ubiquity of de…

Chain of custody only covers from when the evidence came into the hands of the police; the real issue here is original provenance, which chain of custody doesn't solve.

Evidence of provenance is already important, to be sure, but the the ability to have some degree of validation of the contents has itself provided some evidence of provenance; lose that and there is a real challenge.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#94

Post by SoftTalker »

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

Have you ever heard some of the wiretap or hidden microphone recordings used to convict mafia bosses back in the 1980s? It was so bad I can't believe it was accepted. It could easily have been faked. The only thing that made it work was the sworn statements of authenticity from the people who did the recording, and the chain of custody thereafter.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#95

Post by SoftTalker »

ARandumGuy wrote: post #45
hiatus wrote: post #42

Earlier quoted context omitted.

Isn't it up to the opposing party, and not the judge, to impeach an expert?

If it was solely up to the opposing party, then they'd strike down every single expert. The judge still is the ultimate decider on what evidence (including expert testimony) is admissible.

It's the jury that ultimately decides whether the evidence is convincing, even if the judge allows it. The defense will try to cast doubt on the evidence when they make their arguments.

This also reminds me of one of Norm Macdonald's bits. He says (very seriously) that if he were on a jury he would not convict someone on the basis of DNA evidence. "What do I know about DNA?" he says.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#96

Post by SoftTalker »

MichaelDickens wrote: post #75
gortok wrote: post #52

What shocks (and irritates!) me is that Charles Schwab keeps wanting me to set up voice ID. Why would I want to set up a voice ID for something that is now trivially spoofed?

Schwab used to have an 8 character maximum on passwords (although at least they changed that). They have never been a paragon of good security practices.

They've presumably been good enough that their losses are less costly than better security.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#97

Post by SoftTalker »

klipklop wrote: post #66
_DeadFred_ wrote: post #57

Earlier quoted context omitted.

You mean like expert witness polygraphers that were treated as fact by the courts for years and still used to re-incarcerate people on parole/probation? Or gun matching (ballistics) that are no longer considered conclusive but subjective? https://www.mdcourts.gov/data/opinions/coa/2023/10a22.pdf Hair and Fiber expert analysis that was wrong? https://innocenceproject.org/fbi-agents-gave-erroneous-testi... Or do you me…

What always worried me is that nobody ever challenged these methods. We just accepted it as fact. Endless crime tv shows reenforce that these methods are infallible. People that watch these shows then become jurors. Scary.

Cops looking at blurry 240p video frame. "Enhance!" and suddenly it's 4k.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#98

Post by cperciva »

pharrington wrote: post #41
treetalker wrote: post #29

Earlier quoted context omitted.

To the contrary, in US courts the jury determines whether evidence (documentary, testimonial) is credible or not, and what weight (if any) to assign it. (Experts, à la Daubert etc., are a different matter because they give expert opinions, not factual evidence based on personal witnessing of the events in the case, so the judge does perform a gatekeeping function, essentially to ensure the underlying field/science is…

IANAL, and I am assuming you are a lawyer - I interpreted @NoMoreNicksLeft as saying that it's a judge's responsibility to determine whether or not evidence is admissible - before it gets to a jury. And that's also what the article is talking about - "The examples should illustrate circumstances that may satisfy the authentication requirement while still leaving judges discretion to exclude an item of evidence if the…

Judges have a role in excluding evidence which is more prejudicial than probative. In the case of a fake voice recording, hearing someone who sounds like the accused participating in a crime may prejudice the jury even if they later hear evidence that the recording is fake. (This is probably even more true for faked videos.)

Re: AI-Generated Voice Evidence Poses Dangers in Court

#99

Post by DrFalkyn »

AnthonyMouse wrote: post #36
bluGill wrote: post #20

Earlier quoted context omitted.

Or perhaps camera manufactures will start putting traces in. Anyone who makes surveillance systems (like stores use) should put some end to end things in so they can say "this camera took that recording and we can see that it wasn't tampered with by...". (if anyone works for a surveillance company please run with this!) With encryption we can verify a lot of things, but it sets the bar higher than someone took a pict…

Cryptography doesn't really fix it. There are a zillion camera makers and all it takes is one of them to have poor security and leak the keys. Then the forger uses any of the cameras with extractable keys to sign their forgery. Or they just point a camera at a high resolution playback of a forged video. This also assumes you can trust all the camera makers, because by doing this you're implicitly giving them each aut…

Maybe go back to old school analog. Like they did in Altered Carbon

Re: AI-Generated Voice Evidence Poses Dangers in Court

#100

Post by john2x »

carra wrote: post #7

Wait some more time and photos or even video recordings will be deemed just as dangerous. And then what? Even if there is real evidence, it will have to be discarded unless it can't be sufficiently validated. It will get very hard to prove anything.

I wonder if film photography (slide film in particular) will become viable again and that photojournalists will once again become necessary for society.

Rather than cryptography which could be difficult to grok for a non-technical jury, a physical slide of film would be the source of truth.

It can still be faked by photographing a manufactured/generated scene though.

Post reply on HN