AI-Generated Voice Evidence Poses Dangers in Court • Hacker News Board

Re: AI-Generated Voice Evidence Poses Dangers in Court

#101

Post by qingcharles » Wed, Mar 12, 2025, 6:26 AM UTC

SoftTalker wrote: post #94

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

Have you ever heard some of the wiretap or hidden microphone recordings used to convict mafia bosses back in the 1980s? It was so bad I can't believe it was accepted. It could easily have been faked. The only thing that made it work was the sworn statements of authenticity from the people who did the recording, and the chain of custody thereafter.

I know a lawyer who was convicted on one of those. The detectives had the mic + transmitter taped to his groin so it would get through a pat down. Then the undercover just spoke both sides of the conversation (the defendant's side was whispered and muffled). Someone testified it sounded like the defendant. Conviction was overturned on appeal on a simpler issue unrelated to the evidence.

In fact, I remember a drug dealer I was helping with his defense. Hidden mic on undercover was taped under his armpit. He arrived to defendant's hotel room for a deal and defendant made him undress. The recording is hilarious because defendant is like "You fucking snitch, what's that under your armpit?" and the undercover says "It's my .. er .. MP3 player?" LOL

Re: AI-Generated Voice Evidence Poses Dangers in Court

#102

Post by coppsilgold » Wed, Mar 12, 2025, 6:45 AM UTC

recursive wrote: post #28

Here's my plan. 1. Cryptographically hash each piece of media when it's recorded. 2. Submit the hash to a "trusted" authority. 3. It will add a timestamp and sign the result. 4. Now, as long as you keep the original, without re-compressing, and you trust the authority, you have some evidence that the media existed at a timestamp. On or before. This doesn't prove authenticity, but in many cases, establishing a timesta…

You can do this already by yourself or use a service which you need not even trust.

Every x seconds, collect all the pieces of data that need to be notarized as existing in the world prior to some time t. It can be an arbitrary amount of data.

Construct a Merkle Tree[1] of all the hashes (or HMACs) of all the data to be notarized. Compute the Merkle Root. Make sure that everyone gets their Merkle Proof (path from leaf to root) or publish the Merkle Tree publicly.

Embed the Merkle Root into a one or more cryptocurrency blockchains to exploit their immutability guarantees. By either including it as "additional data" to some transaction or just straight up as a fictional cryptocurrency address.

Every piece of data processed in this way will have a Merkle Proof (cryptographic path from hash of the data to the Merkle Root) that proves it existed prior to the creation of the Merkle Root. The Merkle Root will have its creation time bounded by the proof of work conducted on the cryptocurrency blockchain.

[1] https://en.wikipedia.org/wiki/Merkle_tree>

Re: AI-Generated Voice Evidence Poses Dangers in Court

#103

Post by vasco » Wed, Mar 12, 2025, 6:54 AM UTC

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

> But ultimately: Humanity has coped without photo, audio or video evidence for most of its existence. I suppose it will cope again.

Same argument for electricity, the internet, sanitation, democracy... doesn't seem like a great test for stuff that we just didn't have it before and survived.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#104

Post by RataNova » Wed, Mar 12, 2025, 7:12 AM UTC

This is a pretty terrifying problem, especially considering how easily accessible AI voice cloning tech has become. The fact that courts still operate under rules written before AI-generated evidence was even a concept is concerning

Re: AI-Generated Voice Evidence Poses Dangers in Court

#105

Post by RataNova » Wed, Mar 12, 2025, 7:15 AM UTC

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

That's a great point. The ability to undermine real evidence by claiming it's AI-generated could be just as (if not more) damaging than fake evidence itself

Re: AI-Generated Voice Evidence Poses Dangers in Court

#106

Post by RataNova » Wed, Mar 12, 2025, 7:18 AM UTC

treetalker wrote: post #15

I question the wisdom of setting the judge up as a superjury / gatekeeper for this kind of situation. This seems like a reliability / weight of the evidence scenario, not a reliability / qualification of the witness scenario (as with an expert witness). Why would the judge be better qualified to determine whether the voice was authentic, as opposed to the witness? And why should the judge effectively determine the wi…

But I think the argument here is less about judges making definitive calls on authenticity and more about ensuring that clearly questionable evidence isn't automatically admitted just because a witness vouches for it.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#107

Post by trhway » Wed, Mar 12, 2025, 7:26 AM UTC

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

>But ultimately: Humanity has coped without photo, audio or video evidence for most of its existence.

which allowed to burn witches based on testimony of citizens in good standing.

And that leads us to using Neuralink and similar tech and the next gen lie detectors (like say the defendant's fMRI (most probably interpreted by AI too:)) to look into the brain and extract confession. No need for evidence, deposition and all that expensive time consuming stuff standing in the way of truth especially given that it can't be trusted anymore in the face of AI capabilities.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#108

Post by HPsquared » Wed, Mar 12, 2025, 8:07 AM UTC

fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

At this stage it's more a risk for people who have their likeness out in the public domain where it can be copied. But that's just about everyone these days.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#109

Post by HPsquared » Wed, Mar 12, 2025, 8:21 AM UTC

recursive wrote: post #28

Here's my plan. 1. Cryptographically hash each piece of media when it's recorded. 2. Submit the hash to a "trusted" authority. 3. It will add a timestamp and sign the result. 4. Now, as long as you keep the original, without re-compressing, and you trust the authority, you have some evidence that the media existed at a timestamp. On or before. This doesn't prove authenticity, but in many cases, establishing a timesta…

Isn't this basically achieved by standard cloud storage on phones? Photos and videos get uploaded automatically (with authentication for the user account) and presumably that action is logged somewhere the user can't edit. Just need to prove those logs are secure, and there you go.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#110

Post by krainboltgreene » Wed, Mar 12, 2025, 8:43 AM UTC

qwertox wrote: post #58

krainboltgreene wrote: post #35

Earlier quoted context omitted.

What an incredible line to end on. “It’s not normal to read/write”.

I'm not aware of any other species which is capable of reading and writing. But many can certainly hear and make sounds.

What a truly fascinating perspective. I wonder if you yearn to build complex tunnel systems underground at the behest of a queen? I’m of course describing the most common habit on the planet.