Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI-Generated Voice Evidence Poses Dangers in Court

lawfaremedia.org

Post reply on HN
Print view View original

161–170 of 171 posts

Re: AI-Generated Voice Evidence Poses Dangers in Court

#161

Post by rickandmortyy »

vasco wrote: post #103
fhd2 wrote: post #2

Sounds like reasonable changes. Generally speaking, I think evidence tampering is not a new problem, and even though it's easy in some cases, I don't think it's _that_ widespread. Just like it's possible to lie on the stand, but people usually think twice before they do it, because _if_ they are found to have lied, they're in trouble. My main concern is rather that legit evidence can now easily be called into questio…

> But ultimately: Humanity has coped without photo, audio or video evidence for most of its existence. I suppose it will cope again. Same argument for electricity, the internet, sanitation, democracy... doesn't seem like a great test for stuff that we just didn't have it before and survived.

[dead]

Re: AI-Generated Voice Evidence Poses Dangers in Court

#162

Post by gortok »

alamortsubite wrote: post #55
gortok wrote: post #52

What shocks (and irritates!) me is that Charles Schwab keeps wanting me to set up voice ID. Why would I want to set up a voice ID for something that is now trivially spoofed?

When was the last time you encountered this? I remember getting nags up until around the end of last year, but not lately. I like to think they dropped the program because I expressed concerns about it to so many reps, but more likely I've just been dialing in on a different number.

Today. I get an interstitial every time I log in that tries to get me to enable their VoiceID system (here's a screenshot): https://georgestocker.com/wp-content/uploads/2025/03/schwabv...

Re: AI-Generated Voice Evidence Poses Dangers in Court

#163

Post by aziaziazi »

stogot wrote: post #84
zusammen wrote: post #68

Earlier quoted context omitted.

There’s a built-in design paradox. How does a judge assess an expert in a field where he is not one? There’s probably some improvement that comes from experience but it’s not perfect.

Qualifications=academics usually & career experience. No way to know if they actually learned something and aren’t a fraud. Even corporations that do thousands of interviews get duped

Some thought

- perhaps the judge make less experts interviews than the corporations, leading to less experience in that but also takes each of them more seriously.

- one way to remove/add some credit to someone claim is to ask some of their peers opinion and see if there’s a strong majority.

- the judge personal expertise may help him forge a precise opinion but that wouldn’t clear him of making a mistake. For important matters it’s always a good idea to ask for peers reviews. Academics knows that too.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#164

Post by NoMoreNicksLeft »

pixl97 wrote: post #141
NoMoreNicksLeft wrote: post #49

Earlier quoted context omitted.

>While certainly Fox News headlines would not reach the jury in most instances, that is on account of hearsay, Nor should obviously fake evidence reach the jury. They can judge for themselves whether testimony is credible, but this is far different than admitting faked evidence. And if you can't see the difference, I'm not sure I'm qualified to explain it to you.

Exactly how is one supposed to determine what evidence is faked versus what evidence is not? Especially in the case where high technology has been used to mask that it is a fake?

>Exactly how is one supposed to determine what evidence is faked versus what evidence is not?

It's pretty simple. If the "evidence" can be created by software, it's not evidence. Perhaps some specially designed recording hardware might digitally sign a voice recording, and one might be inclined to trust that it was a real recording if the design of that hardware/software system was vetted.

But just for a recording? There's no point. Allowing a jury to decide that this recording is bad, but this other one is okay when they have no expertise to be able to determine if it is fake or not and none of the technical details either is just asking for prejudicial and even superstitious deliberation.

We are at the point where audio (and probably video) recordings no longer count as evidence of real world events. It's been this way for photographs for a long time already.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#165

Post by recursive »

coppsilgold wrote: post #102
recursive wrote: post #28

Here's my plan. 1. Cryptographically hash each piece of media when it's recorded. 2. Submit the hash to a "trusted" authority. 3. It will add a timestamp and sign the result. 4. Now, as long as you keep the original, without re-compressing, and you trust the authority, you have some evidence that the media existed at a timestamp. On or before. This doesn't prove authenticity, but in many cases, establishing a timesta…

You can do this already by yourself or use a service which you need not even trust. Every x seconds, collect all the pieces of data that need to be notarized as existing in the world prior to some time t. It can be an arbitrary amount of data. Construct a Merkle Tree[1] of all the hashes (or HMACs) of all the data to be notarized. Compute the Merkle Root. Make sure that everyone gets their Merkle Proof (path from lea…

There are some tradeoffs here. You need to do currency transfers get a timestamp, with all the constraints that comes with. But you get probably a more trustworthy authority.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#166

Post by CMCDragonkai »

LPisGood wrote: post #39
thereddaikon wrote: post #22

Earlier quoted context omitted.

There's already a process for this, its called chain of custody. If you cant prove the evidence has a solid chain of custody then it was potentially tampered with and isn't reliable.

This is unironically a usecase for blockchain.

More than just a blockchain, you need a decentralized set of oracles to mutually corroborate information. https://polykey.com/blog/ai-detection-versus-cryptographic-p...

Re: AI-Generated Voice Evidence Poses Dangers in Court

#167

Post by consteval »

EGreg wrote: post #149
consteval wrote: post #134

Earlier quoted context omitted.

Everyone misses this all the time. Most problems can be overlooked because they’re not that prolific. There’s a chain of events that really hinges on that assumption. For example, piracy is really not a problem. In fact I’m pro-piracy most of the time. But if everyone could pirate with no thought and no down time, I don’t think our economy will survive. Just because something is okay in small doses doesn’t mean it sc…

Do you think the people who keep commenting "X was always a problem" are really unaware of this? Or they are just trying to dampen any alarm on purpose?

I think they’re genuinely unaware of this because many, maybe most, people who analyze risk casually don’t think about ease or probability. Because it’s easier that way, and typically allows them to maintain some beliefs about personal freedoms.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#168

Post by alamortsubite »

gortok wrote: post #162
alamortsubite wrote: post #55

Earlier quoted context omitted.

When was the last time you encountered this? I remember getting nags up until around the end of last year, but not lately. I like to think they dropped the program because I expressed concerns about it to so many reps, but more likely I've just been dialing in on a different number.

Today. I get an interstitial every time I log in that tries to get me to enable their VoiceID system (here's a screenshot): https://georgestocker.com/wp-content/uploads/2025/03/schwabv...

That's crazy. Well, I guess I'll keep at them.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#169

Post by AnthonyMouse »

Edman274 wrote: post #126
AnthonyMouse wrote: post #38

Earlier quoted context omitted.

Thieves are planning an inside job. They forge the surveillance video ahead of time, do the theft and submit the forgery to be timestamped while it's happening. Also, a lot of surveillance systems are purposely kept offline to prevent them from being compromised, but your system doesn't allow that because they would need external connectivity to get signatures.

This technique is known as an Ella Fitzgerald to those in the heisting industry, as popularized in the documentary Oceans 11.

Oceans 11 is a work of fiction because most people don't have that skillset and the ones who do can make a lot of money doing honest work without risking a long prison sentence, so they're not tempted to rob casinos to heisting music.

The issue here is that the skillset required to produce the forgery has fallen to the level that anybody can do it and then that's enough people that somebody actually does.

Re: AI-Generated Voice Evidence Poses Dangers in Court

#170

Post by RataNova »

pixl97 wrote: post #135
RataNova wrote: post #105

Earlier quoted context omitted.

That's a great point. The ability to undermine real evidence by claiming it's AI-generated could be just as (if not more) damaging than fake evidence itself

Well, and to generate fake evidence to lead cops off the trail for the longest period of time possible. Evidence could be said to contain a half life. The longer (real) evidence is not gathered the more it can decay in one way or another. For example security footage gets overwritten. Actual witness memory gets foggy. Physical items get thrown away. If the cops spend the first month going after some poor Mark that ha…

And given how quickly narratives form online, even if the truth eventually comes out, public opinion might not catch up
Post reply on HN