Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A.I. note takers are making lawyers nervous

nytimes.com

Post reply on HN
Print view View original

151–160 of 204 posts

Re: A.I. note takers are making lawyers nervous

#151

Post by chinabot »

skinfaxi wrote: post #72
gwbas1c wrote: post #47

Back when I was in college, in a fraternity, we always assumed that the phones were tapped. Specifically, we never spoke about alcohol or marijuana (now legal) on the phone. Even today, I generally assume that my phone could be tapped; even when talking with my trusted work colleagues, friends, and family. I'm extra careful about dirty jokes or "grey morality" in video conferences and email. The same applies to speak…

> Even today, I generally assume that my phone could be tapped; even when talking with my trusted work colleagues, friends, and family. I'm extra careful about dirty jokes or "grey morality" in video conferences and email. This is horrifying. Why do you feel the necessity to self-censor? What consequences do you anticipate?

Why bother, if the bad actors was that good it could just make up the conversation using your voice and AI.

Re: A.I. note takers are making lawyers nervous

#152

Post by laughing_man »

operation_moose wrote: post #12
coffeebeqn wrote: post #7

Earlier quoted context omitted.

Plus they are super inaccurate. Gemini gets one of its three bullet subtly or very majorly wrong almost every time. Just a few weeks ago Gemini said we’re rolling out our payment setup in Russia. You know the place where we have 20+ sanctions packages on? We were talking about France in the meeting.

We've found they're surprisingly good if everyone on the call is using a decent headset. The problems start when using conference room audio or someone is on their laptop mic. If they miss a word they never do unintelligible , they just start playing madlibs based on the rest of the sentence. We just went through a round of 100+ (non-sensitive) VoC interviews and they really cut down the workload of compiling all of…

>If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence.

Isn't that what people do?

Re: A.I. note takers are making lawyers nervous

#153

Post by SlinkyOnStairs »

Semaphor wrote: post #95
SlinkyOnStairs wrote: post #74

Earlier quoted context omitted.

> It's just a token predictor what do you expect? The point isn't that it's unexpected. It's that prior text-to-speech systems were much better about this particular failure mode, prone to spitting out entirely incorrect words but not rephrasing entire sentences. This is a particularly bad failure mode because people don't notice it. > What we need are tools that embrace that and ping the agent to validate what it ju…

I often (ish) translate between English and German, two languages I speak very well. The quality of translation is amazing and far better than what old systems did. Maybe it depends on topics or length, for me it's usually 1-2 paragraphs of a German article to share online.

> Maybe it depends on topics or length, for me it's usually 1-2 paragraphs of a German article to share online.

Same languages, same use case. My experience is different. On both google translate and others. ¯\_(ツ)_/¯

Re: A.I. note takers are making lawyers nervous

#154

Post by Lammy »

laughing_man wrote: post #152
operation_moose wrote: post #12

Earlier quoted context omitted.

We've found they're surprisingly good if everyone on the call is using a decent headset. The problems start when using conference room audio or someone is on their laptop mic. If they miss a word they never do unintelligible , they just start playing madlibs based on the rest of the sentence. We just went through a round of 100+ (non-sensitive) VoC interviews and they really cut down the workload of compiling all of…

>If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Isn't that what people do?

For in-person conversations to keep the conversation flowing, sure, but any good transcription will say [unintelligible] when the scribe couldn't tell despite being able to listen over it again and again.

Nixon tapes for example: https://kagi.com/search?q=site%3Anixonlibrary.gov+%22unintel...

Re: A.I. note takers are making lawyers nervous

#155

Post by buzer »

coffeebeqn wrote: post #7
burningion wrote: post #6

The main point raised in the article is that these bots may void attorney client privileges. But the real danger with these IMO is that they're turning casual conversations into a permanent record, and one that will be completely discoverable in court, should the company get into trouble later.

Plus they are super inaccurate. Gemini gets one of its three bullet subtly or very majorly wrong almost every time. Just a few weeks ago Gemini said we’re rolling out our payment setup in Russia. You know the place where we have 20+ sanctions packages on? We were talking about France in the meeting.

I wonder what kind of GDPR implications that has given the requirements around the accuracy of personal data.

Re: A.I. note takers are making lawyers nervous

#156

Post by ijustlovemath »

tomasGiden wrote: post #143
feoren wrote: post #76

Earlier quoted context omitted.

The final output of the neural network part of an LLM is a vector with weights for every token, that is then usually softmaxed and picked from. Can we not quantify the uncertainty by looking at the distribution of weights of the top 10 options? Like we expect for a note-taking app that the top choice would be something like 98% certain, and if we see that the model gives a weight of 60% to "Russia" and 30% to "France…

I’ve looked at confidence outputs for the chosen words from several STT providers and it’s definitely so that low confidence indicate that there is a risk that it has misheard. Not always though. Let’s say that someone is saying ”1 2 3 4 6 7 8” then it will happily write 5 in the middle and give it good confidence as based on the context, it is the only likely word. Varies between TTS providers though. Basically, why…

It seems like the problem in this application is that attention itself. Makes me wonder if using a transformer for transcription is the correct architecture.

Re: A.I. note takers are making lawyers nervous

#157

Post by fwbruno »

The main difference between a transcription error and a summarization error is that what was actually said may not get transcribed correctly, but you can always go back to the audio to check. Summarization errors are different because the narrative may sound coherent on the surface but doesn't necessarily represent what actually happened. A coherent summary that isn't accurate may be accepted as fact when in reality it is not. Only if the actual audio is checked would the discrepancies be found.

A lawyer may just accept it, believing the summary accurately represents the transcription. When AI summarizes a meeting, it does not catch the nuances of what actually happened. An offhand or dissenting comment may be critical but not caught in the summary. The AI compresses but can easily miss important details that matter. The consequences of only using the AI summary are potentially catastrophic. Context could be easily misunderstood, critical details may be left out, things that weren't actually said could be accepted as fact.

Re: A.I. note takers are making lawyers nervous

#158

Post by JacobKfromIRC »

1vuio0pswjnm7 wrote: post #148
1vuio0pswjnm7 wrote: post #101

Alternative to archive.is No Javascript, no CAPTCHA, no geoblocking, no DDoS directed at blog https://static.nytimes.com/narrated-articles/synthetic/artic...

Alternative to archive.is Works where archive.is is blocked Text-only, no DDoS directed at blog view-source:https://www.nytimes.com/2026/05/09/business/dealbook/ai-notetakers-legal-risk.html Save as 1.htm Something like egrep -o "(\"text\":\"[^\"]+)|(\"textAlign\":\"LEFT\")|(\"url\":\"[^\"]+)|(\"__typename\":\"TextInline\")" 1.htm \ |sed '/\"url\":\"/{s/??.*//;s/$/\">/;s/.\{7\}/ /;s/.\{24\}//;}; s/\"textAlign\":\"LEF…

Cool!

Would you be willing to license this code as GPL-3.0-or-later, or some other free license? I'd like to include a JavaScript derivative of this for Haketilo (a userscript manager). I would add it to a collection of scripts that aim to replace proprietary JavaScript here: https://codeberg.org/JacobK/unfinished-site-fixes/

Re: A.I. note takers are making lawyers nervous

#159

Post by grey-area »

r_lee wrote: post #29
user_7832 wrote: post #14

Earlier quoted context omitted.

> If they miss a word they never do unintelligible, they just start playing madlibs based on the rest of the sentence. Imo this is the single biggest flaw of LLMs. They're great at a lot of things, but knowing when they're wrong (or don't have enough information to actually work on) is a critical flaw. IMO there's nothing structural about why they shouldn't be able to spot this and correct themselves - I suspect it's…

I don't think it's a training issue, it's simply that there's no inherent "I don't know" in the transformer architecture unless it's really like something completely unknown, otherwise the nearest neighbor will be chosen and that will be whatever sounds similar or is relevant, even if it might cause a problem

Surely they could be built to pit placeholders for low confidence predictions and ignore those bits when predicting the rest?

The reason AI companies won’t do this of course is it would completely ruin the illusion of confident confidence these machines project.

Re: A.I. note takers are making lawyers nervous

#160

Post by feoren »

cyanydeez wrote: post #137
feoren wrote: post #76

Earlier quoted context omitted.

The final output of the neural network part of an LLM is a vector with weights for every token, that is then usually softmaxed and picked from. Can we not quantify the uncertainty by looking at the distribution of weights of the top 10 options? Like we expect for a note-taking app that the top choice would be something like 98% certain, and if we see that the model gives a weight of 60% to "Russia" and 30% to "France…

Unfortunately, that likely just doesn't exist. Everything suggests that these models are confident about their mistakes.

I mean, what I describe absolutely does exist, that's how LLMs work. The question is whether the relative weights are actually a good measure of confidence, and as the other reply to my comment points out, there are examples where it's not -- at least not the kind of "confidence" we really want.
Post reply on HN