Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

arstechnica.com

Post reply on HN
Print view View original

61–70 of 156 posts

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#61

Post by swores »

lolinder wrote: post #59
swores wrote: post #35

Earlier quoted context omitted.

Which version of GPT? I've found that 4o has actually been quite good at this lately, rarely hallucinating links any more. Just two days ago, I gave it a list of a dozen article titles from a newspaper website (The Guardian), asked it to look up their URLs and give me a list, and to summarise each article for me, and it made no mistakes at all. Maybe your task was more complicated to do in some way, maybe you're not…

> asked it to look up their URLs and give me a list Something missing from this conversation is whether we're talking about the raw model or model+tool calls (search). This sounds like tool calls were enabled. And I do think this is a sign that the current UX of the chatbots is deeply flawed: even on HN we don't seem to interact with the UI components to toggle these features frequently enough that they're the intuit…

Ah, yes you're right - I didn't clarify this in my original comment, but my anecdote was indeed the ChatGPT interface and using its ability to browse the web[#], not expecting it to pull URLs out of its original training data. Thanks for pointing that out.

But the reason I suggested model as a potential difference between me and the person I replied to, rather than ChatGPT interface vs. plain use of model without bells and whistles, is that they had said their trouble was while using ChatGPT, not while using a GPT model over the API or through a different service.

[#] (Technically I didn't, and never do, have the "search" button enabled in the chat interface, but it's able to search/browse the web without that focus being selected.)

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#64

Post by cratermoon »

michaelcampbell wrote: post #41
rsynnott wrote: post #32

Earlier quoted context omitted.

It's worse, because the magic robot's output is often _wrong_.

Well wrong more often. It's not like Google et al has a monopoly on truth.

The issue is not truth, though. It's the difference between completely fabricated but plausible text generated through a stochastic process versus a result pointing towards writing at least exists somewhere on the internet and can be referenced. Said source may be have completely unhinged and bonkers content (Time Cube, anyone?), but it at least exists prior to the query.

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#65

Post by mrob »

halgir wrote: post #22
seanhunter wrote: post #17

What I find really strange about this is I use AI a lot as a “smart friend” to work through explanations of things I find difficult etc and I am currently preparing for some exams so I will often give the AI a document and ask for some supporting resources to take the subject further and it almost always produces something that is plausibly close to a real thing but wrong in specifics. As in when you ask for a refere…

I use it in much the same way as you, and it's been extremely beneficial. But I also would not dream of signing my name on something that has been independently produced by AI, it's just too often blatantly wrong on specifics. I think people who do are simply not aware that AI is not deterministic the same way a calculator is. I would feel entirely safe signing my name on a mathematical result produced by a calculato…

LLMs are deterministic [0]. An LLM is a pure function that takes a list of tokens and returns a set of token probabilities. To make it "chat" you use the generated probabilities to pick a token, append that token to the list, and run the LLM again. Any randomness is introduced by the external component that picks a token using the probabilities: the sampler. Always picking the most likely token is a valid strategy.

The problem is that all output is a "hallucination", and only some of it coincidentally matches the truth. There's no internal distinction between hallucination and truth.

[0] Theoretically; race conditions in a parallel implementation could add non-determinism.

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#66

Post by cratermoon »

tiahura wrote: post #28

As an attorney, I’ve found that this isn’t the issue it was a year ago. 1. Use reasoning models and include in the prompt to check the cited cases and verify holdings. 2. Take the draft, run it through ChatGpt deep research , Gemini deep research and Claude , and tell it to verify holdings. I still double check, for now, but this is catching every hallucination.

> still double check, for now

Whew, that's 4 LLM inference requests and still requires manual checking. Criminal levels of waste and inefficiency. Learn how to use LexisNexis, spend some time in a law library handling actual physical casebooks. Learn to do your job.

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#67

Post by mediumsmart »

gyomu wrote: post #26
seanhunter wrote: post #17

What I find really strange about this is I use AI a lot as a “smart friend” to work through explanations of things I find difficult etc and I am currently preparing for some exams so I will often give the AI a document and ask for some supporting resources to take the subject further and it almost always produces something that is plausibly close to a real thing but wrong in specifics. As in when you ask for a refere…

People are lazy. I’m enrolled in a language class in a foreign country right now - so presumably people taking that class want to actually get good at the language so they can actually live their life here - yet a significant portion of students just turn in ChatGPT essays. And I don’t mean essays edited with chatGPT, but essays that are clearly verbatim output. When the teacher asks the students to read them out lou…

could it be that they just have to attend the class for technical reasons? Also - once the gadgets can translate for free in real time ... you can live in places you don't speak the language of, so maybe they are just prepping for that.

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#68

Post by Gracana »

distances wrote: post #24
rokkamokka wrote: post #12

Earlier quoted context omitted.

Shame? It's often constructive! Just treat it for what it is, imperfect information.

It's not constructive to copy-paste LLM slop to discussions. I've yet to see a context where that is welcome, and people should feel shame for doing that.

I see your frustration that these people exist who don’t share your values, but their comments already get downvoted. Take the win and move on.

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#69

Post by [deleted] »

seanhunter wrote: post #17

What I find really strange about this is I use AI a lot as a “smart friend” to work through explanations of things I find difficult etc and I am currently preparing for some exams so I will often give the AI a document and ask for some supporting resources to take the subject further and it almost always produces something that is plausibly close to a real thing but wrong in specifics. As in when you ask for a refere…

[deleted]

Re: Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes

#70

Post by add-sub-mul-div »

tiahura wrote: post #28

As an attorney, I’ve found that this isn’t the issue it was a year ago. 1. Use reasoning models and include in the prompt to check the cited cases and verify holdings. 2. Take the draft, run it through ChatGpt deep research , Gemini deep research and Claude , and tell it to verify holdings. I still double check, for now, but this is catching every hallucination.

You can't tell what is and isn't parody anymore.
Post reply on HN