Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lawyers blame ChatGPT for tricking them into citing bogus case law

apnews.com

Post reply on HN
Print view View original

91–100 of 180 posts

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#91

Post by 6gvONxR4sf7o »

I don't understand how they messed this up so bad. They say they didn't know it could hallucinate and that they thought it was just like any other search engine. But it seems like even if it worked like they thought, they'd still have fucked up?

If it's just like a normal person, if that person isn't a lawyer, you wouldn't ask them to do your lawyery work. I'd hope this lawyer doesn't his kids to do his work for him.

If it's just like a normal search engine, we all know how much bullshit, spam, and misinformation there is on the internet (mostly written by normal good old fashioned humans!). So that wouldn't have been trustworthy either!

There's no way this kind of thing is excusable.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#92

Post by jakelazaroff »

uoaei wrote: post #73
jakelazaroff wrote: post #69

Earlier quoted context omitted.

You say that as a joke but the lawyer literally asked ChatGPT whether the cases it provided were fake! There are screenshots! https://simonwillison.net/2023/May/27/lawyer-chatgpt/ Lawyer: Is varghese a real case ChatGPT: Yes, Varghese v. China Southern Airlines Co Ltd, 925 F.3d 1339 (11th Cir. 2019) is a real case. Lawyer: What is your source ChatGPT: I apologize for the confusion earlier. Upon double-checking, I fou…

The new LLMs have been advertised in layman circles often as something like "a chatbot with access to all the information on the web" or something similarly ambiguous. So it is certainly easy to imagine why people think it literally goes out and checks its sources by re-accessing the webpage and summarizing it for the user. The responses you quoted seem to simulate/imply that kind of behavior through its natural lang…

Yeah, I've spoken to people who've had trouble internalizing that it can't search the web even after I've explicitly told them that. The "upon double-checking" message from ChatGPT is especially egregious — it's straight-up lying about how it arrived at that response. There really should be a layer on top of the chat UI to alert the user if it detects a response like that.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#93

Post by elforce002 »

This is interesting. How long untill someone gets sick because he/she was following what chatGPT told him/her to do? Medical advice? Political misinformation?

How things will unfold this decade? Banning chatgpt from certain topics (medicine, law, etc...)? This decade will be really interesting indeed.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#94

Post by mcguire »

spondylosaurus wrote: post #65
TeMPOraL wrote: post #36

Earlier quoted context omitted.

> People are dumb. People are dumb. Lawyers shouldn't be. If distribution of stupidity among certain professions is similar to that of people in general, something has gone very bad somewhere in the education and certification pipeline.

What do you call someone who graduated at the bottom of their class in law school? A lawyer.

What do you call someone who didn't graduate from law school?

Not a lawyer.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#95

Post by jamesliudotcc »

jcranmer wrote: post #52

While the lawyers blamed ChatGPT, the totality of the circumstances seem to indicate that they're less than honest in doing so. There is a live-tweet of the hearing here: https://twitter.com/innercitypress/status/166683852676213965... , and you can follow along with the lawyerly cringe there. Okay, lawyer #1 (LoDuca, the one on the case in the first place) appears to have played essentially no role in the entire case…

Hilariously, the explanation of the notarization typo (actually, I find that believable) is a 1746 "declaration." It's an old federal law which provides that as long as you say you are signing under oath, it is as good as an affidavit in federal court.

Why didn't he just make a declaration in the first place? Also, why would he have the lawyer who is in trouble notarize the document? Now that he has made the typo, he may now need Schwartz's testimony that actually, it was April 25, not January.

In my legal career, I worked for a judge once. He told me to never get a notary stamp. It only creates problems. There's never a good reason for a lawyer to notarize something. You ask your staff to do it instead.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#96

Post by rank0 »

mysterydip wrote: post #68
voakbasda wrote: post #4

If they were able to be tricked by ChatGPT, they are definitely not good at being lawyers. Trying to blame the AI is like trying to blame MS Word for offering an inappropriate homonym when spell checking. The computer did not put the citations in front of the judge.

They should ask themselves why we should pay for lawyers at all if we can ask ChatGPT the same things and get answers of the same quality.

These dudes are getting in trouble specifically for the shit answers spat out of chatGPT.

Or yeah you could just yolo your freedom away and represent yourself using a mathematical expression with zero motivations, perception, or emotion.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#98

Post by jstarfish »

jakelazaroff wrote: post #69
IIAOPSW wrote: post #55

Earlier quoted context omitted.

"hey chatGPT, can you make up some bullshit for me?" "I'm sorry, as an online AI language model I am not able to generate output that is knowingly incorrect. Is there anything else you want help with today". see, it isn't capable of lying. It told me so itself.

You say that as a joke but the lawyer literally asked ChatGPT whether the cases it provided were fake! There are screenshots! https://simonwillison.net/2023/May/27/lawyer-chatgpt/ Lawyer: Is varghese a real case ChatGPT: Yes, Varghese v. China Southern Airlines Co Ltd, 925 F.3d 1339 (11th Cir. 2019) is a real case. Lawyer: What is your source ChatGPT: I apologize for the confusion earlier. Upon double-checking, I fou…

> Why the lawyer didn't go the extra step and check the actually real legal databases ChatGPT pointed out is beyond me.

Because that's work and takes effort. He gets paid the same to delegate the work to AI.

He did the absolute, bare minimum amount of verification needed to [hopefully] cover his ass. He just didn't expect the system to lie (sorry, "hallucinate") to him more than once.

> [...] the lawyers did not act quickly to correct the bogus legal citations when they were first alerted to the problem by Avianca’s lawyers and the court. Avianca pointed out the bogus case law in a March filing.

This is what fraud looks like. He's so checked out he even ignored the red flags being waved in his face. It stopped being a cute case of a student generating a common essay about Steinbeck when he started getting paid $200 an hour to cheat an injured client.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#100

Post by Ukv »

plorg wrote: post #37

A lawyer is certainly at fault if they do not fact check the material they present at trial. But the conmen who are selling ChatGPT and the like are extremely irresponsible for the way they sell LLMs as magical AI that arrives at factually correct answers by reasoning rather than the consequence of the law of large numbers applied to stochastic text generation.

> the conmen who are selling ChatGPT and the like are extremely irresponsible for the way they sell LLMs as magical AI that arrives at factually correct answers

ChatGPT has a pop-up on first use, a warning at the top of each chat, a warning below the chat bar, and a section in the FAQ explaining that it can generate nonsense and can't verify facts, provide references, or complete lookups.

There is probably more OpenAI could do, like detect attempts to generate false references and add a warning in red to that chat message - since it seems there are still people taking its hallucinations as fact (although if there's hundreds of millions of users, maybe only a tiny fraction), but I don't think this is a fair characterization.

Post reply on HN