Lawyer cites fake cases invented by ChatGPT, judge is not amused

Post by coldtea » Sat, May 27, 2023, 9:42 PM UTC

taberiand wrote: post #28

mehwoot wrote: post #7

No, it did not “double-check”—that’s not something it can do! And stating that the cases “can be found on legal research databases” is a flat out lie. What’s harder is explaining why ChatGPT would lie in this way. What possible reason could LLM companies have for shipping a model that does this? It did this because it's copying how humans talk, not what humans do. Humans say "I double checked" when asked to verify so…

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. It did that admirably. It's not its fault, or in my opinion OpenAI's fault, that the output is being misunderstood and misused by people who can't be bothered understanding it and project their own ideas of how it should function…

Post by akiselev » Sat, May 27, 2023, 9:43 PM UTC

jimsimmons wrote: post #53

akiselev wrote: post #39

I went ahead and asked ChatGPT with the browsing plugin [1] because I was curious and it answered that it was a real case citing an article about the fake citations! After some prodding ("Are you sure?") it spat out something slightly saner citing this very article! > The case "Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019)" was cited in court documents, but it appears that there might…

In the loop there indeed was a allegedly trained human in this instance

Post by acka » Sat, May 27, 2023, 9:43 PM UTC

leereeves wrote: post #41

londons_explore wrote: post #11

Earlier quoted context omitted.

I wonder if this is a tactic so the court to deems this lawyer incompetent rather than giving the (presumably much harsher) penalty for deliberately lying to the court?

Why assume malice? Asking ChatGPT to verify is exactly what someone who trusts ChatGPT might do. I'm not surprised this lawyer trusted ChatGPT too much. People trust their lives to self driving cars, trust their businesses to AI risk models, trust criminal prosecution to facial recognition. People outside the AI field seem to be either far too trusting or far too suspicious of AI.

Post by hoistbypetard » Sat, May 27, 2023, 9:43 PM UTC

zdragnar wrote: post #56

grumple wrote: post #40

Earlier quoted context omitted.

"It doesn't lie, it just generates lies and printed them to the screen!" I don't think there's a difference.

The difference is everything. It doesn't understand intent, it doesn't have a motivation. This is no different than what fiction authors, songwriters, poets and painters do. The fact that people assume what it produces must always be real because it is sometimes real is not its fault. That lies with the people who uncritically accept what they are told.

Post by [deleted] » Sat, May 27, 2023, 9:44 PM UTC

kordlessagain wrote: post #67

This is why it is very important to have the prompts fill in relevant fragments from a quality corpus. That people think these models “tell the truth” or “hallucinate” is only half the story. It’s like expecting your language center to know all the facts your visual consciousness contains, or your visual consciousness to be able to talk in full sentences. It’s only when all models are working well together the truth…