Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lawyer cites fake cases invented by ChatGPT, judge is not amused

simonwillison.net

Post reply on HN
Print view View original

1–10 of 319 posts

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#6

Post by TechBro8615 »

B1FF_PSUVM wrote: post #3

> ChatGPT, as it often does, hallucinated wildly Plausible bullshit generation for free, as if there's not enough already available cheap.

The lawyer was just trying to outsource his job of plausible bullshit generation. He would have gotten away with it, too, if it weren't for the fake cases.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#7

Post by mehwoot »

No, it did not “double-check”—that’s not something it can do! And stating that the cases “can be found on legal research databases” is a flat out lie.

What’s harder is explaining why ChatGPT would lie in this way. What possible reason could LLM companies have for shipping a model that does this?

It did this because it's copying how humans talk, not what humans do. Humans say "I double checked" when asked to verify something, that's all GPT knows or cares about.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#8

Post by metadat »

dpifke wrote: post #5
metadat wrote: post #2

Discussed today: A man sued Avianca Airline – his lawyer used ChatGPT https://news.ycombinator.com/item?id=36095352 (100 comments)

Also: https://news.ycombinator.com/item?id=36092509

Thanks! Macro-expanded:

ChatGPT-Authored Legal Filing “Replete with Citations to Non-Existent Cases"

https://news.ycombinator.com/item?id=36092509 (67 comments, 1 day ago)

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#9

Post by leereeves »

In the other discussion of this topic, a lot of people said the lawyer should be disbarred, but personally I think people should be able to trust the tools marketed by trillion dollar tech companies, and a lot of the blame should be placed on Microsoft/OpenAI for overhyping ChatGPT and understating how likely it is to mislead.

And every response from ChatGPT should be preceded by a warning that it cannot be trusted.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#10

Post by ubj »

> No, it did not “double-check”—that’s not something it can do! And stating that the cases “can be found on legal research databases” is a flat out lie. What’s harder is explaining why ChatGPT would lie in this way. What possible reason could LLM companies have for shipping a model that does this?

At what point does OpenAI (or any other company) become legally responsible for this kind of behavior from their LLM's? I'm not excusing the lawyer for their reckless and irresponsible use of a tool they didn't understand, but it's becoming increasingly clear that people are trusting LLM's far more than they should.

In my opinion it's dangerous to keep experimenting on the general population without holding the experimenters accountable for the harm that occurs.

Post reply on HN