Lawyer cites fake cases invented by ChatGPT, judge is not amused • Hacker News Board

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#271

Post by rvz » Sun, May 28, 2023, 2:05 AM UTC

Very unsurprising.

Relying on AI sophists like ChatGPT for legal work is still just as risky for normal users and even for legal experts. The difference is, these legal experts are more qualified to carefully review and check over the outputs than the average joe / jane trying to 'replace their lawyer, solicitor, etc' with ChatGPT.

I keep emphasising this importance, and to never fully trust the output of LLMs such as ChatGPT, unless a human has reviewed and checked if it is hallucinating or bullshitting. [0]

[0] https://news.ycombinator.com/item?id=36091468

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#272

Post by [deleted] » Sun, May 28, 2023, 2:08 AM UTC

clnq wrote: post #263

RhysU wrote: post #207

Earlier quoted context omitted.

> Large Language Models (LLMs) are never wrong, and they do not make mistakes. I call B.S. If LLMs never made mistakes we wouldn't train them. Any random initialization would work.

I've noticed that there's a lot of shallow fulmination on HN recently. People say things like "I call bullshit", or "I don't believe this for a second", or even call others demeaning things. My brother (and I say this with empathy), no one is here to hear your vehement judgement. If you have anything of substance to contribute, there are a million different ways to express it with kindness and constructively. As for…

[deleted]

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#273

Post by FactKnower69 » Sun, May 28, 2023, 2:13 AM UTC

noahlt wrote: post #137

zouhair wrote: post #124

Earlier quoted context omitted.

Punishment is not technically the point of prisons. Your unhinged comment will pass as normal to so many people who think punishment is the point and that's scary as hell.

Punishment is absolutely one purpose of prisons. Your unhinged comment will pass as normal to many people who don’t believe in retribution, and that’s why you all belong in Hell.

Insane comment to put your real name on, Noah! You realize this will show up when people Google you?

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#274

Post by shagie » Sun, May 28, 2023, 2:18 AM UTC

RhysU wrote: post #270

clnq wrote: post #265

Earlier quoted context omitted.

I agree for the most part, but I wish to underscore the primary function inherent in each tool. For a LLM, it is to generate plausible language. For a bolt, it is to offer structural integrity. For a car, it is to provide mobility. Should these tools fail to do what they were designed for, we can rightfully deem them as defective. GPT was not primarily made to produce factual statements. While factual accuracy certai…

> For a LLM, it is to generate plausible language. LLMs are not being sold as delivering only plausible language. Is it the craftsman's fault when the salesman lies?

Who (and with links showing it) is selling it as delivering more than that?

From https://simonwillison.net/2023/May/27/lawyer-chatgpt/ (which has the dates)

> Mar 1st, 2023 is where things get interesting. This document was filed—“Affirmation in Opposition to Motion”—and it cites entirely fictional cases! One example quoted from that document (emphasis mine):

GPT-4 wasn't available until March 14th ( https://openai.com/research/gpt-4 ), so we're dealing with 3.5 here.

Where was 3.5 advertised to do more than generate plausible language in a conversation and who is making those claims?

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#275

Post by RobRivera » Sun, May 28, 2023, 2:20 AM UTC

that lawyer should be disbarred.

Is it really so hard to cross reference a chatgpt response with a source?

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#276

Post by shagie » Sun, May 28, 2023, 2:21 AM UTC

mulmen wrote: post #73

la64710 wrote: post #60

Earlier quoted context omitted.

ChatGPT did exactly what it is supposed to do. The lawyers who cited them are fools in my opinion. Of course OpenAI is also an irresponsible company to enable such a powerful technology without adequate warnings. With each chatGPT response they should provide citations (like Google does) and provide a clearly visible disclaimer that what it just spewed may be utter BS. I only hope the judge passes an anecdotal order…

The remedy here seems to be expecting lawyers to do their jobs. Citations would be nice but I don’t see a reason to legislate that requirement, especially from the bench. Let the market sort this one out. Discipline the lawyers using existing mechanisms.

From the NYT article on it: https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-...

> Judge Castel said in an order that he had been presented with “an unprecedented circumstance,” a legal submission replete with “bogus judicial decisions, with bogus quotes and bogus internal citations.” He ordered a hearing for June 8 to discuss potential sanctions.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#277

Post by clnq » Sun, May 28, 2023, 2:29 AM UTC

RhysU wrote: post #270

clnq wrote: post #265

Earlier quoted context omitted.

I agree for the most part, but I wish to underscore the primary function inherent in each tool. For a LLM, it is to generate plausible language. For a bolt, it is to offer structural integrity. For a car, it is to provide mobility. Should these tools fail to do what they were designed for, we can rightfully deem them as defective. GPT was not primarily made to produce factual statements. While factual accuracy certai…

> For a LLM, it is to generate plausible language. LLMs are not being sold as delivering only plausible language. Is it the craftsman's fault when the salesman lies?

The very first limitation listed on the ChatGPT introduction post is about incorrect answers - https://openai.com/blog/chatgpt. This has not changed since ChatGPT was announced. OpenAI is advertising that it will generate more than plausible language.

I think you are barking up the wrong tree here. As much as I understand your scepticism, OpenAI have been very transparent about the limitations of GPT and it is not truthful to say otherwise.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#278

Post by clnq » Sun, May 28, 2023, 2:38 AM UTC

internetguy wrote: post #86

I don't get it... These are PROFESSIONAL lawyers that have gone to years of law school. How do they not know how to double check an answer from ChatGPT? I'm sure there's databases of cases online, it shouldn't be this difficult.

We are all really stupid in a surprising number of ways. That's not all we are, but we are that, too.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#279

Post by RhysU » Sun, May 28, 2023, 2:42 AM UTC

clnq wrote: post #277

RhysU wrote: post #270

Earlier quoted context omitted.

> For a LLM, it is to generate plausible language. LLMs are not being sold as delivering only plausible language. Is it the craftsman's fault when the salesman lies?

The very first limitation listed on the ChatGPT introduction post is about incorrect answers - https://openai.com/blog/chatgpt . This has not changed since ChatGPT was announced. OpenAI is advertising that it will generate more than plausible language. I think you are barking up the wrong tree here. As much as I understand your scepticism, OpenAI have been very transparent about the limitations of GPT and it is not t…

A definition of "plausible" is "apparently reasonable and credible, and therefore convincing".

In what limit does "apparently reasonable and credible" diverge from "true"?

We'd make the LLM not lie if we could. All this "plausible" language constitutes practitioner weasel words. We'd collectively love if the LLMs were more truthful than a 5-year-old.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#280

Post by theamk » Sun, May 28, 2023, 2:50 AM UTC

clnq wrote: post #142

taberiand wrote: post #28

Earlier quoted context omitted.

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. It did that admirably. It's not its fault, or in my opinion OpenAI's fault, that the output is being misunderstood and misused by people who can't be bothered understanding it and project their own ideas of how it should function…

This harks back to around 1999 when people would often blame computers for mistakes in their math, documents, reports, sworn filings, and so on. Then, a thousand different permutations of "computers don't make mistakes" or "computers are never wrong" became popular sayings. Large Language Models (LLMs) are never wrong, and they do not make mistakes. They are not fact machines. Their purpose is to abstract knowledge a…

I just went to ChatGPT page, and was presented with the text:

"ChatGPT: get instant answers, find creative inspiration, and learn something new. Use ChatGPT for free today."

If something claims to give you answers, and those answers are incorrect, that something is wrong. Does not matter what it is -- model, human, dictionary, book.

Claiming that their purpose is "to produce plausible language" is just wrong.. no one (except maybe AI researchers) say: "I need some plausible language, I am going to open ChatGPT".