Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lawyer cites fake cases invented by ChatGPT, judge is not amused

simonwillison.net

Post reply on HN
Print view View original

121–130 of 319 posts

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#121

Post by emodendroket »

bobsmooth wrote: post #114
agnosticmantis wrote: post #107

Earlier quoted context omitted.

Whether a statement is true or false doesn’t depend on the mechanism generating the statement. We should hold these models (or more realistically, their creators) to the same standard as humans. What do we do with a human that generates plausible-sounding sentences without regard for their truth? Let’s hold the creators of these models accountable, and everything will be better.

>Let’s hold the creators of these models accountable, and everything will be better. Shall we hold Adobe responsible for people photoshopping their ex's face into porn as well?

Maybe we should if they were trying to make something completely different and that was the output

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#122

Post by _Microft »

metadat wrote: post #51
_Microft wrote: post #32

Earlier quoted context omitted.

dang is doing that frequently - but what does "macro-expanded" mean for normal users? Do you have a bookmarklet, extension or something similar for this or is there any secret incantation that makes HN generate such an entry automatically?

I just do it manually.. I've gotten pretty efficient at it, takes only a moment. Imo including title, timeline / age, and comment volume provides helpful context to readers (I always appreciate when others do this, rather than, in the most severe cases, leaving a wall of unadorned HN links). Cheers _Microft (and cool username, btw ;D)

Thanks for shedding light on this and thanks for the compliment! Have a nice day

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#123

Post by Karellen »

coldtea wrote: post #81
taberiand wrote: post #28

Earlier quoted context omitted.

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. It did that admirably. It's not its fault, or in my opinion OpenAI's fault, that the output is being misunderstood and misused by people who can't be bothered understanding it and project their own ideas of how it should function…

> ChatGPT did not lie; it cannot lie. If it lies like a duck, it is a lying duck.

Lying implies intent, and knowing what the truth is. Saying something you believe to be true, but is wrong, is generally not considered a lie but a mistake.

A better description of what ChatGPT does is described well by one definition of bullshit:

> bullshit is speech intended to persuade without regard for truth. The liar cares about the truth and attempts to hide it; the bullshitter doesn't care if what they say is true or false

-- Harry Frankfurt, On Bullshit, 2005

https://en.wikipedia.org/wiki/On_Bullshit

ChatGPT neither knows nor cares what the truth is. If it bullshits like a duck, it is a bullshitting duck.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#124

Post by zouhair »

londons_explore wrote: post #15

If I were the judge in this case, I would be looking to throw this lawyer in prison for a month, and ban him from ever being a lawyer again... Deliberately lying to the court, as a professional who should understand the consequences, in a way likely to not be detected, and likely to change the outcome of the case, ought to be met with a really strict punishment.

Punishment is not technically the point of prisons. Your unhinged comment will pass as normal to so many people who think punishment is the point and that's scary as hell.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#126

Post by bobsmooth »

hristov wrote: post #113

It is very concerning how people are anthropomorphizing chat gpt. It will get a lot of people into trouble. The media is largely to blame for this. Never mind most gushing media stories about AI, even the few stories criticizing AI treat it as a human. Chat GPT is racist, they say, it is sexist or a liar. Well it is neither of these things, because all of the above require consciousness and intent and it has none. It…

It's really disconcerting how many people don't understand that these LLMs are programs not people.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#127

Post by ytreacj »

jiggawatts wrote: post #97
mehwoot wrote: post #7

No, it did not “double-check”—that’s not something it can do! And stating that the cases “can be found on legal research databases” is a flat out lie. What’s harder is explaining why ChatGPT would lie in this way. What possible reason could LLM companies have for shipping a model that does this? It did this because it's copying how humans talk, not what humans do. Humans say "I double checked" when asked to verify so…

There are systems built on top of LLMs that can reach out to a vector database or do a keyword search as a plug in. There’s already companies selling these things, backed by databases of real cases. These work as advertised. If you go to ChatGPT and just ask it, you’ll get the equivalent of asking Reddit: a decent chance of someone writing you some fan-fiction, or providing plausible bullshit for the lulz. The real s…

[dead]

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#128

Post by coldtea »

Karellen wrote: post #123
coldtea wrote: post #81

Earlier quoted context omitted.

> ChatGPT did not lie; it cannot lie. If it lies like a duck, it is a lying duck.

Lying implies intent, and knowing what the truth is. Saying something you believe to be true, but is wrong, is generally not considered a lie but a mistake. A better description of what ChatGPT does is described well by one definition of bullshit: > bullshit is speech intended to persuade without regard for truth. The liar cares about the truth and attempts to hide it; the bullshitter doesn't care if what they say is…

>Lying implies intent, and knowing what the truth is. Saying something you believe to be true, but is wrong, is generally not considered a lie but a mistake.

Those are the semantics of lying.

But "X like a duck" is about ignoring semantics, and focusing not on intent or any other subtletly, but only on the outward results (whether something has the external trappings of a duck).

So, if it produces things that look like lies, then it is lying.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#129

Post by manquer »

Everyone is talking about ChatGPT , but is it not possible to train a model with only actual court documents and keep “temp” low and get accuracy levels as high or better than humans?

Most legal (all formal really) documents are very predictably structured and should be easy to generate

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#130

Post by armchairhacker »

akiselev wrote: post #125
armchairhacker wrote: post #106

[flagged]

Did you just have ChatGPT make up all these cases for you? My ChatGPT can't find any of them.

What? I told mine to double-check and it was absolutely certain they are real court cases
Post reply on HN