Lawyer cites fake cases invented by ChatGPT, judge is not amused • Hacker News Board

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#201

Post by shagie » Sat, May 27, 2023, 10:56 PM UTC

flangola7 wrote: post #119

taberiand wrote: post #28

Earlier quoted context omitted.

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. It did that admirably. It's not its fault, or in my opinion OpenAI's fault, that the output is being misunderstood and misused by people who can't be bothered understanding it and project their own ideas of how it should function…

ChatGPT isn't a legal entity but OpenAI is, and Altman has already recommend to Congress that coming regulations should make AI companies liable for produced text and be 230 exempt. I can see it already happening even without legislation, 230 shields liability from user-generated content but ChatGPT output isn't user generated. It's not even a recommendation algorithm steering you into other users' content telling wh…

Why should OpenAI be more liable for a tool that they've created than any other tool creator where the tool is intentionally misused and warnings on the tool ignored?

If I wrote text in Microsoft Word and in doing so, I had a typo in (for example) the name of a drug that Word corrected to something that was incorrect, is Microsoft liable for the use of autocorrect?

If I was copying and pasting data into excel and some of it was interpreted as a date rather than some other data format resulting in an incorrect calculation that I didn't check at the end, is Microsoft again liable for that?

At the bottom of the ChatGPT page, there's the text:

    ChatGPT may produce inaccurate information about people, places, or facts.

If I can make an instance of Eliza say obscene or incorrect things, does that make the estate of Weizenbaum liable?

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#202

Post by lowbloodsugar » Sat, May 27, 2023, 10:56 PM UTC

macrael wrote: post #170

Genuine question: why have these models all been trained to sound so confident? Is it not possible to have rewarded models that announced their own ignorance? Or is even that question belying an "intelligence" view of these models that isn't accurate?

I think you are confusing ChatGPT with AI. ChatGPT is a statistical fiction generator. It sounds confident because it is writing fiction. It sounds confident for precisely the same reason that billions of ignorant people world wide post "facts" on line sound confident: they are incapable of understanding their ignorance. They are just systems that have inputs and then generate outputs.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#203

Post by whitemary » Sat, May 27, 2023, 10:58 PM UTC

nemo44x wrote: post #196

whitemary wrote: post #175

Earlier quoted context omitted.

Liars (humans) are often unconcerned with facts or truth either. More often these days, they are only concerned with profit. Are unconfirmed lies in the pursuit of mere profit somehow not lies?

This book [1] covers it. [1] https://en.m.wikipedia.org/wiki/On_Bullshit#:~:text=The%20li... .

No, it does not. I have read the full text and it was garbage to my mind; another nonsense apology for liberal society's degeneracy.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#204

Post by tankenmate » Sat, May 27, 2023, 11:02 PM UTC

massysett wrote: post #189

Karellen wrote: post #123

Earlier quoted context omitted.

Lying implies intent, and knowing what the truth is. Saying something you believe to be true, but is wrong, is generally not considered a lie but a mistake. A better description of what ChatGPT does is described well by one definition of bullshit: > bullshit is speech intended to persuade without regard for truth. The liar cares about the truth and attempts to hide it; the bullshitter doesn't care if what they say is…

I’ve come to the belief that making statements that may or may not be true, but with reckless disregard for whether or not they actually are true, is indeed lying. Of course we know ChatGPT cannot lie like a human can, but a big reason the thing exists is to assemble text the same way humans do. So I think it’s useful rhetorically to say that ChatGPT, quite simply, lies.

But since this is about the law; ChatGPT can't lie because there is no mens rea. And of course this is a common failing with the common person when it comes to the law, a reckless disregard there of (until it is too late of course). And recklessness is also about intent, it is a form of wantonness, i.e. selfishness. This is why an insane person cannot be found guilty, you can't be reckless if you are incapable of discerning your impact on others.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#205

Post by ineedasername » Sat, May 27, 2023, 11:02 PM UTC

lolinder wrote: post #152

> I apologize for the confusion earlier. Upon double-checking, I found that the case Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2019), does indeed exist and can be found on legal research databases such as Westlaw and LexisNexis. This is the part that stood out to me the most. I've seen this "I apologize for the confusion earlier" language many times when using ChatGPT, and it's always whe…

>”I apologize… does indeed exist”

This showed me that people don’t yet understand how to practice good “hygiene” when using these tools.

This apparent doubling down is (usually) the product of asking it to verify something it previously output in the same chat session

It tends toward logical consistency unless directly told something is wrong. As such, asking it “were you correct when you told me X?” is bad hygiene.

You can “sanitize” the validation process by opening a new chat session and asking it if something is correct. You can also ask it to be adversarial and attempt to prove its prior output is wrong.

Even then it’s just a quick way to see if it’s output was garbage. A positive result is not a confirmation and independent verification is necessary.

Also, especially with ChatGPT, you have to understand that its role has been fine-tuned to be helpful and, to some extent, positively affirmative. This means, in my experience, that if you at all “show your hand” with a leading question or any (even unintended) indication of the answer you’re seeking, it is much more likely to output something that affirms any biases in your prompt.

People keep saying that it’s trained on human conversations/texts/etc and so everything it outputs is a reflection. But that’s not quite true:

ChatGPT in particular, unless you run up against firm guardrails of hate speech etc., appears to be fine tuned to a very large degree to be non confrontational. It generally won’t challenge your assumptions, so if your prompts have underlying assumptions in them (they almost always will) then ChatGPT will play along.

If you’re going to ask it for anything resembling factual information you have to be as neutral and open ended in tone as possible in your prompts. And if you’re trying to do something like check a hunch you have, you should probably not be neutral and instead ask it to be adversarial. Don’t ask “Is X true?”, ask “Analyze the possibility that X is false.”

Those are overly simplistic formulations of prompts but that’s the attitude you need to go into with it if you’re doing anything research-ish.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#206

Post by CatWChainsaw » Sat, May 27, 2023, 11:03 PM UTC

lamp987 wrote: post #23

this is the future of lawyers, doctors, engineers, school teachers... dark times are ahead.

We retreat into an age of darkness, clutching our crystals and consulting our horoscopes because everything has been made equally untrue with the Greatest Invention Man Ever Made. We should be so proud of ourselves.

Those who trust this tripe deserve the consequences they invite on themselves.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#207

Post by RhysU » Sat, May 27, 2023, 11:04 PM UTC

clnq wrote: post #142

taberiand wrote: post #28

Earlier quoted context omitted.

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. It did that admirably. It's not its fault, or in my opinion OpenAI's fault, that the output is being misunderstood and misused by people who can't be bothered understanding it and project their own ideas of how it should function…

This harks back to around 1999 when people would often blame computers for mistakes in their math, documents, reports, sworn filings, and so on. Then, a thousand different permutations of "computers don't make mistakes" or "computers are never wrong" became popular sayings. Large Language Models (LLMs) are never wrong, and they do not make mistakes. They are not fact machines. Their purpose is to abstract knowledge a…

> Large Language Models (LLMs) are never wrong, and they do not make mistakes.

I call B.S. If LLMs never made mistakes we wouldn't train them. Any random initialization would work.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#208

Post by suddenclarity » Sat, May 27, 2023, 11:05 PM UTC

mikestew wrote: post #90

kordlessagain wrote: post #72

Earlier quoted context omitted.

The example of asking it things like counting or sequences isn't a great one because it's been solved by asking it to "translate" to code and then run the code. I took this up as a challenge a while back with a similar line of reasoning on Reddit (that it couldn't do such a thing) and ended up implementing it in my AI web shell thing. heavy-magpie|> I am feeling excited. system=> History has been loaded. pastel-matur…

Shouldn’t the answer be zero?

Thanks. I'll steal this and refer to it in the future as an example of ambiguous project orders.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#209

Post by vitobcn » Sat, May 27, 2023, 11:06 PM UTC

mehwoot wrote: post #7

No, it did not “double-check”—that’s not something it can do! And stating that the cases “can be found on legal research databases” is a flat out lie. What’s harder is explaining why ChatGPT would lie in this way. What possible reason could LLM companies have for shipping a model that does this? It did this because it's copying how humans talk, not what humans do. Humans say "I double checked" when asked to verify so…

As obvious as it is once you think about it, people don't seem to realize ChatGPT is an LLM, a large LANGUAGE model, not a large knowledge model.

Its response from a linguistic perspective, was valid and "human-like", which is what it was trained for.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#210

Post by pylua » Sat, May 27, 2023, 11:08 PM UTC

inopinatus wrote: post #171

pylua wrote: post #131

What needs to exist is a legal compiler that lawyers can plug responses into from these tools. Sort of how a developer would use chat gpt. A giant rules engine for the law. I’m surprised one doesn’t exist or isn’t in progress that I know of. Seems like it would be very helpful

There isn't one because the law is not a programming language. Believing otherwise is a common misconception amongst engineers, but representing law as such is (as I have said in this forum before) a leading cause of disappointment, frustration, bickering, anger, conflict, and vexatiously long and mostly unenforceable contracts. Observance of law is fundamentally about alignment with principles, not blindly following…

This is less about observance with the laws and more about how to verify that lawyers are doing their jobs properly, sort of tools like ide for developer. Often times lawyers make mistakes like in the article or even file things in the wrong court or deficient forms . More like an ide for developers .

That said, it is a great disappointment of mine that the law is not based on an objective, static measure.