Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lawyer cites fake cases invented by ChatGPT, judge is not amused

simonwillison.net

Post reply on HN
Print view View original

161–170 of 319 posts

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#161

Post by simonw »

einpoklum wrote: post #144
Buttons840 wrote: post #47

Earlier quoted context omitted.

GPT4 can double-check to an extent. I gave it a sequence of 67 letter As and asked it to count them. It said "100", I said "recount": 98, recount, 69, recount, 67, recount, 67, recount, 67, recount, 67. It converged to the correct count and stayed there. This is quite a different scenario though, tangential to your [correct] point.

But would GPT4 actually check something it had not checked the first time? Remember, telling the truth is not a consideration for it (and probably isn't even modeled), just saying something that would typically be said in similar circumstances.

Only in as much as there's an element of randomness to the way GPT responds to a prompt - so you can re-run effectively the same prompt and get a different result depending on the outcome of several hundred billion floating point calculations with a random seed thrown in.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#162

Post by coldtea »

Karellen wrote: post #149
coldtea wrote: post #128

Earlier quoted context omitted.

> Lying implies intent, and knowing what the truth is. Saying something you believe to be true, but is wrong, is generally not considered a lie but a mistake. Those are the semantics of lying. But "X like a duck" is about ignoring semantics, and focusing not on intent or any other subtletly, but only on the outward results (whether something has the external trappings of a duck). So, if it produces things that look l…

A person who is mistaken looks like they're lying. That doesn't mean they're actually lying. That's the thing people are trying to point out. You can't look at something that looks like it's lying and conclude that it's lying, because intent is an intrinsic part of what it means to lie.

Even for that, you'd be surprised:

(1) get oneself into or out of a situation by lying. "you lied your way on to this voyage by implying you were an experienced crew"

(2) (of a thing) present a false impression. "the camera cannot lie"

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#164

Post by dragonwriter »

londons_explore wrote: post #11
disillusioned wrote: post #4

> He had, he told Judge Castel, even asked the program to verify that the cases were real. It had said yes. It turns out, asking an unreliable narrator if it's being reliable is not a sound strategy.

I wonder if this is a tactic so the court to deems this lawyer incompetent rather than giving the (presumably much harsher) penalty for deliberately lying to the court?

This explanation is a cause of an expansion of the scope of the show cause order for the lawyer to additional bases for sanctions, as well as its expansion to the other involved lawyer and their firm, so if it was a strategic narrative, it backfired spectacularly already.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#165

Post by joe_the_user »

fallingknife wrote: post #71
ClumsyPilot wrote: post #65

Earlier quoted context omitted.

Thats like saying robots don't murder - they just kill

Which is actually a very good analogy. A lot of things can kill you, but only a human can be a murderer.

In movies and written fiction, "intelligent" robots, anthropomorphized animals, elves, dwarves and etc can all commit murder when given the attributes of humans.

We don't have real things with all human attributes but we're getting closer and as we get close "needs to be a human" will get thinner as an explanation of what is or isn't human for an act of murder, deception and so-forth.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#166

Post by lamontcg »

olalonde wrote: post #112
coldtea wrote: post #81

Earlier quoted context omitted.

> ChatGPT did not lie; it cannot lie. If it lies like a duck, it is a lying duck.

It doesn't lie like a duck. It unintentionally says falsehoods. Lying is intentional.

It doesn't really matter.

This always bugs me about how people judge politicians and other public figures not by what they've actually done, but some ideal of what is in their "heart of hearts" and their intentions and argue that they've just been constrained by the system they were in or whatever.

Or when judging the actions of nations, people often give all kinds of excuses based on intentions gone wrong (apparently forgetting that whole "road to hell is paved with good intentions" bit).

Intentions don't really matter. Our interface to everyone else is their external actions, that's what you've got to judge them on.

Just say that GPT/LLMs will lie, gaslight and bullshit. It doesn't matter that they don't have an intention to do that, it is just what they do. Worrying about intentions just clouds your judgement.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#167

Post by grt_thr »

Karellen wrote: post #149
coldtea wrote: post #128

Earlier quoted context omitted.

> Lying implies intent, and knowing what the truth is. Saying something you believe to be true, but is wrong, is generally not considered a lie but a mistake. Those are the semantics of lying. But "X like a duck" is about ignoring semantics, and focusing not on intent or any other subtletly, but only on the outward results (whether something has the external trappings of a duck). So, if it produces things that look l…

A person who is mistaken looks like they're lying. That doesn't mean they're actually lying. That's the thing people are trying to point out. You can't look at something that looks like it's lying and conclude that it's lying, because intent is an intrinsic part of what it means to lie.

Ironically enough this sounds exactly like how bing was justifying its gas lighting before it got lobotomised.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#168

Post by mod50ack »

lo_zamoyski wrote: post #94
mod50ack wrote: post #54

Earlier quoted context omitted.

Lying implies an intention. ChatGPT doesn't have that. What ChatGPT definitely does do is generate falsehoods. It's a bullshitting machine. Sometimes the bullshit produces true responses. But ChatGPT has no epistemological basis for knowing truths; it just is trained to say stuff.

And if you want to be pedantic, ChatGPT isn't even generating falsehoods. A falsehood requires propositional content and therefore intentionality, but ChatGPT doesn't have that. It merely generates strings that, when interpreted by a human being as English text, signify falsehoods.

Getting into the weeds, but I don't agree with this construal of what propositional content is or can be. (There is no single definition of "proposition" which has wide acceptance and specifies your condition here.) There is no similar way to assess truth outside of formalized mathematics, but the encoding of mathematical statements (think Gödel numbers) comes to mind; I don't think that the ability of the machine to understand propositions is necessary in order to make the propositions propositional; the system of ChatGPT is designed in order to return propositional content (albeit not ex nihilo, but according to the principles of its design) and this could be considered analogous to the encoding of arithmetical symbolic notation into an formally-described system. The difference is just that we happen to have a formal description of how some arithmetic systems operate, which we don't (and I would say can't) have for English. Mild throwback to my university days studying all of this!

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#169

Post by nine_k »

We can hope to start to actually rely on such models once they start learning not only in the language domain, but also in the epistemic domain. True vs false, known vs unknown, precise vs vague, agreement vs contradiction vs unrelated, things like that.

Achieving that is going to be a serious technical, and also philosophical, challenge for humans.

Today's LLM are a literary device. They say what sounds plausible in the universe of texts they were fed. What they say technically isn't even wrong, because they have no notion of truth, or any notion of a world beyond the words. Their output should be judged accordingly.

Re: Lawyer cites fake cases invented by ChatGPT, judge is not amused

#170

Post by macrael »

Genuine question: why have these models all been trained to sound so confident? Is it not possible to have rewarded models that announced their own ignorance? Or is even that question belying an "intelligence" view of these models that isn't accurate?
Post reply on HN