Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

arstechnica.com

Post reply on HN
Print view View original

121–130 of 212 posts

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#121

Post by [deleted] »

BugsJustFindMe wrote: post #106
cj wrote: post #19

I asked ChatGPT to tell me a riddle. It was “What is always hungry, needs to be fed, and makes your hands red?” (Or something like that) I asked for a hint about 5 times and it kept giving more legitimate sounding hints. Finally I gave up and asked for the answer to the riddle, and it spit out a random fruit which made no sense as the answer to the riddle. I then repeated the riddle and asked ChatGPT what the answer…

I need to know what version of ChatGPT you were using, because this is a critical piece of information that everyone just blatantly ignores, and I can only imagine that it's out of ignorance of the significance of the difference. This is what happened when I asked ChatGPT 4... ME Give me hints without outright telling me the answer to the riddle: "What is always hungry, needs to be fed, and makes your hands red?" Cha…

[deleted]

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#122

Post by BugsJustFindMe »

sn0wf1re wrote: post #114
BugsJustFindMe wrote: post #106

Earlier quoted context omitted.

I need to know what version of ChatGPT you were using, because this is a critical piece of information that everyone just blatantly ignores, and I can only imagine that it's out of ignorance of the significance of the difference. This is what happened when I asked ChatGPT 4... ME Give me hints without outright telling me the answer to the riddle: "What is always hungry, needs to be fed, and makes your hands red?" Cha…

That's GPT-4, not ChatGPT (3.5-turbo I think). Also, yes you can get correct information by tailoring your prompts, but that isn't the issue. The issue is that some prompts lead to bad results and confusing/incorrect answers. You changed what OP queried by providing the riddle and asking for hints to that riddle, whereas OP asked for a random riddle and then hints to that riddle.

> That's GPT-4, not ChatGPT

It absolutely is ChatGPT, the paid monthly "Plus" version, using the GPT4 model instead of the 3.5 model.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#123

Post by kbenson »

mabbo wrote: post #56
xahrepap wrote: post #52

Earlier quoted context omitted.

I've had a few moments with ChatGPT that are great anecdotes similar to your own: - Asked it to generate a MadLib for me to play that was no more than a paragraph long. It produced something that was several paragraphs wrong. I told it "no. That's X paragraphs. I asked for one that is only 1 paragraph long" and it would respond "I'm sorry for the misunderstanding. Let me try again" and then would make the same mistak…

There's definitely a potential for a D&D DM with an LLM, but you'd need a lot of careful prompting and processing to handle the token limits today's models have. Simply put: a d&d game has more story and state than the 30,000-ish words an LLM can think about at once. I think there's a lot of interesting opportunities there.

I've also heard (here) that after you get 20-ish questions into an instance you start getting the really weird output. Some of the conjecture was because that's about how deep they trained.

In any case, if that's true, that's a very short role playing session, unless there's a good way to retain info but reset the state that accrues and causes problems (if indeed that happens).

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#125

Post by hospitalhusband »

jiggawatts wrote: post #120
cj wrote: post #19

I asked ChatGPT to tell me a riddle. It was “What is always hungry, needs to be fed, and makes your hands red?” (Or something like that) I asked for a hint about 5 times and it kept giving more legitimate sounding hints. Finally I gave up and asked for the answer to the riddle, and it spit out a random fruit which made no sense as the answer to the riddle. I then repeated the riddle and asked ChatGPT what the answer…

ChatGPT 3.5 or GPT 4? Almost every negative comment about LLMs is by someone using an older, weaker model and making generalisations. Here’s GPT 4 giving me a riddle: https://chat.openai.com/share/1753ce5a-d44d-44ac-bc97-599a26...

> But was it GPT4

I keep seeing this cop-out, which ignores that it's fundamentally the same architecture, and has the same flaws. More wallpaper to hide the cracks better makes it an even worse tool for these use cases because all it does is fool more people into thinking it has capabilities that it fundamentally doesn't.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#126

Post by jiggawatts »

BugsJustFindMe wrote: post #113
selcuka wrote: post #110

Earlier quoted context omitted.

I guess the point is GPT-4 hallucinates, too. Maybe it did well for this example but still a lawyer should not trust its output.

Maybe, but it's surprisingly good in the face of all the non-version-indicating complaints about how terrible people think it is. Mostly I doubt that the lawyer was using GPT4, because the lawyer sounds like the kind of person who would be ignorant of the significance of the difference.

The kind of person too lazy to check the output of a computer program before submitting it to a court of law is the type of person too cheap to pay $20 for the good version of the program.

Think: Lionel Hutz.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#127

Post by kbenson »

Bilal_io wrote: post #97
travisjungroth wrote: post #87

Earlier quoted context omitted.

> That was a great way to show my non-tech family members the limitations of AI and why they shouldn’t trust it. These are the limitations of the version of ChatGPT you were using at that moment. They are not categorical limitations of AI or even LLMs. It’s amazing to me how many people are sleeping on AI, mixing up the failing cases of a freemium chatbot for the full capability of the tech, even on HN. LLMs can say…

> LLMs can say “I don’t know”. Even ChatGPT can do it. That's the problem in my opinion. When you know something is capable of saying "I don't know" but confidently spits out some hallucinated BS is when the average person eats it up.

I don't know exactly why, but for some reason this made me think of qAnon, and now I'm thinking of an AI trained on qAnon theories that people can form a community around like they did qAnon, and frankly that's one of the most terrifying things I've thought in quite a while.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#128

Post by bunabhucan »

lettergram wrote: post #91
travisjungroth wrote: post #87

Earlier quoted context omitted.

> That was a great way to show my non-tech family members the limitations of AI and why they shouldn’t trust it. These are the limitations of the version of ChatGPT you were using at that moment. They are not categorical limitations of AI or even LLMs. It’s amazing to me how many people are sleeping on AI, mixing up the failing cases of a freemium chatbot for the full capability of the tech, even on HN. LLMs can say…

I made https://AskHN.ai What it does is not try to answer, but collect previous topics discussed by experts. Then answer the question based on the text, a far more reliable approach.

How does it qualify experts? I love the discussion here but if it turns to international nuclear strategy or the minutae of electrical networks (or presumably anything outside the regular wheelhouse) I notice that the quality goes down but the confidence stays the same.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#129

Post by oblio »

jimnotgym wrote: post #30
cj wrote: post #19

I asked ChatGPT to tell me a riddle. It was “What is always hungry, needs to be fed, and makes your hands red?” (Or something like that) I asked for a hint about 5 times and it kept giving more legitimate sounding hints. Finally I gave up and asked for the answer to the riddle, and it spit out a random fruit which made no sense as the answer to the riddle. I then repeated the riddle and asked ChatGPT what the answer…

> Or, learn how to say “I don’t know” This is the correct answer. It is like a sad salesman who is out of his depth, but decides to keep bullshiting!

Well, that tells you a lot about:

1. The people designing it (either optimists or looking for a quick exit).

2. The learning set they're using, which I believe is some kind of internet crawl of sorts? I imagine humanity, as a whole, bullshits its way through most of its life.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#130

Post by Zetice »

So how do LLMs fit into the legal profession, if at all?

Do legal tools that make use of LLMs just need to come with big ol' disclaimers at the top saying, "This tool does not represent a legal opinion, please verify the output independently."?

Post reply on HN