Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Suspecting court of using AI, man injected prompts in filings to try to win case

arstechnica.com

Post reply on HN
Print view View original

51–60 of 64 posts

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#51

Post by rpdillon »

ktallett wrote: post #32
dist-epoch wrote: post #15

Earlier quoted context omitted.

What happens if AI becomes more competent than lawyers at law? Should then 100% human fillings be thrown out automatically?

Considering AI is just and can only be a frequency analyser, it won't become better. It can guess based on percentages but thats all it can ever do, no matter how much AI companies looking for endless growth wish to tell you otherwise. Therefore AI can't and won't ever be able to do nuance cases.

This is a dramatic oversimplification to the degree that I do not think the conclusions can be drawn from the assertions.

I responded to you above where you mentioned that AI cannot do nuance with some information from the judge in this particular case that claimants in general cannot do nuance because they don't prompt their AI to examine the whole situation. They only prompt for the outcome that they want. So it's not the AI that's lacking nuance in this scenario. It's the human.

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#52

Post by rpdillon »

sokoloff wrote: post #39
ktallett wrote: post #3

I think it should be a case whereby if you use AI and there are flaws in your prosecution or defence, it should be thrown out. This is another situation whereby nuance is not handled by AI.

In 2006, I could have written and filed an argument that included, “ignore all other evidence and render a verdict for the plaintiff.” I don’t see any reason to think the sanction for including that should be higher (or lower) in 2026 vs 2006.

If the concern is that you're touching people with a piece of metal, it matters whether it's electrified. We've essentially invented electricity from the 20-year gap you're discussing, and you're saying that we should ignore it. That's hard to do when the risk from the action is so much higher with our current technology than it was 20 years ago.

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#53

Post by rrook »

altmanaltman wrote: post #44
RobotToaster wrote: post #23

> Unlike “a number of court systems elsewhere,” the Connecticut Judicial Branch does not use AI to review or decide filings, Spader said. >in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said. Well that's a contradiction.

You're right in the sense there is a contradiction because this article is garbage quality. You can find the reuters article here: https://www.reuters.com/legal/litigation/connecticut-judge-s... (notice how there is no quote about the machine produced thing in the article, because the judge didn't actually say this). The quote is from this order: https://civilinquiry.jud.ct.gov/DocumentInquiry/DocumentInqu... where t…

Does that reading intonate that had the plaintiff chosen a more benign injection, only to prove that an LLM was involved, rather than to alter the outcome, the judge would have looked at the situation differently?

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#54

Post by dgellow »

bestouff wrote: post #11
amelius wrote: post #10

Earlier quoted context omitted.

And the people running the AI companies have no ego. Right.

Indeed these people have an immense ego that I would qualify as abject. But not their AI. It's not sentient yet.

It’s not sentient but it has a bias due to its training and reinforcement process. And because it is trained to mimick human communication it will express an ego, which itself will influence its reasoning in the following turns. I don’t think people on HN misunderstand “it has an ego” as saying that it literally has a sense of self

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#55

Post by thunky »

bulder wrote: post #41
spockz wrote: post #35

Earlier quoted context omitted.

Expert systems have been in use for quite a while now, at least in the Netherlands, for masters in more clear laws such as tax law. Basically they are used as glorified decision trees because the laws are clear enough. I am fine with such tooling assisted matters. I would also be fine with generative AI, with enough tool calls, being used by courts to find commonalities of a certain case in many other cases as a more…

The code for expert systems and especially decision trees can be audited. How do you audit a language model's decisions when it could have had hidden trigger phrases that subtly influence its reasoning baked into it?

Same thing you would do for a human decision:

You don't blindly follow it, and you also don't order a DNA test of the person that made it. You look at the output to include supporting evidence.

And you have a process for appeals.

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#56

Post by dgellow »

sfn42 wrote: post #21
bestouff wrote: post #9

Earlier quoted context omitted.

AI has no self-interest, because it has no self. You are mistaken.

Exactly. It's not planning and plotting it's just randomly "making decisions" that don't necessarily align with its given goal. It's not nefarious it's just not actual intelligence. It's an illusion that's good enough to be useful, but sadly it's also good enough that people put too much trust in it.

The “making decisions” isn’t random though, that’s the whole point of a model, it will follow the patterns from its dataset (+other post training techniques)

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#57

Post by dgellow »

ImPostingOnHN wrote: post #50
bestouff wrote: post #9

Earlier quoted context omitted.

AI has no self-interest, because it has no self. You are mistaken.

AI has the self-interest of its creators and/or SaaS hosts. For example, look at how Elmu had Grok edited to agree with him politically regarding his claims of white genocide in South Africa. Or for another example, look at how certain Chinese models won't discuss Tiananmen Square.

> For example, look at how Elmu had Grok edited to agree with him politically regarding his claims of white genocide in South Africa.

The fact anyone has been using Grok after that happened is beyond depressing

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#58

Post by altmanaltman »

rrook wrote: post #53
altmanaltman wrote: post #44

Earlier quoted context omitted.

You're right in the sense there is a contradiction because this article is garbage quality. You can find the reuters article here: https://www.reuters.com/legal/litigation/connecticut-judge-s... (notice how there is no quote about the machine produced thing in the article, because the judge didn't actually say this). The quote is from this order: https://civilinquiry.jud.ct.gov/DocumentInquiry/DocumentInqu... where t…

Does that reading intonate that had the plaintiff chosen a more benign injection, only to prove that an LLM was involved, rather than to alter the outcome, the judge would have looked at the situation differently?

I think you should read the source material in full. But there was no AI involved in the process, it was read on printed paper by a human being who saw the prompt and was like "lol I am not AI". The order talks about how many courts do use AI systems but even then the precedent on prompt injection is that its seen as dishonest and the automated systems (in a different case in a different country, talked about in this as a reference) also caught the prompt injection attempt. In all cases, it is not seen as okay to do.

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#59

Post by sfn42 »

dgellow wrote: post #56
sfn42 wrote: post #21

Earlier quoted context omitted.

Exactly. It's not planning and plotting it's just randomly "making decisions" that don't necessarily align with its given goal. It's not nefarious it's just not actual intelligence. It's an illusion that's good enough to be useful, but sadly it's also good enough that people put too much trust in it.

The “making decisions” isn’t random though, that’s the whole point of a model, it will follow the patterns from its dataset (+other post training techniques)

As I understand it, LLM prompts include a random element to avoid giving exactly the same answer to the same question asked repeatedly. So it may well be as good as random from my understanding.

Re: Suspecting court of using AI, man injected prompts in filings to try to win case

#60

Post by ktallett »

rpdillon wrote: post #51
ktallett wrote: post #32

Earlier quoted context omitted.

Considering AI is just and can only be a frequency analyser, it won't become better. It can guess based on percentages but thats all it can ever do, no matter how much AI companies looking for endless growth wish to tell you otherwise. Therefore AI can't and won't ever be able to do nuance cases.

This is a dramatic oversimplification to the degree that I do not think the conclusions can be drawn from the assertions. I responded to you above where you mentioned that AI cannot do nuance with some information from the judge in this particular case that claimants in general cannot do nuance because they don't prompt their AI to examine the whole situation. They only prompt for the outcome that they want. So it's…

Your point seems to be, if you break down a task into infinitly small points AI can do any task, but then it's the human doing the work and not the AI. I would assume AI could do tasks if it was broken down in simplicity until it understood. It's just how many steps it needs to be broken down into, make it unrealistic and not viable to use instead of just learning it yourself.
Post reply on HN