Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 outperforms federal judges in legal reasoning experiment

papers.ssrn.com

Post reply on HN
Print view View original

21–30 of 254 posts

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#21

Post by unyttigfjelltol »

swisniewski wrote: post #14

The premise seems flawed. From the paper: “we find that the LLM adheres to the legally correct outcome significantly more often than human judges” That presupposes that a “legally correct” outcome exists The Common Law, which is the foundation of federal law and the law of 49/50 states, is a “bottom up” legal system. Legal principals flow from the specific to the general. That is, judges decided specific cases based…

A Socratic law professor will demoralize students by leading them, no matter the principle or reasoning, to a decision that stands for exactly the opposite. GPT or I can make excuses and advocate for our pet theories, but these contrary decisions exist, everywhere.

I am comforted that folks still are trying to separate right from wrong. Maybe it’s that effort and intention that is the thread of legitimacy our courts dangle from.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#23

Post by qwertox »

codingdave wrote: post #2

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

> If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-and-white thinking.

You can have a team of agents exchange views and maybe the protocol would even allow for settling the cases automatically. The more agents you have, the higher the nuances.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#25

Post by deepsun »

codingdave wrote: post #2

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

The main job of a judicial system is to appear just to people. As long as people think it's just -- everyone is happy. But if it's strictly by the law, but people consider it's unjust -- revolutions happen.

In both cases, lawmakers must adapt the law to reflect what people think is "just". That's why there are jury duty in some countries -- to involve people to the ruling, so they see it's just.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#28

Post by pcj-github »

sinuhe69 wrote: post #22
nacozarina wrote: post #15

The ability of ai to serve as impartial mediators could become the greatest civil rights advance in modern history.

I’d argue it’s the greatest nightmare and the ultimate contempt for human life and values.

Compared to the judicial landscape we're facing in the US right now, it sounds like a safeguard.

Until this administration forces OpenAI to comply by secret government LLM training protocols that is...

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#30

Post by toolslive »

deepsun wrote: post #25
codingdave wrote: post #2

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

The main job of a judicial system is to appear just to people. As long as people think it's just -- everyone is happy. But if it's strictly by the law, but people consider it's unjust -- revolutions happen. In both cases, lawmakers must adapt the law to reflect what people think is "just". That's why there are jury duty in some countries -- to involve people to the ruling, so they see it's just .

Being just (as in the right thing happened) and being legal (as in the judicial system does not object) are 2 totally different things. They overlap, but less than people would like to believe.
Post reply on HN