Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI outperforms law professors in Stanford Law study

law.stanford.edu

Post reply on HN
Print view View original

41–50 of 384 posts

Re: AI outperforms law professors in Stanford Law study

#41

Post by jay_kyburz »

causal wrote: post #7

As a software engineer I have some intuition for what the risks are of letting agents do some tasks vs others. I don't have a similar intuition calibrated for what could go wrong when asking AI to draft a legal document. Some things seem harmless, i.e. drafting a will, but I don't really know- our legal system is notoriously rife with footguns.

I imagine it's really hard to spot a comma in the wrong place, or a missing sentence in a 10 page contract unless you wrote it yourself, or you assembled it from some battle tested templates.

Re: AI outperforms law professors in Stanford Law study

#43

Post by wilg »

mitkebes wrote: post #37
wilg wrote: post #13

Earlier quoted context omitted.

What criteria would you use for judging legal arguments?

The arguments need to be based on actual law, and any cited reference cases need to be real. There's been a lot of news stories about lawyers using AI, and then getting in trouble for citing hallucinated laws or cases. It doesn't matter if the AI response is "preferred" over the human one if it gets thrown out when put under the scrutiny of a real case.

Who's gonna determine that? A bunch of law professors?

Re: AI outperforms law professors in Stanford Law study

#44

Post by ares623 »

Esophagus4 wrote: post #23

Yeah this could be interesting. A lot of the spotlight has been on “law firm stuff” like demand letters and writing contracts… But imagine if a dev team didn’t have to go engineer -> product manager -> legal team to get a question answered on local data retention requirements. You could ship that much faster.

Would you take responsibility for missing details about local data retention requirements?

Re: AI outperforms law professors in Stanford Law study

#47

Post by zuzululu »

ares623 wrote: post #44
Esophagus4 wrote: post #23

Yeah this could be interesting. A lot of the spotlight has been on “law firm stuff” like demand letters and writing contracts… But imagine if a dev team didn’t have to go engineer -> product manager -> legal team to get a question answered on local data retention requirements. You could ship that much faster.

Would you take responsibility for missing details about local data retention requirements?

honestly if you just avoid EU and China

you can get away with anything

Re: AI outperforms law professors in Stanford Law study

#49

Post by paulmist »

bko wrote: post #30

Marc Andreessen argued that we've already reached AGI. He says that the top AI models give better answers than 99% of people he has access to, and he has access to some of the best people in their field. I'm getting more convinced. I mean, sure it makes dumb mistakes sometimes but its a particular set of self serving mistakes, commenting out tests in order to pass. We obv don't want this behavior but I wouldn't say i…

Knowing the question is half of the answer. LLMs are great at scoping your context and answering precisely what you asked; it's also why they go off the rails when they misunderstand a part of your question. Incidentally, they're great at "knowing" and reaching for knowledge.

Humans have the advantage of perspective. We always lack some knowledge and answer broadly. This is bad if you have a particular goal in mind, but better if you're just generally learning, because you see more and learn to discriminate the correct from the wrong. And most importantly, being wrong is part of human ingenuity - because sometimes we turn something "obviously" wrong into something right.

Post reply on HN