Earlier quoted context omitted.
I do wish they'd used some more objective criteria. Simply being preferable one of the things LLMs have trained for since the beginning, hence its sycophantic nature.
What criteria would you use for judging legal arguments?
AI outperforms law professors in Stanford Law study
21–30 of 384 posts
Re: AI outperforms law professors in Stanford Law study
#22As a software engineer I have some intuition for what the risks are of letting agents do some tasks vs others. I don't have a similar intuition calibrated for what could go wrong when asking AI to draft a legal document. Some things seem harmless, i.e. drafting a will, but I don't really know- our legal system is notoriously rife with footguns.
Re: AI outperforms law professors in Stanford Law study
#23But imagine if a dev team didn’t have to go engineer -> product manager -> legal team to get a question answered on local data retention requirements. You could ship that much faster.
Re: AI outperforms law professors in Stanford Law study
#24I think, in the right hands, this could be huge.
Re: AI outperforms law professors in Stanford Law study
#25[flagged]
Re: AI outperforms law professors in Stanford Law study
#26Re: AI outperforms law professors in Stanford Law study
#27Earlier quoted context omitted.
If you want human connection the legal system is not where you are going to find it, period. I don't think there will be any such market for "non ai" law. If I'm involved with the legal system I just want out as quick as possible as cheap as possible.
Bad legal advice will keep you dealing with the legal system for much longer and at much greater cost. Something being cheap and quick upfront doesn't mean it will be cheap and quick by the end of the process.
A bit of extrapolation from the study, but not a crazy stretch.
Re: AI outperforms law professors in Stanford Law study
#28I think there will be a market for firms that aggressively market themselves as non-AI, and then as more people turn towards that human connection we'll go full circle
Re: AI outperforms law professors in Stanford Law study
#29As a software engineer I have some intuition for what the risks are of letting agents do some tasks vs others. I don't have a similar intuition calibrated for what could go wrong when asking AI to draft a legal document. Some things seem harmless, i.e. drafting a will, but I don't really know- our legal system is notoriously rife with footguns.
Re: AI outperforms law professors in Stanford Law study
#30I'm getting more convinced. I mean, sure it makes dumb mistakes sometimes but its a particular set of self serving mistakes, commenting out tests in order to pass. We obv don't want this behavior but I wouldn't say it's dumb.
It'll be like the Turing test, which we just blew past years ago and no one cared. After all the hand-wringing about sentience and rights of the AI if it passes the Turing test, and now we just have AI bots running 24/7 writing slop.
How does everyone else feel?