Earlier quoted context omitted.
'Bread *and butter'. The English expression requires the second part—but otherwise fits perfectly in your well-stated point, with which I wholeheartedly agree. Source: AAL.
Thank you! As a non native speaker I was not sure if “and butter” is a mandatory part but didn’t want (nor had time) to llm the comment for the sake of authenticity :) TIL
AI outperforms law professors in Stanford Law study
301–310 of 384 posts
Re: AI outperforms law professors in Stanford Law study
#302Earlier quoted context omitted.
On the other hand we keep seeing only marginal generational imorovements in CPU space, yet performance gains over last 10 years in CPUs are very material. Every new model might not be a leap like it used to be, but give it enough time and improvements add up.
Nobody is disputing that. I specifically said that I can see the improvements from the last six months. What I’m saying is we can’t assume that every two years it will improve at the same rate. The further we get into this, the more AI feels like 3-D printing. Significantly bigger and will be more widely used for sure. But nowhere near the “new industrial revolution” that all these companies are making it out to be
Re: AI outperforms law professors in Stanford Law study
#303I find this study quite suspect. I'd have to dive deeper but there's definitely significant alarm bells that should be going off for anyone reading. Figure 2 (page 6) screams problems. There's only 16 professors (3k comparisons each?!?!) and the professors are all over the place. That's very high variance, suggesting the study has no meaningful statistical power. Poor instructor 16 can't catch a break lol There's als…
Re: AI outperforms law professors in Stanford Law study
#304I find this study quite suspect. I'd have to dive deeper but there's definitely significant alarm bells that should be going off for anyone reading. Figure 2 (page 6) screams problems. There's only 16 professors (3k comparisons each?!?!) and the professors are all over the place. That's very high variance, suggesting the study has no meaningful statistical power. Poor instructor 16 can't catch a break lol There's als…
Sure, but in two years AI has gone from “impressive tool, but not a replacement for knowledge workers” to “the study where it beats our highest caliber of knowledge workers may have some methodological deficits.” In another two years it’s going to be curtains.
Re: AI outperforms law professors in Stanford Law study
#305I find this study quite suspect. I'd have to dive deeper but there's definitely significant alarm bells that should be going off for anyone reading. Figure 2 (page 6) screams problems. There's only 16 professors (3k comparisons each?!?!) and the professors are all over the place. That's very high variance, suggesting the study has no meaningful statistical power. Poor instructor 16 can't catch a break lol There's als…
Re: AI outperforms law professors in Stanford Law study
#306Earlier quoted context omitted.
I kinda disagree. High risk environments just means that they will have to have a human-in-the-loop for a longer time which drastically reduce the skill required for such human (which is still requires high skill just not stupidly high).
> drastically reduce the skill required for such human I mean thats what is wanted by some companies. The problem, especially for things like legal is that it requires someone more skilled to read through and understand that the argument is bollocks, or the law/precedent they are banking on is in fact the right one. We have a tool that auto-writes letters to our management companies when they break SLAs. We have a sl…
Re: AI outperforms law professors in Stanford Law study
#307Earlier quoted context omitted.
> Just a humble opinion that I would love to see be wrong Out of curiosity, why would you love to be wrong about that? What possible outcome could you see being a net positive for society if the vast majority of knowledge workers (and ultimately, as robotics progress, most workers in general) are replaced by AI?
I believe it was Blink-182 who said, "Work sucks". You have to pay people to do that stuff; they don't want to be there. And then you get into second order effects- costs plummet for anything labor intensive, including medical care, prepared food, cleaning, and private tutors. Then onto tertiary effects- if you can spin up a million genius researchers to attack a problem, you start seeing massive progress in every im…
Re: AI outperforms law professors in Stanford Law study
#308I find this study quite suspect. I'd have to dive deeper but there's definitely significant alarm bells that should be going off for anyone reading. Figure 2 (page 6) screams problems. There's only 16 professors (3k comparisons each?!?!) and the professors are all over the place. That's very high variance, suggesting the study has no meaningful statistical power. Poor instructor 16 can't catch a break lol There's als…
The main results also don’t seem to know what a “model” is, as the two “models” it refers to are “stock Gemini 2.5 Pro” and “a retrieval-augmented version of NotebookLM”.
One of which is a model, and the other of which is an interface backed by different models depending on exactly when the analysis was performed.
Re: AI outperforms law professors in Stanford Law study
#309In general it is not surprising. Even if this particular study is bad. There are certain areas of law work that are about analyzing large amounts of texts, drawing conclusions and writing other texts based on that and nothing more. That is literally the bread of LLMs. Those types of lawyers should be the first in line for unemployment, not programmers, not even close.
I see the same problem with AI in both programming and law though. AI is like a scab on a wound: it's a temporary filler, it rushes in to fill a void, but it's not going to be the final solution. Models showed us that there was huuuge unmet demand for literacy, both in software and in law. But now we have a choice to either address the systemic causes of the unmet demand, or just try to paper over them with layers an…
Yeah, but in my experience it won't come down to "which is the better solution" but "which is cheaper/easier"
So I look forward to lots of layers of papered over AI scabs in the future. It won't be cheaper in the long run, but it will pump someone's quarterly numbers enough that they get a promotion before the problem they introduce come back to them
Re: AI outperforms law professors in Stanford Law study
#310I find this study quite suspect. I'd have to dive deeper but there's definitely significant alarm bells that should be going off for anyone reading. Figure 2 (page 6) screams problems. There's only 16 professors (3k comparisons each?!?!) and the professors are all over the place. That's very high variance, suggesting the study has no meaningful statistical power. Poor instructor 16 can't catch a break lol There's als…
Sure, but in two years AI has gone from “impressive tool, but not a replacement for knowledge workers” to “the study where it beats our highest caliber of knowledge workers may have some methodological deficits.” In another two years it’s going to be curtains.