Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 could pass bar exam, AI researchers say

the-decoder.com

Post reply on HN
Print view View original

61–70 of 167 posts

Re: GPT-4 could pass bar exam, AI researchers say

#61

Post by mdp2021 »

forgetfulness wrote: post #45
Workaccount2 wrote: post #25

I feel like I can now see the event horizon of commoditized intelligence. No idea what society (is "society" even the right word? Who knows) is going to look like on the other side of it, but it is going to be wildly different. Perhaps a brief period where everyone is using an AI to do their job, uh, I mean, assist their work, but beyond that it's unknowable. Moreover, this looks like it is going to be happening soon…

We were perhaps a bit too enamored with the idea that it was intellect that made us unique, and thus knowledge workers would be the last to be replaced. Pouring our brains out by the Petabytes for neural networks to pick them up made the economics just work for an AI industrial revolution to start from there.

No, we were enamored with the idea that intelligence was well distributed between people, as if following Descartes' massive incipit "Good sense must be the best distributed thing in the world, given that nobody seems to be asking for more".

Inability to recognize intelligence is and will be devastating.

Re: GPT-4 could pass bar exam, AI researchers say

#63

Post by BitwiseFool »

Workaccount2 wrote: post #25

I feel like I can now see the event horizon of commoditized intelligence. No idea what society (is "society" even the right word? Who knows) is going to look like on the other side of it, but it is going to be wildly different. Perhaps a brief period where everyone is using an AI to do their job, uh, I mean, assist their work, but beyond that it's unknowable. Moreover, this looks like it is going to be happening soon…

I feel like there is a difference between being able to pass a bar exam and being a "good" lawyer. I suspect AI tools would enhance the jobs of clerks rather than attorneys, mostly because clerks spend a great deal of time going over case law, text, and doing research.

Re: GPT-4 could pass bar exam, AI researchers say

#64

Post by munchler »

pyb wrote: post #9

Sounds like they didn't have access to GPT-4, but "Based on anecdotal evidence"... they still predict this.

They’re extrapolating from the performance of GPT-3.5. It’s speculative, but not anecdotal. GPT has improved rapidly over time, so it's not a huge leap to predict that GPT-4 will be even better.

Re: GPT-4 could pass bar exam, AI researchers say

#65

Post by mdp2021 »

ok123456 wrote: post #55
Aperocky wrote: post #53

Earlier quoted context omitted.

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

Most people's reasoning ability functions at this level.

And in fact we want them to act outside a professional capacity in which they will be dangerous liabilities. ...Is not this piece about "Bar Exams"?

Re: GPT-4 could pass bar exam, AI researchers say

#66

Post by sebzim4500 »

Aperocky wrote: post #53
Workaccount2 wrote: post #25

I feel like I can now see the event horizon of commoditized intelligence. No idea what society (is "society" even the right word? Who knows) is going to look like on the other side of it, but it is going to be wildly different. Perhaps a brief period where everyone is using an AI to do their job, uh, I mean, assist their work, but beyond that it's unknowable. Moreover, this looks like it is going to be happening soon…

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

>As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question

Is this true even if you tell it to show its working? In my experience that drastically improves its ability to do math problems.

Re: GPT-4 could pass bar exam, AI researchers say

#67

Post by czzr »

There’s a gap between passing the bar exam and actually practicing law - I’m pretty certain that I (someone with no legal training whatsoever) could pass the bar exam if you gave me unlimited access to the internet and a couple of additional hours to write the test. However, I don’t think that would make me an effective lawyer.

Ultimately standardised tests are proxy measurements of legal ability - it’s easy to see how a LLM could subvert the proxy without being sufficiently reliable in real life.

I do expect that even unreliable versions will be very useful tools for practicing lawyers, though.

Re: GPT-4 could pass bar exam, AI researchers say

#68

Post by mdp2021 »

BitwiseFool wrote: post #63
Workaccount2 wrote: post #25

I feel like I can now see the event horizon of commoditized intelligence. No idea what society (is "society" even the right word? Who knows) is going to look like on the other side of it, but it is going to be wildly different. Perhaps a brief period where everyone is using an AI to do their job, uh, I mean, assist their work, but beyond that it's unknowable. Moreover, this looks like it is going to be happening soon…

I feel like there is a difference between being able to pass a bar exam and being a "good" lawyer. I suspect AI tools would enhance the jobs of clerks rather than attorneys, mostly because clerks spend a great deal of time going over case law, text, and doing research.

> enhance the jobs of clerks

We already did, it is called "Case Based Reasoning" within Decision Support Systems.

Re: GPT-4 could pass bar exam, AI researchers say

#69

Post by etangent »

ok123456 wrote: post #55
Aperocky wrote: post #53

Earlier quoted context omitted.

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

Most people's reasoning ability functions at this level.

I would argue that it does not matter. The AI could even be "smarter" on pure IQ/reasoning, but in terms of practical reasoning that humans need that depends on exposure to real world, the AIs will still take decades to catch up.

The radiology AIs are technically more accurate than radiologists on any sufficiently large dataset, and yet they still have not replaced radiologists (or even are anywhere close to).

Re: GPT-4 could pass bar exam, AI researchers say

#70

Post by ragebol »

ok123456 wrote: post #55
Aperocky wrote: post #53

Earlier quoted context omitted.

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

Most people's reasoning ability functions at this level.

Which should be worrying. It's also why most people don't pass (nor try) the bar exam.
Post reply on HN