Live data from Hacker News

GPT-4 could pass bar exam, AI researchers say

the-decoder.com

71–80 of 167 posts

Re: GPT-4 could pass bar exam, AI researchers say

#71
post #9

Sounds like they didn't have access to GPT-4, but "Based on anecdotal evidence"... they still predict this.

For some reason, there's a thought-leader sect of Twitter talking about how good GPT-4 is, despite OpenAI having provided zero hints of what GPT-4 could entail or be differentiated from GPT-3/chatGPT.

Re: GPT-4 could pass bar exam, AI researchers say

#72

I feel like I can now see the event horizon of commoditized intelligence. No idea what society (is "society" even the right word? Who knows) is going to look like on the other side of it, but it is going to be wildly different. Perhaps a brief period where everyone is using an AI to do their job, uh, I mean, assist their work, but beyond that it's unknowable. Moreover, this looks like it is going to be happening soon…

I think we're very close to Saturday from Clippy[0].

By this I don't mean an AI as in the story acting by itself with its own motivations, I'm only talking about the subversion of established verification & communication methods used by it by humans with malicious purposes.

Essentially, if you do anything security related, we might only be O(months) away from you needing to stop using basically any electronic communication for your purposes. Companies can't have online meetings anymore in which decisions are made, everything will have to be more analog, more in-person.

Look at the kind of access the Russian comedians Vovan & Lexus [1] have gotten. Without advanced AI, just a little social engineering, they got heads of state on the phone. Now combine this with the kind of text/audio/video synthesis we're not too far away from, and you have an absolute recipe for disaster ...

[0]: https://www.gwern.net/fiction/Clippy#saturday

[1]: https://en.wikipedia.org/wiki/Vovan_and_Lexus

Re: GPT-4 could pass bar exam, AI researchers say

#73
post #29
post #17

Earlier quoted context omitted.

I got the same feeling asking ChatGPT about some basic logic and maths concepts. IMO GPT can find the relevant training data to regurgitate, but i don't think it connects concepts.

I mean, it's a bullshit generator. It'll grab whatever it find in training set that kinda fits the topic and make sure it hits the word count - like a lazy student before deadline. And that's also the result - sometimes it hits something good. Sometimes it spews up utter crock and it doesn't have any notion or understanding of the difference. However, it does look good to the lazy and uninformed and it'll soon render…

Erm. Yeah. Which is precisely what many lawyers and judges do, too, unfortunately. It often has little to do with logic and a lot to do with thinking in boxes and using words as nothing but triggers for other words. Some lawyers are far above that, of course. But what percentage of them works just like your “lazy student”? 80 percent?

Re: GPT-4 could pass bar exam, AI researchers say

#74

Earlier quoted context omitted.

I knew we were going to replace 9/10 doctors and 9/10 lawyers the same day I got to try ChatGPT. It's just a matter of time - whoever does it properly first. I am talking about the first line of defense here, like a chat bot. Courtrooms will probably still work the same way for a long time. It's not like most lawyers or doctors are great. Most are completely average - which is fine. Not everyone wants to read the lat…

It will be exactly the opposite. You are missing what doctors are mostly doing. Caring for elderly people. They will still need human communication, attention and care, the expertise can come from something like ChatGPT.

You do realize that a large part of of patient interaction is from nurses and physician's assistants and not doctors, right?

Re: GPT-4 could pass bar exam, AI researchers say

#75
The Bar Exam is multiple choice, right?

This isn't grading some freeform essay or generating arbitrary legal opinion. It's answering from a limited set of answers.

IMO it's cool, but not THAT shocking given what we've seen from ChatGPT? Especially given GPT 3.5 is only 17% below human test takers?

Re: GPT-4 could pass bar exam, AI researchers say

#76

Earlier quoted context omitted.

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

>As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question Is this true even if you tell it to show its working? In my experience that drastically improves its ability to do math problems.

Largely yes. Some prompts can make it better or worse.

Re: GPT-4 could pass bar exam, AI researchers say

#77
post #9

Sounds like they didn't have access to GPT-4, but "Based on anecdotal evidence"... they still predict this.

They’re extrapolating from the performance of GPT-3.5. It’s speculative, but not anecdotal. GPT has improved rapidly over time, so it's not a huge leap to predict that GPT-4 will be even better.

[dead]

Re: GPT-4 could pass bar exam, AI researchers say

#79
post #6

Earlier quoted context omitted.

Defender is probably good. Prosecutor is what would worry me, given I don't know better than to blindly trust the meme that the average person commits 6 felonies before breakfast.

Defender is a TERRIBLE idea. I can already see the Supreme Court cases down the line: Defendant was provided a state of the art, 50 trillion parameter, neural network for their defense. The internals of this network are not auditable, but it does not tire, engage in substance abuse, or get distracted, so it will by definition represent effective assistance of counsel, even if for some unfathomable reason it decides t…

I'm willing to bet a chat bot would perform better than most public defenders.

Re: GPT-4 could pass bar exam, AI researchers say

#80

Earlier quoted context omitted.

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

>As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question Is this true even if you tell it to show its working? In my experience that drastically improves its ability to do math problems.

It can't solve quadratic equations.
Post reply on HN