Live data from Hacker News

GPT Takes the Bar Exam

github.com

41–50 of 147 posts

Re: GPT Takes the Bar Exam

#41

I cant wait for GPT to be added to a DAO. Imagine the dao setting objectives and the chat gpt generating the communication. Wage negotiations with a machine..

As long as GPT is just a fancy bullshit generator that is able to make up convincing looking facts without having to worry about repercussions I will not trust its reasoning.

> just a fancy bullshit generator

Well, it seems a good fit for DAO, then.

(With apologies to HN DAO enthusiasts.)

Re: GPT Takes the Bar Exam

#42
post #32
post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

I don't get why you're overengineering this so much - with how intelligent AIs currently are, it's much easier to just input everything you did into ChatGPT and have it answer whether you should be executed or sent to a penal colony building iPhones directly. This way we can use the same approach for AIs we already use for banning your Google, iCloud accounts and approving your mortgage and insurance claims. Much eas…

haha, it's absolutism v republicanism wearing new flesh!

Re: GPT Takes the Bar Exam

#43
post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law.

No saying it's better or worse, but maybe more suited to AI assistants?

That said, I'm sure ChatGPT can make emotional appeals.

Re: GPT Takes the Bar Exam

#44

I cant wait for GPT to be added to a DAO. Imagine the dao setting objectives and the chat gpt generating the communication. Wage negotiations with a machine..

As long as GPT is just a fancy bullshit generator that is able to make up convincing looking facts without having to worry about repercussions I will not trust its reasoning.

Good, but unfortunately that description applies to far too many elected politicians recently, so that may not be as widespread an attitude as one might hope.

Re: GPT Takes the Bar Exam

#46
post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

I think ~100 years is an overestimate by an order of magnitude. I know that people like to bash GPT-3 and ChatGPT for not being perfect. But people like to bash everything, especially things they don't understand. Technology tends to grow much closer to an exponential speed than linear. What ChatGPT can do today is truly, utterly, astonishing. That's the new baseline.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

Re: GPT Takes the Bar Exam

#47
post #43

Earlier quoted context omitted.

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law. No saying it's better or worse, but maybe more suited to AI assistants? That said, I'm sure ChatGPT can make emotional appeals.

[flagged]

Re: GPT Takes the Bar Exam

#48

Earlier quoted context omitted.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans. It's like the same thing every 5 ye…

I don’t really know to what extent AI is going to take over. But crypto is a Terri comparison, it didn’t really solve any problems. There’s nothing you can do with a blockchain that’s you couldn’t already do before. The potential with Chatgpt if it can be refined is wild.

> The potential with Chatgpt if it can be refined is wild.

potential for good or bad ? Last time they tried to use AI for police/justice it ended up exacerbating existing biases.

> There’s nothing you can do with a blockchain that’s you couldn’t already do before.

What can you do with an AI you can't with humans in a court of law ? When a human is wrong you get a single judicial error, when the only software used is wrong (bug, bias, &c.) you'll get thousands every day.

What happens if an update makes the same case get a better defence and as an outcome a lesser sentence ? do you retrial all the other similar cases ?

The only potential I see is saving money and this is certainly not on the top of the list of things I would like the justice system to focus on

Re: GPT Takes the Bar Exam

#49

Earlier quoted context omitted.

I would also note that the paper only covers the multiple choice portion of a bar exam, the Uniform Bar Examination (UBE) that has been adopted by most, but not all states. The UBE consists of a multiple choice portion (the Multistate Bar Exam, or MBE), an essay portion and a scenario-based performance test. GPT-3.5 gets a 50% success rate on a practice version of the MBE. It's impressive, but I wouldn't go so far as…

The success rate is not what’s impressive in itself. It’s the speed of improving on the test. We can expect it to get to 90% soon, and that will have real world impact (replacing lawyers for easy advice questions).

> We can expect it to get to 90% soon

I won't be surprised, but that's less than "can expect", and I disagree that this is straightforward to forecast… unless you're currently playing with another similar model that hasn't been published and which can do this. As the saying goes, "forecasting is hard, especially if it's about the future".

AI progress has always been this weird combination of two sides, one saying for every breakthrough "this is just around the corner", the other saying "this is impossible".

This even happens anachronistically, with some people convinced AI can already do things they can't, and others that they could never do things they already do.

Re: GPT Takes the Bar Exam

#50
post #39

Earlier quoted context omitted.

I don’t really know to what extent AI is going to take over. But crypto is a Terri comparison, it didn’t really solve any problems. There’s nothing you can do with a blockchain that’s you couldn’t already do before. The potential with Chatgpt if it can be refined is wild.

- I can send someone 1 million USDC for 0.000001 US cent with confirmation in Couple examples that have come up recently for me personally. I can agree that there is something comparable (i.e. international wire) but the cost and speed is in a different ballpark and international wires are not available to some parts of the world.

> I can send someone 1 million USDC for 0.000001 US cent with confirmation in This is clearly a very big problem for the majority of the world population!

It's like saying tesla solved the problem of going from 0 to 60 in 3 second in a 2t+ car, it's neato but virtually nobody had that problem, if you're not into illegal activities you will most likely never need or want to transfer 1m usd outside of a safe environment anyways

Post reply on HN