Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

91–100 of 147 posts

Re: GPT Takes the Bar Exam

#91

Post by ranting-moth »

andrewinardeer wrote: post #74
ranting-moth wrote: post #38

Earlier quoted context omitted.

I think ~100 years is an overestimate by an order of magnitude. I know that people like to bash GPT-3 and ChatGPT for not being perfect. But people like to bash everything, especially things they don't understand. Technology tends to grow much closer to an exponential speed than linear. What ChatGPT can do today is truly, utterly, astonishing. That's the new baseline.

The release of GPT-3 I believe will be looked back along the lines of a monumental advancement in line with the Trinity nuclear test. Legislation and policy has mainly kept nuclear weapons under check, however, nuclear technology does provide us with a reasonably clean energy which is beneficial for the masses. Perhaps regulation at some point will need to be crafted to ensure that only crippled AI or AI within a def…

With nuclear weapons, we can very much see and feel the power and destruction they can unleash. A proof is trivial, dig it in the ground and hold your ears when it goes off.

With AI we have to debate how destructive it is. The debate itself has been weaponized. You'll be debating against deniers and AI using all the logical fallacies and information overload it can feed you.

I, being the greatest armchair expert in human behavior I know, say it will destroy democracy as we know it.

Re: GPT Takes the Bar Exam

#92

Post by lordnacho »

BoxOfRain wrote: post #63
teekert wrote: post #43

Earlier quoted context omitted.

Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law. No saying it's better or worse, but maybe more suited to AI assistants? That said, I'm sure ChatGPT can make emotional appeals.

I quite like jury trials personally, they're a good mechanism as an authentic people's check on officialdom in my opinion. A similar concept could also serve as a good check on AI by humans I think!

Already exists in those systems, there's "lay judges" who sit with the judge and have stuff explained to them.

IMO the 12 jurors thing is not ideal. Too many people, and cases can get technical.

Re: GPT Takes the Bar Exam

#93

Post by lordnacho »

vasco wrote: post #40
lordnacho wrote: post #28

Earlier quoted context omitted.

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

On legitimacy just because a human did it, I think you are right, and I think the rate of appeal for judgements done by an AI would be higher than judgements done by humans, but if you can even have a 10% "AI-only" close rate for cases, how much time and money are you going to save that can be invested in better justice for the "human required" cases? Over time though, 150, 200, 250 years from now, there's no other w…

Well I'm on the tech side so I agree with you, but it's a massive social change. To people who don't understand tech it's like introducing magic, something they clearly are going to fight against.

But yeah there must be low hanging fruit there that in theory could be picked.

Or maybe get that judge who codes to help educate some of his colleagues.

Re: GPT Takes the Bar Exam

#94

Post by BlueTemplar »

teekert wrote: post #43
lordnacho wrote: post #28

Earlier quoted context omitted.

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law. No saying it's better or worse, but maybe more suited to AI assistants? That said, I'm sure ChatGPT can make emotional appeals.

Sure, but until computers get citizenship (and maybe even then), it will be illegal to use non-algorithmic programs for decision-making, because it runs afoul of the laws that mandate that you must be able to explain how the program helped you to came to a decision (tracing the algorithm step by step) in language that your average person can understand.

Re: GPT Takes the Bar Exam

#95

Post by notahacker »

candiodari wrote: post #79
newaccount74 wrote: post #59

Earlier quoted context omitted.

Aside from games, AI can do all these things only with human assistance. It's like saying "Excel can do your taxes". Sure, it can, if you put the correct numbers and formulas into the cells first.

You forget that a lot of these are moving the goal posts. AI beat the top players at all those games, but new rules were introduced until AI researchers mostly abandoned those efforts. For instance, the starcraft AI found a really good build order that let it quickly build "Stalkers", a versatile but relatively weak unit. I will note that that build order had several factors that humans found very surprising (consist…

Citing software like autoland which slavishly applies the rules humans have programmed into it as a reason why neural networks should be considered better than humans at solving general intelligence problems is moving the goalposts even further the other way...

Re: GPT Takes the Bar Exam

#96

Post by ctrwu2843 »

ranting-moth wrote: post #86
lm28469 wrote: post #46

Earlier quoted context omitted.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

It can be tricky to grasp exponential growth. Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year. Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.

Useful analogy but what makes you think what we're dealing with is exponential ?

Human intelligence isn't growing exponentially and moores law is said to be breaking down.

Re: GPT Takes the Bar Exam

#97

Post by BlueTemplar »

SturgeonsLaw wrote: post #45
qikInNdOutReply wrote: post #19

I cant wait for GPT to be added to a DAO. Imagine the dao setting objectives and the chat gpt generating the communication. Wage negotiations with a machine..

What if the objectives involve increasing paperclip production...

Hardly different from any public company then ?

Re: GPT Takes the Bar Exam

#98

Post by Maken »

alfiedotwtf wrote: post #70
vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

> Potentially in ~100 years you could have two AIs sort out the cases. Marty: Within two hours of his arrest, Martin McFly Jr was tried, convicted, and sentenced to 15 years in the state penitentiary. Marty: Within two hours? Doc: The justice system works swiftly now that they've abolished lawyers. Marty: This is heavy

Funny how that has became real way faster than the hoverboards.

Re: GPT Takes the Bar Exam

#99

Post by Maken »

mrweasel wrote: post #66

One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused th…

I would argue that GPT3 is way better at the last task, while also baking it with with a dozen or so made-up previous rulings that nobody will bother to check.

Re: GPT Takes the Bar Exam

#100

Post by cudgy »

lordnacho wrote: post #92
BoxOfRain wrote: post #63

Earlier quoted context omitted.

I quite like jury trials personally, they're a good mechanism as an authentic people's check on officialdom in my opinion. A similar concept could also serve as a good check on AI by humans I think!

Already exists in those systems, there's "lay judges" who sit with the judge and have stuff explained to them. IMO the 12 jurors thing is not ideal. Too many people, and cases can get technical.

One major flaw in jurist system is the deliberation phase where jurists argue with each other and sometimes bully / browbeat / shame one another into voting a certain way. If jurors could simply vote after hearing the case with no group dynamics, it would be more objective.
Post reply on HN