Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

31–40 of 147 posts

Re: GPT Takes the Bar Exam

#31

Post by blitzar »

heywhatupboys wrote: post #20
vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

While HN often have over the top, "computers will solve everything" takes, your comment is why I keep coming here. What a great fantasy, which at least is a good discussion. We see this already now i high frequency trading, where companies essentially battle trading bots against each other

> companies essentially battle trading bots against each other

two computers playing heads or tails is about the extent of it

Re: GPT Takes the Bar Exam

#32

Post by izacus »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

I don't get why you're overengineering this so much - with how intelligent AIs currently are, it's much easier to just input everything you did into ChatGPT and have it answer whether you should be executed or sent to a penal colony building iPhones directly. This way we can use the same approach for AIs we already use for banning your Google, iCloud accounts and approving your mortgage and insurance claims.

Much easier and it'll bring a new era of effective policing and prosperity for the corporations.

Re: GPT Takes the Bar Exam

#33

Post by Semaphor »

lordnacho wrote: post #28
vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

They said in ~100 years. Think back 100 years ago and how different society was. The war to end all wars recently ended.

Re: GPT Takes the Bar Exam

#34

Post by izacus »

fullstackchris wrote: post #12
smrtinsert wrote: post #6

The next ten years will be very interesting.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans. It's like the same thing every 5 ye…

Oh it won't take all the jobs, it'll just make your life and your job a bigger hell than they already are.

There's so many creative ways in how an arbitrary and empathyless AI can make someone's profit a bit bigger and your life a bit worse :)

We can start with practically unlimited amount of personalised advertising spam that can be directed at you at any point. Articles, phone calls, billboards, conversations with your partners... so many chances to cheaply increase profits and engagements.

The AI takeover isn't going to be evil in Skynet sense... it'll be a constant stream of lowkey bullshit on every aspect of your life you won't be able to fight because "computer said so". Think on how Google randomly bans people from YouTube/Play Store/Drive over AI and put this into every aspect of your life.

Re: GPT Takes the Bar Exam

#35

Post by blitzar »

PartiallyTyped wrote: post #15
fullstackchris wrote: post #13

Earlier quoted context omitted.

So I'm going to put a laptop on the stand when I go to trial? Give me a break.

Nowhere near what I said. Companies have hordes of lawyers scouring through contracts, e.g. think associates and paralegals (which are not technically lawyers). Expect that a non trivial portion will get axed.

One day in the future we will look back at the multi volume contracts (and legislation) we were having written to satisfy our delicate egos and laugh at our past self.

Re: GPT Takes the Bar Exam

#36

Post by angusturner »

fullstackchris wrote: post #12
smrtinsert wrote: post #6

The next ten years will be very interesting.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans. It's like the same thing every 5 ye…

I agree that adoption tends to be slow and there’s a lot of legacy systems that will need humans to look after them for a good while.

But I think you are selling GPT a bit short. Partly because transformers are incredibly general architectures, which make few assumptions about the domain. They are basically SOTA on every domain for which learning-based methods are applicable.

And secondly because ChatGPT can perform “in-context learning”. That is to say, purely by scaling next-token prediction, there is an emergent phenomena where GPT gains the ability to learn from the prompt it is given, and solve new tasks outside of the training distribution.

Hattie Zhou has a cool paper showing how you can teach GPT addition by prompting - a toy example, but it signifies something profound in my opinion. In the last 6-months I have shifted my own view from “scaling is a waste of compute” to a recognition that LLMs are a serious leap forward.

For something very hyped, I think it’s still under hyped and under appreciated.

Re: GPT Takes the Bar Exam

#37

Post by meindnoch »

PartiallyTyped wrote: post #11
fullstackchris wrote: post #8

Earlier quoted context omitted.

Did you look at the results? All models that were tested failed significantly (10%+) to acheive the passing range.

Project to the next 10 years given the trends.

The beginning of an exponential curve looks the same as the beginning of an S-curve.

Re: GPT Takes the Bar Exam

#38

Post by ranting-moth »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

I think ~100 years is an overestimate by an order of magnitude.

I know that people like to bash GPT-3 and ChatGPT for not being perfect. But people like to bash everything, especially things they don't understand.

Technology tends to grow much closer to an exponential speed than linear. What ChatGPT can do today is truly, utterly, astonishing. That's the new baseline.

Re: GPT Takes the Bar Exam

#39

Post by lui8906 »

hilbertseries wrote: post #30
fullstackchris wrote: post #12

Earlier quoted context omitted.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans. It's like the same thing every 5 ye…

I don’t really know to what extent AI is going to take over. But crypto is a Terri comparison, it didn’t really solve any problems. There’s nothing you can do with a blockchain that’s you couldn’t already do before. The potential with Chatgpt if it can be refined is wild.

- I can send someone 1 million USDC for 0.000001 US cent with confirmation in Couple examples that have come up recently for me personally. I can agree that there is something comparable (i.e. international wire) but the cost and speed is in a different ballpark and international wires are not available to some parts of the world.

Re: GPT Takes the Bar Exam

#40

Post by vasco »

lordnacho wrote: post #28
vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

On legitimacy just because a human did it, I think you are right, and I think the rate of appeal for judgements done by an AI would be higher than judgements done by humans, but if you can even have a 10% "AI-only" close rate for cases, how much time and money are you going to save that can be invested in better justice for the "human required" cases? Over time though, 150, 200, 250 years from now, there's no other way around it, I think.
Post reply on HN