Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

121–130 of 147 posts

Re: GPT Takes the Bar Exam

#121

Post by lm28469 »

candiodari wrote: post #79
newaccount74 wrote: post #59

Earlier quoted context omitted.

Aside from games, AI can do all these things only with human assistance. It's like saying "Excel can do your taxes". Sure, it can, if you put the correct numbers and formulas into the cells first.

You forget that a lot of these are moving the goal posts. AI beat the top players at all those games, but new rules were introduced until AI researchers mostly abandoned those efforts. For instance, the starcraft AI found a really good build order that let it quickly build "Stalkers", a versatile but relatively weak unit. I will note that that build order had several factors that humans found very surprising (consist…

> but we're pretty far along.

Go out literally anywhere outside of a major western city and you'll witness just how incredibly wrong you are about this statement.

> and Waymo's "chauffeur" is apparently much better than that

You forgot a major detail, on clear straight and wide californian roads with perfect conditions. Put them in a historical european city or in inda's traffic hell and witness the mayhem

Even waymo's previous ceo was full of doubt: https://www.cnet.com/roadshow/news/alphabet-google-waymo-ceo...

We're still so far away from autonomous vehicles, to the point most still doubt we'll get there one day without rebuilding our entire infrastructure to accommodate it

Teslas can't even drive in snowy or foggy condition, that removes half the planet for 1/3rd of the year, but of course for a FAANG engineer living in the bay that's not a problem, for them we already have fully autonomous cars, it's just an extremely tiny subset of it.

I spent my holidays in a Eastern european country, I'd pay big money to film Musk in a fully autonomous tesla there, it would probably be the funniest shit ever, I bet it would crash in the first mile

Re: GPT Takes the Bar Exam

#122

Post by lm28469 »

ranting-moth wrote: post #86
lm28469 wrote: post #46

Earlier quoted context omitted.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

It can be tricky to grasp exponential growth. Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year. Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.

> It can be tricky to grasp exponential growth.

It can also be tricky to determine if something is growing exponentially, or if you're full of wishful thinking.

We landed on the moon in 69, basically haven't done anything since then.

All our passenger planes are shrink down or blown up version of a boing 707 from 1957.

You car isn't much different than cars in the 70s, and it certainly isn't flying as we were expecting them to be back then.

Musk promised fully autonomous cars "in two years" in 2012.

I'd say the pool isn't being filled any faster than it was back then, if anything it looks like someone is pumping water out of it, straight in the sewers

Re: GPT Takes the Bar Exam

#123

Post by blitzar »

heywhatupboys wrote: post #104
blitzar wrote: post #31

Earlier quoted context omitted.

> companies essentially battle trading bots against each other two computers playing heads or tails is about the extent of it

surely they are making the traders money though. The smart thing about the free markets is that companies are generally not pursuing things that are not of benefit to them

> companies essentially battle trading bots against each other

Bot A is trading against Bot B.

> surely they are making the traders money though

It is not possible in a trading battle, exclusively between two parties for them to both make money.

Re: GPT Takes the Bar Exam

#124

Post by pfsalter »

ben_w wrote: post #49
xiphias2 wrote: post #17

Earlier quoted context omitted.

The success rate is not what’s impressive in itself. It’s the speed of improving on the test. We can expect it to get to 90% soon, and that will have real world impact (replacing lawyers for easy advice questions).

> We can expect it to get to 90% soon I won't be surprised , but that's less than "can expect", and I disagree that this is straightforward to forecast… unless you're currently playing with another similar model that hasn't been published and which can do this. As the saying goes, "forecasting is hard, especially if it's about the future". AI progress has always been this weird combination of two sides, one saying fo…

Completely agree with this. There's a short sentence in the abstract:

> hyperparameter optimization and prompt engineering

Prompt engineering seems a lot like "tweaking the question format until the AI gets the answer" which is the first lesson in ANNs; don't train on your test set. If you can't put the actual question with the only context being "this question concerns US law" then there's an awful lot of reasoning and thinking that the human is doing which the AI cannot.

Let's not have another Moore's law fallacy, it's not reasonable to extrapolate progress based on existing results. It's like building a car which can go at 200mph then saying 300mph is just around the corner.

Re: GPT Takes the Bar Exam

#125

Post by random_upvoter »

ranting-moth wrote: post #86
lm28469 wrote: post #46

Earlier quoted context omitted.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

It can be tricky to grasp exponential growth. Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year. Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.

Have you considered that GPT may not be at 50% but maybe still needs an exponential improvement to achieve human-like understanding?

Re: GPT Takes the Bar Exam

#126

Post by melagonster »

lm28469 wrote: post #48
hilbertseries wrote: post #30

Earlier quoted context omitted.

I don’t really know to what extent AI is going to take over. But crypto is a Terri comparison, it didn’t really solve any problems. There’s nothing you can do with a blockchain that’s you couldn’t already do before. The potential with Chatgpt if it can be refined is wild.

> The potential with Chatgpt if it can be refined is wild. potential for good or bad ? Last time they tried to use AI for police/justice it ended up exacerbating existing biases. > There’s nothing you can do with a blockchain that’s you couldn’t already do before. What can you do with an AI you can't with humans in a court of law ? When a human is wrong you get a single judicial error, when the only software used is…

did not crime prediction system let them arrest more African American?

Re: GPT Takes the Bar Exam

#127

Post by carapace »

I think the machines should do the scut work to free up humans to do the really important human stuff, like parenting, teaching, and, yeah, lawyering and judging too.

Another angle: if there is some legal task that's simple enough for machines to do reliably then (almost by definition) that task will turn out to be pointless bureaucratic busywork, eh?

Last but not least, who gets to keep lawyer-bot's pay?

- - - -

edit to add a link to James Mickens' USENIX Security Keynote address: "Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?"

https://www.youtube.com/watch?v=ajGX7odA87k

If you think AI lawyers and judges are a good idea please watch it.

Re: GPT Takes the Bar Exam

#128

Post by imtringued »

ranting-moth wrote: post #107
ctrwu2843 wrote: post #96

Earlier quoted context omitted.

Useful analogy but what makes you think what we're dealing with is exponential ? Human intelligence isn't growing exponentially and moores law is said to be breaking down.

https://www.kurzweilai.net/the-law-of-accelerating-returns

Where is my human brain capability for $1000 by 2023?

Re: GPT Takes the Bar Exam

#129

Post by imtringued »

ilaksh wrote: post #61
lui8906 wrote: post #39

Earlier quoted context omitted.

- I can send someone 1 million USDC for 0.000001 US cent with confirmation in Couple examples that have come up recently for me personally. I can agree that there is something comparable (i.e. international wire) but the cost and speed is in a different ballpark and international wires are not available to some parts of the world.

Which chain and service is that?

Doesn't matter, cryptocurrency won apparently.

Re: GPT Takes the Bar Exam

#130

Post by nradov »

random_upvoter wrote: post #125
ranting-moth wrote: post #86

Earlier quoted context omitted.

It can be tricky to grasp exponential growth. Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year. Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.

Have you considered that GPT may not be at 50% but maybe still needs an exponential improvement to achieve human-like understanding?

An exponential improvement in GPT along the same axis will still not achieve human-like understanding. It's a very useful tool with the potential to augment or even replace human knowledge workers in many fields. But to achieve true AGI we're going to need to go in a completely different direction.
Post reply on HN