Sounds like they didn't have access to GPT-4, but "Based on anecdotal evidence"... they still predict this.
Sounds like they're writing science fiction then.
GPT-4 could pass bar exam, AI researchers say
131–140 of 167 posts
Re: GPT-4 could pass bar exam, AI researchers say
#132Earlier quoted context omitted.
No, we were enamored with the idea that intelligence was well distributed between people, as if following Descartes' massive incipit "Good sense must be the best distributed thing in the world, given that nobody seems to be asking for more". Inability to recognize intelligence is and will be devastating.
> Inability to recognize intelligence is and will be devastating. It's a pop-culture quote from a movie that was no masterpiece, I know, but "I, Robot" presented in two sentences an argument for having more sober expectations on what machine intelligence could be capable of, and of our own > Detective Del Spooner: "Can a robot write a symphony? Can a robot turn a… canvas into a beautiful masterpiece?" > Sonny: "Can y…
-- Pseudo Detective Del Spooner: "Can a robot lift that object?"
-- Pseudo Sonny: "Can you?"
-- Pseudo Detective Del Spooner: "Ha-ha. So what the #!@! is a robot doing there, not doing what is required? I cannot, and I do not stand there clueless"
What is being engineered, toys for the satisfaction of some idle decadent sympathy urge? Have cats disappeared from the world?
> We're discrediting
We are shocked that an overly large number of individuals expect stones to bleed, and intelligence to pour out of machines that do not have intelligence coded inside, and that instead have unintelligence - acritical repetition - coded inside.
> what does it say about Average Joe
That he should catch up with his nature, if he shows the critical capacities of a simulacrum that has none.
> not at the level
No, no, no: it is not a matter of quantity but of quality: if you do not implement it or its origin, it will not be there.
> [Asimov]
Asimov is relevant. For example, I remember his idea that the State comes from Agriculture (~10000 BC), in the need to plan irrigation, or that the Abel vs Cain story could be a parallel of the political consequences of lands denied to pastors. Now: those seem to be good ideas, and their production can be an interesting goal. But there is something /before/ "creativity", or "advanced pattern recognition": it is /intelligence/, meaning that Asimov, after having spawned those hypoteses, has /vetted/ them as a required duly activity before confirming them in his set of founded hypotheses. You have to use intelligence, you have to have intelligence, and if you want to do AGI, you have to implement intelligence!!!
Re: GPT-4 could pass bar exam, AI researchers say
#133IF it could (I wouldn't know one way or the other), I'd consider that a damning indictment of the Bar Exam failing to test for sentience, rather than evidence of GPT-4 having attained the same.
Bar exam is not a test of sentience but of the ability to recall, interpret, and apply the law. Because law is an entirely textual thing, I would expect GPT to be exceedingly well suited for it. I've said for a long time that most doctors and lawyers are just databases with quick and imperfect retrieval.
And so as AI advances, the goal posts for what counts as intelligence are moved yet again.
Re: GPT-4 could pass bar exam, AI researchers say
#134I feel like I can now see the event horizon of commoditized intelligence. No idea what society (is "society" even the right word? Who knows) is going to look like on the other side of it, but it is going to be wildly different. Perhaps a brief period where everyone is using an AI to do their job, uh, I mean, assist their work, but beyond that it's unknowable. Moreover, this looks like it is going to be happening soon…
Kurzweil's "Singularity" is upon us, but he's now being cagey about it. He says it's still years away. His interview with Lex Fridman[0] was pretty tame - I didn't learn much new from it. Kurzweil deflected the Singularity segment to be a discussion about the history of computer power. Remember that Kurzweil is Director of Engineering[1] at Google, with the mandate to "bring natural language understanding to Google"[…
>It's hard to believe that OpenAI is actually ahead of Google.
Are Google's LLMs available for us to test out? From what I've gleaned, they've locked them up - I'd love to compare GPT vs Google's LLMs.
Re: GPT-4 could pass bar exam, AI researchers say
#135Earlier quoted context omitted.
> Inability to recognize intelligence is and will be devastating. It's a pop-culture quote from a movie that was no masterpiece, I know, but "I, Robot" presented in two sentences an argument for having more sober expectations on what machine intelligence could be capable of, and of our own > Detective Del Spooner: "Can a robot write a symphony? Can a robot turn a… canvas into a beautiful masterpiece?" > Sonny: "Can y…
-- Pseudo Detective Del Spooner: "Can a robot lift that object?" -- Pseudo Sonny: "Can you?" -- Pseudo Detective Del Spooner: "Ha-ha. So what the #!@! is a robot doing there, not doing what is required? I cannot, and I do not stand there clueless" What is being engineered, toys for the satisfaction of some idle decadent sympathy urge? Have cats disappeared from the world? > We're discrediting We are shocked that an o…
And many among the Average Joes will not measure up to this ideal man of science and art. In truth you'll see that I'm not advocating for the rights of as-of-yet non-sapient computer programs, but to think of what this means about people.
If we're setting the bar of personhood or dignity to being exceptional researchers and engineers, it doesn't bode well for the masses that aren't and won't be. Maybe this will result in a society of leisure where everyone can be that! I wouldn't bet on it, there's already more PhDs in the sciences and humanities than society can fit, and humans may just not work that way.
You're already dismissing concerns about the welfare of the merely average, for being unfit when competing with the Machine Learning models we may have in the near future.
Re: GPT-4 could pass bar exam, AI researchers say
#136Earlier quoted context omitted.
> Inability to recognize intelligence is and will be devastating. It's a pop-culture quote from a movie that was no masterpiece, I know, but "I, Robot" presented in two sentences an argument for having more sober expectations on what machine intelligence could be capable of, and of our own > Detective Del Spooner: "Can a robot write a symphony? Can a robot turn a… canvas into a beautiful masterpiece?" > Sonny: "Can y…
-- Pseudo Detective Del Spooner: "Can a robot lift that object?" -- Pseudo Sonny: "Can you?" -- Pseudo Detective Del Spooner: "Ha-ha. So what the #!@! is a robot doing there, not doing what is required? I cannot, and I do not stand there clueless" What is being engineered, toys for the satisfaction of some idle decadent sympathy urge? Have cats disappeared from the world? > We're discrediting We are shocked that an o…
> if you do not implement it or its origin, it will not be there.
Who created ours?
And if it's god (which god?), who created theirs?
Re: GPT-4 could pass bar exam, AI researchers say
#137IF it could (I wouldn't know one way or the other), I'd consider that a damning indictment of the Bar Exam failing to test for sentience, rather than evidence of GPT-4 having attained the same.
Right. Would we be impressed if a layman could pass the bar, given infinite time and access to the entire Internet (including the copious amount of bar exam study guides and worked example problems)? If not why are we impressed that a language model trained on that data can? Meanwhile when I ask ChatGPT which of six numbers are odd, it confidently reports a mix of even numbers, odd numbers, and letters. This is a fun…
Maybe it understood "odd" in a different sense of the word? As in "unusual", whatever the "unusual" is for an AI...
Re: GPT-4 could pass bar exam, AI researchers say
#138I want to perform some research of my own on which exams chatGPT can and can't do. It's multilingual, so can people from outside the UK (I already know where to get those) point me at some example exams and marking schemes? Any level, not just top. Currently have Polish school maths: https://news.ycombinator.com/item?id=34205732
Ireland Leaving Cert (17-18 year olds) https://theleavingcert.com/exam-papers/
Thanks :)
Re: GPT-4 could pass bar exam, AI researchers say
#139Earlier quoted context omitted.
-- Pseudo Detective Del Spooner: "Can a robot lift that object?" -- Pseudo Sonny: "Can you?" -- Pseudo Detective Del Spooner: "Ha-ha. So what the #!@! is a robot doing there, not doing what is required? I cannot, and I do not stand there clueless" What is being engineered, toys for the satisfaction of some idle decadent sympathy urge? Have cats disappeared from the world? > We're discrediting We are shocked that an o…
And many among the Average Joes will not measure up to this ideal man of science and art. In truth you'll see that I'm not advocating for the rights of as-of-yet non-sapient computer programs, but to think of what this means about people . If we're setting the bar of personhood or dignity to being exceptional researchers and engineers, it doesn't bode well for the masses that aren't and won't be. Maybe this will resu…
> You're already dismissing concerns about the welfare of the merely average
This writer individually: no, not literally «dismissing», it is just that I could not grasp precisely your point in this specific area. And I would say, as I wrote just earlier, «Inability to recognize intelligence is and will be devastating»: it already happens that an inability to discriminate ("It takes it to see it") will hide from the sight to some manager the critical risks that the underdeveloped sense of some workers will pose, and such risk will increase when they will have to compete with even riskier and less endowed entities that may be confused for acceptable - since this is what has been showing even here in the past times.
This issue comes from a devaluation of actual intelligence.
> If we're setting the bar of personhood or dignity to being exceptional researchers and engineers
Not really. Look, a few weeks ago this HN member had some heavy exchange with others to which it was said "there is no intelligence if there is no critical thinking", and some arrived to call that position "delirious". Now a rebuttal would have been, "Ask your grandmother". Because there is a "high culture", that of the Professor and the Professional, and "low culture", that of the Teacher and of the Relative, it does not take the former to have good judgement - the latter suffices plentifully, when not polluted.
So, you do not need to have the bar set to «exceptional researchers and engineers» - just a good grandmother. Who could have been an «exceptional researchers and engineer», in case, if life so determined - because "the requirements were there", available.
Re: GPT-4 could pass bar exam, AI researchers say
#140Earlier quoted context omitted.
Kurzweil's "Singularity" is upon us, but he's now being cagey about it. He says it's still years away. His interview with Lex Fridman[0] was pretty tame - I didn't learn much new from it. Kurzweil deflected the Singularity segment to be a discussion about the history of computer power. Remember that Kurzweil is Director of Engineering[1] at Google, with the mandate to "bring natural language understanding to Google"[…
>It's hard to believe that OpenAI is actually ahead of Google. Are Google's LLMs available for us to test out? From what I've gleaned, they've locked them up - I'd love to compare GPT vs Google's LLMs.
Google really doesn't share much publicly except for papers and preset tech demos.
However we know they have been working on AI longer than OpenAI, with better datasets than anyone, with top shelf talent, essentially infinite funding, custom hardware, and what we do see publicly is incredible.
It's a pretty safe bet that Google is ahead of the pack, perhaps even with some distance, but it's not yet clear what they intend to do long term with their projects. What is clear is that they don't want or need the public playing with it.