Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4

openai.com

Post reply on HN
Print view View original

421–430 of 1001 posts

Re: GPT-4

#421

Post by new2yc »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

#unpopularOpinion GPT-4 is not as strong as "we" anticipated, it was just the hype

Re: GPT-4

#423

Post by Analemma_ »

busyant wrote: post #339

What I don't understand is how GPT-4 is able to do reasonably well on tests like the AMC12: Many of the AMC12 questions require a number of logical/deductive steps. If GPT-4 is simply trained on a large corpus of text, how is it able to do this? Does this imply that there is some emergent deductive ability that you get simply by learning "language?" Or am I missing something? Obviously, I'm assuming that GPT-4 wasn't…

It's totally possible: Daniel Dennett's theory of sentient consciousness-- specifically, what we have that animals do not-- is that it is "ignited" by language acquisition. It's within the realm of possibility that LLMs provide empirical proof or disproof of this hypothesis.

Re: GPT-4

#424

Post by kirill5pol »

unit_circle wrote: post #384
nealabq wrote: post #227

Test taking will change. In the future I could see the student engaging in a conversation with an AI and the AI producing an evaluation. This conversation may be focused on a single subject, or more likely range over many fields and ideas. And may stretch out over months. Eventually teaching and scoring could also be integrated as the AI becomes a life-long tutor. Even in a future where human testing/learning is no l…

Immediately I'm very much looking forward to a day where language learning is like this. No Duolingo gamification nonsense... I want something that remembers what words I know, what words I kinda know and what I should know next and has an ongoing conversation with me. I think this will totally change the way we educate and test. As someone for whom the education system really didn't serve well, I am very excited.

This is what I’m actually working on!

One major problem with LLMs is that they don’t have a long term way of figuring out what your “knowledge space” is so no matter how much good the LLM is at explaining, it won’t be able to give you custom explanations without a model of the human’s knowledge to guide the teaching (basically giving the LLM the knowledge of the learner to guide it)

Re: GPT-4

#425

Post by sva_ »

From the paper:

> Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.

"Open"AI, ladies and gentlemen

Re: GPT-4

#426

Post by groestl »

rychco wrote: post #367
nealabq wrote: post #227

Test taking will change. In the future I could see the student engaging in a conversation with an AI and the AI producing an evaluation. This conversation may be focused on a single subject, or more likely range over many fields and ideas. And may stretch out over months. Eventually teaching and scoring could also be integrated as the AI becomes a life-long tutor. Even in a future where human testing/learning is no l…

Completely agree. I've been frequently using ChatGPT to learn new things in my free time. I realize that there's a huge amount of downplay regarding the accuracy of responses, but unless you're asking specifically for verified references or quotes, it does remarkably well in smoothly guiding you towards new keywords/concepts/ideas. Treat it like a map, rather than a full-self-driving tesla, and it's tremendously usef…

True in some regard, but for me, it also just invented words / phrases that nobody else uses. So "treat with caution" is definitely appropriate.

Re: GPT-4

#427

Post by wpietri »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

I'm pretty sanguine. Back in high school, I spent a lot of time with two sorts of people: the ultra-nerdy and people who also came from chaotic backgrounds. One of my friends in the latter group was incredibly bright; she went on to become a lawyer. But she would sometimes despair of our very academic friends and their ability to function in the world, describing them as "book smart but not street smart".

I think the GPT things are a much magnified version of that. For a long time, we got to use skill with text as a proxy for other skills. It was never perfect; we've always had bullshitters and frauds and the extremely glib. Heck, before I even hit puberty I read a lot of dirty joke books, so I could make people laugh with all sorts of jokes that I fundamentally did not understand.

LLMs have now absolutely wrecked that proxy. We've created the world's most advanced bullshitters, able to talk persuasively about things that they cannot do and do not and never will understand. There will be a period of chaos as we learn new ways to take the measure of people. But that's good, in that it's now much easier to see that those old measures were always flawed.

Re: GPT-4

#429

Post by gniv »

sebzim4500 wrote: post #372
Imnimo wrote: post #218

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…

I think you could distinguish those two cases by asking the question in a way that did not sound so similar to an existing (slightly different) problem. What if you refer to the items with letters rather than words and replace the river with something else entirely?

Good observation. It seems that it comes up with the right solution:

"Suppose I have an object of type C, an object of type G and an object of type L, and I need to get them across a river. I have a boat that can only carry myself and a single other object. I am not allowed to leave the objects of type C and L alone together, and I am not allowed to leave the objects of type L and G alone together. How can I safely get all three across?"

"This is a classic puzzle known as the "river crossing problem". Here's how you can safely get all three objects across the river:

First, take the object of type L across the river. Return alone to the original side of the river. Take the object of type C across the river. Return with the object of type L to the original side of the river. Take the object of type G across the river. Return alone to the original side of the river. Finally, take the object of type L across the river again. By following these steps, you can safely transport all three objects across the river without violating the restrictions given in the problem. Note that you have to start with the object of type L because it cannot be left alone with either of the other two objects. Once L is transported to the other side, you can use it to ensure that C and G are never left alone together."

Post reply on HN