let's just hope a student doesn't ask an arithmetic problem!
There's other models that are doing this at a much better rate than GPT, so I don't think this is an unsolvable problem.
Khan Academy integrates GPT-4 as every student’s customized tutor
161–170 of 211 posts
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#162Earlier quoted context omitted.
Human teachers make mistakes too - perhaps even at higher rates than GPT in some cases. Instead of isolating a single factor of hallucinations I think you have to also consider the cost and experience quality and make a more holistic judgement on whether these models are good or bad.
Yes, but human teachers can eventually come to understand why they made a mistake. And they probably are more able to communicate how confident they are in their answers. I think that's because human teachers have a mental model that they're working from. On the other hand, we have no idea if transformer models are working from anything like a "mental model." Just because humans make mistakes (or even as many or more…
>Just because humans make mistakes (or even as many or more mistakes than machines) doesn't mean the nature or consequences of the mistakes are the same.
True, but this shouldn't be hard to measure. It's not like we will go from these AIs will go from teaching no one to replacing all teachers overnight.
Give a random sample of classes access to a GPT-n powered tutor, and see if they do better or worse over the next few years.
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#163At this point I won't be surprised if over the next few weeks and months hacker news is going to be full {insert org here} integrates GPT-4.
I mean honestly, if I took the corpus of our companies support data and used it as a training set for our first line support, I believe that GPT could answer the questions better than they do first shot.
No idea what the cost difference would be, but for many businesses I could see the AI response be a costs savings.
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#164Highly recommend the Neal Stephenson book The Diamond Age: Or, A Young Lady's Illustrated Primer , for an interesting exploration of a custom AI tutor for each student. In that book, students have a "magic book" that teaches them lessons in a story form while encouraging certain life paths. It's pretty fascinating to consider the implications and whether that's something we'll want, bc it may soon be possible.
(I havent read The Diamond Age)
The eventuality I see is everybody talking to each other through LLM's, and the world ending up in some kind of tower of babel situation. You wear an AR/MR headset, and all conversation is dual translated, with some kind of bytecode intermediary. Each persons language evolves to be incompatible with anyone else's, and only your LLM can understand you. The power goes out, and nobody can talk to anybody.
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#165Earlier quoted context omitted.
My thoughts exactly. What is the baseline here? Teachers already give biased and/or inaccurate information at a rate much higher than I have seen in ChatGPT.
Is there a name for this fallacy (is it a fallacy?). I feel like we see the same issue with self driving cars crash rates. People are hesitant to adopt because they aren't perfect, but ignore that it's better than the status quo.
Colloquially: "Don't let perfection stand in the way of greatness."
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#166Earlier quoted context omitted.
My thoughts exactly. What is the baseline here? Teachers already give biased and/or inaccurate information at a rate much higher than I have seen in ChatGPT.
Is there a name for this fallacy (is it a fallacy?). I feel like we see the same issue with self driving cars crash rates. People are hesitant to adopt because they aren't perfect, but ignore that it's better than the status quo.
I'll don't think it's a fallacy. People are more predictable and can be accountable for their actions, and even admit ignorance, eg answering "I don't know ask someone else".
It's worth discussing if we want AIs replacing them - maybe the scalability is better, but these are usually worse.
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#167Am I the only one somewhat alarmed that we are choosing to rely on AIs that are vulnerable to hallucinations so much? Personally I rather have a more limited but reliable tool than a more powerful but unreliable one.
That is a feature, not a bug. It's an efficient way to brainwash a population.
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#168A lot of what GPT-4 can return is ideas based on assumptions and beliefs. Who chooses what of these to train the system? Those subtle ideas will then lead students who are mostly children.
And that is any different from the school books we currently use how?
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#169Earlier quoted context omitted.
Yes, but human teachers can eventually come to understand why they made a mistake. And they probably are more able to communicate how confident they are in their answers. I think that's because human teachers have a mental model that they're working from. On the other hand, we have no idea if transformer models are working from anything like a "mental model." Just because humans make mistakes (or even as many or more…
>Just because humans make mistakes (or even as many or more mistakes than machines) doesn't mean the nature or consequences of the mistakes are the same. True, but this shouldn't be hard to measure. It's not like we will go from these AIs will go from teaching no one to replacing all teachers overnight. Give a random sample of classes access to a GPT-n powered tutor, and see if they do better or worse over the next f…
> this shouldn't be hard to measure
> see if they do better or worse over the next few years
This doesn't strike me as entirely ethical. Also, it may be hard to gauge the long term effects of subtle misinformation generated by these models.
Re: Khan Academy integrates GPT-4 as every student’s customized tutor
#170Am I the only one somewhat alarmed that we are choosing to rely on AIs that are vulnerable to hallucinations so much? Personally I rather have a more limited but reliable tool than a more powerful but unreliable one.
Are the tolerances in education really that tight?
How many terrible teachers are allowed to continue to teach after decades of disastrous results?
How many ideologues with no interest in teaching, but every interest in indoctrinating young minds, are tolerated because the alternative is "no teacher" and the class wouldn't run?
How many teachers who would fail state exams, teach, despite relying on answer sheets to be "competent"?
I agree with you that on the top end of education, this is no replacement and at best a supplementary tool. For the poor kid in a bad neighbourhood whose teacher is more interested in "de-colonizing" mathematics than teaching mathematics, this is a Godsend.