Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

intextbooks.science.uu.nl

Post reply on HN
Print view View original

21–30 of 126 posts

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#23

Post by syou1024 »

rusbus wrote: post #6

This is exciting because the effect size is so large. But as the author's acknowledged, selection bias is nearly impossible to control for in this non-randomized study: > and lacks randomized controls. Self-selection is the central threat: students who complete more quizzes may be more motivated or higher-performing generally But this is still a strong result. I'm excited to see more in this space.

[flagged]

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#24

Post by terribleperson »

baq wrote: post #11

I'm on record saying that a system like this with some extra hardware (i.e. a way for the LLM to have live understanding of the student's paper notebook or handout which are being written in with a plain old pencil) combines the best of both worlds - individual tutoring with approximately zero screen time which scales linearly with the number of students. The role of the teacher or professor then becomes a manager of…

A 'smart pen' that records the student's writing in some way, maybe? My first thought was a tablet that boots straight into a writing software but students should not be subjected to any amount of latency in their writing.

Practically, I think if you want the AI system to have a live view of what the student's doing you're going to have to replace one of either the tablet or the writing instrument. A wearable camera could work as well but there are issues with that.

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#25

Post by Buttons840 »

baq wrote: post #11

I'm on record saying that a system like this with some extra hardware (i.e. a way for the LLM to have live understanding of the student's paper notebook or handout which are being written in with a plain old pencil) combines the best of both worlds - individual tutoring with approximately zero screen time which scales linearly with the number of students. The role of the teacher or professor then becomes a manager of…

I would add that somewhere in there should be a spaced repetition algorithm.

Spaced repetition is very effective, but it's really really clunky to use. My unpopular opinion is that we all have Stockholm syndrome when it comes to creating "cards", and people talk about how valuable creating cards is; but I think it stucks, it takes a lot of time.

If AI is already teaching me math (let's say), it would be nice to tell the AI/app "quiz me on this periodically", and then the AI makes up a fresh polynomial to factor (or whatever) and presents that to you according to a spaced repetition algorithm.

Behind the scenes, the AI should have access to what has happened the last several times a specific topic has been quized, so the AI can watch to see that certain mistakes are resolved, and the AI might also know better how to correct the user if it has context about previous quizzes of that topic.

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#26

Post by tired-turtle »

Buttons840 wrote: post #25
baq wrote: post #11

I'm on record saying that a system like this with some extra hardware (i.e. a way for the LLM to have live understanding of the student's paper notebook or handout which are being written in with a plain old pencil) combines the best of both worlds - individual tutoring with approximately zero screen time which scales linearly with the number of students. The role of the teacher or professor then becomes a manager of…

I would add that somewhere in there should be a spaced repetition algorithm. Spaced repetition is very effective, but it's really really clunky to use. My unpopular opinion is that we all have Stockholm syndrome when it comes to creating "cards", and people talk about how valuable creating cards is; but I think it stucks, it takes a lot of time. If AI is already teaching me math (let's say), it would be nice to tell…

But the very act of making and organizing your card deck is part of the SRS! It “sucks” because you get no dopamine hit from a fresh desk, as the reward system is not yet in place.

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#27

Post by jasondigitized »

kubb wrote: post #4

Too bad the educational use case doesn't make any money. Good LLMs are a game changer for people motivated to learn.

Not sure if its education, but there is huge money in the college admissions process, e.g., SAT prep.

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#28

Post by universa1 »

terribleperson wrote: post #24
baq wrote: post #11

I'm on record saying that a system like this with some extra hardware (i.e. a way for the LLM to have live understanding of the student's paper notebook or handout which are being written in with a plain old pencil) combines the best of both worlds - individual tutoring with approximately zero screen time which scales linearly with the number of students. The role of the teacher or professor then becomes a manager of…

A 'smart pen' that records the student's writing in some way, maybe? My first thought was a tablet that boots straight into a writing software but students should not be subjected to any amount of latency in their writing. Practically, I think if you want the AI system to have a live view of what the student's doing you're going to have to replace one of either the tablet or the writing instrument. A wearable camera…

there was a pen that used special paper to directly record your notes (15-20years ago)... should be possible nowadays to directly transfer this to a connected device and have it feed it to an llm.

and after looking it up, it appears they are still available: https://www.livescribe.com/landingpage/ls3_onenote/

Re: New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]

#30

Post by KaiserPro »

I'm not an expert, but how much of this is down to novelty, ie https://en.wikipedia.org/wiki/Hawthorne_effect ?

(ie changing the environment can lead to short term productivity gains because either participants are aware they are being watch, or it breaks up the monotony and makes people work a bit harder. )

Post reply on HN