The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
blendmylearning.files.wordpress.com
The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
1–10 of 13 posts
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#2To Quote Martin Fowler
"Often when you come across something new it can be a good idea to overuse it in order to find out it's boundaries. This is a quite reasonable learning strategy. It's also why people always tend to overuse new technologies or techniques in the early days. People often criticize this but it's a natural part of learning. "
http://www.google.com/url?sa=t&rct=j&q=&esrc=s&#...
It is good to see some studies being done in this area (even if the one in the article is not particularly rigorous). I am convinced that there is tremendous value in the new online opportunities (Khan Academy, Stanford, Coursera, Udacity, etc), but there are limits to what they can accomplish. For one thing, an online course being done individually requires a higher level of student discipline than a learning activity involving the personal interaction of a teacher. My mind is less likely to wander when a teacher is staring me down :).
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#3To Quote Martin Fowler "Often when you come across something new it can be a good idea to overuse it in order to find out it's boundaries. This is a quite reasonable learning strategy. It's also why people always tend to overuse new technologies or techniques in the early days. People often criticize this but it's a natural part of learning. " http://www.google.com/url?sa=t&rct=j&q=&esrc=s&#... It is good to see some…
As mentioned in the paper, there are Khan Academy "coaching" tools for monitoring the students. The site allows you to monitor the relative progress of the students so if one started to slow down you could pick them up on that: http://www.khanacademy.org/images/class-report-medium.png
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#4The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers.
The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#5To Quote Martin Fowler "Often when you come across something new it can be a good idea to overuse it in order to find out it's boundaries. This is a quite reasonable learning strategy. It's also why people always tend to overuse new technologies or techniques in the early days. People often criticize this but it's a natural part of learning. " http://www.google.com/url?sa=t&rct=j&q=&esrc=s&#... It is good to see some…
As mentioned in the paper, there are Khan Academy "coaching" tools for monitoring the students. The site allows you to monitor the relative progress of the students so if one started to slow down you could pick them up on that: http://www.khanacademy.org/images/class-report-medium.png
Good point. This does imply that the teacher is watching the monitor rather than helping another student and then actively intervenes. So it is likely a different dynamic than a classroom where a teacher is observing the whole class at once and can take a somewhat more subtle action (i.e. speak louder in a direction of a student who is falling asleep). Khan academy may well be much more effective. It would be great to see studies that make clear comparisons of where it is most effective.
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#6The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers. The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
Deep down in the report it is revealed that that 5.2% consists of a third of the class improving by 10% or more... and the rest remaining essentially flat. So it appears to me that it's particularly effective for a subset of learners.
Also note: the teacher doesn't have to really lead the class. The report says that the teacher spent most of their time 1 on 1 helping students who were stuck. Teachers may see value in simply having to do less work to achieve the same (or slightly better) result.
Also, this class was only a couple months. There is a question whether the small improvement would have continued and would have become wider when applied over a whole school year.
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#7The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers. The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
The Khan Academy group spent 50% of their time on basic mathematical skills while the control group spend all their time on algebra. That might be a reason as to why we don't see better improvement in their score.
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#8The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers. The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
6.4/5.2 = 1.23 .
Is 23% improvement considered that small(assuming it's statistically significant) ?
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#9The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers. The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
Deep down in the report it is revealed that that 5.2% consists of a third of the class improving by 10% or more... and the rest remaining essentially flat. So it appears to me that it's particularly effective for a subset of learners. Also note: the teacher doesn't have to really lead the class. The report says that the teacher spent most of their time 1 on 1 helping students who were stuck. Teachers may see value in…
The class only took 5 weeks (24X2 hours sessions).
Re: The end-of-pilot report from using Khan Academy for 80-90% of class time[pdf]
#10The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers. The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
6.4/5.2 = 1.23 . Is 23% improvement considered that small(assuming it's statistically significant) ?
I think you're looking at the wrong derivative. This is 23% improvement of improvement. It's still just 6.4% improvement.
Might be easier to see if you think of it as 106.6/105.2, in order to include the baseline.