How Khan Academy is using Machine Learning to Assess Student Mastery

This is sooooo cool. The actual statistical methods they are using are pretty crude, but that’s fine. What’s important is their focus on the important goal. It’s sort of like Bill James or Nate Silver: if you’re using good information, and you’re focused on good questions, then the fancy statistics can come later (or from others).

In most educational efforts I know of (including my own), very little is done to target assessments to improvements for individual students. I really like what they’re doing here and it reminds me how I want to figure out how to do something similar in my own teaching and course materials.

6 thoughts on “How Khan Academy is using Machine Learning to Assess Student Mastery

  1. Any tips to what can be done to improve their logistic regression? Or is it the usage of Logistic Regression itself that you describe as crude?

  2. Steve:

    Yup. Note this bit from the end of that post: “Do you want to make 0.1% improvements in ad click-thru rates for the rest of your life, or come with us and change the world of education?

    Marco:

    Logistic regression is fine. I didn’t look at their work in detail and don’t have any particular suggestions for improvements. I just thought the analysis had a let’s-try-this-let’s-try-that feeling, sort of like the work of Bill James and Nate Silver. That’s not a bad thing at all!

  3. This is cool. However, pretty sure that these folks are rediscovering item response theory and computer adaptive testing, which is a mature body of methods for estimating student proficiency in the educational/psychological testing literature.

    • Dave:

      Yup. But what’s special here is the idea of doing it in a “retail” rather than “wholesale” setting, that is, focusing on improving learning for individual students in the class rather than on assessment for large numbers of anonymous students.

Comments are closed.