Khanmigo两年实验：AI辅导效果有限，瓶颈在参与度
论证要点
- 试验设计：18所田纳西中学的集群随机试验，学生在每日补救数学课中使用配置为辅导而非直接给答案的Khanmigo，持续两年。
- 效果数据：每学期数学成绩提升1.3个全国百分位，一学年约0.06-0.08个标准差；全年积极参与的隐含效果达0.14个标准差。
- 对比发现：这些增益与无AI辅助的Khan Academy练习效果相似，暗示AI辅导未带来显著额外收益。
- 参与度低：96%学生至少尝试一次Khanmigo，但中位数学生仅在其练习日的三分之一发消息，仅在17%的犯错练习环节中使用。
- 对话质量差：学生发送的消息多为裸答案或点击建议提示，很少进行实质性数学对话。
- 核心结论：约束在于参与度，实现AI辅导潜力需让学生真正使用，而不仅仅是提供访问权限。
对实践的启示
对AI教育应用开发者而言，仅提供AI辅导工具不足以提升学习效果，必须设计机制提升学生参与度和对话深度；对教育从业者，应关注如何激励学生主动使用AI，而非单纯部署技术。
作者承认的反方与局限
作者指出，效果与无AI的Khan Academy练习相似，可能因为学生使用频率低且对话浅；但未否认AI辅导在更高参与度下可能有效。
原文摘引
The binding constraint appears to be engagement: realizing the promise of AI tutoring will require getting students to use it, not just giving them access.
These gains resemble those from Khan Academy practice without AI assistance.
96 percent of students tried Khanmigo at least once, but the median student messaged it on only a third of the days they practiced, and in only 17 percent of the exercise sessions in which they made a mistake.
▸ 展开原文全文（12 段，抓取自 Hacker News）
Student Opinion Scale
Category: Student Learning
Generative AI has been promoted as the technology that could transform education by providing every student a personal tutor. We provide some of the first large-scale experimental evidence, from a two-year cluster randomized trial in 18 Tennessee middle schools in which randomly assigned students used Khan Academy with its AI tutor, Khanmigo, configured to coach rather than give answers, during existing daily remedial mathematics sessions. Assignment raises math achievement by 1.3 national percentile ranks per term, or about 0.06 to 0.08 standard deviations over a school year; the implied effect of a full year of active participation reaches 0.14 standard deviations. These gains resemble those from Khan Academy practice without AI assistance. One explanation is that students used the tutor infrequently and, when they did, rarely engaged it in substantive mathematical dialogue: 96 percent of students tried Khanmigo at least once, but the median student messaged it on only a third of the days they practiced, and in only 17 percent of the exercise sessions in which they made a mistake. Messages that students did send were mostly bare answers or clicks on suggested prompts. The binding constraint appears to be engagement: realizing the promise of AI tutoring will require getting students to use it, not just giving them access.
Keywords
Artificial intelligence; AI tutoring; Khanmigo; computer-assisted learning; educational technology; field experiment; math achievement; student engagement; response to intervention
Education level
Topics
Document Object Identifier (DOI)
10.26300/kner-hv33
EdWorkingPaper suggested citation:
Oreopoulos, Philip, and Nina Low. (). One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment. (EdWorkingPaper: -1551). Retrieved from
Annenberg Institute at Brown University: https://doi.org/10.26300/kner-hv33