Khan Academy Improves Khanmigo Tutoring With Learning-History Context and a Faster Math Agent

Khan Academy launched Khanmigo, a generative AI tutor for students and assistant for teachers, three years before publishing a May 2026 post on its most recent round of improvements. From October 2025 to April 2026 it ran roughly 20 substantive product tests, covering more than 15 million tutoring threads, to find out which changes made the tutor more effective. Every test was run through an A/B testing platform that estimates a "chance to win" against the control version; a change is implemented only if that probability exceeds 0.95 and no guardrail metric is harmed. The core metrics were response latency, next-item correctness (whether the student answers the next problem on the same skill correctly without help) and cognitive engagement quality, rated automatically as passive, active or constructive. Guardrails included giving away the answer, math error rates and interactions per thread. One set of tests targeted the math agent, a behind-the-scenes system that verifies calculations and expressions in real time. Switching it to a faster AI model cut response time by 0.3 seconds across 1.35 million threads; asking it for more concise output cut mean response time by three seconds across 352,000 threads, and limiting it to the math the student had already done saved another 400 milliseconds and halved answer give-aways; a pre-check that skips unnecessary verification saved about 0.3 seconds across 1.04 million threads. Math accuracy held steady in each case. A second set gave Khanmigo more of the student's Khan Academy learning history. A summary of recent problem attempts raised next-item correctness by 3.4% across 608,000 threads, and surfacing unmastered prerequisite skills with a brief review raised it by 2.7% across 1.36 million threads, for a combined 6.1%. Passing the in-session conversation log as hard-to-parse JSON had no measurable effect, but switching to plain text and including the previous 24 hours of threads on the skill raised cognitive engagement by 5.09%. Adding examples of other problem types showed no effect. A full paper is due at the 27th International Conference on AI in Education.