Harvard Experiment: Custom AI Tutor Doubles Learning Gains Over Active-Learning Classroom in Physics

A Harvard randomized crossover experiment with 194 undergraduates compared a custom AI tutor with an active-learning classroom. Led by Gregory Kestin and Kelly Miller, it covered physics topics including surface tension and fluid flow, with students experiencing both methods. Students using the AI tutor recorded more than twice the learning gains of the classroom group, with a median post-test score of 4.5 versus 3.5, while spending less time and reporting higher engagement and motivation.

The study moves the discussion of AI tutor effectiveness from general opinion to verifiable randomized crossover data, offering reference value for university teaching design and AI education product positioning. Its limits are clear: 194 students at one university, only two physics lessons, immediate assessments, and no verification across other subjects, institutions, longer courses, or long-term retention. It also did not compare an ordinary chatbot with every aspect of university education. The research points to combining AI tutoring with classroom instruction rather than replacing teachers. Universities and teachers can use this to assess embedding AI tutors in course preview and difficult-concept explanation while preserving discussion, collaboration, and instructor guidance. AI education product developers can focus on personalized feedback and self-paced design rather than generic chat-based Q&A as a product differentiator.