Hamsa Bastani, O. Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, Rei Mariman
Indiscriminate use of generative AI can boost short-term performance but may harm long-term learning ability, and appropriate safeguards are necessary, as demonstrated by a large-scale field experiment.
While there is ample evidence that generative AI (e.g., GPT-4) enhances productivity, its impact on learning new skills remains unclear. A key question in education is whether AI aids learning or fosters dependency that undermines students' ability to solve problems independently.
A large-scale field experiment was conducted in high school math classes. Students used a GPT-4-based tutor to solve problems, with some groups receiving direct answers from the AI and others receiving only teacher-designed hints. Subsequently, AI access was removed, and learning outcomes were measured through assessments.
Students who used the AI tutor performed better during practice but scored lower on tests without AI. In contrast, the group that received only hints maintained good performance even without AI. This study emphasizes that for educational use of generative AI, designed interactions aligned with learning goals are crucial, rather than unrestricted use.