Fabrizio Dell’Acqua, Edward McFowland, Ethan Mollick, Hila Lifshitz, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, François Candelon et al.
This study experimentally demonstrates the 'jagged technological frontier' of AI, where AI assistance unevenly impacts knowledge worker performance, boosting it on some tasks while hindering it on others.
The research aimed to empirically verify whether AI helps with complex knowledge work and how its effects vary by task type and difficulty.
Researchers developed 18 realistic consulting tasks and randomly assigned 758 participants to three conditions: no AI, GPT-4 access, or GPT-4 access with prompt engineering training, then compared their performance.
The AI group completed 12.2% more tasks, 25.1% faster, and with higher quality on average. However, for a complex managerial task deemed outside AI's frontier, the AI group was 19% less likely to produce correct solutions. This provides crucial empirical evidence that AI's impact is not uniform and depends heavily on task context.