Harvard Study Finds Staggering Productivity Gains From AI at Work

Is AI really that big a deal for the future of work?

A new Harvard Business School paper from nine authors, including AI expert Ethan Mollick, answers with a resounding YES.

The paper is titled Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. It details a multi-month experiment where the researchers worked with BCG to determine how AI is transforming knowledge work.

And boy does AI change knowledge work.

The researchers measured performance across 18 different tasks that represented realistic samples of work done at an elite consulting company. These included creative, analytical, and writing and marketing tasks.

Mollick and team found that consultants using ChatGPT “outperformed those who did not, by a lot. On every dimension. Every way we measured performance.”

What’s more, they found that consultants using AI finished 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results.

However, the study also found that when people used AI for tasks it wasn’t good at, they were more likely to make mistakes or put too much trust in AI.

The key, Mollick and team found, is to get good at judging when AI is good or bad at a task. Some consultants were bad at this and their performance suffered. Others were adept at acting as what Mollick and team call “Cyborgs” or “Centaurs,” moving back and forth between AI and human work in ways that combine the strengths of both.

Says Mollick: “I think this is the way work is heading, very quickly.”

Why it matters:

We now have some real data that quantifies the advantages provided by AI for knowledge work. And those advantages are big enough to have disruptive effects on how we work.

Connecting the dots:

In Episode 64 of the Marketing AI Show, Marketing AI Institute founder/CEO Paul Roetzer outlined why this study is so important.

1. These types of studies are needed. We can run projections and thought experiments all day. But the more tangible numbers we have related to AI’s impact on jobs, the better decisions we can make as business owners, professionals, and people, says Roetzer.
2. You need to be an AI experimenter moving forward. One of the critical skills of the near future will be the ability to run lots of experiments with AI to determine what it is and isn’t good at, says Roetzer. A curiosity to experiment plus a willingness to share what you’re finding is a winning combo.
3. Don’t ignore AI’s benefits for lower-skilled workers. One interesting part of the study is that consultants who scored the worst on tasks when assessed at the start saw the biggest jump in performance from AI. (Though top consultants still got a boost.) Says Mollick: “I do not think enough people are considering what it means when a technology raises all workers to the top tiers of performance.”

What to do about it:

This is a brave new world. You will likely need to define your own career path moving forward. That starts by taking the time to learn what AI is and isn’t good at, then deploying that in your current role to increase your productivity and performance.