SCIENCE · RCTs · 2023

The controlled trials agree: AI lifts the least experienced the most

Two randomized studies, one on about 5,000 support agents, one on professional writing, found the same thing. Generative AI raised output and, crucially, helped novices far more than experts, compressing the gap between weak and strong performers.

+35%
PRODUCTIVITY GAIN FOR LESS-EXPERIENCED WORKERS, VERSUS NEAR ZERO FOR TOP PERFORMERS
WHAT IT FOUND
/01

Brynjolfsson, Li, and Raymond studied about 5,000 customer support agents given an AI assistant in a real deployment. Productivity, measured as issues resolved per hour, rose 13.8 percent on average. The average hides the story. Less-experienced and lower-skilled agents gained about 35 percent, while the most experienced gained little or nothing. An agent with two months on the job, using AI, matched one with six months without it.

/02

Noy and Zhang ran a controlled writing experiment, published in Science. ChatGPT cut the time to complete professional writing tasks by about 40 percent and raised graded quality by about 18 percent, and again narrowed the gap between weaker and stronger writers.

WHAT IT MEANS
/01

AI is not a uniform boost. It is a leveller. It brings the bottom up faster than the top, which means the biggest returns sit with the people and teams currently furthest behind. That is the opposite of the usual instinct to invest in the already-strong.

PRACTICAL TAKEAWAYS
/01

Your largest capability gains are in your least-experienced people. Start there, not with the power users.

/02

AI can compress ramp time. New hires can reach competent output faster when the method is built in.

/03

Measure the spread, not just the average. A 14 percent mean can hide a 35 percent gain where it matters most.

TIPS AND TRICKS
/01

Pair every junior with the AI method a senior already uses. The gap closes fastest when the good method is shared, not rediscovered.

/02

Track time-to-competence for new hires as a metric. It is where AI pays back first.

/03

Do not pull the tool from experts because their gain is small. It still holds quality and floors the bad days.

RISKS AND CAVEATS
/01

These are specific tasks, support and writing. Gains vary by job, so do not assume the exact numbers transfer.

/02

Levelling can flatten pay differentials. If juniors match seniors, the reward structure needs a rethink, not denial.

SOURCE

Brynjolfsson, Li and Raymond, Generative AI at Work (NBER, 2023); Noy and Zhang, Experimental evidence on the productivity effects of generative AI (Science, 2023).

Read the source →