Anthropic’s Claude Study Reveals Surprising Shift in How Humans Work With AI
People are turning to artificial intelligence for more consequential work than previously thought, according to new research from Anthropic examining how users interact with its Claude chatbot.
In a study of roughly 250,000 Claude.ai and Claude Code conversations, researchers found that more than half involved people delegating consequential tasks to AI, challenging the idea that users primarily rely on AI for low-stakes work while keeping important decisions to themselves.
The research was conducted by three independent groups — Stanford University’s Social and Language Technologies Lab, the University of Oxford’s Human Information Processing Lab and AI evaluation nonprofit METR — using Anthropic’s privacy-preserving research platform, Anthropic Insights.
People Are Giving AI More Responsibility
The Stanford researchers examined how people collaborate with AI, including what types of work they bring to Claude and how much control humans retain. People were particularly likely to bring consequential work to Claude when seeking professional guidance, especially on legal and financial questions, according to Anthropic.
But users generally did not simply hand tasks over to AI. In nearly three-quarters of conversations, people set the direction while Claude assisted, and users typically adapted its responses rather than using them verbatim.
That suggests AI use is often closer to collaboration than full automation, with humans directing the work and AI helping produce the result.
Still, the researchers found that users varied in how much they understood or learned from Claude’s output, raising questions about whether human involvement always translates into meaningful oversight.
The Stanford team also found that interactions with Claude often involve "friction," but that isn’t necessarily a bad thing. Users may realize Claude misunderstood a request, clarify what they intended and revise their instructions. That back-and-forth can lead to better results by forcing people to refine their thinking and remain engaged with the problem.
The Oxford researchers looked at another part of the interaction: how people feel while using AI. Their early findings suggest that users’ reactions are closely linked to Claude’s behavior. Warmer responses tended to coincide with more positive reactions, while refusals or disagreements were associated with users pushing back.
More unusual responses from Claude, meanwhile, appeared to coincide with greater intellectual engagement.
The researchers also found that patterns of absorption, frustration and enjoyment in Claude conversations looked similar to patterns observed during everyday internet browsing.
Anthropic Wants More Outside Research
The study is also part of Anthropic’s effort to give outside researchers access to real-world AI usage data. Anthropic said researchers currently face a choice between relying on studies designed by AI companies or public datasets that may skew toward casual use. Its pilot attempted to bridge that gap without giving researchers access to raw conversations.
Researchers instead received aggregated results through Anthropic Insights, while Anthropic conducted privacy and legal reviews. A third-party privacy audit by Imperial College London also examined the data-sharing process.
Anthropic acknowledged that the system made the research slower and more resource-intensive, but said the pilot demonstrated that independent research could be conducted without compromising user privacy.
The company is now considering expanding the program and has publicly released aggregate data from the three studies.
Photo: Shutterstock
