URL: https://www.anthropic.com/research/AI-fluency-index Date evaluated: 2026-02-23 Type: Research report — Anthropic Education Science Authors: Kristen Swanson, Drew Bent, Saffron Huang, Zoe Ludwig, Rick Dakan, Joe Feller
- Anthropic research on 9,830 Claude.ai conversations over a 7-day window (January 2026)
- Framework: 4D AI Fluency (24 behaviors total; 11 directly observable in conversation data)
- Finding 1 — Iteration multiplies fluency: 85.7% of conversations showed iteration. Iterative users had 2.67 fluency behaviors vs 1.33 for non-iterative — 2× more effective. They are 5.6× more likely to question reasoning and 4× more likely to identify missing context.
- Finding 2 — Artifact Paradox: Artifact production (code, docs) made users more directive (+14.7pp) but reduced critical evaluation: −5.2pp identifying missing context, −3.7pp fact-checking, −3.1pp questioning reasoning.
- Finding 3 — Explicit collaboration: Only 30% of users set collaboration terms explicitly. Those who do produce structurally more effective sessions.
- Future directions: Explicit mention of Claude Code platform analysis as next research target.
| Score | Meaning |
|---|---|
| 5 | Essential — major gap |
| 4 | High value — significant improvement |
| 3 | Relevant — useful complement |
| 2 | Marginal — secondary info |
| 1 | Out of scope |
Justification: Source Anthropic first-party, published today. Provides empirical backing for three core guide recommendations that were previously asserted without data (plan review, CLAUDE.md investment, critical output evaluation). The Artifact Paradox is particularly critical for Claude Code: 100% of Claude Code outputs are artifacts.
| Aspect | This resource | Guide (before integration) |
|---|---|---|
| Iteration improves results | ✅ 2.67 vs 1.33 behaviors (Anthropic data) | |
| Plan review before execution | ✅ 5.6× ratio quantified | |
| CLAUDE.md ROI | ✅ 30% stat, explicit collaboration benefit | |
| Artifact Paradox | ✅ −5.2pp / −3.7pp / −3.1pp documented | ❌ Absent from guide |
| Claude Code specificity | ❌ Study on Claude.ai (code future) | ✅ CLI-native focus |
| AI Fluency Framework | ✅ 4D, 24 behaviors | ❌ Absent from guide |
What: Callout with 5.6× stat as empirical backing for plan review Why: Justifies the "Rev the Engine" pattern with Anthropic-sourced data instead of anecdotal "30-40% catch rate"
What: Callout with the 30% stat as motivation for explicit collaboration setup Why: The challenge agent correctly flagged this as belonging in the CLAUDE.md section, not generic prompting
What: Full callout with all data points + 5 concrete counter-measures Why: Core finding directly applicable to Claude Code's nominal use case (100% output = artifacts)
The challenge agent raised the score from 3/5 to 4/5 and identified:
- The "Claude.ai vs CLI" gap is a false objection — behavioral patterns are human, not interface-specific
- The Artifact Paradox is the most critical finding for Claude Code (100% of outputs are artifacts)
- The 5.6× ratio justifies plan review more powerfully than existing anecdotal claims
- The 30% stat belongs in the CLAUDE.md section, not generic prompting
| Claim | Verified | Source |
|---|---|---|
| 9,830 conversations analyzed | ✅ | Article methodology section |
| 85.7% iteration rate | ✅ | Article finding #1 |
| 2.67 vs 1.33 fluency behaviors | ✅ | Article finding #1 |
| 5.6× more likely to question reasoning | ✅ | Article finding #1 |
| 4× more likely to identify missing context | ✅ | Article finding #1 |
| −5.2pp missing context identification (artifacts) | ✅ | Article finding #2 |
| −3.7pp fact-checking (artifacts) | ✅ | Article finding #2 — exact quote: "check facts (-3.7pp)" |
| −3.1pp questioning reasoning (artifacts) | ✅ | Article finding #2 — exact quote: "question the model's reasoning (-3.1pp)" — NOT "3.1% of users" |
| 30% define collaboration terms explicitly | ✅ | Article exact quote: "In only 30% of conversations do users tell Claude how they'd like it to interact with them" |
| Publication date 2026-02-23 | ✅ | Article byline (today) |
No corrections needed. All stats verified directly in source.
- Score: 4/5
- Action: Integrated (3 insertion points)
- Confidence: High — Anthropic first-party, all stats verified
Evaluation completed: 2026-02-23 Next review: 2026-06-23 (check for follow-up Claude Code platform study)