Can LLMs Do Reflexive Thematic Analysis? A 2026 Study
Qualitati Research Team · 2026-09-09 · 7 min read
Reflexive thematic analysis (RTA) is the part of qualitative work AI helps least with. In a 2025 study in Qualitative Health Research, researchers ran a full RTA of a real interview with an offline large language model across seven structured tests. The model summarized competently but could not interpret, abstract, or stay consistent — and it fabricated quotes.
What did the study actually test?
Vikan, Aryan, Kannelønning, Riegler and Danielsen (2025) took one interview from a primary study on surgical-team perspectives on patient safety culture — a 10-page transcript of 9,262 words with an operating room nurse, originally in Norwegian — and walked an LLM through every phase of Braun and Clarke's reflexive thematic analysis.
The model was Mistral-7B, run offline via Ollama. That choice was deliberate: keeping health data local, and testing what the base text-generation technology can do without a commercial wrapper. The team ran seven primary tests between 11 November 2024 and 6 January 2025, escalating the scaffolding each time — better system prompts, longer summaries, an English translation of the transcript reviewed by a fluent translator, and finally the RTA methodology documents and theoretical frameworks (Donabedian's theory, patient safety culture dimensions) supplied as context.
How did the LLM perform, phase by phase?
It degraded as the analysis moved from description toward interpretation. According to Vikan et al. (2025):
| RTA phase | What the LLM produced | Verdict |
|---|
| Familiarization | Summaries that "consistently excluded essential topics"; missed relationships, trust and professional well-being; misread an ironic, internally humorous sentence | Partial |
| Coding | Keywords such as "anesthesia" and "surgeon" rather than meaning-bearing phrases; dictionary synonyms when asked for concepts; error codes such as "constructive mistake" | Weak |
| Theme generation | Topic summaries, "often inappropriately merged"; themes that overlapped, were unrelated to the data, or lacked substance; codes and themes changed with every conversation | Unstable |
| Theme development | "No contributions to the abstractions of themes and subthemes"; fabricated quotes, complete with invented page and line numbers | Fails |
| Theory integration | A mix of appropriate and inappropriate theories; a false definition of patient safety culture; themes silently rewritten to fit the framework | Fails |
Two details deserve attention from anyone building an AI-assisted workflow. First, more scaffolding did not rescue the weak phases: lengthening the requested summary from 150 to 300 words produced "more repetitive sentences without added substance." Second, raising the temperature setting — the usual reflex when output feels flat — made things worse, yielding "less relevant, fabricated, and contextually inappropriate responses."
Why does the fabricated-quote finding matter most?
Because it is the one failure a reader cannot catch. A shallow theme is visible; an invented quotation that carries a page and line number looks like evidence. In reflexive TA the quote is the warrant for the interpretation, so a fabricated one contaminates the audit trail rather than merely weakening it. Any pipeline that lets model-generated text reach a manuscript without a line-level check against the transcript inherits this risk, regardless of model size.
What are the limits of this study?
The authors frame it as exploratory, and the design is narrow by construction: one interview, one 7B open-weights model, one methodological tradition. A frontier hosted model with a long context window and stronger instruction-following would very likely produce better summaries and cleaner codes. What the study does establish is a floor and a mechanism — the failures cluster in interpretation and abstraction, not in language competence — which is exactly where reflexive TA locates the researcher's contribution.
What this means for researchers
The practical reading is a division of labour rather than a verdict on AI. Three rules follow from the paper's own recommendations, which call for "a human, comprehensive familiarization phase, and methodological competence to preserve epistemological integrity":
- Read your data yourself. Familiarization is where reflexive analysis is actually earned; a model summary is not a substitute for it, and the study shows summaries silently drop what matters.
- Never accept a machine-supplied quote. Verify every quotation against the transcript by string match, not by eye. Tools that anchor each theme to a retrievable source segment — the design behind ThemeLens — make that check mechanical rather than optional.
- Use the model where instability is harmless. Brainstorming candidate angles, stress-testing a codebook, surfacing overlooked topics: contexts where an inconsistent second opinion costs nothing. Not for producing the themes you will publish.
The same logic applies upstream. Interpretation depends on the depth of the material collected, so the leverage in AI is often in the fieldwork — running consistent, well-probed conversations at scale with an AI Interviewer — while the meaning-making stays with the analyst.
FAQ
Can an LLM do reflexive thematic analysis?
Not on the evidence here. Vikan et al. (2025) found that base LLMs "provide limited support and do not increase the efficiency of RTA," failing specifically at abstraction and theme development.
Would a larger model have done better?
Probably on summarization and coding. The study tested Mistral-7B offline for data-security reasons, so its findings speak to base-model capability rather than to the frontier. The interpretive failures, however, are not obviously a scale problem.
Do LLMs really invent quotes?
Yes. The authors report fabricated quotes that included page and line references, some "entirely unrelated to the themes." Treat every model-supplied quotation as unverified until matched against the transcript.
Is there any safe use of AI in thematic analysis?
The paper positions LLMs as useful for brainstorming. Descriptive and organizational tasks with a human check are the defensible zone; interpretation and abstraction are not.
Primary source: Vikan, M., Aryan, R., Kannelønning, M. S., Riegler, M. A., & Danielsen, S. O. (2025). Reflecting on LLM Support in Reflexive Thematic Analysis: An Exploratory Study. Qualitative Health Research, 36(2-3), 191-205. DOI: 10.1177/10497323251365211.
Last updated: 2026-09-09. This is an independent editorial summary of third-party research; Qualitati is not affiliated with the authors.