Is AI Qualitative Coding Biased? What a 2025 Study Found
Qualitati Research Team · 2026-07-28 · 8 min read
Yes — a 2025 study found that using large language models (LLMs) to code qualitative interviews can introduce systematic bias, not just random error. When ChatGPT and Llama 2 annotated interview transcripts, their mistakes were correlated with respondents' characteristics (like refugee status and a child's gender), which distorted downstream conclusions. A smaller supervised model trained on human codes was both more accurate and far less biased.
What did the study test?
The study asked whether LLMs can reliably annotate large-N qualitative data from open-ended interviews. According to Ashwin, Chhabra, and Rao (2025), published in Sociological Methods & Research, the authors analysed transcripts of 2,407 open-ended interviews with Rohingya refugees and their Bangladeshi hosts in Cox's Bazar, Bangladesh. The interviews explored parents' aspirations for their eldest child — the kind of nuanced, interpretive concept where qualitative methods matter most.
Trained sociologists manually annotated 789 of the transcripts using a flexible-coding scheme with 19 codes. The researchers then compared three approaches on a held-out test set of 200 interviews: direct LLM annotation, a bespoke supervised model, and human expert codes as the gold standard.
Which models did they compare?
The team benchmarked closed- and open-source LLMs against a purpose-built classifier. The comparison covered:
- ChatGPT (GPT-3.5) — closed-source, prompted to code each transcript and explain its reasoning.
- Llama 2 13B (chat and base) — Meta's open-source models, prompted the same way.
- iQual — a bespoke supervised classifier (a lightweight BERT-family model) fine-tuned on the human annotations.
How accurate and how biased was AI coding?
The bespoke supervised model beat the LLMs on both accuracy and bias. According to Ashwin, Chhabra, and Rao (2025), averaging out-of-sample F1 scores across all codes, iQual performed best — outperforming ChatGPT and both Llama 2 variants at reproducing the human codes. More striking was the bias test: the authors regressed each code's prevalence on respondent characteristics to check whether annotation errors were random.
iQual showed evidence of bias in only 1 of 19 codes. The LLMs failed the bias test far more often, showing strong evidence of bias across many codes. Because the errors tracked who the respondent was, they were not noise that averages out — they were systematic distortions.
| Approach | Relative accuracy (avg F1) | Codes flagged for bias (of 19) |
|---|
| iQual (supervised, trained on human codes) | Best of all methods | 1 |
| ChatGPT (GPT-3.5) | Below iQual | Many |
| Llama 2 13B (chat / base) | Below iQual | Many |
Source: Ashwin, Chhabra & Rao (2025), Sociological Methods & Research. "Many" reflects the paper's finding of strong bias evidence across numerous codes; exact counts vary by model.
Why does biased coding matter for your findings?
Biased coding can flip a study's conclusions. The authors show that estimates built on LLM annotations sometimes disagreed with estimates built on the human codes — in several cases a relationship that was positive and significant on the expert annotations became negative or insignificant on the ChatGPT annotations. A researcher relying only on the LLM output could report the opposite of what the data actually show, mistaking an algorithmic artifact for a real-world pattern.
This is different from the familiar worry that AI is simply "less accurate." Random error mostly inflates noise. Systematic bias that correlates with respondent demographics can quietly manufacture — or erase — the very group differences qualitative and mixed-methods researchers care about.
What did the authors recommend?
The authors recommend training a bespoke model on human codes rather than annotating directly with an LLM. Their reasoning is practical: you already need a batch of high-quality human annotations just to check whether an LLM is biased, so those annotations are better spent training a smaller, less biased supervised classifier. They frame LLMs and NLP as tools that assist and extend traditional qualitative analysis — not replace the close reading of transcripts that produces a defensible coding tree in the first place.
For teams doing AI-assisted analysis today, the takeaways are concrete:
- Keep a human-coded gold set. Without it you cannot detect bias at all.
- Run a bias check, not just an accuracy check. Test whether coding errors correlate with respondent characteristics, not only whether overall agreement looks high.
- Prefer human-in-the-loop verification over unaudited bulk auto-coding, especially on interpretive or identity-linked constructs.
This is why AI qualitative tools should be built around auditable, human-reviewable codebooks. In ThemeLens, AI-suggested themes and codes stay editable and traceable back to the transcript, so researchers can validate — and correct — the machine's interpretation rather than inherit it blindly. The same principle applies when you scale open coding in a QDA workspace: the AI proposes, the researcher disposes.
FAQ
Does AI always introduce bias into qualitative coding?
Not always, but the risk is real and hard to see. The 2025 study found LLM annotation errors were systematically related to respondents' characteristics across many codes, while a supervised model trained on human codes was biased on only one of 19 codes. The safest assumption is that bias is possible until you test for it.
How can I check whether my AI coding is biased?
Compare AI codes against a human-coded sample, then test whether the disagreements correlate with respondent attributes (age, gender, group membership) rather than being randomly distributed. If errors cluster by subgroup, your downstream estimates may be distorted.
Is a bespoke model really better than a frontier LLM?
In this study, yes. A lightweight supervised classifier fine-tuned on expert annotations achieved the best average F1 and the least bias, outperforming ChatGPT and Llama 2 on both dimensions for this coding task.
Should I stop using LLMs for qualitative analysis?
No — the authors position LLMs as assistants, not replacements. Use them to accelerate coding and surface candidates, but keep human reading, a gold-standard sample, and an explicit bias check in the loop.
Read the original paper: "Using Large Language Models for Qualitative Analysis can Introduce Serious Bias" (Ashwin, Chhabra & Rao, Sociological Methods & Research, 2025).
Last updated: 2026-07-28.
This article is an independent editorial summary of third-party research and is not affiliated with or endorsed by the study's authors.