Does AI Miss the Middle of Your Transcripts?
Qualitati Research Team · 2026-08-27 · 10 min read
Last updated: August 27, 2026
Short answer
Positional bias in AI thematic analysis is the tendency of a language model to code some parts of a transcript more thoroughly than others because of where they sit in the input, not because of what they say. Peer-reviewed testing has observed it in qualitative tools, and long-context benchmarks show the underlying effect has changed shape rather than disappeared. Audit it; do not assume it is solved.
Key takeaways
- Nguyen and Welch, Organizational Research Methods (published online September 30, 2025; Volume 29, Issue 1, 2026) tested ChatGPT and specialized qualitative-AI tools and reported that the software "drew heavily from some parts of the interview and ignored others, with no apparent reason," alongside outputs containing quotes not present in the uploaded data.
- The mechanism was first characterized by Liu et al., "Lost in the Middle" (TACL, 2023): a U-shaped curve where models used information at the start and end of a long context far better than information in the middle.
- That specific U-shape has weakened. Tian et al. (LongPiBench, ACL 2025 Findings) tested 11 models — 5 commercial, 6 open-source — and found most had largely overcome classic lost-in-the-middle, while remaining sensitive to the distance between relevant pieces of information.
- For qualitative coding that second effect matters more, because evidence for a theme is almost always scattered across a transcript rather than sitting in one passage.
- Agreement with human coders remains modest. In Misra et al., International Journal of Qualitative Methods (February 28, 2026), across 34 semi-structured diabetes interviews, Gemma2 matched researcher-identified themes 27.1% of the time and Llama3.1 26.8%; about 45% of generated codes added meaningful context and 22–39% were duplicative.
- Use the Transcript Position Audit and the Chunking Decision Matrix below to measure the effect on your own corpus instead of assuming a number.
Why positional bias matters for AI thematic analysis
Most long-context benchmarks are retrieval tasks: a fact is planted in a long document and the model is asked to find it. Qualitative coding is not retrieval. It is exhaustive segmentation — every meaningful unit should be considered, and the analyst has to notice absences as well as presences.
That difference makes qualitative work more exposed, not less. A retrieval task fails loudly: the answer is wrong. An under-coded stretch of transcript fails silently. The output still looks like a complete set of themes. Nothing in the deliverable says "minutes 18 through 31 were skimmed."
There is also a structural coincidence working against researchers. Interview guides usually open with warm-up and context questions and close with summary or wish-list questions. The analytically richest material — where a participant has relaxed and is narrating a specific past episode — tends to sit in the middle third. If a model treats the middle less carefully, it treats your best data less carefully.
What the evidence supports — and what it does not
This topic attracts overstatement in both directions. Three claims are well supported as of August 27, 2026:
- Position affects long-context performance. Established by Liu et al. (2023) and widely replicated.
- Uneven coverage has been observed in real qualitative tooling. Nguyen and Welch report it directly, together with fabricated quotes and unstable results across identical prompts.
- The classic U-shape has flattened, but distance bias has not. LongPiBench found models handle absolute position far better than 2023-era systems while remaining sensitive to how far apart relevant pieces are.
What is not established: any published figure for how much positional bias degrades thematic-analysis quality on real interview transcripts. No study we could locate as of August 27, 2026 isolates that quantity. Treat the mechanism as real and the magnitude as unmeasured — which is exactly why you should audit your own corpus.
Why a bigger context window is not the answer
Context length is capacity, not attention quality. A model that accepts a million tokens can still weight them unevenly, and LongPiBench's distance finding suggests that spreading relevant evidence further apart is itself a risk factor. Loading fifty raw transcripts into one prompt maximizes that spread: the evidence for any single theme ends up as far apart as it can possibly be.
Capacity and uniform coverage are different properties. Vendors advertise the first.
Original asset: the Transcript Position Audit
A Qualitati-designed, one-afternoon protocol to test whether your pipeline reads a whole transcript evenly. It needs one representative transcript that a human has already coded.
| Step | What you do | What it reveals |
|---|
| 1. Segment | Divide the human-coded transcript into thirds by turn count, not by minutes. | A baseline of where the real content lives. |
| 2. Baseline density | Count human codes per third and note the natural distribution. | Whether the middle is genuinely richer in your guide. |
| 3. Machine run | Run your AI analysis on the full transcript; count machine codes per third. | The raw coverage profile. |
| 4. Rotation test | Re-run with the thirds reordered (3-1-2, then 2-3-1), prompt unchanged. | If a segment gains codes when moved to an edge, position is driving output, not content. |
| 5. Isolation test | Run each third alone as its own input. | The ceiling. Codes found in isolation but missed in the full run are position losses. |
Read it this way: steps 3 and 5 give you a recall gap, and step 4 tells you whether that gap is positional or just hard material. If reordering changes which themes appear, you have a methods problem to disclose, not a tuning problem.
Original asset: the Chunking Decision Matrix
Chunking is the standard mitigation — it is what Raza et al.'s LLM-TA pipeline (GenAI for Health workshop, AAAI 2025) applied to nine parent interviews, and what Misra et al. varied in their 2026 comparison. But chunking trades one error for another: small chunks protect coverage and destroy context.
| Unit of analysis | Chunk at | Main risk | Mitigation |
|---|
| Discrete opinions, feature reactions | Question–answer pair | Loses the arc of a shifting view | Second pass over the whole transcript for trajectory only |
| Narratives, critical incidents | Topic block (several turns) | Boundary cuts mid-story | Overlap adjacent blocks by one turn |
| Identity, contradiction, ambivalence | Whole transcript | Positional under-coding | Rotation test; isolation pass on the middle third |
| Cross-participant patterns | Code level, after per-transcript coding | Distance bias across many documents | Reduce over codes, never over raw transcripts |
The fix: map-reduce thematic analysis
The last row of that matrix is the load-bearing one. Coding each transcript independently and then synthesizing over the resulting codes shortens every relevant distance: the reduce step reads a compact, structured set of codes rather than hundreds of pages of raw talk. It also produces an inspectable intermediate layer — you can see which codes came from which transcript, which is what makes the audit above possible at all.
This does not make positional bias vanish. A single long transcript is still a long context, and the map step still has to read it. Map-reduce shortens distances; it does not eliminate them.
Where Qualitati fits
Qualitati is an AI user research platform for product, UX, and customer insights teams. Its ThemeLens thematic analysis runs a map-reduce pipeline across up to 100 transcripts at once: each transcript is coded on its own, codes are mapped to the stated research questions, and themes are synthesized with participant-anchored quotes. That is the bottom row of the matrix above — synthesis over codes, not over one concatenated mega-context.
The QDA Workspace supports the audit side: AI-assisted inductive and deductive coding, codebook generation, and theme visualization, so a researcher can compare machine coverage against a human-coded baseline segment by segment. Voice Analytics adds acoustic features from the audio — pitch, loudness variability, speech rate, voice quality — which are derived from the recording rather than the text and therefore do not inherit text-position effects.
The honest claim here is architectural, not absolute. Qualitati's pipeline is designed to shorten the distances that LongPiBench identifies as risky; it is not a guarantee of uniform coverage, and the platform's own output should be reviewed by a human analyst.
Limitations and trade-offs
- The audit is descriptive, not diagnostic. It tells you a segment was under-coded. It cannot tell you whether the missed codes mattered to your research question. A human still reads the gap.
- Rotation breaks sequence. Reordering thirds destroys conversational logic, so some code loss in the rotation test is legitimate rather than positional. Interpret direction, not magnitude.
- Chunking has its own bias. Small chunks inflate code counts and produce duplication, consistent with the 22–39% duplicative codes Misra et al. reported.
- Model behavior moves. A 2023 or 2025 benchmark may not describe the model you run today. Re-run the audit when you change models.
- Human review remains required. At 26–27% alignment with researcher themes in the Misra et al. study, AI output is a starting draft, not a finding. Nguyen and Welch go further and argue the instability they observed cannot be remedied without architectural change — a stronger position than this article takes, and one worth reading in full before you commit a methodology.
Methodology note for reviewers: if you report AI-assisted analysis, state your chunking strategy, the model and version, and whether you tested for position effects. Sensitive methodology claims should be verified by a human reviewer before publication.
Who this is for — and when not to bother
Run the audit if: your interviews run longer than about 45 minutes; your analysis is going into a paper, a regulated report, or a high-stakes product decision; or you feed multiple transcripts into a single prompt.
Skip it if: you are analyzing short open-ended survey responses, running exploratory discovery where recall matters more than completeness, or your transcripts already fit in a few thousand tokens. Position effects need length to appear.
FAQ
Is "lost in the middle" still a real problem in 2026?
Partly. LongPiBench (Tian et al.) found most of 11 tested models have largely overcome the classic absolute-position U-shape, but they remain sensitive to the distance between relevant pieces of information. For qualitative coding, where evidence is scattered, that second effect is the more relevant one.
Does a bigger context window fix positional bias in AI thematic analysis?
No. Context length is capacity, not attention quality. A model that accepts a million tokens can still weight them unevenly, and longer inputs spread relevant evidence further apart.
Should I chunk transcripts or send them whole?
It depends on the unit of analysis. Send whole transcripts when the phenomenon is contradiction, ambivalence, or identity work that only appears across the full arc. Chunk when codes are discrete and local. The Chunking Decision Matrix above maps the common cases.
How do I report this in a methods section?
State the model and version, the chunking strategy and chunk boundaries, whether you ran a position or coverage check, and what human review was applied. Reporting standards for AI-assisted qualitative work are still forming, so specificity is your protection.
Does this affect AI-moderated interviews as well as analysis?
Different mechanism, same family of concern. During a live interview the model's context grows turn by turn, so early context can be weighted differently late in a session. That is a reason to design interview guides with explicit section structure rather than relying on the model to remember its own opening.
Can I just ask the model to "read the whole transcript carefully"?
Prompt instructions of that kind are unverified as a fix. Treat them as a hypothesis to test with the rotation and isolation steps above, not as a mitigation you can claim in a methods section.
The bottom line
Positional bias in AI thematic analysis is a quiet failure mode: it produces plausible themes with unmarked gaps. The mechanism is documented, its magnitude on real qualitative data is not, and the responsible move is to measure it on your own corpus with the Transcript Position Audit rather than assume it is either fatal or solved. Match your chunking strategy to your unit of analysis, synthesize over codes rather than concatenated raw text, and disclose what you did.
Start free with 30 credits — no credit card required — and run an AI-moderated interview, conversational survey, or AI thematic analysis project on your own data. See transparent pricing, or compare Qualitati with NVivo and ATLAS.ti and MAXQDA.
This article is an independent editorial summary of publicly available research and publicly available information about third-party products, accurate as of August 27, 2026.