How Accurate Are LLM Digital Twins of Real Respondents?
Qualitati Research Team · 2026-09-08 · 7 min read
Short answer: Not yet at the individual level. Building digital twins from German Socio-Economic Panel microdata, Kinzinger and Hartmann (2026) got 78.8% best-cell accuracy and a rank-order correlation of r = 0.590 across 2.1 million simulated answers. Good enough to rank items; not good enough to stand in for a person.
Key takeaways
- Individual-level digital twins built from rich panel microdata top out around 78.8% accuracy in the best configuration tested (Kinzinger & Hartmann, 2026).
- More biographical detail helps — but with sharp diminishing returns past roughly the 75th percentile of information entropy.
- How you feed the person to the model matters more than most teams assume: raw dialog history of past answers beat a tidy narrative persona summary at maximum depth.
- Turning on explicit reasoning improved rank-order correlation without moving accuracy — a useful hint about what twins are actually good for.
- Practical read: use twins to prioritize and pre-test, use humans to conclude.
What did the study test?
The study asked whether an LLM given a real person's socio-economic history can answer survey questions the way that person would. On June 3, 2026, Leonard Kinzinger and Jochen Hartmann posted "Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?" — a grid study rather than a single demo.
The source data is the German Socio-Economic Panel (SOEP), a long-running household panel with years of answers per respondent. That is the important design choice. Most synthetic-respondent work conditions a model on a demographic sketch — 34, female, urban, university-educated. Here the model gets an actual longitudinal record of one human being, which is the strongest version of the digital-twin premise anyone can realistically build.
According to Kinzinger and Hartmann (2026), the evaluation covered 500 participants and 183 held-out questions, producing over 2.1 million twin responses across the full experimental grid.
The four factors they varied
- Model. Three open-weights LLMs, so the pipeline can be run locally on sensitive panel data rather than shipped to a vendor API.
- Information depth. Five cumulative levels of biographical detail, ranked by normalized Shannon entropy — an explicit measure of how much information the twin was actually given, rather than a vague "rich vs. sparse persona."
- Embedding method. A narrative persona summary versus the raw dialog history of the respondent's past answers.
- Reasoning mode. Standard generation versus an explicit thinking mode.
How accurate were the digital twins?
Accurate enough to be useful, not accurate enough to be a substitute. The best cell in the grid reached 78.8% accuracy with a Fisher-z rank-order correlation of r = 0.590 (Kinzinger & Hartmann, 2026). Read those two numbers together rather than separately: roughly four answers in five land on the right response option, and the ordering of a person's answers across items is moderately — not strongly — preserved.
For a market researcher, that gap is the whole story. A twin panel that is right 79% of the time will usually put your top-scoring concept at the top. It will not reliably tell you whether that concept beats the runner-up by three points or eleven, and it will not tell you what an individual respondent believes.
What moved the numbers
| Design decision | Reported effect | What to do with it |
|---|
| More information depth | Helps, with diminishing returns past ~the 75th entropy percentile | Stop collecting biography past the knee of the curve; spend the budget on more items instead |
| Raw dialog history vs. narrative persona | Raw history improved accuracy in every configuration at maximum depth | Do not summarize your respondent into a tidy paragraph — feed the actual prior answers |
| Explicit thinking mode | Raised rank-order correlation; accuracy unchanged | Turn it on for ranking and prioritization tasks; do not expect it to fix point accuracy |
| Model choice | One of the remaining live constraints named by the authors | Benchmark models on your own held-out items before committing |
The persona-summary result is the one most likely to change how teams build these systems. Compressing a respondent into a readable character sketch is exactly what feels professional — and it discards the granular, sometimes contradictory answer patterns that make one person distinguishable from a demographic average.
What this means for researchers
The authors' own conclusion is that twin-based market research "is no longer gated by data design" but by item volume, model selection, and a handful of construction-level decisions. That is a meaningful reframing: the bottleneck has moved from can we describe the person well enough to how many questions can we afford to ask and which model do we point at them.
Three practical implications:
- Treat twins as a prioritization layer, not an evidence layer. Moderate rank-order correlation supports screening twenty concepts down to five. It does not support a go/no-go decision.
- Pre-test instruments, then field them with humans. Running a draft questionnaire past a twin panel surfaces confusing items and dead-end branches cheaply — which is why teams increasingly pair a Digital Twin Panel with real fielding rather than instead of it.
- Report the construction decisions. Depth, embedding format, and reasoning mode each moved results here. A synthetic-data appendix that omits them is not reproducible.
There is also a caution worth stating plainly. This study used one of the richest individual-level panels available anywhere and still landed at moderate correlation. Twins built from a demographic sketch and a purchase history should be expected to do worse, not better.
Where twins fit next to real qualitative data
Synthetic respondents are best understood as a hypothesis generator sitting upstream of real data collection. They are fast, cheap, and repeatable, and they carry the entire bias of their training and construction. Real interviews are slow and expensive and carry the participant's own reasoning — including the part you did not think to ask about.
A workable division of labor: generate and rank hypotheses against a twin panel, then test the survivors with real participants — an AI Interviewer makes that second stage affordable enough that the synthetic stage does not have to carry weight it cannot bear.
FAQ
Can LLM digital twins replace survey respondents?
No. The best configuration in this study reached 78.8% accuracy with r = 0.590 rank-order correlation on held-out items, which supports screening and prioritization but not individual-level or close-margin conclusions.
Does giving the model more personal data make the twin better?
Up to a point. Kinzinger and Hartmann (2026) report diminishing returns beyond roughly the 75th percentile of information entropy, so additional biography buys progressively less accuracy.
Should I write a persona summary for each twin?
The evidence here says no. Raw dialog history of a respondent's past answers outperformed a narrative persona summary at maximum information depth.
Does reasoning mode help?
It raised rank-order correlation without improving accuracy — useful when your output is a ranking, largely irrelevant when you need the point estimate to be right.
Source
Kinzinger, L., & Hartmann, J. (2026). Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata? arXiv preprint, June 3, 2026.
Last updated: 2026-09-08. This is an independent editorial summary of third-party research; it is not affiliated with or endorsed by the authors.