Short answer: Often, yes. A 2026 study by Chae, Choi, Lee and Im (Yonsei University) found that LLM-simulated survey populations either shrink the differences between countries (cultural flattening) or blow them up (cultural caricature). The persona method that scored best on standard fidelity metrics flattened cross-country differences the most, keeping only 14–59% of the divergence seen in real human data.
Last updated: October 23, 2026
Synthetic respondents are increasingly used to pilot cross-national surveys, extend samples into under-surveyed countries and test questionnaires before fieldwork. Most validation work asks one question: does the synthetic answer distribution for each country look like the real one? The paper "Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations" (Chae, Choi, Lee and Im, 2026; accepted to the EMNLP 2026 Workshop on Pluralistic AI & NLP) asks a second one that comparative researchers care about more: are the differences between countries preserved?
Key takeaways
- Matching each country's answer distribution and preserving the gaps between countries are two separate properties. A synthetic population can do well on the first and fail the second.
- The authors propose Cultural Divergence Preservation (CDP): synthetic cross-country divergence expressed as a percentage of human cross-country divergence. Below 100% means flattening; above 100% means caricature.
- Rich, DeepPersona-style profiles won most fidelity comparisons but kept only 14–59% of human cross-country divergence in all eight model–domain blocks.
- A bare "you were born and live in country X" prompt did the opposite, inflating divergence to 184–501% of the human level.
- Standard fidelity metrics barely move when divergence is flattened, so you can lose most of the cultural signal without your usual validation scores noticing.
What did the study test?
The study tested whether three common ways of prompting synthetic respondents preserve the size of cross-country differences in two well-known surveys. According to Chae et al. (2026), the design was:
- Surveys: six World Values Survey (Wave 7) items for Argentina, Australia, Germany, India, Kenya and the United States, covering topics such as trust, happiness, national pride and the justifiability of abortion; and the 50-item IPIP Big Five personality inventory for Argentina, Australia and India.
- Models: four open-weight LLMs: Gemma-3-4B, Qwen3.5-9B, Qwen3.5-27B and Llama-2-13B.
- Persona methods: Cultural prompting (country of birth and residence only), a PersonaHub-inspired method (a one-line persona expanded into a character profile), and a DeepPersona-inspired method (a long template of demographics, personality, values, social life and lifestyle attributes).
- Sample: 300 simulated respondents per country, method and model, compared against real national response distributions.
- Metrics: conventional fidelity scores (Wasserstein distance, Jensen–Shannon divergence and Kolmogorov–Smirnov) alongside the new CDP measure.
What is Cultural Divergence Preservation (CDP)?
CDP measures how much of the real between-country variation a synthetic population keeps. For each survey item, the authors average the country answer distributions into a cross-country "centroid" and measure how far each country sits from it. That average distance is the population's cultural divergence. CDP divides the synthetic divergence by the human divergence for the same items and countries.
A CDP of 100% means the synthetic countries are exactly as different from each other as the real ones. A CDP of 30% means about 70% of the cross-country contrast has disappeared; a CDP of 300% means the countries look three times more different than they really are. Because the human reference is computed once per survey domain, it can be reused to audit many models and prompting methods.
Do synthetic respondents flatten cultural differences?
Yes, with the persona method most researchers would consider the most realistic. The table below summarises the main results reported by Chae et al. (2026) across the eight model–domain blocks.
| Persona method | What the prompt contains | Fidelity ranking | CDP range | Distortion |
|---|---|---|---|---|
| Cultural prompting | Country of birth and residence only | Highest JSD in all 8 blocks | 184–501% | Caricature (exaggerated differences) |
| PersonaHub-inspired | Short persona expanded into a character profile | Intermediate | 44–99% | Mild to moderate flattening |
| DeepPersona-inspired | Long structured profile of demographics, traits, values and lifestyle | Lowest raw WD in 7 of 8 blocks; lowest JSD in 6 of 8 | 14–59% | Strongest flattening in every block |
The pattern held at the country level too. Cultural prompting was over-divergent in 33 of 36 country–model cells, while the DeepPersona-inspired method was under-divergent in 33 of 36. The PersonaHub-inspired method had the most cells inside the human band, 20 of 36. A respondent-level bootstrap confirmed the DeepPersona flattening gap in seven of eight blocks; the exception was Llama-2-13B on the Big Five.
The authors also found 37 of 108 country conditions where a synthetic country matched its human distribution better than the median while showing less than half of the human cross-country divergence. In other words, "good fidelity" and "lost cultural signal" frequently appeared together.
Why don't standard fidelity metrics catch flattening?
Because flattening is cheap in distribution-matching terms. In a controlled experiment, the authors took the real human data and pulled each country toward the cross-country average. On the World Values Survey, mixing just 20% of the centroid into each country left only 62% of human divergence, yet added roughly 0.5% of the typical JSD error that real LLM conditions already showed. Collapsing all countries to the identical answer distribution, which removes every cultural difference, cost only 9.3% (WVS) and 2.3% (Big Five) of that typical LLM error.
Exaggeration behaves the same way: amplifying country deviations pushed CDP to 231–254% while adding only 0.6–3.9% of the typical LLM JSD. The practical point is that a per-country fidelity score is dominated by other generation errors, so it is close to blind to whether cross-country contrasts survive.
What this means for researchers
If your research question is comparative, such as country differences, segment differences or group contrasts, validate the contrast and not only each group on its own. Four practical steps follow from the study:
- Report a between-group diagnostic. Alongside per-group fidelity, compute how much of the human between-group divergence your synthetic sample retains. CDP is one option; a rank correlation of group orderings covers direction, which CDP does not.
- Do not assume richer personas are safer. In this study, the most detailed persona template produced the most homogenised countries. The authors note this may partly reflect that its attribute taxonomy was not conditioned on the target country, which is itself a design lesson: country-specific attributes need country-specific distributions.
- Watch for caricature with thin prompts. A bare nationality cue made countries look far more different than they are, which can manufacture stereotypes in cross-cultural findings.
- Keep humans in the loop for cross-cultural claims. Synthetic panels such as a Digital Twin Panel are useful for piloting instruments and spotting confusing items, but country or group comparisons should be confirmed with real participants, for example through multilingual AI-moderated interviews in each market.
What are the study's limitations?
The authors are explicit about scope. CDP measures only the magnitude of cross-country divergence, not its direction, so two countries could swap positions without changing the score. It requires a one-time human reference that must be recomputed whenever countries, items or response scales change. It cannot separate culture from other country-level factors such as translation or response styles. The evidence comes from four open-weight models, two survey domains and English-language prompts, and the Big Five analysis includes only three countries. Frontier commercial models were not tested.
FAQ
What is cultural flattening in LLM survey simulation?
Cultural flattening is when synthetic respondents from different countries answer more similarly than real people from those countries do. In Chae et al. (2026), the DeepPersona-inspired method retained only 14–59% of real cross-country divergence.
What is cultural caricature?
Cultural caricature is the opposite distortion: synthetic countries differ more than real ones, exaggerating national stereotypes. A nationality-only prompt produced 184–501% of human cross-country divergence in the study.
Can I use synthetic respondents for cross-national research?
For piloting and question testing, yes, with care. For substantive claims about how countries differ, this study suggests synthetic data can distort the contrast in either direction, so real-participant data should anchor the conclusion.
Is Jensen–Shannon divergence enough to validate a synthetic population?
Not for comparative questions. The study shows per-country JSD changes very little even when all cross-country differences are removed, so it should be paired with a between-group diagnostic.
Source: Chae, Y., Choi, Y., Lee, S., & Im, I. (2026). Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations. arXiv:2609.29928. Accepted to the EMNLP 2026 Workshop on Pluralistic AI & NLP (PANDORA).
Last updated: October 23, 2026. This article is an independent editorial summary of third-party research by the Qualitati Research Team. It is not affiliated with or endorsed by the paper's authors; consult the original paper for full methods and results.