Can LLMs Do Projective Techniques? A 2026 Six-Model Test
Qualitati Research Team · 2026-09-21 · 7 min read
Large language models can produce synthetic responses to projective techniques such as word association and sentence completion that cover many of the same themes human consumers raise — but they express those themes in a different, more stylized voice, and they cannot tell you how common a perception is. That is the central finding of a July 2026 study comparing six LLMs against 173 human respondents.
What are projective techniques, and why test LLMs on them?
Projective techniques are indirect elicitation methods: instead of asking people to rate a brand or pick from a list, researchers ask them to complete a sentence, say the first words that come to mind, or react to an ambiguous scenario. The goal is to surface associations, emotions, and needs that respondents might not volunteer in a direct survey item.
They are a harder test for synthetic data than a Likert scale. A useful synthetic respondent here must reproduce not just an average preference but the spread of associations — the breadth, the repetition, the odd one-off idea. That is precisely what Stephen L. France (Mississippi State University) and Pia A. Albinsson (Appalachian State University) set out to measure in Synthetic Consumer Insight Generation with Large Language Models, posted to arXiv on 7 July 2026.
What did the study test?
The authors took human data from an earlier tourism study and asked LLMs to perform the same tasks. The human benchmark came from a student sample at a U.S. state university: 173 respondents completed word-association and sentence-completion tasks for five destinations — Las Vegas, Los Angeles, Nashville, New Orleans, and New York. The LLM side was a full factorial experiment, with roughly 173 synthetic responses generated per condition to match the human sample.
| Factor | Levels tested |
|---|
| Projective task | Word association; positive sentence completion; negative sentence completion |
| LLM | ChatGPT 3.5, ChatGPT 4.1, ChatGPT 5.1, Gemini 2.5 Flash, Grok 4.1 (non-reasoning), Mistral Medium 3.1 |
| Temperature | 1.0, 1.2, 1.4, 1.6, 1.8 |
| Prompt | Basic (the human wording); Extended; Extended + “sample from the full distribution”; the latter with 10 or 50 human example answers (few-shot) |
Responses were compared on basic text features (word count, word length, stop-word share, vocabulary size), six diversity and concentration metrics (including normalized entropy and Simpson concentration), structural topic models, and a “lift” analysis of the terms most distinctive to each city.
How close were LLM responses to human ones?
Close on content, different on style. According to France and Albinsson (2026), with the right prompt and temperature settings LLMs generated responses with diversity characteristics similar to the human data and covering similar topics. When the authors fit nine-topic models to human word associations and to the most human-like LLM condition (Gemini 2.5 Flash, extended prompt, temperature 1.8), they found high concordance — Hollywood crowds and partying in New Orleans and Nashville appeared in both.
The differences are instructive, though:
- Different emphasis. For Nashville, LLM topics centred on hot chicken and bachelorette parties. Humans mentioned bachelorette parties too, but not among their top ten distinctive terms.
- Diversity of a different kind. Much of the LLM’s apparent variety came from invented compound adjectives — “walletemptying”, “bachelorettepacked”, “beignetfueled” — rather than from genuinely distinct ideas.
- Longer answers. Human responses were at or near the shortest across all three tasks; LLMs, especially with extended prompts at high temperature, wrote longer and more elaborate text.
On normalized entropy, human responses scored highest for both sentence-completion tasks and second highest for word association — beaten only by that Gemini high-temperature condition.
Which settings matter most?
The researcher’s design choices mattered as much as the model. The authors’ regressions explained a large share of variation in the text features, with R² from 0.538 (vocabulary size) to 0.896 (stop-word share), and from 0.498 to 0.688 for the diversity measures. The main levers:
- Temperature. Higher temperature raised all three diversity measures and lowered all three concentration measures (p < .001). Below 1.0 the OpenAI models were almost deterministic; above 1.8 degenerate, incoherent output became unacceptably common.
- Prompt design. All four extended prompts significantly increased diversity over the basic prompt on all six metrics (p < .001 in all 24 prompt-by-metric comparisons). A prompt written for humans is not automatically a good prompt for an LLM.
- Few-shot human examples. Seeding the prompt with 10 or 50 real answers kept diversity gains while pulling style and content toward the human benchmark — a “more targeted” kind of diversity.
- Model choice. Relative to Gemini 2.5 Flash, ChatGPT 3.5, ChatGPT 4.1 and Grok 4.1 showed significantly lower diversity (p < .001). ChatGPT 5.1 was more verbose and diverse than earlier ChatGPT generations. No single model won on every metric.
What this means for researchers
The authors’ conclusion is deliberately not a simple yes or no. Synthetic projective data looks useful for exploring the range of possible associations — for ideation, for piloting a qualitative study, or for supplementing a small human sample when budget is tight. It is not a basis for estimating prevalence: the paper explicitly warns against using LLM output to claim what percentage of consumers hold a given view.
Three practical takeaways follow:
- Treat temperature, prompt wording, and seed examples as part of your method, and report them. Synthetic data is the product of the whole configuration, not the model alone.
- Do not mistake lexical novelty for new insight. Check whether “diverse” synthetic answers contain new ideas or just new adjectives.
- Keep a human benchmark. The study’s value came from having 173 real answers to compare against — the same discipline applies to any synthetic sample.
If you are using synthetic participants for early-stage exploration, tools such as the Synthetic Focus Group and Digital Twin Panel fit best in that same role: generating hypotheses to test with real people, not replacing them.
Limitations to keep in mind
The human sample was college students at a single university, the domain was city tourism, and cost constraints ruled out the most expensive frontier models (Gemini was tested at its mid-tier Flash level). The results describe how LLM output compares to one population on one topic; they should be replicated before generalizing.
FAQ
Can LLMs replace human respondents in projective research?
No. In this 2026 study LLMs reproduced many of the same broad associations, but with a different style and emphasis, and the authors caution against using synthetic answers to estimate how widespread any perception is.
What temperature works best for synthetic open-ended responses?
In this experiment, higher temperatures (up to 1.8) produced more human-like diversity, while settings below 1.0 were nearly deterministic for OpenAI models and settings above 1.8 produced too much incoherent text. The best value will vary by model and task.
Do few-shot examples make synthetic data more realistic?
They helped steer it. Adding 10 or 50 real human answers preserved the diversity gains of extended prompts while guiding style and content toward the human data.
Which LLM produced the most diverse projective responses?
It depended on the metric. Gemini 2.5 Flash and ChatGPT 4.1 at high temperature with extended prompts did well on word association, while ChatGPT 3.5, ChatGPT 4.1 and Grok 4.1 were less diverse than Gemini on the concentration measures overall.
Last updated: September 21, 2026. This is an independent editorial summary of third-party research by France and Albinsson (2026); it is not affiliated with or endorsed by the authors. Consult the original paper for full methods and results.