How to Screen Fraudulent Participants in AI Research

Qualitati Research Team · 2026-09-05 · 8 min read

Short answer: Fraudulent participants in AI-moderated research are bots, LLM-assisted humans, and people who lie through screeners to reach an incentive. Screen them in three layers — recruit through controlled channels, run a screener designed to be hard to fake, and audit the transcript itself for the tells that only long-form qualitative data exposes. Detection after the fact is cheaper than a retracted finding.

Why AI-moderated research is a fraud target

Any study that pays participants and takes applications through a public link is an incentive market, and incentive markets attract fraud. What changed between 2024 and 2026 is that the tools for faking a plausible participant became free and general-purpose. A respondent no longer needs to write convincing prose about their onboarding experience; they can paste the question into a chatbot.

The measured scale varies enormously by channel. Xu and Malkin (2026), presented at SOUPS 2026, ran 250 surveys and found LLM-assisted responses ranging from under 10% on Prolific to over 80% on Mechanical Turk — a spread wide enough that "what is the fraud rate" is an unanswerable question without naming the recruiting source. In a separate May 2026 investigation, Conjointly estimated 4–8% suspected bot activity in its own platform data, dropping to 2–3% once speeders were excluded, and noted that the suspicious respondents looked more like coordinated human fraud than sophisticated automation.

Qualitative research was slower to notice because it was assumed to be self-protecting: surely a 30-minute interview filters out anyone who is not real. That assumption is now the vulnerability.

Three kinds of fraudulent participant

They need different countermeasures, so it is worth separating them.

Type	What it is	Primary defense
Automated bot	A script or browser agent completing the study end to end	Recruitment controls, timing analysis, tasks LLMs fail
LLM-assisted human	A real person pasting questions into a chatbot and answers back	Transcript-level analysis, probing, voice mode
Criteria fraud	A real person answering honestly but misrepresenting eligibility to get the incentive	Screener design, verification against an independent record

Criteria fraud is the most common and the least discussed. Ziminski and Liddell-Quinty, writing in BMJ Open Quality in December 2025, describe exactly this pattern in online qualitative recruitment: applications arriving in rapid succession, Gmail addresses with similar naming conventions, and clustered or implausible IP geolocations. Their recommended defenses are unglamorous — a deliberate outreach and recruitment plan, a short screener, and community-engaged recruiting rather than open public links.

What actually detects LLM-assisted responses

Two findings from the 2026 literature are worth internalizing because they cut against intuition.

Asking people not to use AI does not solve it. Xu and Malkin tested platform choice, survey length, explicit requests not to use AI, and disabling copy-paste. The mitigations reduced LLM usage — but the authors report that they did not necessarily improve data quality. A deterrent that pushes a marginal participant from pasting to guessing has not bought you a better dataset.

Traps beat classifiers. Conjointly's "cognitive trap" approach embeds tasks where humans reliably succeed and LLMs reliably fail — their example is a modified Müller-Lyer illusion, which models answer from memorized pattern rather than from the stimulus in front of them. Reported failure rates were 51–100% for LLMs against 78–96% human success. The principle generalizes past optical illusions: any item whose correct answer depends on a detail the model cannot see, or contradicts the canonical version of a famous problem, is a usable trap.

For qualitative work specifically, the transcript is itself a detection surface that surveys do not have. Chatbot-mediated answers tend to be uniformly well-structured across every question, to summarize rather than narrate, to resist specificity when probed, and to answer the question that was asked rather than drifting the way real people do. A follow-up like "walk me through the last time that happened, including what you did right before" is hard to satisfy from a paste-and-return loop.

The AI Interview Fraud Screen

An original three-layer checklist you can apply to any AI-moderated study. Each layer catches what the previous one misses; none is sufficient alone.

Layer 1 — Recruitment (before anyone applies)

Prefer a known list, a customer database, or a vetted panel over an open public link.
Do not publish the incentive amount in the recruitment copy where it can be scraped.
Use unique, single-use interview links rather than one shared URL.
Set an expected recruitment velocity in advance — if 40 applications arrive in an hour for a niche B2B segment, that is a signal, not luck.

Layer 2 — Screener (before the interview)

Include at least one open-ended screener question that requires domain-specific lived detail, not a definition.
Include one verifiable item you can check against an independent record (an account, an order, a role).
Add a low-stakes attention or trap item that a model answers from memory rather than from the prompt.
Log application timestamp, email pattern, and coarse geolocation — and review them as a batch, not one by one. Fraud is visible in the distribution, not the individual case.

Layer 3 — Transcript audit (after the interview)

Flag transcripts where response length and structure are near-constant across all questions.
Flag answers that stay abstract after two consecutive requests for a specific episode.
Check for internal contradictions between the screener claim and the narrative.
Compare response latency to the length of the answer — a long, polished response delivered instantly is worth a second look.
Decide the exclusion rule before reading the data, and record every exclusion with its reason.

That last point is a methodological safeguard, not a fraud one. Excluding participants after seeing whether their answers support your hypothesis is a researcher-degrees-of-freedom problem regardless of how justified each individual exclusion feels.

Where Qualitati fits

Qualitati is an AI user research platform for product, UX, and customer insights teams, running AI-moderated interviews and focus groups in text and voice, conversational surveys, and AI thematic analysis across up to 100 transcripts at a time.

Several parts of the platform map onto the layers above. AI-moderated interviews can be configured to probe for specific episodes rather than accept a first answer, which is the transcript-level defense in Layer 3 — a moderator that asks "when exactly, and what happened next" produces data that is measurably harder to fake. Voice mode raises the cost of a paste-and-return loop, though it does not eliminate it. Conversational surveys with AI-driven follow-ups are a natural place to put a Layer 2 screener that adapts to the answer given. And because ThemeLens anchors every theme to participant quotes, an excluded transcript can be removed and the analysis re-run without unpicking a hand-built codebook.

We do not claim automated fraud detection as a product feature. The screen above is a research-operations practice, and the honest framing is that the platform makes some layers easier to run, not that it removes the need for them.

Limitations and trade-offs

Every control here costs something.

Tighter recruitment reduces sample diversity. Closed lists and customer databases are safer and less representative. For exploratory research on populations you do not already reach, that trade is expensive.
Traps degrade. A cognitive trap that circulates publicly stops working, and model capabilities move. Treat any specific trap item as perishable.
False positives harm real participants. Non-native speakers, people using assistive technology, and anyone who simply writes tidily can look "too structured." Exclusion should require converging signals, never one flag.
Prevalence estimates do not transfer. The 4–8% and the 10%-to-80% figures above describe specific platforms at specific dates. Your rate is an empirical question about your channel.
None of this is a compliance or security claim. Fraud screening is about data validity, and it is separate from consent, privacy, and IRB obligations.

Human-review note: exclusion criteria and any participant-level fraud determination should be reviewed by a named researcher before they affect published findings or compensation decisions.

Who this is for, and when not to use it

This is for research operations leads, UX researchers, and insights teams running incentivized studies through open or panel-based recruitment. It is over-engineered for internal research with colleagues, for unpaid studies with no incentive to fake, and for interviews scheduled individually with people you already know. In those settings, Layer 3 alone is plenty.

FAQ

How common are fraudulent participants in online research?

It depends almost entirely on the channel. Xu and Malkin (2026) found LLM-assisted responses under 10% on Prolific and over 80% on Mechanical Turk; Conjointly's May 2026 analysis estimated 4–8% suspected bots on its own platform, or 2–3% excluding speeders.

Does asking participants not to use AI work?

Partly. Xu and Malkin tested explicit requests along with disabling copy-paste and shortening surveys; usage fell, but the authors report data quality did not necessarily improve.

Can an AI moderator detect a fraudulent participant during the interview?

Not reliably, and no responsible platform should claim it can. What an AI moderator can do is probe for specific episodes and concrete detail, which produces transcripts where fraud is easier for a human reviewer to spot afterward.

Is voice interviewing safer than text?

It raises the effort required to route questions through a chatbot, but it is not a solution — voice can be read aloud, and voice adds its own accessibility and transcription-accuracy considerations.

Should I exclude a participant based on one red flag?

No. Single indicators produce false positives against non-native speakers and neatly-writing participants. Require converging evidence, define the rule before analysis, and document every exclusion.

Bottom line

Fraudulent participants are now a routine operating condition of incentivized online research, not an edge case, and the AI-moderated interview is not self-protecting. The workable defense is layered and boring: control the recruiting channel, design a screener that is hard to fake, and audit the transcript with a rule you wrote before you looked at the data. None of the three layers is optional, and none of them is sufficient.

Start free with 30 credits — no credit card required — or see transparent pricing before you run your next AI-moderated study.

Sources

Xu, Z., & Malkin, N. (2026). A Penny for Your Prompts: Experiments Detecting and Mitigating LLM Usage by Survey Respondents. SOUPS 2026. arXiv:2607.00403
Conjointly (18 May 2026, updated 10 August 2026). We went looking for bots. conjointly.com
Ziminski, D., & Liddell-Quinty, E. (December 2025). Strategies to identify potential fraudulent participants in online qualitative research. BMJ Open Quality. Rutgers School of Public Health summary
NORC at the University of Chicago (2026). Fraudulent respondents and bots in nonprobability surveys: A literature review. norc.org

Last updated: 5 September 2026.

This article summarizes third-party research. Qualitati is not affiliated with the authors or platforms cited, and the interpretation above is our own. Competitor and platform statements reflect publicly available information as of 5 September 2026.