Can an Open-Source 7B Model Run Research Interviews?

Qualitati Research Team · 2026-08-17 · 7 min read

An AI interviewer built on a 7-billion-parameter open-source model collected qualitative interview data that was just as relevant and nearly as specific as data collected by trained human interviewers. In a 2026 pilot study of 40 participants and 946 responses, humans elicited longer individual answers, but the AI asked more questions in the same time — producing a larger corpus overall.

What was the study?

The study introduces AInterviewer, an open-source platform for designing and running AI-led qualitative interviews, described by Gardhus, Vitsakis, Frederiksen, Rogers, and Carlsen (2026) in a paper released on arXiv and demonstrated at ACL 2026. The authors' motivation is methodological rather than commercial: most AI interviewing systems run on proprietary models behind an API, which they argue "compromises reproducibility and data security" — a researcher cannot pin the model version, cannot inspect it, and often cannot keep transcripts on their own infrastructure.

Their answer is a system that can run a locally hosted model. The pilot reported in the paper used OpenHermes-2.5-Mistral-7B, an open-weights model small enough to run on a single workstation GPU — several orders of magnitude smaller than the frontier models most commercial AI interviewers rely on.

How does the system work?

Rather than handing a whole interview guide to one model and hoping it behaves, AInterviewer splits the job across a multi-agent pipeline. Each agent has one narrow, checkable responsibility:

Probing agent — generates contextually relevant follow-up questions from what the participant just said.
Reformulation agent — adapts the wording of the researcher's main questions to the conversation that has already happened, so the script does not read as a form being read aloud.
Classification agents — three binary classifiers that manage flow: has this topic already been discussed, has the participant refused to answer, and has this line of probing gone deep enough to move on.

This is the architectural point worth taking away. The controlled question administration of survey software is preserved — question order and core wording stay standardized across participants — while conversational adaptivity is added in a bounded way. It is the same design logic behind structured AI interviewing tools generally, including the AI Interviewer: the researcher's guide is the spine, and the model's freedom is scoped to probing.

How did AI-led interviews compare to human interviews?

Forty university participants were randomly assigned on site to either an AI or a human interviewer, yielding 946 responses. The authors coded the data on three constructs commonly used to judge qualitative interview quality:

Metric	What it captures	Human interviewer	AI interviewer
Exposure	Response length and number of questions answered	Longer individual responses (+38%, p = 0.470)	More questions asked in the same session
Relevance	Does the answer address the interview topic (5-point scale)	M = 4.9	M = 4.9
Specificity	Concrete detail of lived experience (5-point scale)	M = 3.8	M = 3.5

According to Gardhus et al. (2026), the 38% advantage in response length for human interviewers was not statistically significant (p = 0.470) at this sample size, and no significant differences emerged on specificity or relevance. Both conditions scored 4.9 out of 5 on relevance, which suggests neither interviewer type had much trouble keeping participants on topic.

The more interesting result is a throughput effect the length comparison hides. Because the AI formulated its next question faster than a human moderator, AI-led sessions covered more ground and produced a substantially larger overall corpus — even though each individual answer trended shorter.

What does this mean for researchers?

Three practical implications follow, in descending order of confidence.

Model size may not be the binding constraint on interview quality. A 7B open-weights model matched trained humans on relevance and came within 0.3 scale points on specificity. If that holds up in larger studies, the deciding factors for AI interviewing are architecture, prompt design, and interview-guide quality — not access to the largest available model. That is good news for anyone whose ethics board or funder rules out sending transcripts to a commercial API.

Corpus size and answer depth are separate outcomes. An AI interviewer that produces more total text is not automatically producing richer text. When you evaluate an AI-moderated study, look at per-response depth alongside volume; a larger transcript that is thinner per answer changes what your analysis can support. This matters most downstream, where the depth of individual accounts — not word count — determines whether thematic analysis yields interpretable themes.

Reproducibility is a real, under-discussed argument for local models. A proprietary model that is silently updated between waves of data collection is an uncontrolled variable in your method section. A pinned open-weights checkpoint is not.

What are the limits of this evidence?

The authors are explicit that these are pilot results and "should not be interpreted as the final verdict" on AI-led interviewing quality. Several caveats deserve weight:

Small, narrow sample. 40 participants, all university students recruited through campus outreach — not a population that generalizes to clinical, organizational, or vulnerable groups.
Null results are not equivalence. A non-significant difference at n = 40 is weak evidence of no difference; the study is underpowered to detect a modest real gap.
The constructs themselves are unsettled. The paper notes that evaluation methodology for AI-led interviewing remains underdeveloped, with no consensus on which constructs to measure or how to operationalize them.
No safeguards for sensitive disclosure. The authors state the system currently lacks protections for vulnerable populations and cannot handle disclosures requiring professional intervention — a hard constraint on the topics it should be pointed at.

FAQ

Can a small open-source model really conduct research interviews?

In this pilot, yes — a 7B open-weights model matched trained human interviewers on topic relevance (4.9 vs 4.9 on a 5-point scale) and came close on specificity (3.5 vs 3.8). But this is one study of 40 participants on a general-interest topic, not evidence that small models suffice for every research context.

Why would a researcher prefer a locally hosted model?

Two reasons the paper emphasizes: data security — transcripts never leave your infrastructure — and reproducibility, since a pinned open-weights checkpoint does not change under you between data-collection waves the way a commercial API endpoint can.

Do AI interviewers collect more or less data than humans?

Both, depending on what you count. In this study human interviewers drew longer individual responses (38% longer, though not statistically significant), while AI-led sessions asked more questions and produced a larger overall corpus.

Is a multi-agent pipeline better than a single prompted model?

This study did not test that comparison directly. Its design argument is that splitting probing, reformulation, and flow control into separate agents keeps question wording standardized across participants while still allowing adaptive follow-ups — a control property single-prompt systems are harder to guarantee.

Source

Gardhus, T. P., Vitsakis, N., Frederiksen, F. L., Rogers, A., & Carlsen, H. B. (2026). AInterviewer: A Platform for Designing and Conducting AI-led Qualitative Interviews. arXiv:2606.20588. Read the paper.

Last updated: 17 August 2026.

This is an independent editorial summary of third-party research. Qualitati is not affiliated with the authors, and all figures are drawn from the published paper.