{"slug":"delphi-method-qualitative-research-2026","title":"Delphi Method in Research: Steps, Consensus and Examples","description":"What is the Delphi method? How a Delphi study works step by step, how to define consensus, ACCORD and CREDES reporting, AI-assisted rounds and limits.","keywords":"delphi method, delphi study, delphi technique, modified delphi, delphi consensus definition, expert panel consensus, ACCORD reporting guideline, CREDES, consensus methods research, AI-assisted delphi","date":"2026-11-15","author":"Qualitati Research Team","category":"Qualitative Methods","readTime":"10 min read","content":"<div class=\"short-answer\"><p><strong>Short answer:</strong> The Delphi method is a structured way to reach expert consensus through several anonymous rounds. A panel answers questions, the researcher summarizes the group's responses, and panelists revise their ratings after seeing that feedback. Rounds stop when a consensus rule set in advance is met or a fixed number of rounds ends. Its strength is transparency. Its weakness is that consensus is not the same as being correct.</p></div>\n\n<p><em>Last updated: November 15, 2026</em></p>\n\n<p>When evidence is thin, contested or still emerging, researchers often turn to expert judgment. The <strong>Delphi method</strong> turns that judgment into a documented, repeatable process instead of one loud meeting. It is used to set research priorities, build reporting guidelines, define competencies, forecast trends and agree on clinical or educational standards. This guide explains how a Delphi study works, the decisions you have to make before round one, how to define consensus, and where AI tools help or should be kept out. It ends with a design sheet you can copy into a protocol.</p>\n\n<h2>Key takeaways</h2>\n<ul>\n<li>A Delphi study has four defining features: a selected expert panel, anonymity between panelists, two or more rounds, and controlled feedback of group results between rounds.</li>\n<li>The consensus rule (for example, at least 75% of panelists rating an item 7 to 9 on a 9-point scale) must be fixed in the protocol before data collection.</li>\n<li>A \"classic\" Delphi opens with an open-ended qualitative round. A \"modified\" Delphi starts from statements drawn from a literature review or an earlier study.</li>\n<li>Reporting is now guided by checklists such as ACCORD (2024) and, in palliative care, CREDES (2017).</li>\n<li>AI can help draft statements and summarize open comments, but recent evidence supports AI assistance with human review, not AI replacing the panel.</li>\n</ul>\n\n<h2>What is the Delphi method?</h2>\n\n<p>The Delphi method is a consensus technique in which a panel of experts answers a series of questionnaires in rounds, receives an anonymous summary of the group's answers after each round, and may revise its own answers in light of that summary. It was developed at the RAND Corporation in the 1950s and 1960s for forecasting questions where data did not exist; RAND keeps a <a href=\"https://www.rand.org/topics/delphi-method.html\" target=\"_blank\" rel=\"noopener\">collection of its Delphi research</a>, including Dalkey and Helmer's early experimental application.</p>\n\n<p>The design tries to keep the benefit of group judgment while removing the usual problems of face-to-face meetings: dominant personalities, status effects, and pressure to agree quickly. Because panelists never see who said what, a junior researcher's view carries the same weight as a well-known professor's.</p>\n\n<h3>Delphi vs focus groups vs nominal group technique</h3>\n\n<table>\n<thead><tr><th>Feature</th><th>Delphi</th><th>Nominal group technique</th><th>Focus group</th></tr></thead>\n<tbody>\n<tr><td>Goal</td><td>Measure and build consensus</td><td>Generate and rank ideas</td><td>Explore views and meanings</td></tr>\n<tr><td>Anonymity</td><td>Yes, between panelists</td><td>Partial (silent writing, then open discussion)</td><td>No</td></tr>\n<tr><td>Interaction</td><td>Indirect, via summarized feedback</td><td>Structured, in one meeting</td><td>Direct, moderated discussion</td></tr>\n<tr><td>Typical format</td><td>Online questionnaires over weeks</td><td>One session of a few hours</td><td>One session of 60 to 120 minutes</td></tr>\n<tr><td>Output</td><td>Items with agreement levels per round</td><td>Ranked list</td><td>Themes and quotes</td></tr>\n</tbody>\n</table>\n\n<p>If you want to understand <em>why</em> people hold a view, a <a href=\"/blog/what-is-a-focus-group-2026\">focus group</a> or interviews are a better fit. If you need a defensible statement of what a field agrees on, Delphi is the usual choice.</p>\n\n<h2>How a Delphi study works, step by step</h2>\n\n<h3>1. Define the question and the type of Delphi</h3>\n<p>Write one clear aim, such as \"agree on the core reporting items for AI-assisted qualitative coding.\" Then decide whether you run a classic Delphi, with an open first round, or a modified Delphi that starts with draft statements. Modified designs are faster; classic designs are better when you do not yet know the right questions.</p>\n\n<h3>2. Select the panel</h3>\n<p>Write explicit expertise criteria (publications, years of practice, role, lived experience) and decide which stakeholder groups must be represented. There is no single correct panel size. Hsu and Sandford's <a href=\"https://doi.org/10.7275/pdz9-th90\" target=\"_blank\" rel=\"noopener\">2007 overview of the Delphi technique</a> notes that the literature gives no firm rule on panel size. Published panels range widely; the two studies cited later in this guide recruited 50 and 72 panelists. Plan for attrition: the ACCORD panel, for example, started with 72 people and finished its three rounds with 51.</p>\n\n<h3>3. Round 1: elicit or rate</h3>\n<p>In a classic Delphi, round 1 asks open questions (\"What should a methods section report when AI coded part of the data?\"). The researcher codes the answers into a list of distinct statements. This is qualitative analysis, and it deserves the same care as any <a href=\"/blog/thematic-analysis-guide\">thematic analysis</a>: keep an audit trail of how each statement came from the raw comments. In a modified Delphi, panelists rate the draft statements straight away and can suggest new ones.</p>\n\n<h3>4. Feedback and re-rating</h3>\n<p>Each later round shows panelists the group result for every item (for example, the median and the share rating it 7 to 9), often with their own previous rating and a summary of anonymous comments. They then rate again. Items that reached consensus may be removed or kept for confirmation; items far from consensus may be reworded.</p>\n\n<h3>5. Stop and report</h3>\n<p>Stop when your pre-set rule is met: consensus reached on all items, stability between rounds, or a maximum number of rounds (two to three is common). Report every item, including those that never reached consensus.</p>\n\n<h2>How to define consensus in a Delphi study</h2>\n\n<p>Consensus is the most criticized part of Delphi practice. A <a href=\"https://doi.org/10.1016/j.jclinepi.2013.12.002\" target=\"_blank\" rel=\"noopener\">systematic review by Diamond and colleagues</a> (Journal of Clinical Epidemiology, 2014) looked at 100 Delphi studies and found that only 72 defined consensus at all. The most common definition was percent agreement, and thresholds varied widely.</p>\n\n<p>A workable rule has three parts:</p>\n<ul>\n<li><strong>Scale:</strong> for example, a 9-point scale where 1 to 3 means \"not important,\" 4 to 6 \"important but not critical\" and 7 to 9 \"critical.\"</li>\n<li><strong>Threshold:</strong> for example, \"consensus in\" if at least 75% rate 7 to 9 and fewer than 15% rate 1 to 3; \"consensus out\" if at least 75% rate 1 to 3.</li>\n<li><strong>Stopping:</strong> for example, a maximum of three rounds, or stop when no item changes category between two rounds.</li>\n</ul>\n\n<p>The 75% figure is common rather than correct. What matters is that you choose it before seeing the data and justify it in the protocol. Changing the threshold after round 2 to get a cleaner result is the Delphi equivalent of p-hacking.</p>\n\n<h2>Delphi Design Decision Sheet</h2>\n\n<p>Use this ten-point sheet when drafting or reviewing a Delphi protocol. Each line should have a written answer before round 1 opens.</p>\n\n<ol>\n<li><strong>Aim:</strong> one sentence. What will the consensus be used for?</li>\n<li><strong>Type:</strong> classic (open round 1) or modified (seeded statements). Source of the seed statements.</li>\n<li><strong>Expertise criteria:</strong> who counts as an expert, and how you verify it.</li>\n<li><strong>Stakeholder mix:</strong> required groups and minimum numbers per group.</li>\n<li><strong>Recruitment target:</strong> number invited, number needed to finish, expected attrition.</li>\n<li><strong>Rating scale:</strong> points, anchors, and an \"unable to judge\" option.</li>\n<li><strong>Consensus rule:</strong> threshold for \"in,\" threshold for \"out,\" and how sub-group disagreement is handled.</li>\n<li><strong>Feedback content:</strong> which statistics, whether own previous rating is shown, how comments are summarized and by whom.</li>\n<li><strong>Stopping rule:</strong> maximum rounds and stability criterion.</li>\n<li><strong>Reporting:</strong> which checklist you will follow, and where the full item list will be published.</li>\n</ol>\n\n<h2>Reporting standards: ACCORD and CREDES</h2>\n\n<p>Two checklists are worth knowing. <a href=\"https://doi.org/10.1371/journal.pmed.1004326\" target=\"_blank\" rel=\"noopener\">ACCORD</a> (Gattrell et al., PLOS Medicine, 2024) is a 35-item reporting guideline for consensus methods in biomedicine, itself developed through a modified Delphi. It covers Delphi, nominal group technique and consensus meetings. <a href=\"https://doi.org/10.1177/0269216317690685\" target=\"_blank\" rel=\"noopener\">CREDES</a> (Jünger et al., Palliative Medicine, 2017) gives guidance on conducting and reporting Delphi studies, with specific attention to how the method is adapted. Even outside health research, they work well as a checklist for reviewers' predictable questions.</p>\n\n<h2>Can AI help run a Delphi study?</h2>\n\n<p>The evidence is early but growing. In a <a href=\"https://medinform.jmir.org/2026/1/e90228\" target=\"_blank\" rel=\"noopener\">2026 study in JMIR Medical Informatics</a>, Nello and colleagues randomized 50 panelists into two arms on 28 statements about RNA-based medicines. Statements below a 75% agreement threshold were revised either by humans or by an AI pipeline whose output humans checked. Consensus after the first round was 20 of 28 statements in the human arm and 13 of 28 in the AI-assisted arm. After revision it rose to 26 of 28 and 24 of 28. Both arms left two or three statements unresolved because the disagreement was substantive, not a wording problem.</p>\n\n<p>Our reading: AI is useful for the labor-heavy steps (turning hundreds of open comments into candidate statements, drafting rewordings, summarizing round feedback) when a person checks every output. It cannot supply the expertise the panel is there for, and it should never be used to simulate panelists. This needs human review in each study: check that AI-drafted statements do not drift toward the model's own framing.</p>\n\n<h2>Where Qualitati fits</h2>\n\n<p>Qualitati is a European, budget-friendly qualitative research platform for universities and research companies, built with data privacy and GDPR as priorities. It does not run a Delphi study for you, but it can support the qualitative parts:</p>\n<ul>\n<li><strong>Round 1 elicitation:</strong> run the open round as an <a href=\"/blog/ai-moderated-interview-discussion-guide-2026\">AI-moderated interview</a> in text or voice, or as a conversational survey that asks a follow-up when an answer is vague. Panelists answer on their own schedule.</li>\n<li><strong>Turning comments into statements:</strong> ThemeLens analyzes up to 100 transcripts at once and returns themes with participant-anchored quotes, which gives you a traceable starting list of candidate statements to edit by hand.</li>\n<li><strong>Rationales between rounds:</strong> conversational surveys can ask panelists to explain a rating that departs from the group, giving you richer comments to feed back anonymously.</li>\n<li><strong>International panels:</strong> interviews and surveys run in 10 languages, which helps when a panel spans countries.</li>\n</ul>\n<p>Who this is for: PhD students, research labs and research firms running Delphi or modified Delphi studies that start with an open qualitative round. Rating rounds can stay in your usual survey tool. A free account includes 30 monthly credits with no credit card; see <a href=\"/pricing\">pricing</a> for paid plans.</p>\n\n<h2>Limitations and when not to use Delphi</h2>\n<ul>\n<li><strong>Consensus is not truth.</strong> A panel can agree confidently on something wrong, especially if it lacks diversity.</li>\n<li><strong>Panel selection drives the result.</strong> Who you invite largely decides what \"the field\" agrees on. Report selection criteria in full.</li>\n<li><strong>Feedback pushes people toward the median.</strong> Some convergence reflects conformity, not persuasion. Showing reasons, not just numbers, helps.</li>\n<li><strong>Attrition is selective.</strong> Panelists who disagree may be more likely to drop out, which inflates agreement.</li>\n<li><strong>Not for exploring experience.</strong> If your question is about how people live or interpret something, use interviews, focus groups or <a href=\"/blog/ethnography-qualitative-research-guide-2026\">ethnography</a> instead.</li>\n<li><strong>Not when good evidence exists.</strong> If a systematic review can answer the question, expert opinion should not replace it.</li>\n</ul>\n\n<h2>Frequently asked questions</h2>\n\n<h3>What is the Delphi method in simple terms?</h3>\n<p>It is a way of reaching expert agreement through several rounds of anonymous questionnaires. After each round, panelists see a summary of the group's answers and can change their own.</p>\n\n<h3>Is the Delphi method qualitative or quantitative?</h3>\n<p>It is usually mixed. Open first rounds and comment analysis are qualitative; ratings, percentages and stopping rules are quantitative.</p>\n\n<h3>How many rounds should a Delphi study have?</h3>\n<p>Two to three rounds is common. Set a maximum in advance, and stop earlier if your consensus or stability rule is met.</p>\n\n<h3>How many experts do you need for a Delphi panel?</h3>\n<p>There is no fixed rule, and published panels range from small specialist groups to more than 70 people in guideline work. Recruit more than you need, because people drop out between rounds.</p>\n\n<h3>What is a modified Delphi?</h3>\n<p>A modified Delphi starts from statements drawn from a literature review or earlier research instead of an open first round. Some modified designs also add a final meeting.</p>\n\n<h3>What percentage counts as consensus in Delphi?</h3>\n<p>Percent agreement is the most common definition, and thresholds around 70% to 80% are frequent. The right choice is the one you justify and fix before data collection.</p>\n\n<h2>Bottom line</h2>\n\n<p>The <strong>Delphi method</strong> is a transparent way to find out what a defined group of experts agrees on, and how strongly. Its value depends on decisions made before round 1: who is on the panel, what counts as consensus and when to stop. Write those down, follow ACCORD or CREDES when you report, and treat AI as a drafting assistant under human review, not a panelist.</p>\n\n<p>Planning an open first round? <a href=\"/register\">Start free with 30 credits</a> and run it as an AI-moderated interview or conversational survey, then use ThemeLens to build your statement list. Or <a href=\"/pricing\">view transparent pricing</a>.</p>\n","related":[{"slug":"vignette-technique-qualitative-interviews-2026","title":"Vignette Technique in Qualitative Interviews (2026 Guide)","description":"What is the vignette technique in qualitative interviews? Definition, when to use it, a 10-point design checklist, a worked example and its limits.","category":"Qualitative Methods","date":"2026-11-14","readTime":"9 min read","author":"Qualitati Research Team"},{"slug":"ethnography-qualitative-research-guide-2026","title":"Ethnography in Qualitative Research: Types & Steps","description":"Ethnography in qualitative research explained: types (focused, rapid, digital, netnography), participant observation, fieldnotes, 8 steps and what AI can do.","category":"Qualitative Methods","date":"2026-11-08","readTime":"14 min read","author":"Qualitati Research Team"},{"slug":"critical-incident-technique-qualitative-research-2026","title":"Critical Incident Technique: Steps, Guide & Example","description":"The critical incident technique (Flanagan, 1954) explained: five steps, an interview guide with probes, a worked example, and the nine ECIT credibility checks.","category":"Qualitative Methods","date":"2026-10-27","readTime":"10 min read","author":"Qualitati Research Team"}]}