Study questions reliability of mental health diagnostic interviews
8 reported
A new study published in Jama Network Open has found that diagnostic interviews, the most common method for diagnosing substance use and mental disorders, vary in reliability depending on the condition. The review, led by Laura Duncan of McMaster University, analyzed test-retest reliability data from studies reviewed between February 2024 and September 2025. The authors used Cohen’s kappa coefficient to measure how often patients received the same diagnosis when given the same interview twice. The average reliability was highest for substance use disorders, particularly opioid use disorder, which Duncan attributed to criteria based on observable behavior. Dr. Michael First of Columbia University, author of the SCID diagnostic tool, criticized the study for not providing enough detail to compare specific instruments. Duncan responded that the necessary data to address those concerns does not yet exist in the available research.
What’s reported
The study was published in Jama Network Open.
It reviewed test-retest reliability of diagnostic interviews from February 2024 to September 2025.
Reliability was measured using Cohen’s kappa coefficient.
Substance use disorders, especially opioid use disorder, showed the highest average reliability.
Laura Duncan is a psychiatry professor at McMaster University in Ontario, Canada.
Dr. Michael First is a psychiatrist and professor at Columbia University who authored the Structured Clinical Interview for DSM 5 (SCID).
First said the study did not provide enough information to compare specific diagnostic instruments.
Duncan said the data needed to address First’s concerns does not exist yet.
Key figures
Laura Duncan, psychiatry professor at McMaster University, study author
Dr. Michael First, psychiatrist and professor at Columbia University, author of the SCID
Sources: The Guardian
