How well does decision support software perform in the emergency department?

Objective: To determine how well general decision support systems perform given the data collected in an emergency department (ED). Methods: A convenience sample of 25 patients was selected from those patients having a diagnostic question on presentation to the ED. All interactions with the patients were audiotaped and abstracted into a structured data form. All other data such as written notes, laboratory, and EKG results were also abstracted. All data were entered into two general diagnostic decision support programs (Quick Medical Reference (QMR Version 3.82, Knowledge Base 10-07-1998 Copyright University of Pittsburgh and The Hearst Corporation) and Iliad (Version 4.5 Copyright 1996 Applied Medical Informatics)). The diagnoses generated by the computer programs were compared with the final diagnoses of the ED attending. Results: The final ED diagnosis was found in the differential diagnosis generated by Iliad and QMR 72% and 52% of the time respectively. The final ED diagnosis was found in the top 10 diagnoses 51% and 44% of the time and in the top five diagnoses 36% and 32% of the time for each program respectively. This approximates to the performance of these programs in other clinical settings. Conclusions: Diagnostic decision support software has the same success in finding the "correct" diagnosis in the ED as in other clinical settings where more extensive clinical data are available. The accuracy is not sufficiently high to permit the use of these programs as an arbiter in any individual case. However, they may be useful, prompting additional investigation in particularly difficult cases.