Reliability and validity of estimating the NIH Stroke Scale score from medical records

Background and Purpose-The aim of our study was to determine whether the National Institutes of Health Stroke Scale (NIHSS) can be estimated retrospectively from medical records. The NIHSS is a quantitative measure of stroke-related neurological deficit with established reliability and validity for use in prospective clinical research. Recently, retrospective observational studies have estimated NIHSS scores from medical records for quantitative outcome analysis. The reliability and validity of estimation based on chart review has not been determined. Methods-Thirty-nine patients were selected because their NIHSS scores were formally measured at admission and discharge. Handwritten notes from medical records were abstracted and NIHSS scores were estimated by 6 raters who were blinded to the actual scores. Estimated scores were compared among raters and with the actual measured scores. Results-Interrater reliability was excellent, with an intraclass correlation coefficient of 0.82. Scores were well calibrated among the 6 raters. Estimated NIHSS scores closely approximated the actual scores, with a probability of 0.86 of correctly ranking a set of patients according to 5-point interval categories (as determined by the area under the receiver-operator characteristic curve). Patients with excellent outcomes (NIHSS score of less than or equal to 5) could be identified with sensitivity of 0.72 and specificity of 0.89. There were no significant differences between these parameters at admission and discharge. Conclusions-For the purposes of retrospective studies of acute stroke outcome, the NIHSS can be abstracted from medical records with a high degree of reliability and validity.