NIHSS training and certification using a new digital video disk is reliable

Background and Purpose: NIH Stroke Scale certification is required for participation in modern stroke clinical trials and as part of good clinical care in stroke centers. The existing training and certification videotapes, however, are more than 10 years old and do not contain an adequate balance of patient findings. Methods: After producing a new NIHSS training and demonstration DVD, we selected 18 patients representing all possible scores on 15 scale items for a new certification DVD. Patients were divided into 3 certification groups of 6 patients each, balanced for lesion side, distribution of scale item findings, and total score. We sought to measure interrater reliability of the certification DVD using methodology previously published for the original videotapes. Raters were recruited from 3 experienced stroke centers. Each rater watched the new training DVD and then evaluated one of the 3 certification groups. Results: Responses were received from 112 raters: 26.2% of all responses came from stroke nurses, 34.1% from emergency departments/other physicians, and 39.6% from neurologists. One half (50%) of raters were previously NIHSS-certified. Item responses were tabulated, scoring performed as previously published, and agreement measured with unweighted kappa coefficients for individual items and an intraclass correlation coefficient for the overall score. kappa ranged from 0.21 +/- 0.05 (ataxia) to 0.92 +/- 0.09 (LOC-C questions). Of 15 items, 2 showed poor, 11 moderate, and 2 excellent agreement based on kappa scores. The intraclass correlation coefficient for total score was 0.94 (95% confidence interval, 0.84 to 1.00). Reliability scores were similar among specialists and centers, and there were no differences between nurses and physicians. kappa scores trended higher among raters previously certified. Conclusions: These certification DVDs are reliable for NIHSS certification, and scoring sheets have been posted on a web site for real-time, online certification.