Intercenter agreement in reading Doppler embolic signals - A multicenter international study

Background and Purpose Different frequencies of asymptomatic Doppler embolic signals have been reported in studies. There has been concern that different criteria for identification may account for some of this variation. A previous reproducibility study between two centers found good agreement, but no studies among large numbers of centers have been performed. We performed an international reproducibility study among nine centers, each of which had published recent studies of embolic signal detection in peer-reviewed journals. Methods Each center performed blinded analysis of a taped audio Doppler signal composed of transcranial Doppler middle cerebral artery recordings from 6 patients with symptomatic carotid artery stenosis. The exact time of any embolic signal was recorded. Six centers also measured the intensity increase of any embolic signals detected. Interobserver agreement was determined by a method based on the proportion of specific agreement. Results Seven centers reported between 39 and 55 signals, but one center reported 142 embolic signals. The probability of agreement between observers was .675, which rose to .791 when the data from the highest reporting center were excluded. Introducing a decibel threshold resulted in a significant increase in the probability of agreement; a decibel threshold of >7 dB resulted in a probability of agreement of .902. Intensity measurements made by different centers were usually highly correlated, but this was not always the case, and 3 of the 15 correlations were not significant. The absolute values of the intensities measured varied between centers by as much as 40%. Conclusions Although most centers report similar numbers of embolic signals, some use less specific criteria and report more events. The use of a decibel threshold improves reproducibility. However, intensity thresholds developed by one center cannot be directly transferred without validation to another center; differing methods of measurement are being used, and this results in different intensity values for the same embolic signals, even when the same equipment is used.