EFFECTS OF COLLINEARITY ON INFORMATION ABOUT REGRESSION-COEFFICIENTS

This paper uses information theory to examine effects of nonorthogonality of a regression matrix on data and posterior information about the regression coefficients. In the absence of an informative prior, the entropy analysis results in collinearity measures that are akin to some well-known diagnostics available for the least squares regressions. Entropy difference and cross-entropy analyses are used to quantify collinearity effects on the sample information and posterior distribution. Information-theoretic diagnostics are developed for examining the incremental contribution of additional prior precision to reduction of collinearity effects on the sample and posterior information. The proposed collinearity measures are applied to an illustrative example. © 1990.