SMOOTHING CATEGORICAL-DATA

Statistical analysis of categorical data (contingency tables) has a long history, and a good deal of work has been done formulating parametric models for such data. Unfortunately, such analyses are often not appropriate, due to sparseness of the table. An alternative to these parametric models is smoothing the table, by 'borrowing' information from neighboring cells. In this paper, various strategies that have been proposed for such smoothing are discussed. It is shown that these strategies have close ties to other areas of statistical methodology, including shrinkage estimation, Bayes methods, penalized likelihood, spline estimation, and kernel density and regression estimation. Probability estimates based on smoothing methods can outperform the unsmoothed frequency estimates when the table is sparse (often, dramatically so). Methods for one-dimensional tables are discussed, as well as generalizations to higher-dimensional tables. Attempts to use smoothed probability estimates in statistical functionals are identified. Finally, potential future work in categorical data smoothing is also mentioned.