Fast meta-models for local fusion of multiple predictive models

被引:41
作者
Bonissone, Piero P. [1 ]
Xue, Feng [1 ]
Subbu, Raj [1 ]
机构
[1] Gen Elect Global Res, Niskayuna, NY 12309 USA
关键词
Bootstrap; Fusion; Neural networks; Ensemble; CART; Prediction; NEURAL-NETWORKS; CLASSIFIERS; COMBINATION; CONFIDENCE;
D O I
10.1016/j.asoc.2008.03.006
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Fusing the outputs of an ensemble of diverse predictive models usually boosts overall prediction accuracy. Such fusion is guided by each model's local performance, i.e., each model's prediction accuracy in the neighborhood of the probe point. Therefore, for each probe we instantiate a customized fusion mechanism. The fusion mechanism is a meta-model, i.e., a model that operates one level above the object-level models whose predictions we want to fuse. Like these models, such a meta-model is defined by structural and parametric information. In this paper, we focus on the definition of the parametric information for a given structure. For each probe point, we either retrieve or compute the parameters to instantiate the associated meta-model. The retrieval approach is based on a CART-derived segmentation of the probe's state space, which contains the meta-model parameters. The computation approach is based on a run-time evaluation of each model's local performance in the neighborhood of the probe. We explore various structures for the meta-model, and for each structure we compare the pre-compiled (retrieval) or run-time (computation) approaches. We demonstrate this fusion methodology in the context of multiple neural network models. However, our methodology is broadly applicable to other predictive modeling approaches. This fusion method is illustrated in the development of highly accurate models for emissions, efficiency, and load prediction in a complex power plant. The locally weighted fusion method boosts the predictive performance by 30-50% over the baseline single model approach for the various prediction targets. Relative to this approach, typical fusion strategies that use averaging or globally weighting schemes only produce a 2-6% performance boost over the same baseline. (C) 2008 Elsevier B. V. All rights reserved.
引用
收藏
页码:1529 / 1539
页数:11
相关论文
共 26 条
[1]  
[Anonymous], 1989, GERNALIZED LINEAR MO
[2]  
Bonissone P, 2004, LECT NOTES COMPUT SC, V3077, P154
[3]  
BONISSONE P, 2005, P 2005 MULT CLASS SY, P376
[4]   Random forests [J].
Breiman, L .
MACHINE LEARNING, 2001, 45 (01) :5-32
[5]   Random forests [J].
Breiman, L .
MACHINE LEARNING, 2001, 45 (01) :5-32
[6]  
Breiman L., 1984, Classification and regression trees, DOI DOI 10.1201/9781315139470
[7]   Prediction intervals for neural networks via nonlinear regression [J].
De Veaux, RD ;
Schumi, J ;
Schweinsberg, J ;
Ungar, LH .
TECHNOMETRICS, 1998, 40 (04) :273-282
[8]  
Efron B., 1993, INTRO BOOTSTRAP, DOI 10.1007/978-1-4899-4541-9
[9]   NEURAL NETWORKS AND THE BIAS VARIANCE DILEMMA [J].
GEMAN, S ;
BIENENSTOCK, E ;
DOURSAT, R .
NEURAL COMPUTATION, 1992, 4 (01) :1-58
[10]   Optimal linear combinations of neural networks [J].
Hashem, S .
NEURAL NETWORKS, 1997, 10 (04) :599-614