Combining supervised classifiers with unlabeled data

Abstract

Ensemble learning is a wildly concerned issue. Traditional ensemble techniques are always adopted to seek better results with labeled data and base classifiers. They fail to address the ensemble task where only unlabeled data are available. A label propagation based ensemble (LPBE) approach is proposed to further combine base classification results with unlabeled data. First, a graph is constructed by taking unlabeled data as vertexes, and the weights in the graph are calculated by correntropy function. Average prediction results are gained from base classifiers, and then propagated under a regularization framework and adaptively enhanced over the graph. The proposed approach is further enriched when small labeled data are available. The proposed algorithms are evaluated on several UCI benchmark data sets. Results of simulations show that the proposed algorithms achieve satisfactory performance compared with existing ensemble methods.

Keywords

correntropy / unlabeled data / regularization framework / ensemble learning

References

[1] ZhangJ, OuJ-p, ZhanR-hui. Automatic target recognition of moving target based on empirical mode decomposition and genetic algorithm support vector machine [J]. Journal of Central South University, 2015, 22(4): 1389-1396

[2] YuanY-f, ShawM J. Induction of fuzzy decision trees [J]. Fuzzy Sets Syst, 1995, 69(2): 125-139

[3] MitchellT MMachine learning [M], 1997New YorkMcGraw-Hill Press

[4] KunchevaL I, WhitakerC J. Measures of diversity in classifier ensembles [J]. Machine Learning, 2003, 51: 181-207

[5] SchapireR. The strength of weak learn ability [J]. Machine Learning, 1990, 5(2): 197-227

[6] YinX-c, LiuC-p, HanZhi. Feature combination using boosting [J]. Pattern Recognition Letters, 2005, 26(13): 2195-2205

[7] BreimanL. Bagging predictors [J]. Machine Learning, 1996, 24(1): 123-140

[8] BelletA, HabrardA, MorvantE, SebbanM. Learning a priori constrained weighted majority votes [J]. Machine Learning, 2014, 97: 129-154

[9] MorvantE, HabrardA, AyacheS. Majority vote of diverse classifiers for late fusion [J]. Structural, Syntactic, and Statistical Pattern Recognition Lecture Notes in Computer Science, 2014, 8621153-162

[10] HeY-l, LiuJ, HuY-x, WangX-zhao. OWA operator based link prediction ensemble for social network [J]. Expert Systems with Applications, 2015, 42(1): 21-50

[11] LudwigO, NunesU, RibeiroB, PremebidaC. Improving the generalization capacity of cascade classifiers [J]. IEEE Transactions on Cybernetics, 2013, 43(6): 2135-2146

[12] KunchevaL ICombining pattern classifiers: Methods and algorithms [M], 2003HobokenWiley-Interscience Press

[13] KunchevaL I. Switching between selection and fusion in combining classifiers: An experiment [J]. IEEE Transaction on Cybernetics, 2002, 32(2): 146-156

[14] MehdiS H, AbedinV, HadiS Y. Extending Dempster-Shafer method by multilayer decision template in classifier fusion [J]. Expert System with Applications, 2011, 38(7): 8414-8418

[15] ShaferGA Mathematical theory of evidence [M], 1976Princeton, NJ, USAPrinceton Univ Press

[16] ZhangL-y, PengZ-r, LiL, WangHua. Road boundary estimation to improve vehicle detection and tracking in UAV video [J]. Journal of Central South University, 2014, 21(12): 4732-4741

[17] DengX-y, ZhengX, SuX-y, ChanF, HuY, SadiqR, DengYong. An evidential game theory framework in multi-criteria decision making process [J]. Applied Mathematics and Computation, 2014, 244: 783-793

[18] LiuC-lin. Classifier combination based on confidence transformation [J]. Pattern Recognition, 2005, 38(1): 11-28

[19] ZhangL, ZhouW-da. Sparse ensembles using weighted combination methods based on linear programming [J]. Pattern Recognition, 2011, 44(1): 97-106

[20] YinX-c, HuangK-z, YangC, HaoH-wei. Convex ensemble learning with sparsity and diversity [J]. Information Fusion, 2014, 20: 49-59

[21] KessentiniY, BurgerT, PaquetT. A Dempster–Shafer theory based combination of handwriting recognition systems with multiple rejection strategies [J]. Pattern Recognition, 2015, 48: 534-544

[22] WangX-z, WangR, FengH-m, WangH-chao. A new approach to classifier fusion based on upper integral [J]. IEEE Trans on Cybernetics, 2014, 44(5): 620-635

[23] KangX-d, LiS-t, FangL-y, LiM-x, BenediktssonJ. Extended random walker-based classification of hyperspectral images [J]. IEEE Trans on Geoscience and Remote Sensing, 2015, 53(1): 144-153

[24] SunZ-b, SongQ-b, ZhuX-y, SunH-l, XuB-w, ZhouY-ming. A novel ensemble method for classifying imbalanced data [J]. Pattern Recognition, 2015, 48(5): 1623-1637

[25] LiuW-f, PokharelP P, PrincipeJ C. Correntropy: Properties and applications in non-gaussian signal processing [J]. IEEE Trans. Signal Processing, 2007, 55(11): 5286-5298

[26] GardeA, SornmoL, JaneR, GiraldoB F. Correntropy- based analysis of respiratory patterns in patients with chronic heart failure [C]//. 31st Annual International Conference of the IEEE EMBS Minneapolis, 2009

[27] VladimirVThe nature of statistical learning theory [M], 1995New YorkSpringer-Verlag Press

[28] SantamariaI, PokharelP P, PrincipeJ C. Generalized correlation function: definition, properties, and application to blind equalization [J]. IEEE Transactions on Signal Processing, 2006, 54(6): 2187-2197

[29] ZhuX-jin. Semi-supervised learning literature survey [R]. Computer Sciences, 2008

[30] ZhouD-y, BousquetO, LalT N, WestonJ, ScholkopfB. Learning with local and global consistency [J]. Advances in Neural Information Processing Systems, 2004, 16: 321-328

[31] FRANK A, ASUNCION A. UCI machine learning repository [EB/OL] [2013-07-29]. http://archive.ics.uci.edu/html.

[32] PolikarR. Ensemble based systems in decision making [J]. IEEE Circuits and Systems Magazine, 2006, 6: 21-45