Learning with privileged information using Bayesian networks

Abstract

For many supervised learning applications, additional information, besides the labels, is often available during training, but not available during testing. Such additional information, referred to the privileged information, can be exploited during training to construct a better classifier. In this paper, we propose a Bayesian network (BN) approach for learning with privileged information. We propose to incorporate the privileged information through a three-node BN. We further mathematically evaluate different topologies of the three-node BN and identify those structures, through which the privileged information can benefit the classification. Experimental results on handwritten digit recognition, spontaneous versus posed expression recognition, and gender recognition demonstrate the effectiveness of our approach.

Keywords

Bayesian network / privileged information / classification / maximum likelihood estimation

References

[1] Bishop C M, Nasrabadi N M. Pattern recognition and machine learning, Volume 1. New York: Springer, 2006

[2] Vapnik V, Vashist A. A new learning paradigm: learning using privileged information. Neural networks: the official journal of the International Neural Network Society, 2009, 22(5−6): 544−557

[3] Chen J, Liu X, Lyu S. Boosting with side information. In: Computer Vision − ACCV 2012. 2013, 563−577

[4] Yang H, Patras I. Privileged information-based conditional regression forest for facial feature detection. In: Proceedings of 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). 2013, 1−6

[5] Fouad S, Tino P, Raychaudhury S, Schneider P. Incorporating privileged information through metric learning. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(7): 1086−1098

[6] Pechyony D, Izmailov R, Vashist A, Vapnik V. SMO-style algorithms for learning using privileged information. In: Proceedings of the 2010 International Conference on Data Mining. 2010, 235−241

[7] Pechyony D, Vapnik V. Fast optimization algorithms for solving SVM. Statistical Learning and Data Science, 2011

[8] Niu L, Wu J. Nonlinear l-1 support vector machines for learning using privileged information. In: Proceedings of the IEEE 12th International Conference on Data Mining Workshops. 2012, 495−499

[9] Liu J, Zhu W, Zhong P. A new multi-class support vector algorithm based on privileged information. Journal of Information & Computational Science, 2013, 10: 443−450

[10] Zhong P, Fukushima M. A new multi-class support vector algorithm. Optimisation Methods and Software, 2006, 21(3): 359−372

[11] Ji Y, Sun S, Lu Y. Multitask multiclass privileged information support vector machines. In: Proceedings of 21st International Conference on Pattern Recognition. 2012, 2323−2326

[12] Cai F. Advanced Learning Approaches Based on SVM+ Methodology. PhD thesis, University of Minnesota, 2011

[13] Koller D, Friedman N. Probabilistic graphical models: principles and techniques. MIT press, 2009

[14] Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278−2324

[15] Vapnik V, Vashist A, Pavlovitch N. Learning using hidden information: Master-class learning. NATO Science for Peace and Security Series, D: Information and Communication Security, 2008, 19: 3−14

[16] Smith C A, McHugo G J, Lanzetta J T. The facial muscle patterning of posed and imagery-induced expressions of emotion by expressive and nonexpressive posers. Motivation and Emotion, 1986, 10: 133−157

[17] Valstar M, Gunes H, Pantic M. How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the 9th International Conference on Multimodal Interfaces. 2007, 38−45

[18] Zhang L, Tjondronegoro D, Chandran V. Geometry vs. appearance for discriminating between posed and spontaneous emotions. In: Neural Information Processing, volume 7064, 431−440. Springer Berlin Heidelberg, 2011

[19] Cohn J, Schmidt K. The timing of facial motion in posed and spontaneous smiles. International Journal of Wavelets, Multiresolution and Information Processing, 2004, 2(02): 121−132

[20] Delannoy J, McDonald J. Estimation of the temporal dynamics of posed and spontaneous facial expression formation using LLE. In: Proceedings of 13th International Machine Vision and Image Processing Conference. 2009, 139−144

[21] Lithari C, Frantzidis C, Papadelis C, Vivas A, Klados M, Kourtidou- Papadeli C, Pappas C, Ioannides A, Bamidis P. Are females more responsive to emotional stimuli? a neurophysiological study across arousal and valence dimensions. Brain topography, 2010, 23(1): 27−40

[22] Saatci Y, Town C. Cascaded classification of gender and facial expression using active appearance models. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, 2006, 393−400

[23] Wang S, Liu Z, Lv S, Lv Y, Wu G, Peng P, Chen F, Wang X. A natural visible and infrared facial expression database for expression recognition and emotion inference. IEEE Transactions on Multimedia, 2010, 12(7): 682−691

[24] Wang P, Ji Q. Multi-view face and eye detection using discriminant features. Computer Vision and Image Understanding, 2007, 105(2): 99−111

[25] Zhang Y, Ji Q. Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(5): 699−714

[26] Smeeton N C. Early history of the kappa statistic, 1985

[27] Makinen E, Raisamo R. Evaluation of gender classification methods with automatically detected and aligned faces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(3): 541−547

[28] Chen C, Ross A. Evaluation of gender classification methods on thermal and near-infrared face images. In: Proceedings of 2011 International Joint Conference on Biometrics. 2011, 1−8

[29] Cootes T. Modelling and search software.

[30] Ding C. Analysis of gene expression profiles: class discovery and leaf ordering. In: Proceedings of the 6th Annual International Conference on Computational Biology. 2002, 127−136

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg