Combat data shift in few-shot learning with knowledge graph

Abstract

Many few-shot learning approaches have been designed under the meta-learning framework, which learns from a variety of learning tasks and generalizes to new tasks. These meta-learning approaches achieve the expected performance in the scenario where all samples are drawn from the same distributions (i.i.d. observations). However, in real-world applications, few-shot learning paradigm often suffers from data shift, i.e., samples in different tasks, even in the same task, could be drawn from various data distributions. Most existing few-shot learning approaches are not designed with the consideration of data shift, and thus show downgraded performance when data distribution shifts. However, it is non-trivial to address the data shift problem in few-shot learning, due to the limited number of labeled samples in each task. Targeting at addressing this problem, we propose a novel metric-based meta-learning framework to extract task-specific representations and task-shared representations with the help of knowledge graph. The data shift within/between tasks can thus be combated by the combination of task-shared and task-specific representations. The proposed model is evaluated on popular benchmarks and two constructed new challenging datasets. The evaluation results demonstrate its remarkable performance.

Graphical abstract

Keywords

few-shot / data shift / knowledge graph

Yongchun Zhu is currently pursuing his MS degree in the Institute of Computing Technology, Chinese Academy of Sciences, China. He has published more than 10 papers in journals and conference proceedings including KDD, AAAI, WWW, SIGIR and so on. He received his BS degree from Beijing Normal University, China in 2018. His main research interests include transfer learning, meta learning and recommendation system

Fuzhen Zhuang is a professor in Institute of Artificial Intelligence, Beihang University, China. His research interests include transfer learning, machine learning, data mining, multi-task learning and recommendation systems. He has published more than 100 papers in the prestigious refereed journals and conference proceedings, such as IEEE TKDE, IEEE Transactions on Cybernetics, IEEE TNNLS, ACM TIST, SIGKDD, IJCAI, AAAI, WWW, and ICDE

Xiangliang Zhang is currently an Associate Professor and directs the Machine Intelligence and Knowledge Engineering (MINE) Laboratory at the Department of Computer Science and Engineering in University of Notre Dame, USA. She received the PhD degree in computer science from INRIA-University Paris-Sud, France in July 2010. She has authored or co-authored over 170 refereed papers in various journals and conferences. Her current research interests lie in designing machine learning algorithms for learning from complex and large-scale streaming data and graph data

Zhiyuan Qi is currently pursuing his MS degree in the University of California, USA. He received the BE degree in software engineering from Sun Yat-sen University, China in 2019. He has published several papers in journals and conference proceedings including Proceedings of the IEEE, IEEE Computational Intelligence Magazine, Neurocomputing, IJCAI, etc

Zhiping Shi is currently a professor in the College of Information Engineering at the Capital Normal University, China. From 2005 to 2010, he was on the faculty at the Institute of Computing Technology, Chinese Academy of Sciences where he received his PhD degree in computer software and theory in 2005. His research interests include formal verification and visual information analysis. He is the (co-) author of more than 100 research papers. He is a Member of the IEEE and the ACM

Juan Cao received the PhD degree from the Institute of Computing Technology, Chinese Academy of Sciences, China in 2008. She is currently working as an Professor with the Institute of Computing Technology, Chinese Academy of Sciences, China. Her research interests include multimedia content analysis, fake news detection, and forgery detection

Qing He is a Professor in the Institute of Computing Technology, Chinese Academy of Science (CAS), and a Professor at the Graduate University of Chinese (GUCAS), China. He received the BS degree from Hebei Normal University, China in 1985, and the MS degree from Zhengzhou University, China in 1987, both in mathematics. He received the PhD degree in 2000 from Beijing Normal University, China in fuzzy mathematics and artificial intelligence, China. From 1987 to 1997, he had been with Hebei University of Science and Technology, China. He is currently a doctoral tutor at the Institute of Computing and Technology, CAS, China. His interests include data mining, machine learning, classification, and fuzzy clustering

References

[1] Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D. Matching networks for one shot learning. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3637− 3645

[2] Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 1126− 1135

[3] Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 4080− 4090

[4] Sung F, Yang Y, Zhang L, Xiang T, Torr P H S, Hospedales T M. Learning to compare: Relation network for few-shot learning. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 1199− 1208

[5] Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009, 248– 255

[6] Pan S J , Yang Q . A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22( 10): 1345– 1359

[7] Chen W Y, Liu Y C, Kira Z, Wang Y C F, Huang J B. A closer look at few-shot classification. In: Proceedings of the 7th International Conference on Learning Representations. 2019

[8] Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S. Deep hashing network for unsupervised domain adaptation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5385− 5394

[9] Fei-Fei L , Fergus R , Perona P . One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28( 4): 594– 611

[10] Lake B, Salakhutdinov R, Gross J, Tenenbaum J B. One shot learning of simple visual concepts. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society. 2011

[11] Koch G, Zemel R, Salakhutdinov R. Siamese neural networks for one-shot image recognition. In: Proceedings of the 32nd International Conference on Machine Learning. 2015

[12] Oreshkin B N, Rodriguez P, Lacoste A. TADAM: task dependent adaptive metric for improved few-shot learning. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 719– 729

[13] Li H, Dong W, Mei X, Ma C, Huang F, Hu B G. LGM-Net: learning to generate matching networks for few-shot learning. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 3825− 3834

[14] Allen K, Shelhamer E, Shin H, Tenenbaum J. Infinite mixture prototypes for few-shot learning. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 232– 241

[15] Liu L, Zhou T, Long G, Jiang J, Zhang C. Learning to propagate for graph meta-learning. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019

[16] Ravi S, Larochelle H. Optimization as a model for few-shot learning. In: Proceedings of the ICLR 2017. 2017

[17] Lee Y, Choi S. Gradient-based meta-learning with learned layerwise metric and subspace. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 2933− 2942

[18] Sun Q, Liu Y, Chua T S, Schiele B. Meta-transfer learning for few-shot learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 403– 412

[19] Cai Q, Pan Y, Yao T, Yan C, Mei T. Memory matching networks for one-shot image recognition. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4080− 4088

[20] Munkhdalai T, Yu H. Meta networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 2554− 2563

[21] Munkhdalai T, Yuan X, Mehri S, Trischler A. Rapid adaptation with conditionally shifted neurons. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 3664− 3673

[22] Peng Z, Li Z, Zhang J, Li Y, Qi G J, Tang J. Few-shot image recognition with knowledge transfer. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 441– 449

[23] Dong N, Xing E P. Domain adaption in one-shot learning. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2018, 573– 588

[24] Guan J, Lu Z, Xiang T, Wen J R. Few-shot learning as domain adaptation: algorithm and analysis. 2020, arXiv preprint arXiv: 2002.02050

[25] Tseng H Y, Lee H Y, Huang J B, Yang M H. Cross-domain few-shot classification via learned feature-wise transformation. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[26] Wang X, Ye Y, Gupta A. Zero-shot recognition via semantic embeddings and knowledge graphs. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 6857− 6866

[27] Kampffmeyer M, Chen Y, Liang X, Wang H, Zhang Y, Xing E P. Rethinking knowledge graph propagation for zero-shot learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019

[28] Zhuang F , Qi Z , Duan K , Xi D , Zhu Y , Zhu H , Xiong H , He Q . A comprehensive survey on transfer learning. Proceedings of the IEEE, 2021, 109( 1): 43– 76

[29] Wang J, Lan C, Liu C, Ouyang Y, Qin T. Generalizing to unseen domains: a survey on domain generalization. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence. 2021, 4627−4635

[30] Zhuang F, Cheng X, Luo P, Pan S J, He Q. Supervised representation learning: transfer learning with deep autoencoders. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 4119− 4125

[31] Ganin Y , Ustinova E , Ajakan H , Germain P , Larochelle H , Laviolette F , Marchand M , Lempitsky V . Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016, 17( 1): 2096– 2030

[32] Wang J, Chen Y, Hao S, Feng W, Shen Z. Balanced distribution adaptation for transfer learning. In: Proceedings of 2017 IEEE International Conference on Data Mining. 2017, 1129−1134

[33] Wang J, Feng W, Chen Y, Yu H, Huang M, Yu P S. Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM International Conference on Multimedia. 2018, 402–410

[34] Zhu Y, Zhuang F, Wang J, Chen J, Shi Z, Wu W, He Q. Multi-representation adaptation network for cross-domain image classification. Neural Networks, 2019, 119: 214– 221

[35] Xi D, Zhuang F, Zhou G, Cheng X, Lin F, He Q. Domain adaptation with category attention network for deep sentiment analysis. In: Proceedings of the Web Conference 2020. 2020, 3133− 3139

[36] Zhu Y, Ge K, Zhuang F, Xie R, Xi D, Zhang X, Lin L, He Q. Transfer-meta framework for cross-domain recommendation to cold-start users. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 1813− 1817

[37] Tzeng E, Hoffman J, Darrell T, Saenko K. Simultaneous deep transfer across domains and tasks. In: Proceedings of 2015 IEEE International Conference on Computer Vision. 2015, 4068− 4076

[38] Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2962− 2971

[39] Long M, Cao Y, Wang J, Jordan M. Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 97– 105

[40] Long M, Zhu H, Wang J, Jordan M I. Deep transfer learning with joint adaptation networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 2208− 2217

[41] Zhu Y , Zhuang F , Wang J , Ke G , Chen J , Bian J , Xiong H , He Q . Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32( 4): 1713– 1722

[42] Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D. Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 95– 104

[43] Hoffman J, Tzeng E, Park T, Zhu J Y, Isola P, Saenko K, Efros A, Darrell T. CyCADA: cycle-consistent adversarial domain adaptation. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1989− 1998

[44] Ghifary M, Kleijn W B, Zhang M, Balduzzi D, Li W. Deep reconstruction-classification networks for unsupervised domain adaptation. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 597– 613

[45] Scarselli F , Gori M , Tsoi A C , Hagenbuchner M , Monfardini G . The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20( 1): 61– 80

[46] Bruna J, Zaremba W, Szlam A, LeCun Y. Spectral networks and locally connected networks on graphs. In: Proceedings of the 2nd International Conference on Learning Representations. 2014

[47] Henaff M, Bruna J, LeCun Y. Deep convolutional networks on graph-structured data. 2015, arXiv preprint arXiv: 1506.05163

[48] Satorras V G, Estrach J B. Few-shot learning with graph neural networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[49] Kim J, Kim T, Kim S, Yoo C D. Edge-labeling graph neural network for few-shot learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 11– 20

[50] Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3844− 3852

[51] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[52] Zhuo J, Wang S, Cui S, Huang Q. Unsupervised open domain recognition by semantic discrepancy minimization. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 750– 759

[53] Salakhutdinov R, Torralba A, Tenenbaum J. Learning to share visual appearance for multiclass object detection. In: Proceedings of the CVPR 2011. 2011, 1481− 1488

[54] Wu Q, Wang P, Shen C, Dick A, Van Den Hengel A. Ask me anything: free-form visual question answering based on knowledge from external sources. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4622− 4630

[55] Long M, Cao Z, Wang J, Jordan M I. Conditional adversarial domain adaptation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 1647−1657

[56] Zhu Y , Zhuang F , Wang D . Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33 : 5989– 5996

[57] Zhang R, Che T, Ghahramani Z, Bengio Y, Song Y. MetaGAN: an adversarial approach to few-shot learning. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 2371− 2380

[58] Franceschi L, Frasconi P, Salzo S, Grazzi R, Pontil M. Bilevel programming for hyperparameter optimization and meta-learning. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1568− 1577

[59] Jiang X, Havaei M, Varno F, Chartrand G, Chapados N, Matwin S. Learning to learn with conditional class dependencies. In: Proceedings of the 7th International Conference on Learning Representations. 2019

[60] Sun B, Saenko K. Deep CORAL: correlation alignment for deep domain adaptation. In: Proceedings of the European Conference on Computer Vision. 2016, 443– 450

[61] Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on Machine Learning. 2013, 1139− 1147

[62] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770– 778

[63] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[64] Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A. Automatic differentiation in PyTorch. In: Proceedings of the 31st Conference on Neural Information Processing Systems. 2017

[65] Miller G A . WordNet: a lexical database for English. Communications of the ACM, 1995, 38( 11): 39– 41

[66] Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 1532− 1543

[67] Hariharan B, Girshick R. Low-shot visual recognition by shrinking and hallucinating features. In: Proceedings of 2017 IEEE International Conference on Computer Vision. 2017, 3037− 3046

[68] Gidaris S, Komodakis N. Dynamic few-shot visual learning without forgetting. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4367− 4375

Acknowledgements

The research work was supported by the National Natural Science Foundation of China (Grant Nos. 62176014, U1836206, 61773361, U1811461).

RIGHTS & PERMISSIONS

2021 Higher Education Press 2021