Combat data shift in few-shot learning with knowledge graph

Abstract

Many few-shot learning approaches have been designed under the meta-learning framework, which learns from a variety of learning tasks and generalizes to new tasks. These meta-learning approaches achieve the expected performance in the scenario where all samples are drawn from the same distributions (i.i.d. observations). However, in real-world applications, few-shot learning paradigm often suffers from data shift, i.e., samples in different tasks, even in the same task, could be drawn from various data distributions. Most existing few-shot learning approaches are not designed with the consideration of data shift, and thus show downgraded performance when data distribution shifts. However, it is non-trivial to address the data shift problem in few-shot learning, due to the limited number of labeled samples in each task. Targeting at addressing this problem, we propose a novel metric-based meta-learning framework to extract task-specific representations and task-shared representations with the help of knowledge graph. The data shift within/between tasks can thus be combated by the combination of task-shared and task-specific representations. The proposed model is evaluated on popular benchmarks and two constructed new challenging datasets. The evaluation results demonstrate its remarkable performance.

Graphical abstract

Keywords

few-shot / data shift / knowledge graph

References

[1] Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D. Matching networks for one shot learning. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3637− 3645

[2] Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 1126− 1135

[3] Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 4080− 4090

[4] Sung F, Yang Y, Zhang L, Xiang T, Torr P H S, Hospedales T M. Learning to compare: Relation network for few-shot learning. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 1199− 1208

[5] Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009, 248– 255

[6] Pan S J , Yang Q . A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22( 10): 1345– 1359

[7] Chen W Y, Liu Y C, Kira Z, Wang Y C F, Huang J B. A closer look at few-shot classification. In: Proceedings of the 7th International Conference on Learning Representations. 2019

[8] Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S. Deep hashing network for unsupervised domain adaptation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5385− 5394

[9] Fei-Fei L , Fergus R , Perona P . One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28( 4): 594– 611

[10] Lake B, Salakhutdinov R, Gross J, Tenenbaum J B. One shot learning of simple visual concepts. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society. 2011

[11] Koch G, Zemel R, Salakhutdinov R. Siamese neural networks for one-shot image recognition. In: Proceedings of the 32nd International Conference on Machine Learning. 2015

[12] Oreshkin B N, Rodriguez P, Lacoste A. TADAM: task dependent adaptive metric for improved few-shot learning. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 719– 729

[13] Li H, Dong W, Mei X, Ma C, Huang F, Hu B G. LGM-Net: learning to generate matching networks for few-shot learning. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 3825− 3834

[14] Allen K, Shelhamer E, Shin H, Tenenbaum J. Infinite mixture prototypes for few-shot learning. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 232– 241

[15] Liu L, Zhou T, Long G, Jiang J, Zhang C. Learning to propagate for graph meta-learning. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019

[16] Ravi S, Larochelle H. Optimization as a model for few-shot learning. In: Proceedings of the ICLR 2017. 2017

[17] Lee Y, Choi S. Gradient-based meta-learning with learned layerwise metric and subspace. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 2933− 2942

[18] Sun Q, Liu Y, Chua T S, Schiele B. Meta-transfer learning for few-shot learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 403– 412

[19] Cai Q, Pan Y, Yao T, Yan C, Mei T. Memory matching networks for one-shot image recognition. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4080− 4088

[20] Munkhdalai T, Yu H. Meta networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 2554− 2563

[21] Munkhdalai T, Yuan X, Mehri S, Trischler A. Rapid adaptation with conditionally shifted neurons. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 3664− 3673

[22] Peng Z, Li Z, Zhang J, Li Y, Qi G J, Tang J. Few-shot image recognition with knowledge transfer. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 441– 449

[23] Dong N, Xing E P. Domain adaption in one-shot learning. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2018, 573– 588

[24] Guan J, Lu Z, Xiang T, Wen J R. Few-shot learning as domain adaptation: algorithm and analysis. 2020, arXiv preprint arXiv: 2002.02050

[25] Tseng H Y, Lee H Y, Huang J B, Yang M H. Cross-domain few-shot classification via learned feature-wise transformation. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[26] Wang X, Ye Y, Gupta A. Zero-shot recognition via semantic embeddings and knowledge graphs. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 6857− 6866

[27] Kampffmeyer M, Chen Y, Liang X, Wang H, Zhang Y, Xing E P. Rethinking knowledge graph propagation for zero-shot learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019

[28] Zhuang F , Qi Z , Duan K , Xi D , Zhu Y , Zhu H , Xiong H , He Q . A comprehensive survey on transfer learning. Proceedings of the IEEE, 2021, 109( 1): 43– 76

[29] Wang J, Lan C, Liu C, Ouyang Y, Qin T. Generalizing to unseen domains: a survey on domain generalization. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence. 2021, 4627−4635

[30] Zhuang F, Cheng X, Luo P, Pan S J, He Q. Supervised representation learning: transfer learning with deep autoencoders. In: Proceedings of the 24th International Conference on Artificial Intelligence. 2015, 4119− 4125

[31] Ganin Y , Ustinova E , Ajakan H , Germain P , Larochelle H , Laviolette F , Marchand M , Lempitsky V . Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016, 17( 1): 2096– 2030

[32] Wang J, Chen Y, Hao S, Feng W, Shen Z. Balanced distribution adaptation for transfer learning. In: Proceedings of 2017 IEEE International Conference on Data Mining. 2017, 1129−1134

[33] Wang J, Feng W, Chen Y, Yu H, Huang M, Yu P S. Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM International Conference on Multimedia. 2018, 402–410

[34] Zhu Y, Zhuang F, Wang J, Chen J, Shi Z, Wu W, He Q. Multi-representation adaptation network for cross-domain image classification. Neural Networks, 2019, 119: 214– 221

[35] Xi D, Zhuang F, Zhou G, Cheng X, Lin F, He Q. Domain adaptation with category attention network for deep sentiment analysis. In: Proceedings of the Web Conference 2020. 2020, 3133− 3139

[36] Zhu Y, Ge K, Zhuang F, Xie R, Xi D, Zhang X, Lin L, He Q. Transfer-meta framework for cross-domain recommendation to cold-start users. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 1813− 1817

[37] Tzeng E, Hoffman J, Darrell T, Saenko K. Simultaneous deep transfer across domains and tasks. In: Proceedings of 2015 IEEE International Conference on Computer Vision. 2015, 4068− 4076

[38] Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2962− 2971

[39] Long M, Cao Y, Wang J, Jordan M. Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 97– 105

[40] Long M, Zhu H, Wang J, Jordan M I. Deep transfer learning with joint adaptation networks. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 2208− 2217

[41] Zhu Y , Zhuang F , Wang J , Ke G , Chen J , Bian J , Xiong H , He Q . Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32( 4): 1713– 1722

[42] Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D. Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 95– 104

[43] Hoffman J, Tzeng E, Park T, Zhu J Y, Isola P, Saenko K, Efros A, Darrell T. CyCADA: cycle-consistent adversarial domain adaptation. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1989− 1998

[44] Ghifary M, Kleijn W B, Zhang M, Balduzzi D, Li W. Deep reconstruction-classification networks for unsupervised domain adaptation. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 597– 613

[45] Scarselli F , Gori M , Tsoi A C , Hagenbuchner M , Monfardini G . The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20( 1): 61– 80

[46] Bruna J, Zaremba W, Szlam A, LeCun Y. Spectral networks and locally connected networks on graphs. In: Proceedings of the 2nd International Conference on Learning Representations. 2014

[47] Henaff M, Bruna J, LeCun Y. Deep convolutional networks on graph-structured data. 2015, arXiv preprint arXiv: 1506.05163

[48] Satorras V G, Estrach J B. Few-shot learning with graph neural networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[49] Kim J, Kim T, Kim S, Yoo C D. Edge-labeling graph neural network for few-shot learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 11– 20

[50] Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3844− 3852

[51] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[52] Zhuo J, Wang S, Cui S, Huang Q. Unsupervised open domain recognition by semantic discrepancy minimization. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 750– 759

[53] Salakhutdinov R, Torralba A, Tenenbaum J. Learning to share visual appearance for multiclass object detection. In: Proceedings of the CVPR 2011. 2011, 1481− 1488

[54] Wu Q, Wang P, Shen C, Dick A, Van Den Hengel A. Ask me anything: free-form visual question answering based on knowledge from external sources. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4622− 4630

[55] Long M, Cao Z, Wang J, Jordan M I. Conditional adversarial domain adaptation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 1647−1657

[56] Zhu Y , Zhuang F , Wang D . Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33 : 5989– 5996

[57] Zhang R, Che T, Ghahramani Z, Bengio Y, Song Y. MetaGAN: an adversarial approach to few-shot learning. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 2371− 2380

[58] Franceschi L, Frasconi P, Salzo S, Grazzi R, Pontil M. Bilevel programming for hyperparameter optimization and meta-learning. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1568− 1577

[59] Jiang X, Havaei M, Varno F, Chartrand G, Chapados N, Matwin S. Learning to learn with conditional class dependencies. In: Proceedings of the 7th International Conference on Learning Representations. 2019

[60] Sun B, Saenko K. Deep CORAL: correlation alignment for deep domain adaptation. In: Proceedings of the European Conference on Computer Vision. 2016, 443– 450

[61] Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on Machine Learning. 2013, 1139− 1147

[62] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770– 778

[63] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[64] Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A. Automatic differentiation in PyTorch. In: Proceedings of the 31st Conference on Neural Information Processing Systems. 2017

[65] Miller G A . WordNet: a lexical database for English. Communications of the ACM, 1995, 38( 11): 39– 41

[66] Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 1532− 1543

[67] Hariharan B, Girshick R. Low-shot visual recognition by shrinking and hallucinating features. In: Proceedings of 2017 IEEE International Conference on Computer Vision. 2017, 3037− 3046

[68] Gidaris S, Komodakis N. Dynamic few-shot visual learning without forgetting. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4367− 4375

RIGHTS & PERMISSIONS

Higher Education Press 2021