Two-level hierarchical feature learning for image classification

Abstract

In some image classification tasks, similarities among different categories are different and the samples are usually misclassified as highly similar categories. To distinguish highly similar categories, more specific features are required so that the classifier can improve the classification performance. In this paper, we propose a novel two-level hierarchical feature learning framework based on the deep convolutional neural network (CNN), which is simple and effective. First, the deep feature extractors of different levels are trained using the transfer learning method that fine-tunes the pre-trained deep CNN model toward the new target dataset. Second, the general feature extracted from all the categories and the specific feature extracted from highly similar categories are fused into a feature vector. Then the final feature representation is fed into a linear classifier. Finally, experiments using the Caltech-256, Oxford Flower-102, and Tasmania Coral Point Count (CPC) datasets demonstrate that the expression ability of the deep features resulting from two-level hierarchical feature learning is powerful. Our proposed method effectively increases the classification accuracy in comparison with flat multiple classification methods.

Keywords

Transfer learning / Feature learning / Deep convolutional neural network / Hierarchical classification / Spectral clustering

References

[1] Angelova, A., Zhu, S., Lin, Y., 2013. Image segmentation for large-scale subcategory flower recognition. Proc. IEEE Workshop on Applications of Computer Vision, p.39–45. http://dx.doi.org/10.1109/WACV.2013.6474997

[2] Barrett, N., Meyer, L., Hill, N., , 2011. Methods for the Processing and Scoring of AUV Digital Imagery from South Eastern Tasmania. Technical Report. University of Tasmania, Hobart.

[3] Bewley, M.S., Douillard, B., Nourani-Vatani, N., , 2012. Automated species detection: an experimental approach to kelp detection from sea-floor AUV images. Proc. Australasian Conf. on Robotics and Automation, p.1–10.

[4] Bewley, M.S., Nourani-Vatani, N., Rao, D., , 2015. Hierarchical classification in AUV imagery. Proc. 9th Int. Conf. on Field and Service Robotics, p.3–16. http://dx.doi.org/10.1007/978-3-319-07488-7_1

[5] Cai, Y., Yang, M.L., Li, J., 2015. Multiclass classification based on a deep convolutional network for head pose estimation. Front. Inform. Technol. Electron. Eng., 16(11):930–939. http://dx.doi.org/10.1631/FITEE.1500125

[6] Deng, J., Satheesh, S., Berg, A.C., , 2011. Fast and balanced: efficient label tree learning for large scale object recognition. Advances in Neural Information Processing Systems 24, p.567–575.

[7] Deng, J., Ding, N., Jia, Y.Q., , 2014. Large-scale object classification using label relation graphs. Proc. 13th European Conf. on Computer Vision, p.48–64. http://dx.doi.org/10.1007/978-3-319-10590-1_4

[8] Donahue, J., Jia, Y., Vinyals, O., , 2014. DeCAF: a deep convolutional activation feature for generic visual recognition. Proc. 31st Int. Conf. on Machine Learning, p.1–9.

[9] Fergus, R., Bernal, H., Weiss, Y., , 2010. Semantic label sharing for learning with many categories. Proc. 11th European Conf. on Computer Vision, p.762–775. http://dx.doi.org/10.1007/978-3-642-15549-9_55

[10] Ge, Z.Y., McCool, C., Sanderson, C., , 2015. Subset feature learning for fine-grained category classification. Proc. IEEE Conf. on Computer Vision and Pattern Recognition Workshops, p.46–52. http://dx.doi.org/10.1109/CVPRW.2015.7301271

[11] Girshick, R., Donahue, J., Darrell, T., , 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.580–587. http://dx.doi.org/10.1109/cvpr.2014.81

[12] Griffin, G., Perona, P., 2008. Learning and using taxonomies for fast visual categorization. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.1–8. http://dx.doi.org/10.1109/cvpr.2008.4587410

[13] Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, p.1097–1105.

[14] Liu, B.Y., Sadeghi, F., Tappen, M., , 2013. Probabilistic label trees for efficient large scale image classification. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.843–850. http://dx.doi.org/10.1109/CVPR.2013.114

[15] Nilsback, M.E., 2009. An Automatic Visual Flora—Segmentation and Classification of Flower Images. PhD Thesis, University of Oxford, UK.

[16] Nilsback, M.E., Zisserman, A., 2008. Automated flower classification over a large number of classes. Proc. 6th Indian Conf. on Computer Vision, p.722–729. http://dx.doi.org/10.1109/ICVGIP.2008.47

[17] Razavian, A.S., Azizpour, H., Sullivan, J., , 2014. CNN features off-the-shelf: an astounding baseline for recognition. Proc. IEEE Conf. on Computer Vision and Pattern Recognition Workshops, p.512–519. http://dx.doi.org/10.1109/CVPRW.2014.131

[18] Srivastava, N., Salakhutdinov, R.R., 2013. Discriminative transfer learning with tree-based priors. Advances in Neural Information Processing Systems 26, p.2094–2102.

[19] Tousch, A.M., Herbin, S., Audibert, J.Y., 2012. Semantic hierarchies for image annotation: a survey. Patt. Recogn., 45(1):333–345. http://dx.doi.org/10.1016/j.patcog.2011.05.017

[20] Yan, Z.C., Zhang, H., Piramuthu, R., , 2015. HDCNN: hierarchical deep convolutional neural networks for large scale visual recognition. Proc. Int. Conf. on Computer Vision, p.2740–2748.

[21] Yosinski, J., Clune, J., Bengio, Y., , 2014. How transferable are features in deep neural networks? Advances in Neural Information Processing Systems 27, p.3320–3328.

[22] Zeiler, M.D., Fergus, R., 2014. Visualizing and understanding convolutional networks. Proc. 13th European Conf. on Computer Vision, p.818–833. http://dx.doi.org/10.1007/978-3-319-10590-1_53

[23] Zhao, B., Li, F., Xing, E.P., 2011. Large-scale category structure aware image categorization. Advances in Neural Information Processing Systems 24, p.1251–1259.

RIGHTS & PERMISSIONS

2016 Zhejiang University and Springer-Verlag Berlin Heidelberg