Image aesthetic quality evaluation using convolution neural network embedded learning

Abstract

A way of embedded learning convolution neural network (ELCNN) based on the image content is proposed to evaluate the image aesthetic quality in this paper. Our approach can not only solve the problem of small-scale data but also score the image aesthetic quality. First, we chose Alexnet and VGG_S to compare for confirming which is more suitable for this image aesthetic quality evaluation task. Second, to further boost the image aesthetic quality classification performance, we employ the image content to train aesthetic quality classification models. But the training samples become smaller and only using once fine-tuning cannot make full use of the small-scale data set. Third, to solve the problem in second step, a way of using twice fine-tuning continually based on the aesthetic quality label and content label respective is proposed, the classification probability of the trained CNN models is used to evaluate the image aesthetic quality. The experiments are carried on the small-scale data set of Photo Quality. The experiment results show that the classification accuracy rates of our approach are higher than the existing image aesthetic quality evaluation approaches.

References

[1] DattaR, JoshiD, LiJ. Studying Aesthetics in Photographic Images Using a Computational Approach, 2006, 288

[2] LuoW, WangX, TangX. Content-Based Photo Quality Assessment, 2011, 2206

[3] ShaoJ, ZhouY. Journal of Computational Information Systems, 2013, 9: 3209

[4] DharS, OrdonezV, BergL T. High Level Describable Attributes for Predicting Aesthetics and Interestingness, 2011, 1657

[5] ObradorP, Schmidt-HackenbergL, OliverN. The role of Image Composition in Image Aesthetics, 2010, 3185

[6] WangC, PuY, XuD. Evaluating aesthetics quality in scenery images, 2015, 141

[7] WangC, PuY, XuD, ZhuJ, TaoZ. Journal of Software, 2015, 26: 20

[8] LuX, LinZ, JinH, YangJ, WangJ. RAPID: Rating Pictorial Aesthetics using Deep Learning, 2014, 457

[9] GuoL, LiF. Image Aesthetic Evaluation Using Paralleled Deep Convolution Neural Network, 2015,

[10] ZhouY, LuX, ZhangJ, WangJ. Joint Image and Text Representation for Aesthetics Analysis, 2016, 262

[11] DongZ, ShenX, LiH, TianX. Photo Quality Assessment with DCNN that Understands Image Well, 2015, 524

[12] TianX, DongZ, YangK, MeiT. IEEE Transactions on Multimedia, 2015, 17: 2035

[13] MurrayN, MarchesottiL, PerronninF. AVA: A Large-Scale Database for Aesthetic Visual Analysis, 2012, 2408

[14] GengH-q, ZhangH, XueY-b, ZhouM, XuG-p, GaoZ. Optoelectronics Letters, 2017, 13: 381

[15] XuF, LiuJ-h. Optoelectronic Letters, 2016, 12: 473

[16] KrizhevskyA, SutskeverI, HintonG. ImageNet Classification with Deep Convolutional Neural Networks, 2012, 1097

[17] ChatfieldK, SimonyanK, VedaldiA, ZissermanA. Return of the Devil in the Details: Delving Deep into Convolutional Nets, 2014,

[18] JiaY, ShelhamerE, DonahueJ, KarayevS, LongJ, GirshickR, GuadarramaS, DarrellT. Caffe: Convolutional Architecture for Fast Feature Embedding, 2014, 675

This work has been supported by the National Natural Science Foundation of China (Nos.61271361, 61163019, 61462093 and 61761046), the Research Foundation of Yunnan Province (Nos.2014FA021 and 2014FB113), and the Digital Media Technology Key Laboratory of Universities in Yunnan Province. This paper was presented in part at the CCF Chinese Conference on Computer Vision, Tianjin, 2017. This paper was recommended by the program committee.