Tile selection method based on error minimization for photomosaic image creation

Abstract

Photomosaic images are composite images composed of many small images called tiles. In its overall visual effect, a photomosaic image is similar to the target image, and photomosaics are also called “montage art”. Noisy blocks and the loss of local information are the major obstacles in most methods or programs that create photomosaic images. To solve these problems and generate a photomosaic image in this study, we propose a tile selection method based on errorminimization. A photomosaic image can be generated by partitioning the target image in a rectangular pattern, selecting appropriate tile images, and then adding them with a weight coefficient. Based on the principles of montage art, the quality of the generated photomosaic image can be evaluated by both global and local error. Under the proposed framework, via an error function analysis, the results show that selecting a tile image using a global minimum distance minimizes both the global error and the local error simultaneously. Moreover, the weight coefficient of the image superposition can be used to adjust the ratio of the global and local errors. Finally, to verify the proposed method, we built a new photomosaic creation dataset during this study. The experimental results show that the proposed method achieves a lowmean absolute error and that the generated photomosaic images have a more artistic effect than do the existing approaches.

Keywords

photomosaic image / tile image / target image / error minimization / mean absolute error

References

[1] Yang X, Mei T, Xu Y Q, Rui Y, Li S. Automatic generation of visualtextual presentation layout. ACM Transactions on Multimedia Computing, Communications, and Applications, 2016, 12(2): 1–22

[2] Gatys L A, Ecker A S, Bethge M. Image style transfer using convolutional neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 2414–2423

[3] Gao P, Wu J, Lin Y, Xia Y, Mao T. Fast Chinese calligraphic character recognition with large-scale data. Multimedia Tools and Applications, 2015, 74(17): 1–18

[4] Seo S, Kang D. A photomosaic image generation method using photo annotation in a social network environment. Multimedia Tools and Applications, 2016, 75(20): 12831–12841

[5] Lee H Y. Generation of photo-mosaic images through block matching and color adjustment. International Journal of Computer and Information Engineering, 2014, 8(3): 457–460

[6] Blasi G D, Petralia M. Fast photomosaic. In: Proceedings of International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision. 2005, 1–2

[7] Deligiannis N, Comelis B, Rodrigues M R, Daubechies I. Multi-modal dictionary learning for image separation with application in art investigation. IEEE Transactions on Image Processing, 2016, 26(2): 751–764

[8] Li C L, Su Y, Wang R Z. Extended photomosaic with QR code capability. In: Proceedings of IEEE International Conference on Multimedia and Expo Workshops. 2017, 345–350

[9] Silvers R, Michael H. Photomosaics. New York: Henry Holt and Company, 1997

[10] Narasimhan H, Satheesh S. A randomized iterative improvement algorithm for photomosaic generation. In: Proceedings of World Congress on Nature and Biologically Inspired Computing. 2009, 777–781

[11] He Y, Zhou J, Yuen S Y. Composing photomosaic images using clustering based evolutionary programming. Multimedia Tools and Applications, 2019, 78(18): 25919–25936

[12] Lee H Y, Automatic photomosaic algorithm through adaptive tiling and block matching. Multimedia Tools and Applications, 2017, 76(22): 24281–24297

[13] Fujisawa M, Amano T, Taketomi T, Yamamoto G, Uranishi Y, Miyazaki J. Interactive photomosaic system using GPU. In: Proceedings of ACM International Conference on Multimedia. 2012, 1297–1298

[14] Yang Y, Ito Y, Nakano K. Photomosaic generation by rearranging subimages, with GPU acceleration. In: Proceedings of IEEE International Parallel and Distributed Processing Symposium Workshops. 2017, 942–951

[15] Chavan A S, Manjrekar A A. Data embedding technique using secret fragment visible mosaic image for covered communication. In: Proceedings of International Conference on Information Processing. 2016, 260–265

[16] William P, Lumsden J, Nabney I T. The Mosaic test: measuring the effectiveness of colour-based image retrieval. Multimedia Tools and Applications, 2013, 64(3): 695–716

[17] Chen D, Yuan L, Liao J, Yu N, Hua G. StyleBank: an explicit representation for neural image style transfer. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2770–2779

[18] Chu H K, Chang C S, Lee R R, Mitra N J. Halftone QR codes. ACM Transactions on Graphics, 2013, 32(6): 217

[19] Xu M, Su H, Li Y, Li X, Liao J, Niu J, Lv P, Zhou B. Stylized aesthetic QR code. IEEE Transactions on Multimedia, 2019, 21(8): 1960–1970

[20] Yu Z, Lu L, Yanwen Guo Y, Fan R, Liu M, Wang W. Content-aware photo collage using circle packing. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(2): 182–195

[21] Liu L, Zhang H, Jing G, Guo Y, Chen Z, Wang W. Correlation-preserving photo collage. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(6): 1956–1968

[22] Wang J, Quan L, Sun J, Tang X, Shum H Y. Picture collage. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2006, 347–354

[23] Liu T, Wang J, Sun J, Zheng N, Xiaoou Tang X, Shum H Y. Picture collage. IEEE Transactions on Multimedia, 2009, 11(7): 1225–1239

[24] Fang F, Yi M, Feng H, Hu S, Xiao C. Narrative collage of image collections by scene graph recombination. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(9): 2559–2572

[25] Kuhn H W. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 1955, 2(1): 83–97

[26] Kumar N, Berg A C, Belhumeur P N, Nayaret S K. Attribute and simile classifiers for face verification. In: Proceedings of IEEE International Conference on Computer Vision. 2009, 365–372

RIGHTS & PERMISSIONS

Higher Education Press