A GAN based method for multiple prohibited items synthesis of X-ray security image

Abstract

Detecting prohibited item based on convolutional neural networks (CNNs) is of great significance to ensure public safety. However, the natural occurrence of such prohibited items is a small-probability event, collecting enough datasets to support CNN training is a big challenge. In this paper, we propose a new method for synthesizing X-ray security image with multiple prohibited items from semantic label images basing on Generative Adversarial Networks (GANs). Theoretically, we can use it to synthesize as many X-ray images as needed. A new generator architecture with Res2Net is presented, which is more effective in learning multi-scale features of different prohibited items images. This method is extended by establishing the semantic label library which contains 14 000 images. So we totally synthesize 14 000 X-ray security images. The experimental results show the super performance (Fréchet Inception Distance (FID) score of 30.55). And we achieve 0.825 of mean average precision (mAP) with Single Shot MultiBox Detector (SSD) for object detection, demonstrating the effectiveness of our approach.

References

[1] MeryD, SvecE, AriasM. Object Recognition in Baggage Inspection Using Adaptive Sparse Representations of X-ray Images, 2015, Cham, Switzerland, Springer: 709

[2] Bhowmik N, Wang Q and Gaus Y F A, The Good, the Bad and the Ugly: Evaluating Convolutional Neural Networks for Prohibited Item Detection Using Real and Synthetically Composited X-ray Imagery, arXiv preprint arXiv:1909.11508, (2019).

[3] MeryD, RiffoV, ZscherpelU. Journal of Nondestructive Evaluation, 2015, 34: 42

[4] Miao C, Xie L, Wan F, Su C, Liu H, Jiao J and Ye Q, SIXray: A Large-scale Security Inspection X-ray Benchmark for Prohibited Item Discovery in Overlapping Images, Conference on Computer Vision and Pattern Recognition, 2119 (2019).

[5] Zhao T, Zhang H, Zhang Y and J Yang, X-Ray Image with Prohibited Items Synthesis Based on Generative Adversarial Network, Chinese Conference on Biometric Recognition, 379 (2019).

[6] Heusel M, Ramsauer H, Unterthiner T and Nessler B, GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, Advances in Neural Information Processing Systems, 6626 (2017).

[7] HintonG E, SalakhutdinovR. Science, 2006, 313: 504

[8] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y and Berg A C, SSD: Single Shot Multibox Detector, European Conference on Computer Vision, 21 (2016).

[9] GoodfellowI, Pouget-AbadieJ, MirzaM, XuB, Warde-FarleyD, OzairS, CourvilleA, BengioY. Generative Adversarial Nets, Advances in Neural Information Processing Systems, 2014, 27: 2672

[10] BauD, StrobeltH, PeeblesW, WulffJ. ACM Transactions on Graphics (TOG), 2019, 38: 1

[11] Mo S, Cho M and Shin J, InstaGAN: Instance-aware Image-to-Image Translation, arXiv preprint arXiv:1812.10889, (2018).

[12] Isola P, Zhu J Y and Zhou T, Image-to-Image Translation with Conditional Adversarial Networks, Conference on Computer Vision and Pattern Recognition, 1125 (2017).

[13] Zhu J Y, Park T and Isola P, Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, IEEE International Conference on Computer Vision (ICCV), 2223 (2017).

[14] Yi Z, Zhang H, Tan P and Gong M, DualGAN: Unsupervised Dual Learning for Image-To-Image Translation, IEEE International Conference on Computer Vision (ICCV), 2849 (2017).

[15] KimT, ChaM, KimH. Learning to Discover Cross-Domain Relations with Generative Adversarial Networks, The 34th International Conference on Machine Learning, 2017, 70: 1857

[16] Wang T C, Liu M Y and Zhu J Y, pix2pixHD: HighResolution Image Synthesis and Semantic Manipulation with Conditional GANs, IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018).

[17] Park T, Liu M Y, Wang T C and Zhu J Y, Semantic Image Synthesis with Spatially-Adaptive Normalization, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2337 (2019).

[18] Gao S H, Cheng M M and Zhao K, Res2Net: A New Multi-scale Backbone Architecture, arXiv preprint arXiv:1904.01169, (2019).

[19] Mejjati Y A, Richardt C, Tompkin J, Cosker D and Kim K I, Unsupervised Attention-Guided Image-to-Image Translation, The 32nd International Conference on Neural Information Processing, 3693 (2018).