Cross-scale backscattered-electron imaging and its application in revealing the microstructure-property relations

Abstract

Scanning electron microscopy (SEM) has been widely utilized in the field of materials science due to its significant advantages, such as large depth of field, wide field of view, and excellent stereoscopic imaging. However, at high magnification, the limited imaging range in SEM cannot cover all the possible inhomogeneous microstructures. In this research, we propose a novel approach for generating high-resolution SEM images across multiple scales, enabling a single image to capture physical dimensions at the centimeter level while preserving submicron-level details. We adopted the SEM imaging on the AlCoCrFeNi_2.1 eutectic high entropy alloy as an example. SEM videos and image stitching are combined to fulfill this goal, and the video-extracted low-definition images are clarified by a well-trained denoising model. Furthermore, we segment the macroscopic image of the eutectic high entropy alloy, and the area of various microstructures is distinguished. By combining the segmentation results and hardness experiments, we found that the hardness is positively correlated with the content of the body-centered cubic phase and negatively correlated with the lamella width. The whole procedure is also applied to a thermoelectric gradient material (PbSe-SrSe). Our work provides a feasible solution to generate macroscopic images based on SEM for further analysis of the correlations between the microstructures and spatial distribution, and can be widely applied to other types of microscopes.

Keywords

Cross-scale imaging / imaging denoising / imaging stitching / material microstructures

References

[1] Li X,Lu L,Gao H.Dislocation nucleation governed softening and maximum strength in nano-twinned metals.Nature2010;464:877-80

[2] Shi P,Li Y.Hierarchical crack buffering triples ductility in eutectic herringbone high-entropy alloys.Science2021;373:912-8

[3] Mu Y,Deng S.A high-entropy alloy with dislocation-precipitate skeleton for ultrastrength and ductility.Acta Mater2022;232:117975

[4] Nie J,Li D,Jiang Y.High-throughput characterization of microstructure and corrosion behavior of additively manufactured SS316L-SS431 graded material.Addit Manuf2020;35:101295

[5] Pegues JW,Puckett R,Argibay N.Exploring additive manufacturing as a high-throughput screening tool for multiphase high entropy alloys.Addit Manuf2021;37:101598

[6] Fang TH,Tao NR.Revealing extraordinary intrinsic tensile plasticity in gradient nano-grained copper.Science2011;331:1587-90

[7] Cheng Z,Lu Q,Lu L.Extra strengthening and work hardening in gradient nanotwinned metals.Science2018;362:eaau1925

[8] Kelly SM.Microstructural evolution in laser-deposited multilayer Ti-6Al-4V builds: Part II. Thermal modeling.Metall Mater Trans A2004;35:1869-79

[9] Kok Y,Wang P.Anisotropy and heterogeneity of microstructure and mechanical properties in metal additive manufacturing: a critical review.Mater Des2018;139:565-86

[10] Wang Z,Duan S.An image denoising method based on deep residual GAN.J Phys Conf Ser2020;1550:032127

[11] Singla K,Ghanekar U.A review on single image super resolution techniques using generative adversarial network.Optik2022;266:169607

[12] Yang W,Tian Y,Xue J.Deep learning for single image super-resolution: a brief review.IEEE Trans Multimedia2019;21:3106-21

[13] Chauhan K,Kumhar M.Deep learning-based single-image super-resolution: a comprehensive review.IEEE Access2023;11:21811-30

[14] Chen H,Qing L.Real-world single image super-resolution: a brief review.Inform Fusion2022;79:124-45

[15] Li K,Dong R,Huang J.Survey of single image super-resolution reconstruction.IET Image Process2020;14:2273-90

[16] Lepcha DC,Dogra A.Image super-resolution: a comprehensive review, recent trends, challenges and applications.Inform Fusion2023;91:230-60

[17] Dong C,He K.Image super-resolution using deep convolutional networks.IEEE Trans Pattern Anal Mach Intell2016;38:295-307

[18] Kim J,Lee KM.Accurate image super-resolution using very deep convolutional networks. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 27-30 June 2016; Las Vegas, NV, USA.

[19] Ledig C,Huszár F,Cunningham A.Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 21-26 July 2017; Honolulu, HI, USA.

[20] Zhang Y,Li K,Zhong B.Image super-resolution using very deep residual channel attention networks. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors, Computer Vision - ECCV 2018, Lecture Notes in Computer Science. Cham: Springer International Publishing; 2018. pp. 294-310.

[21] Liang J,Sun G,Van Gool L.SwinIR: image restoration using swin transformer. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW); 11-17 October 2021; Montreal, BC, Canada.

[22] Zhang S,Pan S.A fast medical image super resolution method based on deep learning network.IEEE Access2019;7:12319-27

[23] Li Y,Peyrin F.A review of the deep learning methods for medical images super resolution problems.IRBM2021;42:120-33

[24] Sood R,Choutagunta K,Rusu M.An application of generative adversarial networks for super resolution medical imaging. In Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA); 17-20 December 2018; Orlando, FL, USA.

[25] Liu H,Wu Y,Ibragimov B.Learning deconvolutional deep neural network for high resolution medical image reconstruction.Inf Sci2018;468:142-54

[26] Wang P,Sertel E.A comprehensive review on deep learning based remote sensing image super-resolution methods.Earth-Sci Rev2022;232:104110

[27] Li Z,Cheng Q,You S.Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors.ISPRS J Photogramm Remote Sens2019;150:197-212

[28] Lei S,Zou Z.Super-resolution for remote sensing images via local-global combined network.IEEE Geosci Remote Sensing Lett2017;14:1243-7

[29] Furat O,Yang Z,Smith K.Super-resolving microscopy images of Li-ion electrodes for fine-feature quantification using generative adversarial networks.NPJ Comput Mater2022;8:749

[30] Jangid DK,Goebel MG.Adaptable physics-based super-resolution for electron backscatter diffraction maps.NPJ Comput Mater2022;8:924

[31] Yin W,Borseth J.A petascale automated imaging pipeline for mapping neuronal circuits with high-throughput transmission electron microscopy.Nat Commun2020;11:4949 PMCID:PMC7532165

[32] Ma B,Huang H.A fast algorithm for material image sequential stitching.Comput Mater Sci2019;158:1-13

[33] Yang F,Fan QH.A method for fast automated microscope image stitching.Micron2013;48:17-25

[34] Kaynig V,Müller E.Fully automatic stitching and distortion correction of transmission electron microscope images.J Struct Biol2010;171:163-73

[35] Chalfoun J,Blattner T.MIST: accurate and scalable microscopy image stitching tool with stage modeling and error minimization.Sci Rep2017;7:4988 PMCID:PMC5504007

[36] Singla A,Graeb H.Recovery of 2D and 3D layout information through an advanced image stitching algorithm using scanning electron microscope images. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR); 10-15 January 2021; Milan, Italy.

[37] Ma B,Rohde M.Use of autostitch for automatic stitching of microscope images.Micron2007;38:492-9

[38] Wang X,Wu S.ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé L, Roth S, editors, Computer Vision - ECCV 2018 Workshops, Lecture Notes in Computer Science. Cham: Springer International Publishing; 2019. pp. 63-79.

[39] Goodfellow LJ,Mirza M. Generative adversarial nets. Available from: https://papers.nips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf [Last accessed on 3 Jan 2025]

[40] Ronneberger O,Brox T.U-net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in Computer Science. Cham: Springer International Publishing; 2015. pp. 234-41.

[41] Miyato T,Koyama M.Spectral normalization for generative adversarial networks. In Proceedings of the International Conference on Learning Representations. arXiv2018.

[42] Lu Y,Guo S.A promising new class of high-temperature alloys: eutectic high-entropy alloys.Sci Rep2014;4:6200 PMCID:PMC4145285

[43] Gao X,Zhang B.Microstructural origins of high strength and high ductility in an AlCoCrFeNi_2.1 eutectic high-entropy alloy.Acta Mater2017;141:59-66

[44] Yan P,Wang W,Lin M.Eutectic growth kinetics and microscopic mechanical properties of rapidly solidified CoCrFeNiMo_0.8 high entropy alloy.Acta Mater2022;237:118149

[45] Vikram R,Suwas S.Design of a new cobalt base nano-lamellar eutectic high entropy alloy.Scr Mater2021;202:113993

[46] Wischi M,Starck L,Lopes É.Microstructure and mechanical behavior of the directionally solidified AlCoCrFeNi_2.1 eutectic high-entropy alloy.J Mater Res Technol2022;20:811-20

[47] Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation.IEEE Trans Pattern Anal Mach Intell2017;39:640-51

[48] He K,Dollar P.Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV); 22-29 October 2017; Venice, Italy.

[49] Badrinarayanan V,Cipolla R.SegNet: a deep convolutional encoder-decoder architecture for image segmentation.IEEE Trans Pattern Anal Mach Intell2017;39:2481-95

[50] Wen Q,Yang X.PatchDCT: patch refinement for high quality instance segmentation. arXiv 2023.

[51] Bochkovskiy A,Liao HYM.YOLOv4: optimal speed and accuracy of object detection. arXiv 2024.

[52] Zhou Z,Tajbakhsh N.UNet++: a nested U-net architecture for medical image segmentation. In: Stoyanov D, Taylor Z, Carneiro G, et al., editors. Deep learning in medical image analysis and multimodal learning for clinical decision support. Cham: Springer International Publishing; 2018. pp. 3-11. PMCID:PMC7329239

[53] Bradley D.Adaptive thresholding using the integral image.J Grap Tools2007;12:13-21

[54] Wani IS,Sheikh S.Ultrafine-grained AlCoCrFeNi_2.1 eutectic high-entropy alloy.Mater Res Lett2016;4:174-9

[55] Nassar A,Cochrane R,Micklethwaite S.Rapid solidification of AlCoCrFeNi_2.1 high-entropy alloy.J Alloys Compd2022;900:163350

[56] Schuh CA.Nanoindentation studies of materials.Mater Today2006;9:32-40

[57] Karimzadeh A,Ayatollahi MR,Yahya MY.Assessment of nano-indentation method in mechanical characterization of heterogeneous nanocomposite materials using experimental and computational approaches.Sci Rep2019;9:15763 PMCID:PMC6823492

[58] Miller M,Vandamme M.Surface roughness criteria for cement paste nanoindentation.Cement Concrete Res2008;38:467-76

[59] Zhang R,Efros AA,Wang O.The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition; 18-23 June 2018; Salt Lake City, UT, USA.

[60] Jiang K,Xiao Y,Xu X.Image enhancement via associated perturbation removal and texture reconstruction learning.IEEE/CAA J Autom Sin2024;11:2253-69

[61] Tian C,Lin C,Zhang D.Heterogeneous window transformer for image denoising.IEEE Trans Syst Man Cybern Syst2024;54:6621-32

[62] He S,Li Z.A general strategy for high-throughput experimental screening of promising bulk thermoelectric materials.Sci China Mater2021;64:1751-60

[63] Shi P,Li Y.Multistage work hardening assisted by multi-type twinning in ultrafine-grained heterostructural eutectic high-entropy alloys.Mater Today2020;41:62-71

[64] Shi P,Zheng T.Enhanced strength-ductility synergy in ultrafine-grained eutectic high-entropy alloys by inheriting microstructural lamellae.Nat Commun2019;10:489 PMCID:PMC6353877

[65] Wada K,Buijs M. wkentaro/labelme: v4.6.0. Available from: https://zenodo.org/records/5711226 [Last accessed on 21 Jan 2025]

[66] Kingma DP.Adam: a method for stochastic optimization. arXiv 2024.

[67] BasicSR: open source image and video restoration toolbox; 2022. Available from: https://github.com/XPixelGroup/BasicSR [Last accessed on 3 Jan 2025]

[68] Russakovsky O,Su H.ImageNet large scale visual recognition challenge.Int J Comput Vis2015;115:211-52

[69] Milletari F,Ahmadi SA.V-net: fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV); 25-28 October 2016; Stanford, CA, USA.