A Dense Feature Iterative Fusion Network for Extracting Building Contours from Remote Sensing Imagery

Abstract

Extracting building contours from aerial images is a fundamental task in remote sensing. Current building extraction methods cannot accurately extract building contour information and have errors in extracting small-scale buildings. This paper introduces a novel dense feature iterative(DFI) fusion network, denoted as DFINet, for extracting building contours. The network uses a DFI decoder to fuse semantic information at different scales and learns the building contour knowledge, producing the last features through iterative fusion. The dense feature fusion(DFF) module combines features at multiple scales. We employ the contour reconstruction(CR) module to access the final predictions. Extensive experiments validate the effectiveness of the DFINet on two different remote sensing datasets, INRIA aerial image dataset and Wuhan University(WHU) building dataset. On the INRIA aerial image dataset, our method achieves the highest intersection over union(IoU), overall accuracy(OA) and F1 scores compared to other state-of-the-art methods.

Keywords

remote sensing image / building contour extraction / feature iteration

References

[1] GUO M Q, LIU H, XU Y Y, et al. Building extraction based on U-net with an attention block and multiple losses[J]. Remote Sensing, 2020, 12(9):1400.

[2] CHEN M, WU J J, LIU L Z, et al. DR-net:an improved network for building extraction from high resolution remote sensing image[J]. Remote Sensing, 2021, 13(2):294.

[3] ZHOU D J, WANG G Z, HE G J, et al. Robust building extraction for high spatial resolution remote sensing images with self-attention network[J]. Sensors, 2020, 20(24):7241.

[4] JI S P, WEI S Q, LU M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1):574-586.

[5] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018:833-851.

[6] LIU P H, LIU X P, LIU M X, et al. Building footprint extraction from high-resolution images via spatial residual inception convolutional neural network[J]. Remote Sensing, 2019, 11(7):830.

[7] CAI J H, CHEN Y M. MHA-net:multipath hybrid attention network for building footprint extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021,14:5807-5817.

[8] YUAN W, XU W B. MSST-net:a multi-scale adaptive network for building extraction from remote sensing images based on swin transformer[J]. Remote Sensing, 2021, 13(23):4743.

[9] GUO H N, DU B, ZHANG L P, et al. A coarse-to-fine boundary refinement network for building footprint extraction from remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022,183:240-252.

[10] RONNEBERGER O, FISCHER P, BROX T. U-net:convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham: Springer, 2015:234-241.

[11] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.

[12] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL].(2017-12-5 )[2023-10-12 ]. https://arxiv.org/abs/1706.05587.

[13] FU J, LIU J, TIAN H J, et al. Dual attention network for scene segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitons: IEEE, 2019:6877-6886.

[14] ZHENG S, LU J, ZHAO H, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitons: IEEE, 2021:6881-6890.

[15] LIN J B, JING W P, SONG H B, et al. ESFNet:efficient network for building extraction from high-resolution aerial images[J]. IEEE Access, 2019,7:54285-54294.

[16] ZHU Q, LIAO C, HU H, et al. MAP-net:multiple attending path neural network for building footprint extraction from remote sensed imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7):6169-6181.

[17] SHAO Z F, TANG P H, WANG Z Y, et al. BRRNet:a fully convolutional neural network for automatic building extraction from high-resolution remote sensing images[J]. Remote Sensing, 2020, 12(6):1050.

[18] YANG H, WU P H, YAO X D, et al. Building extraction in very high resolution imagery by dense-attention networks[J]. Remote Sensing, 2018, 10(11):1768.

[19] CHEN K Y, ZOU Z X, SHI Z W. Building extraction from remote sensing images with sparse token transformers[J]. Remote Sensing, 2021, 13(21):4441.

[20] XU L L, LI Y, XU J Z, et al. BCTNet:Bi-branch cross-fusion transformer for building footprint extraction[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023,61:4402014.

Funding

National Natural Science Foundation of China(61903078)

Fundamental Research Funds for the Central Universities, China(2232021A-10)

Shanghai Sailing Program, China(22YF1401300)

Natural Science Foundation of Shanghai, China(20ZR1400400)