Please wait a minute...
img

官方微信

遥感技术与应用  2019, Vol. 34 Issue (4): 736-747    DOI: 10.11873/j.issn.1004-0323.2019.4.0736
CNN 专栏     
深度残差神经网络高分辨率遥感图像建筑物分割
王宇1,2(),杨艺3(),王宝山1,王田4,卜旭辉3,王传云5
1. 河南理工大学 测绘与国土信息工程学院,河南 焦作 454000
2. 河南理工大学 国土资源部野外科学观测研究基地,河南 焦作 454000
3. 河南理工大学 电气工程与自动化学院,河南 焦作 454000
4. 北京航空航天大学 自动化科学与电气工程学院,北京 100191
5. 沈阳航空航天大学 计算机学院,辽宁 沈阳 110136
Building Segmentation in High Resolution Remote Sensing Imageby Deep ResNet
Yu Wang1,2(),Yi Yang3(),Baoshan Wang1,Tian Wang4,Xuhui Bu3,Chuanyun Wang5
1. School of Surveying and Land Information Engineering, Henan Polytechnic University, Jiaozuo 454000, China
2. Field Scientific Observation and Research base of Ministry of Land and Resources, Henan Polytechnic University, Jiaozuo 454000, China
3. School of Electrical Engineering and Automation, Henan Polytechnic University, Jiaozuo 454000, China
4. School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
5. School of computer Science,Shenyang Aerospace University, Shenyang 110136,China
 全文: PDF(5521 KB)   HTML
摘要:

针对高分辨率遥感图像建筑物分割问题,提出一种Encoder-Decoder的深度学习框架,建立输入图像到分割结果之间的端对端的分割模型。其中Encoder以残差网络为基础,自动提取建筑物的特征;Decoder采用反卷积实现对特征图的上采样,从而完成对建筑物的分割;同时引入批量规范化处理,降低了神经网络权重训练过程中的梯度竞争,从而减小了神经网络的训练难度。实验表明:提出的建筑物分割算法能有效提取建筑物的块状特征和边缘信息,降低复杂道路等干扰的影响,提升建筑物的分割精准度,算法对邻近复杂道路的建筑物、规律性建筑物、单体复杂建筑物等3种典型建筑物的分割精度分别为:0.837、0.892和0.630;F值分别为:0.851、0.879和0.730。同时,多分辨率条件下的分割实验结果表明,该算法对于一定范围内的多分辨率遥感图像具有较好的泛化能力。

关键词: 高分辨率遥感图像建筑物分割深度学习残差神经网络批量规范化    
Abstract:

This paper addresses the buildings segmentation in high resolution remote sensing image and proposes an Encoder-Decoder architecture of deep learning with End-to-End model, in which Encoder is based on ResNet, and the features needed by segmentation are exacted automatically, and the Decoder produces the segmentation result by deconvolution. Furthermore, in the training process, batch normalization is employed to decrease the gradient competition, so as to reduce the difficulty of training the deep neural network.The experiment results show that the algorithm effectively exacts the bulk feature and edge information of building in the high resolution remote sensing image, therefore the complex road disturbance is suppressed convincingly, and the building segmentation precision is improved effectively, the segmentation precision for three typical buildings, the building besides complex road, the ordered buildings and the complex single building, are 0.836 5, 0.892 4, and 0.629 7 respectively; and the F-measure are 0.851 4, 0.878 6 and 0.729 8, respectively. Meanwhile, the experiment results for multi-resolution remote sensing images show that the method can be generalized to the multi-resolution image within limits.

Key words: High resolution remote sensing image    Building segmentation    Deep learning    ResNet    Batch normalization
收稿日期: 2018-04-25 出版日期: 2019-10-16
ZTFLH:  P237  
基金资助: 国家自然科学基金项目(61503017);航空科学基金项目(2016ZC51022)
通讯作者: 杨艺     E-mail: wangyu@hpu.edu.cn;yangyi@hpu.edu.cn
作者简介: 王 宇(1978-),女,河南焦作人,讲师,主要从事遥感图像处理和土地利用规划方面的研究。E?mail:wangyu@hpu.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
王宇
杨艺
王宝山
王田
卜旭辉
王传云

引用本文:

王宇,杨艺,王宝山,王田,卜旭辉,王传云. 深度残差神经网络高分辨率遥感图像建筑物分割[J]. 遥感技术与应用, 2019, 34(4): 736-747.

Yu Wang,Yi Yang,Baoshan Wang,Tian Wang,Xuhui Bu,Chuanyun Wang. Building Segmentation in High Resolution Remote Sensing Imageby Deep ResNet. Remote Sensing Technology and Application, 2019, 34(4): 736-747.

链接本文:

http://www.rsta.ac.cn/CN/10.11873/j.issn.1004-0323.2019.4.0736        http://www.rsta.ac.cn/CN/Y2019/V34/I4/736

图1  建筑物分割流程图
图2  遥感图像建筑物分割深度学习框架
图3  深度学习框架
图4  残差神经网络结构图
图5  前馈闭环结构替换方法
图6  前馈闭环及通道扩容替换方法
图7  图像卷积过程
图8  Decoder的反卷积实现
步骤 操作
输入

获取当前批次训练样本:

X = x 1 , x 2 , ? , x m ,容量m

1 计算均值: μ k = 1 m i = 1 m x i
2 计算方差: σ k 2 = 1 m i = 1 m x i - μ k 2
3 样本规范化: x ? i = x i - μ σ k 2 + ε ε 为小数值正常数
4 尺度变换: y i = γ x ? i + β γ β 为可学习参数
5 规范化输出: B N ( x i ) = y i
表1  批量规范化
图9  深度神经网络训练误差
时间 VGG VGGCRF ResNet
网络调用时间/s 25.26 31.23 137.33
单个样本时间/s 0.69 5.11 0.23
样本集一次训练时间/s 3 328.94 23 222.35 1 418.86
表2  深度神经网络训练耗时对比
图10  包含复杂道路的建筑物分割结果
图11  包含复杂道路的建筑物分割性能指标
图12  规律性建筑物分割结果
图13  规律性建筑物分割性能指标
图14  单体复杂建筑物分割结果
图15  单体复杂建筑物分割性能指标
图16  建筑物分割细节
图17  多分辨率感图像建筑物分割细节
图18  多尺度遥感图像建筑物分割性能指标
1 Chen Jie , Deng Min , Xiao Pengfeng , et al . Object-oriented Classification of High Resolution Imagery based on Watershed Transform and Sptial Clustering[J]. Remote Sensing Technology and Application, 2010, 25(5):597-603.
1 陈杰,邓敏,肖鹏峰,等 . 基于分水岭变换与空间聚类的高分辨率遥感影像面向对象分类[J]. 遥感技术与应用, 2010, 25(5):597-603.
2 Wang Yu , Wang Baoshan , Wang Tian , et al . Image Entropy Active Contour Models towards Water Area Segmentation in Remote Sensing Image[J]. Optics and Precision Engineering, 2018
2 王宇,王宝山,王田,等 . 面向遥感图像水域分割的图像熵主动轮廓模型[J]. 光学精密工程,2018, 26(3):698-707.
3 Wu Jun , Wang Yuanyuan , Chen Yue , et al . Speckle Reduction of Ultrasound Images with Anisotropic Diffusion based on Homogeneous Region Automatic Selection[J]. Optics and Precision Engineering, 2014, 22(5):1312-1321.
3 吴俊,汪源源,陈悦,等 . 基于同质区域自动选取的各向异性扩散超声图像去噪[J]. 光学精密工程,2014, 22(5):1312-1321.
4 Chen Kuntang , Dong Xiaolong , Xu Xing’ou , et al . The Study on Oceanic Vector Wind Field Retrieve Technique based on Neural Networks of Microwave Scatterometer [J]. Remote Sensing Technology and Application, 2017,32 (4): 683-690.
4 陈坤堂,董晓龙,徐星欧,等 . 微波散射计反演海面风场的神经网络方法研究[J]. 遥感技术与应用, 2017,32 (4): 683-690.
5 Wang Yu , Li Yu , Zhao Quanhua . Region-based Multiscale Segmentation of Panchromatic Remote Sensing Image[J]. Control and Decision, 2018, 33(3): 535-541.
5 王玉,李玉,赵泉华 . 基于区域的多尺度全色遥感图像分割[J]. 控制与决策, 2018, 33(3): 535-541.
6 Hinton G E , Salakhutdinov R R . Reducing the Dimensionality of Data with Neural Networks[J].Science, 2006, 313(5786): 504-507.
7 Zeng N Y , Zhang H , Song B Y , et al . Facial Expression Recognition via Learning Deep Sparse Autoencoders[J]. Neurocomputing, 2018, 273(17): 643-649.
8 Xu X Y , Pan J S , Zhang Y J , et al . Motion Blur Kernel Estimation via Deep Learning[J]. IEEE Transactions on Image Processing, 2018, 27(1): 194-205.
9 Shao H D , Jiang H K , Zhang H Z , et al . Rolling Bearing Fault Feature Learning Using Improved Convolutional Deep Belief Network with Compressed Sensing[J]. Mechanical Systems and Signal Processing, 2018, 100: 743-765.
10 Long J , Shelhamer E , Darrell T . Fully Convolutional Networks for Semantic Segmentation[C]∥ IEEE Transactions on Pattern Analysis & Machine Intelligence,2015:3431-3440.
11 Shelhamer E , Long J , Darrell T . Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
12 Simonyan K , Zisserman A . Visual Geometry Group[EB/OL]. http:∥,2014.
13 Badrinarayanan V , Kendall A , Cipolla R . SegNet: A Deep Convolutional Encoder-decoder Architecture for Scene Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
14 Yu F , Koltun V . Multi-scale Context Aggregation by Dilated Convolutions [C]∥ International Conference on Learning Representations,2016.
15 Chen L C , Papandreou G , Kokkinos I , et al . Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[C]∥International Conference on Learning Representations, 2015.
16 Deeplab-public [EB/OL], https:∥bitbucket.org/deeplab/deeplab-public/,2017,2018.
17 Vakalopoulou M , Karantzalos K , Komodakis N , et al . Building Detection in very High Resolution Multispectral Data with Deep Learning Features[C]∥IEEE. Geoscience & Remote Sensing Symposium, 2015:1873-1876.
18 Huang Z M , Cheng G L , Wang H Z , et al . Building Extraction from Multi-source Remote Sensing Images via Deep Deconvolution Neural Networks[C]∥IEEE. Geoscience and Remote Sensing Symposium, 2016:1835-1838.
19 Saito S , Aoki Y . Building and Road Detection from Large Aerial Imagery[C]∥Image Processing: Machine Vision Applications VIII , 2015:1814-1821.
20 Yuan J . Learning Building Extraction in Aerial Scenes with Convolutional Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 40(11): 2793-2798.
21 Bittner K , Cui S Y , Reinartz P . Building Extraction from Remote Sensing Data Using Fully Convolutional Networks[C]∥ISPRS Hannover Workshop: Hrigi, 2017:481-486.
22 Bischke B , Helber P , Folz J , et al . Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks[EB/OL].https:∥arxiv.org/abs/ 1709.,2017,2017.
23 Wang Y , Wang C , Zhang H . Integrating H-A-α with Fully Convolutional Networks for Fully PolSAR Classification[C]∥IEEE. International Workshop on Remote Sensing with Intelligent Processing, 2017:1-4.
24 Alshehhi R , Marpu P R , Woon W L , et al . Simultaneous Extraction of Roads and Buildings in Remote Sensing Imagery with Convolutional Neural Networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130: 139-149.
25 Lin H , Shi Z , Zou Z . Fully Convolutional Network With Task Partitioning for Inshore Ship Detection in Optical Remote Sensing Images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1665-1669.
26 Jiao L , Liang M , Chen H , et al . Deep Fully Convolutional Network-based Spatial Distribution Prediction for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10): 5585-5599.
27 Ioffe S , Szegedy C . Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[EB/OL], https:∥arxiv.org/abs/ 1502.,2015,2018.
28 He K M , Zhang X Y , Ren S Q , et al . Deep Residual Learning for Image Recognition [J]. https:∥arxiv.org/abs/ 1512.03385,2015.
29 Nirvana Inria Aerial Image Labeling Dataset [EB/OL]. https:∥project.inria.fr/aerialimagelabeling/,2016,2017..
30 Maggiori E , Tarabalka Y , Charpiat G ,et al . Can Semantic Labeling Methods Generalize to Any City? The Inria Aerial Image Labeling Benchmark[C]∥IEEE International Symposium on Geoscience and Remote Sensing, 2017:3226-3229.
31 Krizhevsky A , Sutskever I , Hinton G E . ImageNet Classification with Deep Convolutional Neural Networks[C]∥Neural Information Processing Systems Conference, 2012:1097-1105
32 Dumoulin V , Visin F . A Guide to Convolution Arithmetic for Deep Learning[EB/OL], https:∥arxiv.org/abs/ 1603. , 2016,2018.
33 Shelhamer E , Long J , Darrell T . Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(4): 640-651.
[1] 王颖洁,张荞,张艳梅,蒙印,郭文. 基于深度卷积神经网络的油罐目标检测研究[J]. 遥感技术与应用, 2019, 34(4): 727-735.
[2] 林志玮,丁启禄,黄嘉航,涂伟豪,胡典,刘金福. 基于DenseNet的无人机光学图像树种分类研究[J]. 遥感技术与应用, 2019, 34(4): 704-711.
[3] 刘天福,陈学泓,董琪,曹鑫,陈晋. 深度学习在GlobeLand30-2010产品分类精度优化中应用研究[J]. 遥感技术与应用, 2019, 34(4): 685-693.
[4] 徐梦竹, 徐佳, 邓鸿儒, 袁春琦. 基于全极化SAR影像的海岛地物分类[J]. 遥感技术与应用, 2019, 34(3): 647-654.
[5] 李淑敏, 冯权泷, 梁其椿, 张学庆. 基于深度学习的国产高分遥感影像飞机目标自动检测[J]. 遥感技术与应用, 2018, 33(6): 1095-1102.
[6] 何海清,庞燕,陈晓勇. 面向遥感影像场景的深度卷积神经网络递归识别模型[J]. 遥感技术与应用, 2017, 32(6): 1078-1082.