基于孪生网络的高速公路雾天能见度识别方法

汤伟; 方嘉楠; 张龙; 杨晓东; 李国强

doi:10.3963/j.jssn.1674-4861.2023.04.013

基于孪生网络的高速公路雾天能见度识别方法

doi: 10.3963/j.jssn.1674-4861.2023.04.013

1.
陕西科技大学电气与控制工程学院西安 710021
2.
西安金路交通工程科技发展有限责任公司西安 710000

基金项目:

国家重点研发计划项目 2017YFC0803907

陕西重点研发计划项目 2022GY-335

详细信息

通讯作者:
汤伟（1971—），博士，教授. 研究方向：智能交通、交通安全等. E-mail：wtang906@163.com

中图分类号: U491.5
计量
- 文章访问数: 482
- HTML全文浏览量: 232
- PDF下载量: 15
- 被引次数: 0
出版历程
- 收稿日期: 2023-03-22
- 网络出版日期: 2023-11-23

A Method for Measuring Visibility under Foggy Weather for Expressways Based on Siamese Network

1.
School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi'an 710021, China
2.
Xi'an Jinlu Traffic Engineering Technology Development Co. LTD, Xi'an 710000, China

摘要

摘要: 从监控视频中准确识别高速公路雾天能见度等级，对于高速公路智能监管具有重要意义。针对当前高速公路能见度识别方法存在的精度低、速率慢、泛化性弱等问题，研究了基于孪生网络的能见度识别方法，重点关注于图像特征提取模块和等级识别主干模块的优化。图像特征提取模块采用改进的VGG16网络作为骨干网络，为增强网络从图像全局信息中提取重要特征的能力，在VGG16网络的5个block中增加卷积块注意力机制，达到强调有效特征和抑制无用特征的作用；为提升网络的泛化能力和训练速率，在网络卷积层之后添加滤波器响应归一化层来消除各维数据之间的差别；为解决网络权重参数冗余问题、防止过拟合，采用全局平均池化将输出特征图直接压缩为1×1向量，代替VGG16网络中前2层全连接层。等级识别主干模块采用孪生网络作为主体框架，将图像特征提取模块提取的有效特征进行前向传播，使用对比损失函数的距离测量方法在高维空间中对比输入图像对的相似程度来进行雾能见度等级识别。实验以2022年8月—2023年1月通过高速公路摄像机采集的陕西省部分高速公路雾天真实图像作为测试集对模型进行验证，结果表明：所提方法的识别准确率为90.3%，相比于单一网络AlexNet，ResNet50，VGG16在准确率上分别提高20.4%，18.9%，18.0%，相比于以单一网络为基准构建的孪生网络模型Simaese-AlexNet，Simaese-ResNet50，Simaese-VGG16在准确率上分别提高16.2%，11.0%，5.4%。所提出的方法对高速公路雾天能见度识别具备较高的准确率，有助于提升高速公路雾天的智能监管能力。
- 交通安全 /
- 能见度识别 /
- Siamese-Enhance-VGG /
- 孪生网络 /
- VGG16
Abstract: Accurately identifying highway visibility levels in foggy weather from surveillance video is important for intelligent highway supervision. Aiming at the problems of low accuracy, slow rate, and weak generalization of the current visibility recognition methods on highways, a visibility recognition method based on Siamese network is proposed, focusing on the optimization of the image feature extraction module and the fog visibility level recognition. The image feature extraction module adopts the improved VGG16 network as the backbone network. In order to enhance the ability of the network to extract important features from the global information of the image, a convolution block attention module is added to the five blocks of the VGG16 network to emphasize the effective features and suppress the useless features. To improve the generalization ability and training rate of the network, a filter response normalization layer is added after the convolutional layer of the network to remove the differences between the dimensional data. In order to solve the redundancy problem of network weight parameters and prevent overfitting, global average pooling is used to compress the output feature map directly into a 1×1 vector instead of the first two fully connected layers in the VGG16 network. A Siamese network is adopted as the main framework of the fog visibility level recognition module, and the effective features extracted by the image feature extraction module are propagated forward. The distance measurement method is utilized in the contrastive loss function to assess the similarity between input image pairs in a high-dimensional space for fog visibility level recognition. Experiments are conducted based on a dataset of actual foggy images collected from August 2022 to January 2023 on highways in Shaanxi Province. The experimental results show that the recognition accuracy of the proposed method is 90.3%, which is an improvement of 20.4%, 18.9%, and 18.0% compared to the single networks AlexNet, ResNet50, and VGG16, respectively. It is also an improvement of 16.2%, 11.0%, and 5.4% compared to the Siamese networks Simaese-AlexNet, Simaese-ResNet50, and Simaese-VGG16, respectively, which constructed based on single networks as benchmark models. In conclusion, this method exhibits a high accuracy, which contributes to enhancing the intelligent supervision capabilities for foggy weather conditions on highways.
- traffic safety /
- visibility recognition /
- Siamese-Enhance-VGG /
- Siamese network /
- VGG16

HTML全文

图 1 不同雾浓度下部分高速公路的监控图像

Figure 1. Surveillance images of some highways at different fog concentrations

下载: 全尺寸图片幻灯片

图 2 雾天能见度识别模型

Figure 2. Recognition model for visibility in foggy weather

下载: 全尺寸图片幻灯片

图 3 VGG16网络结构图

Figure 3. The structure diagram of VGG16 network

下载: 全尺寸图片幻灯片

图 4 Enhance-VGG网络结构图

Figure 4. Enhance-VGG network architecture

下载: 全尺寸图片幻灯片

图 5 CBAM注意力机制整体流程图

Figure 5. Overall flowchart of CBAM attention mechanism

下载: 全尺寸图片幻灯片

图 6 全连接层与全局平均池化对比

Figure 6. Comparison between fully-connected layer and global average pooling

下载: 全尺寸图片幻灯片

图 7 孪生网络架构

Figure 7. The architecture of siamese network

下载: 全尺寸图片幻灯片

图 8 损失函数与样本特征欧氏距离的关系

Figure 8. The relationship between loss function and euclidean distance of sample feature

下载: 全尺寸图片幻灯片

图 9 混淆矩阵

Figure 9. Confusion matrix

下载: 全尺寸图片幻灯片

图 10 消融实验准确率曲线

Figure 10. Accuracy curve of ablation experiments

下载: 全尺寸图片幻灯片

图 11 4种单一网络准确率和损失值曲线

Figure 11. Accuracy and loss value curves of 4 individual networks.

下载: 全尺寸图片幻灯片

图 12 4种孪生网络准确率和损失值曲线

Figure 12. Accuracy and loss value curves of 4 siamese networks

下载: 全尺寸图片幻灯片

图 13 8种网络模型实验对比结果

Figure 13. Comparison of experimental results for 8 network models

下载: 全尺寸图片幻灯片

图 14 识别结果混淆矩阵

Figure 14. Confusion matrix of recognition results

下载: 全尺寸图片幻灯片

图 15 4种孪生网络算法高速公路数据集检测结果对比

Figure 15. Comparison of detection results for 4 siamese network algorithms on highway datasets

下载: 全尺寸图片幻灯片

图 16 2023年5月9日上午07:26往西安西富高速公路团雾变化

Figure 16. Variation of agglomerate fog on Xi'an Xifu highway on May 9, 2023, at 07:26 AM

下载: 全尺寸图片幻灯片

表 1 能见度等级标准

Table 1. Visibility level standards

能见度等级	能见度距离/m
0级（浓雾）	0~50
1级（重雾）	＞50~200
2级（大雾）	＞200~500
3级（轻雾）	＞500~1 000
4级（无雾）	＞1 000

下载: 导出CSV

表 2 样本分布

Table 2. Sample distribution 单位: 张

数据集	能见度等级					总计
数据集	0级（浓雾）	1级（重雾）	2级（大雾）	3级（轻雾）	4级（无雾）	总计
训练集	880	892	909	907	912	4 500
验证集	289	296	300	306	309	1 500
测试集	288	294	303	310	305	1 500

下载: 导出CSV

表 3 消融实验结果

Table 3. Results of ablation experiment

网络	准确率/%
VGG16	72.3
VGG16-CBAM	77.4
VGG16-GAP	75.3
VGG16-FRN	83.6
Enhance-VGG	87.7

下载: 导出CSV

参考文献(21)

[1]	张驰, 周郁茗, 张敏, 等. 交通事故导致的高速公路拥堵状态判别方法[J]. 交通信息与安全, 2023, 41(1): 23-33. doi: 10.3963/j.jssn.1674-4861.2023.01.003 ZHANG C, ZHOU Y M, ZHANG M, et al. A method for identifying traffic congestion resulting from accidents on freeways[J]. Journal of Transport Information and Safety, 2023, 41(1): 23-33. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2023.01.003
[2]	交通强国建设评价指标体系[J]. 中国水运, 2022(4): 12-15. Evaluation index system for the construction of a powerful transportation country[J]. China Water Transport, 2022, 4: 12-15. (in Chinese)
[3]	刘钦, 宋太龙, 李振龙, 等. 小样本下基于迁移学习与LSTM的雾天高速公路车辆跟驰模型[J]. 交通信息与安全, 2023, 41(1): 13-22. doi: 10.3963/j.jssn.1674-4861.2023.01.002 LIU Q, SONG T L, LI Z L, et al. A car-following model for expressway under foggy weather based on transfer learning and LSTM with small-sample[J]. Journal of Transport Information and Safety, 2023, 41(1): 13-22. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2023.01.002
[4]	龙学军, 高枫. 基于深度学习的高速公路气象识别方法[J]. 中国交通信息化, 2021(5): 134-136. https://www.cnki.com.cn/Article/CJFDTOTAL-JTXC202105018.htm LONG X J, GAO F. Highway weather recognition method based on deep learning[J]. China ITS Journal, 2021(5): 134-136. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JTXC202105018.htm
[5]	GU J, WANG Z, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377. doi: 10.1016/j.patcog.2017.10.013
[6]	LI S, FU H, LO W L. Meteorological visibility evaluation on webcam weather image using deep learning features[J]. Int. J. Comput. Theory Eng, 2017, 9(6): 455-461. doi: 10.7763/IJCTE.2017.V9.1186
[7]	WANG H, SHEN K, YU P, et al. Multimodal deep fusion network for visibility assessment with a small training dataset[J]. IEEE Access, 2020(8): 217057-217067.
[8]	刘冬韡, 穆海振, 贺千山, 等. 1种基于实景图像的低能见度识别算法[J]. 应用气象学报, 2022, 33(4): 501-512. https://www.cnki.com.cn/Article/CJFDTOTAL-YYQX202204010.htm LIU D W, MU H Z, HE Q S, et al. A low visibility recognition algorithm based on surveillance video[J]. Journal of Applied Meteorological Science, 2022, 33(4): 501-512. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-YYQX202204010.htm
[9]	曹爽亮, 杨亚莉, 陈浩, 等. 基于卷积神经网络的能见度估算[J]. 软件工程, 2021, 24(8): 2-5. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGC202108002.htm CAO S L, YANG Y L, CHEN H, et al. Visibility estimation based on convolutional neural network[J]. Software Engineering, 2021, 24(8): 2-5. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGC202108002.htm
[10]	苗开超, 周建平, 陶鹏, 等. 自适应混合卷积神经网络的雾图能见度识别[J]. 计算机工程与应用, 2020, 56(10): 205-212. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202010032.htm MIAO K C, ZHOU J P, TAO P, et al. Visibility recognition of fog figure based on self-adaptive hybrid convolutional neural network[J]. Computer Engineering and Applications, 2020, 56(10): 205-212. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202010032.htm
[11]	黄亮, 张振东, 肖鹏飞, 等. 基于深度学习的公路能见度分类及应用[J]. 大气科学学报, 2022, 45(2): 203-211. https://www.cnki.com.cn/Article/CJFDTOTAL-NJQX202202005.htm HUANG L, ZHANG Z D, XIAO P F, et al. Classification and application of highway visibility based on deep learning[J]. Transactions of Atmospheric Sciences, 2022, 45(2): 203-211. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-NJQX202202005.htm
[12]	LI Z, ZOU H, SUN X, et al. 3d expression-invariant face verification based on transfer learning and siamese network for small sample size[J]. Electronics, 2021, 10(17): 2128.
[13]	张伟光, 钟靖涛, 呼延菊, 等. 基于VGG16-UNet语义分割模型的路面龟裂形态提取与量化[J]. 交通运输工程学报, 2023, 23(2): 166-182. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202302012.htm ZHANG W G, ZHONG J T, HU Y J, et al. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202302012.htm
[14]	胡丹丹, 张忠婷, 牛国臣. 融合CBAM注意力机制与可变形卷积的车道线检测[J/OL]. 北京航空航天大学学报: 1-14[2023-03-02]. HU D D, ZHANG Z T, NIU G C. Lane line detection incorporating CBAM attention mechanism and deformable convolution[J/OL]. Journal of Beijing University of Aeronautics and Astronautics: 1-14[2023-03-02]. (in Chinese)
[15]	IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]. International Conference on Machine Learning. Lile, France: IMLS, 2015.
[16]	SINGH S, KRISHNAN S. Filter response normalization layer: eliminating batch dependence in the training of deep neural networks[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual. Online, USA: IEEE, 2020.
[17]	刘超军, 段喜萍, 谢宝文. 应用GhostNet卷积特征的ECO目标跟踪算法改进[J]. 激光技术, 2022, 46(2): 239-247. https://www.cnki.com.cn/Article/CJFDTOTAL-JGJS202202019.htm LIU C J, DUAN X P, XIE B W. Improvement of ECO target tracking algorithm based on GhostNet convolution feature[J]. Laser Technology, 2022, 46(2): 239-247. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JGJS202202019.htm
[18]	陶倩文, 胡钊政, 万金杰, 等. 基于点云极化表征与孪生网络的智能车定位[J]. 电子与信息学报, 2023, 45(4): 1163-1172. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX202304003.htm TAO Q W, HU Z Z, WAN J J, et al. Intelligent vehicle localization based on polarized LiDAR representation and siamese network[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1163-1172. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX202304003.htm
[19]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[20]	张伟光, 钟靖涛, 呼延菊, 等. 基于VGG16-UNet语义分割模型的路面龟裂形态提取与量化[J]. 交通运输工程学报, 2023, 23(2): 166-182. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202302012.htm ZHANG W G, ZHONG J T, HU Y J, et al. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202302012.htm
[21]	徐慧智, 闫卓远, 常梦莹. 1种结合ResNet和迁移学习的交通标志识别方法[J]. 重庆理工大学学报(自然科学), 2023, 37(3): 264-273. https://www.cnki.com.cn/Article/CJFDTOTAL-CGGL202303030.htm XU H Z, YAN Z Y, CHANG M Y. A traffic sign recognition method based on ResNet and transfer learning[J]. Journal of Chongqing University of Technology(Natural Science), 2023, 37(3): 264-273. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CGGL202303030.htm