基于路侧多机视频目标关联与轨迹拼接的车辆连续轨迹构建方法

刘超; 罗如意; 刘春青; 吕能超

doi:10.3963/j.jssn.1674-4861.2023.03.009

基于路侧多机视频目标关联与轨迹拼接的车辆连续轨迹构建方法

doi: 10.3963/j.jssn.1674-4861.2023.03.009

刘超^{1, 2,},
罗如意³,
刘春青^{1, 2},
吕能超^{1, 2, ,}

1.
武汉理工大学智能交通系统研究中心武汉 430063
2.
武汉理工大学交通与物流工程学院武汉 430063
3.
湖北交投科技发展有限公司武汉 430034

基金项目:

国家自然科学基金项目 52072290

国家重点研发计划项目 2020YFB1600302

湖北省杰出青年基金项目 2020CFA081

详细信息

作者简介:
刘超（1998—），硕士研究生. 研究方向：交通信息与安全. E-mail: 15927510744@163.com

通讯作者:
吕能超（1982—），博士，研究员. 研究方向：智能网联交通、智慧公路等. E-mail: lvnengchao@163.com

中图分类号: U491.4
计量
- 文章访问数: 546
- HTML全文浏览量: 266
- PDF下载量: 25
- 被引次数: 0
出版历程
- 收稿日期: 2022-10-25
- 网络出版日期: 2023-09-16

A Method for Developing Continuous Vehicle Trajectories through Target Association and Trajectory Splicing Based on Video Data from Multiple Roadside Cameras

LIU Chao^{1, 2
,},
LUO Ruyi³,
LIU Chunqing^{1, 2},
LYU Nengchao^{1, 2
, ,}

1.
Intelligent Transportation System Research Center, Wuhan University of Technology, Hubei Wuhan 430063, China
2.
School of Transportation and Logistics Engineering, Wuhan University of Technology, Wuhan 430063, China
3.
Hubei Communications Investment Technology Development Co., Ltd Wuhan 430034

摘要

摘要: 针对单个相机覆盖区域有限的问题，本文提出了通过路侧多个相机获取车辆连续轨迹数据的方法。在路侧端布置多台固定相机采集视频数据，采用直接线性变换算法解决相机外参造成的画面畸变问题；通过全时域抽帧提取图片训练样本，采用YOLOv5训练车辆检测模型；针对偶发的车辆漏检情形，通过完整性检查可以筛查出此类情况并修复；针对连续多帧漏检或误检导致的目标关联问题，通过异常轨迹核查算法及数据修复工具解决；针对相机斜下方区域的车辆轮廓变形问题，采用车辆检测轮廓修复算法解决车辆在不同路段检测框大小不一的问题；提出了基于车辆质心坐标匹配的方法实现相邻机位间的车辆轨迹拼接。基于上述多机视频目标关联与轨迹拼接方法，在多机位时间同步下构建了覆盖武汉珞狮路高架桥的车辆连续轨迹数据集，轨迹数据集验证结果表明：数据集涵盖从畅通到拥堵的各种交通状态，包含多段分合流区域，数据集连续时长达到3.5 h，覆盖区域1.41 km；车辆检测模型的召回率达到93.23%，准确率达到98.51%，F1分数为95.80%；数据集包含主路及各处匝道汇入的轨迹共25 734条，其中覆盖道路全域的完整长轨迹15 004条。本研究丰富了路侧多机视频目标关联与轨迹拼接方法，有助于路侧宽域车辆连续轨迹构建及交通管理与控制。
- 智能交通 /
- 交通视频处理 /
- 长时宽域 /
- 轨迹拼接 /
- 车辆目标关联 /
- 透视变换
Abstract: A method for developing continuous vehicle trajectories through multiple roadside cameras is proposed to address the limited coverage of a single camera. This study sets up multiple fixed cameras on the roadside to col-lect video data, and solves the problem of image distortion caused by camera extrinsic parameters through direct linear transformation algorithm. Training samples are evenly extracted from images of all time periods and road areas, and a vehicle detection model is trained using convolutional neural network YOLOv5. For the occasional missed vehicles, an integrity check method can be used to screen missing vehicles and get the problem fixed. In cases where a vehicle is missed or falsely detected in multiple consecutive frames, the target association problem is solved through the use of checking algorithm for abnormal trajectory and data repair plugin. A repair algorithm is proposed to solve the problem of deformation of vehicle profile in the areas diagonally below the camera, which solves the problem of varying detection box sizes for the same vehicles traveling at different road segments. And a method for vehicle trajectory splicing between adjacent cameras is proposed based on the centroid coordinates of vehicle. The development of continuous vehicle trajectory dataset among continuous multiple cameras is achieved under the premise of time synchronization among multiple cameras. By using the methods of target association and trajectory splicing mentioned above, a continuous vehicle trajectory dataset covering Luoshi Road Overpass in Wuhan has been developed using a time synchronization method for different locations. Study results of track data set show that: the dataset covers various traffic flow states from free-flow to congested, including multiple diver-sion and merging areas. The dataset has a continuous duration of 3.5 hours and covers an area of 1.41 km. Study results of the vehicle detection model show that the recall rate of the model is 93.23%, the precision rate is 98.51%, and the F1 score is 95.80%. According to the data self-inspection results, the dataset contains a total of 25 734 trajectories from the arterial roads and ramps, including 15 004 trajectories covering the entire road. The method proposed in this study provides a technical framework for target association and trajectory splicing of video data from multiple roadside cameras, and a way of developing continuous vehicle trajectories for better traffic man-agement and control.
- intelligent transportation /
- traffic video processing /
- continuous long time and wide range /
- trajectory splicing /
- vehicle target association /
- direct linear transform

HTML全文

图 1 目标关联框架

Figure 1. Target association framework

下载: 全尺寸图片幻灯片

图 2 临时缓存区机制

Figure 2. Temporary buffer mechanism

下载: 全尺寸图片幻灯片

图 3 检测框修复说明

Figure 3. Instruction for repairing detection box

下载: 全尺寸图片幻灯片

图 4 轨迹拼接框架

Figure 4. Trajectory splicing framework

下载: 全尺寸图片幻灯片

图 5 观测区域卫星地图

Figure 5. Satellite map of observation area

下载: 全尺寸图片幻灯片

图 6 透视变换结果

Figure 6. Result of direct linear transform

下载: 全尺寸图片幻灯片

图 7 珞狮路高架桥图像

Figure 7. Image of Luoshi Road Overpass

下载: 全尺寸图片幻灯片

图 8 车辆数据集样本

Figure 8. Vehicle dataset samples

下载: 全尺寸图片幻灯片

图 9 桥下被检测车辆

Figure 9. Detected vehicles under the overpass

下载: 全尺寸图片幻灯片

图 10 数据修复插件示例

Figure 10. Example of data repair plugin

下载: 全尺寸图片幻灯片

图 11 轨迹滤波

Figure 11. Trajectory filtering

下载: 全尺寸图片幻灯片

图 12 检测效果示意图

Figure 12. Schematic diagram of detection effect

下载: 全尺寸图片幻灯片

图 13 检测框位置精确性验证

Figure 13. Verification of accuracy of detection box position

下载: 全尺寸图片幻灯片

图 14 轨迹示意图

Figure 14. Trajectory diagram

下载: 全尺寸图片幻灯片

图 15 交通流参数统计

Figure 15. Statistics of traffic flow parameters

下载: 全尺寸图片幻灯片

表 1 模型配置参数

Table 1. Model configuration parameters

训练参数名称	参数值
训练轮次数	200
图像批次大小	8
初始学习率	0.01
最终学习率	0.2
图像输入尺寸/px	640×640

下载: 导出CSV

表 2 检测后输出数据结构表

Table 2. Data structure after detection

列号	数据说明
1	帧号
2	X_left
3	Y_left
4	X_right
5	Y_right

下载: 导出CSV

表 3 关联后轨迹统计

Table 3. Trajectory statistics after association

机位	轨迹总数	异常轨迹
1	20 408	923
2	17 270	1 078
3	21 444	1 434
4	21 690	1 595
5	21 474	1 508
6	25 382	1 233

下载: 导出CSV

表 4 轨迹拼接结果

Table 4. Trajectory splicing result 单位：个

机位	问题①	问题②	问题③
1~2	35	0	9
2~3	38	2	16
3~4	75	0	22
4~5	160	6	360
5~6	31	0	17

下载: 导出CSV

表 5 视频数据集时空范围对比

Table 5. Comparison of spatiotemporal coverage of video datasets

数据集	数据采集方式	数据空间范围/m	数据时间范围
HighD数据集^[14]	无人机航拍	420	30 min以内的离散片段，累计时长16.5 h
南京快速路数据集^[15]	无人机航拍	140~427	20 min以内的离散片段，累计时长50 min
NGSIM数据集^[16]	高空固定机位拍摄	500~640	15 min的离散片段，累计时长2.5 h
珞狮路高架桥数据集	高空固定连续多机位拍摄	1 410	连续覆盖3.5 h

下载: 导出CSV

参考文献(28)

[1]	吕伟, 黄广琛, 汪京辉. 基于元胞自动机的高速公路瓶颈交通演化仿真[J]. 交通运输系统工程与信息, 2022, 22(3): 293-302. https://www.cnki.com.cn/Article/CJFDTOTAL-YSXT202203033.htm LYU W, HUANG G S, WANG J H. Simulation of highway traffic bottleneck via cellular automata[J]. Journal of Transportation Systems Engineering and Information Technology, 2022, 22(3): 293-302. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-YSXT202203033.htm
[2]	SUN D H., CHEN D, ZHAO M., et al. Linear stability and nonlinear analyses of traffic waves for the general nonlinear car-following model with multi-time delays[J]. Physica A: Statistical Mechanics and its Applications. 2018, 501: 293-307. doi: 10.1016/j.physa.2018.02.179
[3]	朱顺应, 蒋若曦, 王红, 等. 机动车交通冲突技术研究综述[J]. 中国公路学报, 2020, 33(2): 15-33. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202002002.htm ZHU S Y, JIANG R X, WANG H, et al. Review of research on traffic conflict techniques[J]. China Journal of Highway and Transport, 2020, 33(2): 15-33. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202002002.htm
[4]	WU K, JOVANIS P. Crashes and crash-surrogate events: Exploratory modeling with naturalistic driving data[J]. Accident Analysis & Prevention, 2012(45): 507-516.
[5]	BAGDADI O. Assessing safety critical braking events in naturalistic driving studies[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2013(16): 117-126.
[6]	PARSA A B, TAGHIPOUR H, DERRIBLE S, et al. Real-time accident detection: Coping with imbalanced data[J]. Accident Analysis & Prevention, 2019(129): 202-210.
[7]	吕能超, 彭凌枫, 吴超仲, 等. 区分冲突类型的路段实时碰撞风险预测模型[J]. 中国公路学报, 2022, 35(1): 93-108. LYU N C, PENG L F, WU C Z, et al. Real-time crash-risk prediction model that distinguishes collision types[J]. China Journal of Highway and Transport, 2022, 35(1): 93-108. (in Chinese)
[8]	关丽敏, 张倩, 楚庆玲, 等. 基于改进ICP算法的路侧双激光雷达数据融合[J]. 激光杂志, 2021, 42(9): 38-44. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202109008.htm GUAN L M, ZHANG Q, CHU Q L, et al. Roadside dual lidar data fusion based on improved ICP algorithm[J]. Laser Journal, 2021, 42(9): 38-44. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202109008.htm
[9]	薛清文, 蒋愚明, 陆键. 基于轨迹数据的危险驾驶行为识别方法[J]. 中国公路学报, 2020, 33(6): 84-94. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202006009.htm XUE Q W, JIANG Y M, LU J. Risky driving behavior recognition based on trajectory data[J]. China Journal of Highway and Transport, 2020, 33(6): 84-94. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202006009.htm
[10]	王俊骅, 宋昊, 景强, 等. 基于毫米波雷达组群的全域车辆轨迹检测技术方法[J]. 中国公路学报, 2022, 35(12): 181-192. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202212015.htm WANG J H, SONG H, JING Q, et al. Road-range tracking of vehicle trajectories based on millimeter-wave radar[J]. China Journal of Highway and Transport, 2022, 35(12): 181-192. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202212015.htm
[11]	李熙莹, 梁靖茹, 郝腾龙. 考虑连锁冲突的城市公交车行车风险量化分析方法[J]. 交通信息与安全, 2022, 40(3): 19-29. doi: 10.3963/j.jssn.1674-4861.2022.03.003 LI X Y, LIANG J R, HAO T L. A method for quantitatively analyzing risks associated with the operation of urban buses considering chained conflicts[J]. Journal of Transport Information and Safety. 2022, 40(3): 19-29. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.03.003
[12]	WANG C, XU C C, DAI Y L. A crash prediction method based on bivariate extreme value theory and video-based vehicle trajectory data[J]. Accident Analysis & Prevention, 2019(123): 365-373.
[13]	房锐, 张琪, 胡澄宇, 等. 基于风险矩阵的干线公路弯道路段交通冲突风险评估模型[J]. 交通运输系统工程与信息, 2021, 21(2): 166-172. https://www.cnki.com.cn/Article/CJFDTOTAL-YSXT202102026.htm FANG R, ZHANG Q, HU C G, et al. Risk assessment model based on risk matrix for traffic conflict on arterial highway bend section[J]. Journal of Transportation Systems Engineering and Information Technology, 2021, 21(2): 166-172. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-YSXT202102026.htm
[14]	KRAJEWSKI R, BOCK J, KLOEKER L, et al. The HighD dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems[C]. 21st International Conference on Intelligent Transportation Systems(ITSC), Maui, HI, USA: IEEE, 2018.
[15]	冯汝怡, 李志斌, 吴启范, 等. 航拍视频车辆检测目标关联与时空轨迹匹配[J]. 交通信息与安全, 2021, 39(2): 61-69, 77. doi: 10.3963/j.jssn.1674-4861.2021.02.008 FENG R Y, LI Z B, WU Q F, et al. Association of vehicle object detection and the time-space trajectory matching from aerial videos[J]. Journal of Transport Information and Safety, 2021, 39(2): 61-69, 77. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2021.02.008
[16]	FHWA, Department of Transportation, America. NGSIM: Next generation simulation[EB/OL]. (2007-5-5)[2015-08-16]. http://www.ngsim-community.org/.
[17]	李枫. 高速公路视频监控系统的应用及构建[J]. 电子技术与软件工程, 2023, 243(1): 159-164. https://www.cnki.com.cn/Article/CJFDTOTAL-DZRU202301033.htm LI F. Application and construction of expressway video monitoring system[J]. Electronic Technology and Software Engineering, 2023, 243(1): 159-164. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DZRU202301033.htm
[18]	李子腾, 施绍武, 张康. 大数据+AI收费稽核系统[J]. 中国交通信息化, 2022, 269(5): 95-98. https://www.cnki.com.cn/Article/CJFDTOTAL-JTXC202205011.htm LI Z T, SHI S W, ZHANG K. Big data + AI fee audit system[J]. China's Transportation Informatization, 2022, 269 (5): 95-98. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JTXC202205011.htm
[19]	李君羡, 童文聪, 沈宙彪, 等. 基于结构化视频数据的交叉口评估及问题自动化诊断[J]. 同济大学学报(自然科学版), 2020, 48(8): 1149-1160. https://www.cnki.com.cn/Article/CJFDTOTAL-TJDZ202008008.htm LI J X, TONG W C, SHEN Z B, et al. Intersection evaluation and automatic problem diagnosis based on structured video data[J]. Journal of Tongji University (Natural Science), 2020, 48(8): 1149-1160. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-TJDZ202008008.htm
[20]	REDMON J, FARHADI A. YOLO9000: Better, faster, Stronger. [C]. 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA: IEEE, 2017.
[21]	李建国, 张睿, 王凯, 等. 基于匈牙利匹配和卡尔曼滤波的动态多目标跟踪[J]. 汽车实用技术, 2022, 47(1): 45-50. https://www.cnki.com.cn/Article/CJFDTOTAL-SXQC202201011.htm LI J G, ZHANG R, WANG K, ET AL. Multi-object tracking algorithm based on Hungarian matching and Kalman filtering[J]. Automobile Applied Technology, 2022, 47(1): 45-50. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-SXQC202201011.htm
[22]	李月峰, 周书仁. 在线多目标视频跟踪算法综述[J]. 计算技术与自动化, 2018, 37(1): 73-82. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJH201801016.htm LI Y F, ZHOU S R. Survey of online multi-object video tracking algorithms[J]. Computing Technology and Automation, 2018, 37(01): 73-82. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJH201801016.htm
[23]	戴雯惠, 樊凌. 基于改进透视变换的畸变图像校正方法研究[J]. 信息通信, 2020, 215(11): 63-65. https://www.cnki.com.cn/Article/CJFDTOTAL-HBYD202011021.htm DAI W H, FAN L. Research on distorted image correction method based on improved perspective transform[J], Information Communication, 2020, 215(11): 63-65. https://www.cnki.com.cn/Article/CJFDTOTAL-HBYD202011021.htm
[24]	MUTHALAGU R, BOLIMERA A, KALAICHELVI V. Lane detection technique based on perspective transformation and histogram analysis for self-driving cars[J]. Computers and Electrical Engineering, 2020, 85(1): 106653.
[25]	王德咏, 葛修润, 罗先启, 等. 基于改进DLT算法的数字近景摄影测量[J]. 上海交通大学学报, 2011, 45(增刊1): 16-20, 26. https://www.cnki.com.cn/Article/CJFDTOTAL-SHJT2011S1005.htm WANG D Y, GE X R, LUO X Q, et al. Study on digital close-range photogrammetry based on improved DLT algorithm[J]. Journal of Shanghai Jiao Tong University 2011, 45 (S1): 16-20, 26. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-SHJT2011S1005.htm
[26]	YUN S, HAN D, CHUN S, et al. CutMix: regularization strategy to train strong classifiers with localizable features[C]. IEEE/CVF International Conference on Computer Vision(ICCV), Seoul, South Korea: IEEE, 2019.
[27]	WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]. The European Conference on Computer Vision(ECCV)Munich, Germany: IEEE, 2018.
[28]	彭丁聪. 卡尔曼滤波的基本原理及应用[J]. 软件导刊, 2009, 8(11): 32-34. https://www.cnki.com.cn/Article/CJFDTOTAL-RJDK200911011.htm PENG D C. Basic principle and application of kalman filter[J]. Software Guide, 2009, 8(11): 32-34. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-RJDK200911011.htm