A hybrid virtual–real traffic simulation approach to reproducing the spatiotemporal distribution of bridge loads

Current traffic simulation approaches analyze vehicle loads and load effects from a statistical perspective; however, they fail to reproduce the spatiotemporal distribution of bridge loads and resultant load effects at every moment, hindering real‐time bridge health management. This paper proposes a two‐step hybrid virtual–real traffic simulation (HvrTS) approach to reproduce the spatiotemporal distribution of bridge loads. Vehicle load sequences are first identified using computer vision based on traffic video from two surveillance cameras installed along the bridge. Next, traffic microsimulation is optimized with the identified vehicle load sequences from the two cameras serving as the known input and validated output. Using a cable‐stayed bridge under free‐flowing traffic, the HvrTS approach achieved a weighted mean square matching error of ≤0.3 m for the vehicle longitudinal location and a matching error of ≤7% for the vehicle lane position, whereas with congested traffic, the matching errors were much higher due to the inherent complexities and challenges associated with reproducing vehicle behavior in heavily congested situations. The traffic load effects calculated via HvrTS presented excellent spatiotemporal matching with those measured by a structural health monitoring system, especially in free‐flow traffic conditions. Applications on a continuous rigid frame girder bridge further validate these findings. Hence, the proposed HvrTS approach can overcome the challenge of spatiotemporal matching between vehicle loads and load effects in field monitoring.

action during the operation stage of transportation infrastructures and is the second-most important risk factor after flood damage for bridge collapse events (Deng et al., 2016).Hence, integrating vehicle load information into traffic simulation is critical for effective infrastructure management.

Review of traffic simulation integrated with vehicle load information
Pavement weigh-in-motion (PWIM) and bridge weigh-inmotion (BWIM) systems help collect vehicle load information without disrupting traffic (Sujon & Dai, 2021;Yu et al., 2016), leading to increased research interest in traffic load simulation (O'Brien et al., 2021).This involves typical technical routes of establishing statistical models of vehicles and vehicle sequences, generating vehicle load sequences using the Monte Carlo algorithm, simulating the spatiotemporal evolution of vehicle load sequences, and calculating time-varying infrastructural load effects (N.Hou et al., 2021;J. Y. Zhou et al., 2021).Traffic load simulation allows the modeling of complex traffic loading scenarios thereby aiding in infrastructure design and management (Testa et al., 2022).
For short structures that can accommodate a limited number of vehicles, the simultaneous presence of multiple vehicles is the decisive loading scenario; hence, traffic load simulation is commonly performed under the assumption of a constant vehicle speed or inter-vehicle gap using the Monte Carlo modeling algorithm (Enright & O'Brien, 2013;Sifre & Lenner, 2019).However, microscopic carfollowing and lane-changing rules must be incorporated to realistically simulate the vehicle load sequences on long structures where vehicles may change their kinematic states.Therefore, the intelligent driver model (IDM; O'Brien et al., 2012;J. Y. Zhou et al., 2021), agent-based model (F.Y. Wang & Xu, 2019), and cellular automaton model (S.R. Chen & Wu, 2011;Ruan et al., 2017) have been integrated into traffic load modeling of long-span bridges.
These traffic load simulation approaches have been verified by field-measured traffic loads or load effects, as seen in O' Brien et al. (2012) and Ruan et al. (2017).However, these verifications focus solely on statistical rather than time-history differences in values.For bridge engineers, the most valuable information is the accurate spatiotemporal distribution of on-bridge vehicles and resulting bridge load effects.Unfortunately, this information cannot be reproduced by these traffic simulation approaches, making real-time management of traffic loads and bridge health conditions a challenge.

Review of vehicle spatiotemporal recognition in structural health monitoring (SHM)
Vehicle load is the most variable action during bridge operation and is, therefore, the primary focus of monitoring in the field of bridge SHM.Recent SHM studies have widely used machine learning techniques (Flah et al., 2021;Rafiei & Adeli, 2017), with computer vision demonstrating outstanding effectiveness in practical SHM projects due to its non-contact nature and low economic cost.Computer vision has successfully been used in several aspects, such as updating the geometry of finite element models (Y.Zhang & Lin, 2022), detecting surface defects (Pan & Zhang, 2022), reconstructing three-dimensional shapes (S.Wei et al., 2021), and measuring dynamic deflections (Zhao et al., 2022;Zhuge et al., 2022).More recently, computer vision has been utilized to detect and track vehicle trajectories across bridges using monitoring cameras, revealing the spatiotemporal distribution of on-bridge traffic loads.
Z. C. Chen et al. (2016) combined the weigh-in-motion (WIM) system with distributed cameras to detect and track vehicles, from entering the WIM system to exiting the bridge using template image matching and particle filtering algorithms.Subsequently, researchers attempted to increase the efficiency and accuracy of vehicle detection and tracking in the framework by improving the computervision algorithm using Gaussian mixture model-based image background subtraction (Dan et al., 2019), a faster region-based convolutional neural network (R-CNN; B. Zhang et al., 2019), Mask R-CNN (B.Zhang & Zhang, 2021), you only look once (YOLO) version three (Ge et al., 2020;G. Yang et al., 2022), and YOLOv5 (Zhu et al., 2022).However, besides being expensive to implement, having a limited sensor lifetime, and requiring regular calibration (Yu et al., 2016;Sujon & Dai, 2021), the PWIM system can cover only a limited number of bridges, as there are many entrances and exits along the highway.
Hence, researchers have estimated vehicle weight by matching the video-identified vehicle features with a collected vehicle load database (Micu et al., 2019;Pan et al., 2021;Y. Zhou et al., 2020).However, vehicle features alone cannot accurately determine the vehicle weight (e.g., vehicles may be fully loaded or empty) and hence cannot accurately reproduce the on-bridge traffic loads.In other studies, monitoring video and the BWIM system were combined to obtain the spatiotemporal distribution evolution of on-bridge vehicle loads (Jian et al., 2022;Xia et al., 2019).However, the feasibility of identifying multi-lane traffic loads for long bridges requires engineering validation, as BWIM is inherently an inverse problem that can be ill-posed under environmental noise.a This approach uses collected vehicle load databases and feature matching to determine vehicle loads and therefore does not require equipment.b Justifications can be found in Table 2.
The technical means and industrial application challenges of current literature studies on spatiotemporal recognition of traffic loads are summarized in Table 1.These studies suggest that the current computer-vision detection technologies based on CNN algorithms are effective in detecting and tracking vehicles using monitoring cameras.However, significant challenges to the efficient and precise reproduction of the spatiotemporal distribution of on-bridge traffic loads remain.First, the current approaches require the installation of dense distributed high-definition cameras along the highway corridors.As each camera must operate without failure to ensure the transmission of vehicle tracking information, the strategy is not only expensive but also lacks robustness for longterm monitoring.Second, vehicle weight identification using the PWIM system is very expensive and is not accurate enough for long-term monitoring.Meanwhile, the BWIM system requires further field validation under dense traffic conditions.Therefore, the acquisition of vehicle weight information to benefit the large number of ordinary bridges in roadway networks remains a significant challenge.

Contribution of this study
While current traffic load simulation approaches are highly efficient in presenting the evolution of on-bridge traffic loads, the generated vehicle load spatiotemporal information does not correspond to reality.Meanwhile, current computer vision approaches can detect and track the spatiotemporal distribution of vehicles; however, they have significant shortcomings in terms of costs, robustness, and long-term monitoring.
To address these limitations, this paper proposes a hybrid virtual-real traffic simulation (HvrTS) approach that involves two critical steps.First, the traffic parameters are identified using computer vision from only two surveillance cameras that are installed along the bridge for traffic monitoring.Additionally, YOLOv5 combined with DeepSort is employed to identify vehicle parameters such as license plate characters, number of axles, axle spacing, width, speed, lane location, and tire position.Vehicle weight information is obtained by mapping the video-identified license plates with the vehicle license plates recorded by the highway electronic toll collection (ETC) system.In China, ETC systems have been densely installed on toll expressways to increase traffic efficiency (e.g., one ETC gantry per 3 km in Guangdong Province; S. Wei et al., 2021).These systems record the weight information (uniquely matched to the vehicle license plate) of all vehicles on the expressway.Second, a microscopic traffic load simulation is conducted, where the identified vehicle load sequences from the two surveillance cameras serve as the known input and validated output.A self-developed grid-based traffic microsimulation program called the multi-axle single-cell cellular automaton (MSCA) is used for the traffic microsimulation, and the IDM and minimizing overall braking induced by lane change (MOBIL) model are used for the car-following and lane-changing rules, respectively.The values of the IDM and MOBIL parameters are optimized to achieve the best match between the simulated and video-identified vehicle load sequences.
The proposed HvrTS approach uses traffic simulation to determine the spatiotemporal distribution of on-bridge traffic loads and produces results consistent with realworld video-identified vehicle load sequences and SHMmeasured bridge load effects.Moreover, it has a low economic cost, considering that it only requires two surveillance cameras to cover several bridges when validated for application to long roadway lengths and that the vehicle weight is recorded using an off-the-shelf ETC system with no additional equipment.HvrTS addresses the difficulty that the current traffic simulation approaches face in realistically reproducing the real-time spatial-temporal distribution of on-bridge vehicle loads.It also solves the long-term monitoring challenge of computer vision methods in recognizing the spatiotemporal distribution of traffic loads on full bridge decks.
Most importantly, HvrTS overcomes the challenge of spatiotemporal matching between video-identified vehicle loads and SHM-measured bridge responses.To the best of the authors' knowledge, current studies on vehicle spatiotemporal recognition have not yet verified its accuracy using monitored vehicle-induced load effects.However, our study performs verification using a longspan cable-stayed bridge.Following the spatiotemporal matching between video-identified vehicle loads and monitored bridge responses using HvrTS, random traffic can be used to load-test the bridge.This is a significant advancement beyond current bridge loading tests that use carefully prepared trucks under closed/controlled traffic.Additionally, the spatiotemporal matching relationship between input traffic loads and output monitored responses can be used to better evaluate bridge health, overcoming the limitations of current SHM methods that mainly rely on output monitoring responses for structural condition assessment.
The remainder of this paper is organized as follows.Section 2 presents the background of the applied bridge and introduces the implementation framework of the proposed HvrTS approach.Section 3 describes the use of computer-vision techniques to identify vehicle parameters, for example, vehicle multi-object detection and tracking, image coordinate calibration, and vehicle weight collection via vehicle license plate mapping.Section 4 presents the microscopic hybrid traffic simulation approach using MSCA, where the video-identified vehicle load sequences at the entrance and exit of the bridge serve as the known input and validated output of the simulation optimization, respectively.Furthermore, the mapping results of traffic parameters from virtual simulation and real monitoring are analyzed.Section 5 describes the simulated spatiotemporal distribution of on-bridge vehicle loads and its mapping with the field monitoring of bridge load effects.
Finally, Section 6 concludes the paper by summarizing major findings and the scope for future research.

Framework
A cable-stayed bridge with a lightweight SHM system was chosen as a testbed to demonstrate the proposed HvrTS approach in Figure 1.The bridge had a span arrangement of 75 + 130 + 365 + 130 + 75 m and was supported by two concrete pylons and a single cable plane.The girder was 38 m in width and carried six-lane bidirectional traffic.The SHM system measured girder deflection, structural strain, and on-bridge traffic, with deflections monitored by a connected pipe system and strains monitored using fiber grating sensors.Four high-definition cameras were installed at the entrance and midspan of the bridge in both traffic directions with a sampling frequency of 25 Hz.The sampling frequency for deflection and strain measurement was 20 Hz.ETC gantries were located about 1.5 km from the bridge in both traffic directions, and there were no highway ramp entrances or exits between the ETC gantry and bridge to ensure that all on-bridge vehicles were captured by the ETC gantry recordings.
The framework of the HvrTS approach is shown in Figure 2. It comprises two critical steps.First, the vehicle load sequences over the monitoring regions are identified by incorporating roadside camera videos and surrounding ETC recordings.The camera videos are processed by computer-vision algorithms to identify vehicle parameters such as license plate characters, number of axles, axle spacing, width, speed, lane location, and tire position.Considering that each vehicle has a unique license plate number, fuzzy matching of the vehicle license plate characters between video identifications and the ETC recordings was conducted to obtain the gross vehicle weight.Finally, vehicle axle weights were inferred according to the vehicle weight, axle number, and axle spacing.Second, the random traffic simulation program was optimized, where the identified vehicle load sequences at the entrance and midspan of the bridge served as the known input and validated output, respectively.The MSCA was introduced as the microscopic traffic simulation approach for HvrTS, and the IDM and MOBIL models were utilized as the carfollowing and lane-changing rules, respectively.Then, a sensitivity analysis of the IDM and MOBIL parameters was performed to select the optimization parameters.Finally, practical optimization procedures for accurate virtual-real matching of vehicle load sequences at the bridge midspan were established.As a result, the spatiotemporal distribution of bridge traffic loads was reproduced, and the corresponding simulated traffic load effects were obtained.Additionally, the bridge condition can be evaluated and predicted by comparing the simulated and monitored bridge traffic load effects.The proposed HvrTS approach uses traffic loads as a carrier to realize the interaction between virtual simulation and real monitoring for the bridge.

Discussion
Table 2 presents a rough cost comparison for monitoring the spatiotemporal information of bridge traffic loads in a single-traffic direction, revealing a considerable economic advantage the proposed HvrTS approach offers over current approaches.In this study, vehicle weight data are obtained through the ETC system.Due to privacy concerns such as vehicle license plates, the ETC data are protected, and their usage requires an application to the administrative department, along with an explanation of the intended purpose.In Guangdong Province, China, the authority responsible for managing expressway ETC systems also oversees expressway infrastructure management, enabling the effective use of ETC data for bridge health monitoring.If ETC data are unavailable or lack weight information, the HvrTS framework can also be used but with alternative vehicle weight identification methods like PWIM or BWIM systems.However, HvrTS still offers cost advantages by reducing the traffic monitoring need for densely distributed cameras as shown in Table 2. Furthermore, the HvrTS approach offers robust vehicle parameter recognition, achieved by employing continuous frame recognition of vehicle parameters while requiring only one recognition result per parameter.This is because the spatiotemporal distribution of vehicles is reproduced by virtual traffic simulation, eliminating the need for precise vehicle trajectory tracking using computer vision.Consequently, the framework enables the identification of temporally masked vehicles and remains applicable even in congested traffic conditions as demonstrated in Section 5.

COMPUTER VISION-BASED VEHICLE LOAD SEQUENCE IDENTIFICATION
The procedure of utilizing computer-vision algorithms to identify vehicle load sequences is summarized in Figure 3. Vehicle detection and trajectory tracking, image coordi-nate calibration, vehicle license plate fuzzy matching, and axle weight inference are the four critical steps to be addressed.

Vehicle detection and trajectory tracking
The combination of YOLO and SORT algorithms for object detection and trajectory tracking, respectively, is widely used in computer vision (Haghighat & Sharma, 2023;Veeramani et al., 2018).In this study, YOLOv5 and Deep-Sort were used for multi-object detection and object trajectory tracking, respectively.A dataset, comprising 5700 pictures from the Stanford Cars Dataset and 2700 pictures captured by surveillance cameras on Chinese freeways, was used to train YOLOv5.Each picture was pre-tagged with the vehicle profile, vehicle head, vehicle tire, and vehicle license plate.Hence, a multi-object detector with minimized network loss was obtained, which was then evaluated using a validation dataset consisting of 1000 pictures.The validation results demonstrate high prediction of 94%, 89%, 97%, 96%, and 93% for car, truck, vehicle tire, vehicle license plate, and vehicle head classes, respectively.Moreover, the F1 score attains 0.95 at a confidence level of 0.574 for all classes, and the mean average precision is 0.978 at an intersection over union threshold of 0.5.Using the trained multi-object detector, any input video frame can be processed to identify these four vehicle objects, including the vehicle profile, vehicle head, vehicle tire, and vehicle license plate.
Figure 4 shows a sample result of vehicle detection and trajectory tracking.Applying the YOLOv5 and Deep-Sort algorithms, bounding boxes were added to the four The identification results of continuous frames are used to match all the vehicle license plate identification results with those in ETC recordings, resulting in high precision in fuzzy matching.For vehicle tire identification, the result from the closest frame containing the maximum number of vehicle tires is selected.The corresponding vehicle profile and vehicle head information from this frame are then utilized to determine vehicle width, tire position, lane location, axle number, and axle spacings.Additionally, the trajectory of the vehicle head from successfully identified frames is used to calculate vehicle speed.Therefore, all vehicle parameters are recognized.

Image coordinate calibration
After the object detection in the camera video, the true values of the vehicle parameters were calculated by transforming the pixel coordinates to world coordinates.
The transformation relationship between the pixel and world coordinates has been established in many computervision studies (Dan et al., 2019;G. Yang et al., 2022).
It is generally assumed that the bridge deck is in a plane in the world coordinate system; hence, the conversion from world coordinates to pixel coordinates can be expressed as where M is the coordinate transformation matrix, and Z c represents the depth value in the camera coordinate system.m 34 = 1 can be obtained through the transformation relationship between these coordinate systems.Therefore, there are eight unknown parameters in M, which should be calibrated by at least four measurement points with known world coordinates.Even if the coordinate transformation matrix M is known, Z c differs among the measurement points; hence, the relationship between Z c and the pixel coordinates should be established.In this study, the Z c values of the unknown measurement points were inferred by a polynomial function regressed by least squares fitted by the data of known measurement points: where a ik represents the polynomial function coefficients; and j and k represent the powers of u and v in the pixel coordinate system, respectively.The coordinate transformation matrix and depth values are generally calibrated by setting artificial points with measured world coordinates.However, because cameras may shift or rotate during operation, they regularly require temporary artificial points to be set on the bridge deck to recalibrate the coordinate conversion relationship, which is uneconomical and disruptive to traffic.In this study, the inherent roadside reflection points were utilized as image coordinate calibration points, as shown in Figure 5, which allowed automatic adaptive calibration without disturbing traffic and was economical as well as convenient.
First, the world coordinates of roadside reflection points were measured as shown in Figure 5b; the coordinates of the roadside reflective points on both sides in the bridge longitudinal direction were found to be the same.Then, the auxiliary lines (central line) of the traffic lane line were drawn, as well as the lines perpendicular to the traffic lane line connecting two reflection points on the two roadway sides.The interaction points of auxiliary and perpendicular lines were used as calibration points.Finally, the coordinate conversion matrix and Z c fitting function were calculated using these calibration points.Thus, the world coordinates of any image point could be determined, and the true vehicle sizes could be obtained.Furthermore, to verify the accuracy of the image coordinate transformation, several verification points were employed, as shown in Figure 5, and the results are presented in Table 3.The identification errors for these verification points in the longitudinal and transverse directions were all within 0.4 m, confirming that the proposed image coordinate transformation approach is accurate for the analysis of traffic load effects on long-span bridges.In preliminary experiments, the accuracy of the LPRNet algorithm was approximately 86% for vehicle license plate recognition.However, character recognition errors were owing to stains on the surface of the license plate or low differentiation of character colors.To address the issue of incomplete matching, a fuzzy matching algorithm is proposed, which incorporates the calculation of the Levenshtein distance to determine a fault tolerance index for character matching.The Levenshtein distance between two characters is calculated as follows: When the Levenshtein distance between two groups of license plate characters is no greater than two, they are considered to be the same characters.With this fuzzy matching algorithm, nearly 98% of vehicle license plates were successfully matched to the ETC recordings in our experiments.Hence, the weight information of the vehicles from the ETC recordings is accurate and can be used.

Vehicle axle weight inference
Once license plate matching is successfully achieved, the vehicle weight information captured by the ETC system is transferred to the video-recognition vehicle.Notably, the ETC system only records the gross weight of the vehicle.
Considering that the vehicle length cannot be neglected, compared with the bridge length, accurate calculations of bridge traffic load effects require precise vehicle axle weights.Therefore, it is necessary to infer axle weights based on the vehicle's gross weight and other relevant vehicle characteristics.
In this study, the K-nearest neighbor (KNN) clustering algorithm was used to learn such a relationship to infer vehicle axle weights.Over 10 million pieces of data previously collected from WIM systems of several highways were used for the training and verification of the KNN model.The WIM data included the vehicle's gross weight, axle number, axle spacings, and axle weights (J.Y. Zhou et al., 2020;J. Y. Zhou et al., 2021).Therefore, the vehicle weight, axle number, and axle spacings served as the model inputs, and the vehicle axle weights served as the model output.A total of 80% of the data was used for training the KNN model, and the remaining 20% of the data was used for model accuracy verification.Figure 6 shows the verification results of the KNN model.The KNN model well captured the features of the vehicle axle weight distribution accurately and inferred some cases of bifurcation of the axle weight distribution that existed owing to the inclusion of multiple sub-vehicle types of the same axle-type vehicle.
The axle weight is conventionally assumed to be in linear proportion to the gross vehicle weight (G.Yang et al., 2022).To verify the superiority of the KNN method, a comparative analysis is conducted to evaluate the accuracy of

Methods
Axle Note: In the format A/B, A is the average error and B is the error at the 95th percentile.The error is calculated as the percentage of the difference between predicted and actual axle weights to the gross vehicle weight.Abbreviation: KNN, K-nearest neighbor.
axle weight prediction using the KNN approach and linear inference approach.Additionally, the performance of other advanced data classification methods, such as multilayer perceptron, XGBoost, and random forest, is also examined.The results are shown in Table 4.The findings demonstrate that the KNN method outperforms the other methods, with an average error of axle weight prediction being no greater than 2.31% and a 95% percentile error no greater than 6.04%.In contrast, the conventional linear inference method yields a maximum average error of 4.82% and a maximum 95% percentile error of 14.57%.Additionally, the computational times for the KNN, multilayer perceptron, XGBoost, random forest, and linear inference methods for the 2 million validation data are 47.9, 24.3, 42.6, and 85.9 s, respectively.Despite not being the fastest, the prediction time of the KNN method is still rather short and has little effect on the forecast in real time.In summary, the KNN method performs well in determining axle weights.

MSCA-BASED HYBRID VIRTUAL-REAL TRAFFIC MICROSIMULATION
Figure 7 shows the HvrTS procedures, where the critical steps of MSCA-based traffic microsimulation, the selection of optimization parameters based on virtual-real matching errors and the optimization processes of traffic microsimulation are addressed.

MSCA-based traffic microsimulation
A cellular automaton is an efficient dynamic modeling tool with discrete time, space, and state that is well suited to the inherent discrete features of the traffic system.Furthermore, traffic cellular automata can generate probabilistic traffic information by simulating individual vehicle behavior, and they have been widely used in traffic engineering since they were proposed by Nagel and Schreckenberg (1992).Conventionally, traffic cellular automata characterize a single vehicle using a single cell or multiple cells with a fixed cell size of 7.5 m and a simulation timestep of 1 s (Esser & Schreckenberg, 1997;G. Hou et al., 2019).All the vehicle parameters, including the vehicle speed, vehicle length, and inter-vehicle gap, are integer multiples of the cell size.Although these assumptions can be adapted to meet the needs of the traffic simulation, they cannot fulfill the accuracy requirement for calculating bridge traffic load effects.Previously, an MSCA was developed to increase the accuracy and efficiency of random traffic load modeling for bridges (Ruan et al., 2017;J. Y. Zhou et al., 2019).In the MSCA, a vehicle is represented by a single cell occupied by the first vehicle axle as shown in Figure 8.All vehicle and bridge information is embedded into the cell states, including the cell-type parameters (vehicle and nonvehicle cells), vehicle characteristic parameters (axle number, gross weight, axle weights, axle spacings, length, and suspension lengths), vehicle dynamic characteristics (tire stiffness and damping, suspension stiffness and damping, vehicle mass, tire mass, and vehicle rotary inertia), vehicle motion parameters (first axle in-cell position, lane position, speed, time headway, inter-vehicle gap, acceleration, deceleration, and politeness factor), and bridge information (road roughness, bridge influence surfaces, and modal properties).
Following the defined cellular neighborhoods and transition rules, the random traffic flow can be simulated by the advancement of vehicle cells on the roadway.Vehicle load sequences can be accurately obtained at each simulation step by expanding the aforementioned vehicle parameters embedded into the cell status.The arbitrary timestep setting, precise vehicle axle loadings, and accurate inter-vehicle gaps provided by the MSCA are crucial for capturing the most adverse bridge load effects during the progression of the traffic flow.The MSCA has been verified by the measured traffic loads and load effects from a statistical perspective, and it is effective for the microsimulation of random traffic loads and the calculation of resultant bridge load effects in complex traffic scenarios (J.Y. Zhou et al., 2019Zhou et al., , 2021)).However, our previous studies have not used MSCA to reproduce the spatiotemporal distribution of vehicle loads.
In traffic microsimulation, the car-following and lane-changing models are fundamental traffic rules that require careful consideration.However, the literature offers numerous car-following and lane-changing models (Barceló, 2010).To optimize traffic simulation in HvrTS, the car-following and lane-changing models need to be thoroughly validated to ensure that they accurately reproduce real-world traffic phenomena while minimizing the number of parameters for convenient optimization.Therefore, the IDM and MOBIL models were, respectively, adopted for car-following and lane-changing rules.The IDM can effectively replicate real-world traffic phenomena using only five parameters (O'Brien et al., 2015;Treiber et al., 2000), and the MOBIL model has been shown to match well with the IDM and uses only four parameters (Kesting et al., 2007;Z. Wang et al., 2015).
For IDM, the vehicle travels at the desired speed for a particular distance, and the following strategy is adjusted according to its maximum acceleration, comfortable deceleration, and speed difference compared with the front vehicle as follows: ) 2 ] (4) where a c represents the current acceleration, a represents the maximum acceleration, v c represents the current speed, v 0 represents the desired speed, s c represents the current inter-vehicle gap to the front vehicle, s* represents the desired minimum inter-vehicle gap, s 0 represents the minimum inter-vehicle gap, T represents the safe time headway, Δv c represents the speed difference between the current vehicle and the front vehicle, and b represents the comfortable deceleration.There are only five parameters (a, v 0 , s 0 , T, and b) in the IDM that should be defined for simulation.
MOBIL is a decision lane-changing model that follows incentive and safety criteria.It measures the lane-changing risk according to the changes in acceleration and is well adapted to the maneuvering and forced lane change requirements in a multi-vehicle following environment (Caprani et al., 2016).The incentive criterion for moving from the slow lane to the fast lane is given as The incentive criterion for moving from the fast lane to the slow lane is given as The safety criterion for lane-changing is given as Here,   and   represent the accelerations of the target vehicle before and after it changes lanes, respectively;   and   represent the accelerations of the following vehicle before and after the target vehicle changes lanes, respectively;   and   represent the accelerations of the new following vehicle in the target lane before and after the target vehicle changes lanes, respectively; p is the politeness factor; b safe represents the maximum deceleration; Δa th represents the acceleration threshold preventing overtaking with a marginal advantage; and Δa bias represents the bias acceleration acting as an incentive to remain in the slow lane.There are only four parameters (p, b safe , Δa th , and Δa bias ) in the MOBIL model that should be defined for simulation.
The on-bridge vehicle load sequences can be obtained by applying the IDM and MOBIL models to individual vehicles in each simulation step.Then, to derive the bridge traffic load effects, the vehicle load sequences are applied to the bridge influence surfaces.The load sequences are given as follows: where gj i represents the jth axle weight of the ith vehicle at simulation time t.xj i represents the bridge longitudinal location corresponding to the jth axle weight of the ith vehicle at simulation time t.n represents the lateral lane location.f(.) represents the bridge influence surface, which is expressed as a function of the longitudinal location and lane location.
According to the definition of the MSCA, gj can be obtained directly from the vehicle characteristic parameters of the vehicle cells, and xj can be calculated using the vehicle characteristic parameters and motion parameters.Because the bridge influence surfaces are embedded into the cell states, the calculation of bridge traffic load effects is the numerical matrix operation of these parameters in the cell states of vehicle cells, which is efficient.

Selection of optimization parameters
Following the MSCA-based traffic microsimulation, the video-identified vehicle load sequences at the bridge entrance can be input to obtain the simulated vehicle load sequences at the bridge midspan.The simulated vehicle load sequences must precisely match the video-identified The more stars the greater the impact of the parameter on the virtual-real matching.
vehicle load sequences at the bridge midspan.Thus, two indicators are established as the objective optimization functions for HvrTS: where ER x represents the weighted mean square matching error of the longitudinal location of the same vehicle at the same time between recognition by the midspan camera and reproduction by traffic microsimulation.ER y represents the matching error of the lateral lane position of the same vehicle at the same time between recognition by the midspan camera and reproduction by traffic microsimulation.A smaller ER x indicates better optimization of IDM parameters and improves the virtual-real matching result.A smaller ER y indicates better optimization of MOBIL parameters and improves the virtual-real matching result.
N represents the number of vehicles for statistics.The superscripts virtual and real denote the simulated and video-identified results, respectively.W i represents the gross weight of the ith vehicle (unit: ton); a heavier vehicle has a larger W i value, with the standard for normalization being 3.5 tons.x i represents the longitudinal location of the ith vehicle.l i represents the lane label of the ith vehicle.χ represents the indicator function, which is 1 if the condition is satisfied and 0 otherwise.According to the literature (Caprani et al., 2016;Kesting et al., 2007;Mian & Jaffry, 2020;O'Brien et al., 2015;Treiber et al., 2000;J. Y. Zhou et al., 2021), the parameter value ranges for the IDM and MOBIL model in traffic microsimulation for various traffic scenarios are presented in Table 5.A sensitivity analysis was performed to investigate the effects of the IDM and MOBIL parameters on the matching errors of the vehicle longitudinal locations (Equation 10) and lane positions (Equation 11).Sensitivity analysis was conducted using the univariate control variables principle, where the studied parameter was varied linearly while keeping other parameters constant.The parameter value was changed within the range specified in Table 5, generating a total of 30 simulation sets for analysis.Initial parameter values for cars and trucks were provided: v 0 = average speed between two cameras, s 0 = 2.0m, T = 1.6 s, a = 2.0 m/s 2 , b = 2.0 m/s 2 , p = 0.1, b safe = 4.0 m/s 2 , Δa th = 0.2 m/s 2 , Δa bias = 0.2 m/s 2 .The parameter optimization followed a specific sequence: v 0 →s 0 →T→a→b→p→b safe →Δa th →Δa bias .After analyzing each parameter, the optimal value resulting in the lowest ER x and ER y was used for further sensitivity analysis.The sensitivity analysis results of IDM and MOBIL parameters on the matching errors of ER x and ER y are given in Figures 9 and 10, respectively.The virtual-real matching error is indicated by the z-axes in both figures, whereas the x-and y-axes indicate the parameter value ranges for the truck and car, respectively.
The results indicated that the desired vehicle speed most significantly influenced both ER x and ER y because the desired vehicle speed affected not only the forward movement of the vehicle but also the aggressiveness of the driver.To characterize the desired speed of vehicles between the two monitoring cameras, there are two options: the instantaneous speed captured by the midspan monitoring camera and the average speed calculated by dividing the distance by the time elapsed.The average speed offers a more reliable measure of the expected speed of vehicles over a specific distance, compared to the instantaneous velocity, which tends to have greater variability.Primary analysis validated that the use of average speed is much better in minimizing the matching error than the instantaneous speed.Hence, the average speed between the two monitoring cameras (i.e., from the bridge entrance to the bridge midspan for the case bridge; Figure 1) for each vehicle was used.Considering the randomness of the desired speed for each vehicle, a coefficient of variance based on a normal distribution was used.Additionally, the maximum acceleration and comfortable deceleration hardly affected the matching errors.Free-flowing traffic was investigated; therefore, it appeared as though the minimum jam distance did not affect the traffic microsimulation.Owing to the free-flowing nature of the traffic under study, these MOBIL parameters had less impact on the lane position matching error than the IDM parameters.Table 5 presents the sensitivities of these parameters, which offer the traffic microsimulation direction to achieve the minimum matching errors.The recommended initial values of the IDM and MOBIL parameters based on the sensitivity analysis are also pre-sented in Table 5, which were determined by the minimum matching errors in Figures 9 and 10.

Hybrid traffic simulation optimization
Based on the sensitivity ratings of the IDM and MOBIL parameters, a practical optimization procedure is proposed for minimizing the virtual-real matching error in the hybrid traffic microsimulation.
Step 1: The recommended initial settings of the IDM and MOBIL parameters in Table 5 are adopted for traffic microsimulation, where the video-identified vehicle load sequences at the bridge entrance are input as shown in Figure 7. Through a microscopic simulation analysis, the simulated vehicle load sequences at the bridge midspan are generated at the same moment as the video-identified vehicle load sequences at the bridge midspan.
Hence, the weighted mean square matching error of the longitudinal location and mean matching error of the lateral lane position are calculated by Equations ( 10) and ( 11), respectively.
Step 2: By modifying the MOBIL parameters, that is, the politeness factor, the changing threshold, and the bias for the slow lane, vehicles with failed lane position matching are forced to move to their target lanes.Consequently, although lane position matching can be easily accomplished, the longitudinal location matching error may increase.
Step 3: The desired speed v 0 is optimized for each vehicle individually, where a randomly generated δ for each vehicle is iterated-particularly for the vehicle with a large longitudinal location matching error.
The best results are obtained with the matching error being minimized.
The sensitivity ratings of these factors are considered in the proposed optimization strategy.The optimal matching results can be obtained quickly if the values of the parameters with the highest sensitivity are first determined.
To demonstrate the application of the HvrTS approach, 1-h monitoring data under free-flow traffic in two traveling directions are used.More details on the data are provided later in Section 5.1.The matching errors of the virtualreal hybrid traffic simulation under critical optimization steps are presented in Figure 11 and Table 6, where "Step 0" corresponds to commonly used IDM and MOBIL settings (Caprani et al., 2016;J. Y. Zhou et al., 2021), and "Steps 1-3" are explained above.There were 446 and 468 vehicles on the left and right sides, respectively, within the 1-h monitoring period.
Figure 11a presents the longitudinal location matching error of each vehicle under the commonly used IDM and MOBIL settings.The vehicle longitudinal position matching error was large, with a maximum value of 170 m.The results indicated that it is generally difficult to accurately reproduce the spatiotemporal evolution of on-bridge vehicles using the common IDM and MOBIL parameters.Because the traffic conditions differ between the two traffic directions, the proposed optimization process is conducted separately for the left and right sides.The matching error of each vehicle under the recommended initial settings of the IDM and MOBIL parameters presented in Table 5 is shown in Figure 11b.Although the vehicle longitudinal position matching error was F I G U R E 1 1 Longitudinal location matching error of each vehicle during critical optimization steps under free-flow traffic.
significantly reduced, a few individual vehicles had relatively large errors owing to insufficient lane changes, which caused vehicle drivers to drive slowly, as indicated by the lane position matching error in Table 6.Through the forced lane change described in Step 2, the lane position matching error was significantly reduced (to 1.6% and 0.4% on the left and right sides, respectively) as shown in Table 6.However, the forced lane changes disrupted traffic operations, increasing the vehicle longitudinal position matching error as shown in Figure 11c.Furthermore, by optimizing the desired speed of each vehicle, as described in Step 3, the vehicle longitudinal position matching error was significantly reduced, and a small lane position matching error was maintained as shown in Figure 11d.
After optimizing the parameters of traffic microsimulation, the weighted matching errors for the longitudinal position of vehicles on both the left and right sides were only 0.2 m, and the matching errors for the lane position of vehicles on the left and right sides were 4.1% and 6.6%, respectively.These results indicated that the accuracy of the proposed method was sufficient for engineering analysis.While the hybrid traffic simulation optimization process proposed in this study was found to be practical and effective, it is worth noting that multiparameter optimization algorithms that can automatically search for the best parameter values would be necessary to achieve highly efficient virtual-real matching in the future.

Results under free-flow traffic
Due to privacy concerns related to vehicle license plates, the ETC data are protected, and their usage requires an application with justifications to the administrative department.As the initial attempt in this topic, 1-h free-flow traffic data on March 13, 2022, are first granted.However, it is important to note that the duration of the data does not impact the methodology.For the free-flow traffic data, the optimization of traffic microsimulation has been detailed in Section 4.3.Using the final parameter values obtained via the HvrTS, the spatiotemporal distribution evolution of on-bridge vehicle loads was reproduced as shown in Figure 12.
The diagram shows the spatiotemporal trajectory of each vehicle on the bridge deck during the 1-h monitoring period.In the figure, the time and location along the bridge are indicated by the horizontal and vertical axes, respectively.The trajectory of each vehicle is indicated by continuous dots, and the slope of these continuous dots represents the vehicle speed.Additionally, symbols are used to distinguish vehicle types.Most vehicles were concentrated in the outer two traffic lanes that were nearest the roadway shoulder.The proportion of fast lane vehicles in both directions was approximately 11%, and the proportions of vehicles in the outer two traffic lanes were 45% and 44%.The two-axle vehicles accounted for the majority of the traffic flow, with a proportion of 91.70%, whereas the proportions for three-axle, fouraxle, five-axle, and six-axle vehicles were 1.35%, 1.57%, 0.45%, and 4.93%, respectively.Most vehicles maintained a uniform speed on the bridge; hence, their trajectories were straight lines.Vehicle lane-changing behaviors were observed, with trajectories alternating between adjacent traffic lanes.The traffic was free-flowing, and no local traffic congestion was observed.The reproduced spatiotemporal distribution evolution of on-bridge vehicle sequences was useful for analyzing vehicle conflicts as well as the microscopic behavior of abnormal vehicles.For example, at approximately 2800 s, a six-axle vehicle followed a three-axle vehicle in the middle lane of the right side, and after a period of following, the six-axle vehicle moved to the fast lane to overtake the three-axle vehicle.
The weights of the vehicles passing the bridge midspan over time are shown in Figure 13.The maximum vehicle weight was no greater than 60 tons owing to the strict overloading management along the highway.Most heavy trucks were distributed in the outer two traffic lanes; however, a few heavy trucks were traveling on the fast lane, considering that the traffic was free-flowing and each vehicle had a high degree of randomness.As shown in the figure, there were closely spaced trucks, which had significant bridge load effects.
The spatiotemporal distribution of on-bridge vehicle loads was applied to the bridge influence surface to calculate the traffic load effects.The spatiotemporal synchronization between the calculated and measured load effects was evaluated by analyzing the bridge deflections.For the bridge deflections measured by the SHM system, factors such as temperature, wind, and environmental noise can impact the bridge load effects; therefore, traffic-induced bridge load effects should be separated from them.The measured girder deflection over 1 h at the bridge midspan (DL#5 in Figure 1) is shown in Figure 14a.The girder deflection exhibits an overall downward deformation trend with many oscillations.
Owing to the flexibility of the studied bridge, the overall downward deformation was mainly caused by the temperature that exhibited a quasi-static characteristic, which was evidenced by the sunny weather on this day.Therefore, by applying a moving-average window function with a window size of 15 min to the measured girder deflection, the quasi-static deflection was filtered out, and the traffic-induced girder deflections were obtained as shown in Figure 14b.The traffic-induced deflection contains two components: static traffic deflection and dynamic traffic deflection.The dynamic traffic deflection belongs to the high-frequency vibration component, which can be filtered out using a moving-average window function with a window size of 1 s.Thus, the measured static traffic deflections were obtained and compared with calculated static traffic deflections via HvrTS.The results are presented in Figure 14c.As shown, both the peak appearance times and overall changing trends were consistent between the measured and calculated static traffic deflections.Although the timing of the peaks exhibited a slight offset between the measured and calculated traffic deflections, the timehistory load effects during the period when the vehicles passed over the bridge were fully reflected.The offset of the matched peaks between the measured and calculated static traffic deflections is attributed to the fact that the bridge cameras were installed at the head and midspan of the bridge.The spatiotemporal distribution evolution of the on-bridge vehicle loads between the midspan and the bridge end was not verified.If cameras installed at the end of the bridge are used for HvrTS calibration, the matching results may be improved.

Results under congested traffic
Another hour of heavy traffic on May 3, 2023, was used to test the practicality of the HvrTS approach applied to complex traffic scenarios.This day was a holiday, and one traffic direction experienced congestion, while the other direction had free-flowing traffic.Figure 15 illustrates the bridge map, along with snapshots from four surveillance cameras.It is shown that the traffic is congested in the direction monitored and #A2.Additionally, the outmost lane is heavily congested, while the two inner lanes are relatively clear, as observed from snapshots taken by cameras #A1 and #A2.This distinctive traffic congestion is caused by a forthcoming road bifurcation as shown in the bridge map, posing significant challenges to traffic microsimulation in HvrTS.In contrast, the traffic in the opposite direction monitored by cameras #B1 and #B2 is sparse, making it comparatively easier to identify The map of the bridge along with the snapshots of four surveillance cameras under congested traffic.
vehicle parameters and perform HvrTS.Notably, the proposed vehicle parameter recognition using computer vision is based on the analysis of consecutive frames.
The successful recognition results of each vehicle object from several frames are combined to calculate the vehicle parameters, making complete identification of the vehicle trajectory redundant.Therefore, even in scenarios where the vehicle is partially masked, the vehicle parameter can still be extracted as long as the vehicle object is detected in certain frames.
Figure 16 gives the weights of vehicles when passing through the bridge midspan over time.There was a total number of 1455 and 3210 vehicles on the left free-flowing side and right congested side, respectively.Among these vehicles, the 2-axle type constitutes over 97%, and the heaviest vehicle weighs approximately 50 tons.Despite the varying traffic volume in the two directions, the truck volume and average truck weights are comparable.Similar to the results observed in free-flowing traffic, in congested traffic scenarios, the outer two traffic lanes predominantly accommodate heavy trucks, while the inner fast lane is occupied by cars.
Traffic microsimulation is optimized following the same numerical procedures described in Section 4.3.A significant challenge is the implementation of mandatory lane changes in vehicles identified at different lane locations by the two cameras.Besides, several other strategies, based on the vehicle behavior observations obtained from the two monitoring cameras are employed, which are not detailed here for simplification.Finally, the longitudinal location matching error of each vehicle after final optimization under congested traffic is shown in Figure 17.The results indicate that the matching errors on the left free-flowing side are small, with ER x of 0.30 m and ER y of 6.5%.However, as for the right congested side, the matching error for each vehicle is relatively significant, with ER x of 16.53 m and ER y of 19.8%.
The large matching errors are attributed to the complex congested traffic condition on the right side of the bridge, that is, one-traffic-lane congested but other lanes free-flowing due to forthcoming road bifurcation.The F I G U R E 1 8 Spatiotemporal synchronization between calculated deflections from the HvrTS and measured deflections from the SHM system, for measurement point DL#5 under congested traffic.
complexity of traffic diversion in this congested state necessitates the incorporation of vehicle destination as a crucial indicator in traffic microsimulation.The unsatisfactory results suggest that the optimization procedures for traffic microsimulation in HvrTS require improvements to accurately reproduce complicated congested traffic with diversion.However, in cases of regular congestion without traffic diversion, the proposed approach is believed to be able to reproduce the spatiotemporal distribution of traffic loads.
Based on the reproduced spatiotemporal distribution of on-bridge vehicle loads, the girder deflections under traffic loadings are calculated using a similar method, and the results are then compared with field-measured girder deflections.Figure 18a displays the 1-h measured total deflections at the same measurement DL#5.The graph indicates that the overall deflection does not exhibit a distinct trend, suggesting that temperatureinduced deflection is not prominent, which is supported by the cloudy weather conditions on that particular day.Furthermore, the total deflections deviate from zero, primarily due to heavy traffic.Under such conditions, the traffic-induced deflections cannot be easily extracted from the temperature-induced deflections.
Figure 18b illustrates the calculated static deflections under left-side traffic, which exhibit oscillations around zero, indicating free-flowing traffic conditions on the bridge deck.Moreover, several peaks coincide with the measured deflections, suggesting that these peak deflections are primarily caused by left-side traffic loadings.Figure 18c displays the calculated static deflections under right-side congested traffic, showing a gradual trend of change.Notably, some peaks align with the measured deflections.The measured total deflections obtained from the SHM system are compared with the calculated total deflection using HvrTS, which combines both left-side and right-side traffic loadings, as depicted in Figure 18d.It should be noted that the measured deflections are raw data and have been adjusted to the mean value of the calculated deflections for comparison purposes.The graph reveals numerous consistent peaks between the measured and calculated deflections.However, the spatiotemporal matching effect of Figure 18d is not as pronounced as that of Figure 14c due to the inadequate reproduction of on-bridge traffic loads during right-side congested traffic conditions.Nevertheless, the spatiotemporal matching of vehicle loads, and monitoring load effects remains favorable for congested traffic conditions.
In summary, the virtual-real matching of the calculated and measured bridge load effects confirms the feasibility of HvrTS for reproducing the spatiotemporal distribution evolution of on-bridge vehicle loads.This method has broad application prospects in engineering, for example, for analyzing the microscopic behaviors of special vehicles such as oil tank trucks over the bridge deck, evaluating vehicle conflict in complex traffic scenarios such as partial lane closure for infrastructure maintenance, assessing bridge health by comparing the calculated and measured load effects, and providing early warnings related to bridge safety during the passage of overloaded trucks.

APPLICATIONS ON ANOTHER BRIDGE
A continuous rigid frame prestressed concrete box-girder bridge, recently equipped with an SHM system as depicted in Figure 19, was utilized to further validate the proposed approach.The bridge comprised two identical yet individual lateral parts, with each part accommodating three lanes in one traffic direction.Both parts of the bridge were equipped with SHM systems, which monitored strains and displacements of critical girder cross-sections.The SHM F I U R E 2 0 Longitudinal location matching error of each vehicle.
systems were similar to those utilized for the cable-stayed bridge discussed in Section 2. However, only the left part of the bridge was equipped with two surveillance cameras for monitoring on-bridge traffic.Cameras #1 and #2 were positioned at the head and midspan of the bridge, respectively.A total monitoring video duration of 11 h, spanning from 6:00 to 17:00 , was granted for investigations.
Following the proposed approach, vehicle parameters in the two surveillance cameras were initially identified using the computer vision algorithms YOLOv5, DeepSort, and LPRNet.Subsequently, load information was assigned to the vehicles through fuzzy matching of license plates obtained from the video-identified and ETC-recorded data.Finally, the traffic microsimulation was optimized to minimize virtual-real matching errors of ER x and ER y , where the longitudinal location matching error of each vehicle is presented in Figure 20.The results indicate that the traffic was free-flow, with a total of 19,630 vehicles.The matching errors for these vehicles ranged from −8 to 16 m, with the ER x of 0.87 m and ER y of 6.85%.The low values of matching errors demonstrate the feasibility of the proposed HvrTS approach for reproducing spatiotemporal traffic loads.
In addition, the time and weight of each vehicle when passing the midspan of the bridge were extracted as illustrated in Figure 21.The data showed that the fast F I G U R E 2 2 Spatiotemporal synchronization between calculated deflections from the HvrTS and measured deflections from the SHM system, for measurement DL#6.ETC, electronic toll collection.lane predominantly accommodated light vehicles weighing generally no more than 20 t.However, there were occasional instances of heavy trucks, exceeding 40 t, appearing in the fast lane.The middle traffic lane exhibited a large number of heavy trucks, approaching 50 t.Moreover, the slow lane featured many heavy trucks with weights nearing 50 t.Notably, there was an exceptionally heavy truck weighing 151.08 t, identified as a specialized vehicle transporting a concrete box girder (Figure 22).
Utilizing the reproduced spatiotemporal on-bridge vehicle loads, girder deflections at the midspan under traffic loadings were calculated.Subsequently, the results were compared with the field-measured girder deflections at measurement point DL#6. Figure 22 illustrates the spatiotemporal synchronization between calculated static traffic deflections and measured static traffic deflections.The measured static traffic deflection was extracted from the raw monitoring data using the filters mentioned previously.It is evident from the figure that the peak appearance times and overall changing trends are consistent between the measured and calculated static traffic deflections.Although the timing of the peaks exhibited a slight offset between the measured and calculated traffic deflections, the time-history load effects during the period when the vehicles passed over the bridge were fully reflected.
Moreover, the matching peaks were zoomed in to illustrate the time-history changes.Snapshots from the midspan Camera #2, corresponding to the peak load effect, were provided, along with vehicle weight information.These details demonstrate the good virtual-real matching effects.Such a long duration (11 h) of spatiotemporal synchronization of virtual-real bridge load effects validates that the proposed HvrTS approach is reliable for reproducing the spatiotemporal distribution of traffic loads.

CONCLUSION
A HvrTS approach is proposed for reproducing the spatiotemporal distribution of on-bridge vehicle loads.The method involves two critical steps.First, the vehicle load sequences are identified via computer vision using two surveillance cameras installed along the bridge for traffic monitoring.Second, a traffic microsimulation is conducted to identify the vehicle load sequences from the two surveillance cameras, which serve as the known input and validated output, respectively.The MSCA is used for traffic microsimulation, and the IDM and MOBIL are used as the car-following and lane-changing models, respectively.The values of the IDM and MOBIL parameters were optimized to achieve the best matching between the simulated and video-identified vehicle load sequences.The major findings of the study are given below.Accurate detection and trajectory tracking of the vehicle, including the license plate, vehicle head, vehicle tires, and contour, were realized using YOLOv5 and DeepSort.The spatiotemporal locations of vehicle parameters were identified with good accuracy, with longitudinal and lateral errors within 0.8 and 0.4 m, respectively.The fuzzy matching rate between the video-identified and ETC-recorded vehicle license plates was >98%, indicating that the proposed method is convenient and efficient for obtaining vehicle weight information on highways with ETC systems installed.After the step-by-step iterative optimization of the IDM and MOBIL parameters via the HvrTS, the weighted mean square matching error of the longitudinal location was within 0.3 m, and the mean matching error of the lane position was within 7% for free-flowing traffic.However, the matching errors are higher under congested traffic due to the inherent complexities and challenges associated with reproducing vehicle behavior in heavily congested situations.The virtual-real matching of the HvrTS had a good consistency between the calculated (obtained from the HvrTS) and measured (obtained from the bridge SHM system) traffic load effects.The extended applications of the proposed approach to a continuous rigid frame girder bridge further validate the above findings.
The proposed HvrTS approach was found to be efficient for reproducing the spatiotemporal distribution of on-bridge vehicle loads both in free-flowing and congested traffic conditions.It linked the on-bridge vehicle loads obtained via the traffic microsimulation with the monitored bridge load effects obtained from the SHM system, thereby realizing spatiotemporal synchronization, which is important for the risk assessment and management of both traffic systems and transportation infrastructure.In future research, the following topics can be considered: (1) An automatic multi-parameter optimization approach for the IDM and MOBIL model in HvrTS is needed to achieve efficient virtual-real matching results, especially in congested traffic conditions.(2) Methods can be developed to fully utilize the spatiotemporal matching load effects for performance analysis and health assessment of bridges after the matching of calculated and measured traffic load effects.(3) The safety risk of traffic systems and bridge superstructures caused by special vehicles such as oil tank trucks over the bridge deck can be analyzed using HvrTS.(4) More complex on-bridge traffic scenarios, such as regional congestion due to bottleneck constraints or fullfield congestion, could be analyzed to further validate and enhance the HvrTS approach.(5) Efficient edge intelligent computing techniques would be proposed to allow direct vehicle license plate matching at the edge without exposing the actual information, thus addressing the privacy concerns of the ETC and video data.

A C K N O W L E D G M E N T S
This research was supported by the National Natural Science Foundation of China (52178280, 51808148), the Natural Science Foundation of Guangdong Province of China, and the Fundamental Research Program of Guangzhou Municipal College Joint Fund (SL2023A03J00897).
Open access publishing facilitated by Monash University, as part of the Wiley -Monash University agreement via the Council of Australian University Librarians.

TA B L E 1
Technical means and industrial application challenges of current literature studies on spatiotemporal recognition of traffic loads.

F
I G U R E 1 Structural health monitoring (SHM) system of the bridge.Proposed hybrid virtual-real traffic simulation (HvrTS) approach framework.ETC, electronic toll collection.MSCA, multi-axle single-cell cellular automaton.

F
Processes of vehicle load sequence identification using computer vision.ETC, electronic toll collection.F I G U R E 4 Multi-object detection and trajectory tracking of a sample vehicle using computer-vision algorithms.YOLO, you only look once.detected objects in each video frame, where the vehicle license plate characters were extracted by the License Plate Recognition via Deep Neural Networks (LPRNet) algorithm (D.Wang et al., 2020).Depending on the lighting conditions, clarity of the license plate, and recognition position, individual license plate character recognition may have errors; however, this does not affect the matching of the entire license plate characters.The bounding box of the vehicle profile is mainly used to separate individual vehicles.The bottom midpoint of the bounding box of the vehicle tire is used as the feature point to determine the lane location, tire position, axle number, and axle spacing.The two corner points on the long side of the bounding box of the vehicle head are used as feature points to calculate the vehicle width and determine the tire position of the other side of the vehicle.Notably, continuous frame recognition of vehicle parameters is performed, with only one recognition result per parameter required.
Automatic image coordinate calibration using roadside reflection points.
, ( − 1, ) + 1  , (,  − 1) + 1  , ( − 1,  − 1) +  (≠) (3) where |a| and |b| represent the lengths of strings a and b, respectively.lev a,b (|a|, |b|) represents the Levenshtein distance between strings a and b. χ (ai≠bj) is the characteristic function, which has an arbitrary value of 1 if a i ≠ b j and 0 otherwise.lev a,b (i, j) represents the Levenshtein distance between the first ith character of string a and the first jth character of string b.

F
Verification of the trained K-nearest neighbor (KNN) model for vehicle axle weight inference.TA B L E 4 Accuracy comparison among different axle weight inference methods: A sample of six-axle vehicles (%).

F
Process of hybrid virtual-real traffic microsimulation.MSCA, multi-axle single-cell cellular automaton.

F
Diagram of the cell neighbors and cell states in the multi-axle single-cell cellular automaton (MSCA).

TA B L E 5
Parameter values of the intelligent driver model (IDM) and minimizing overall braking induced by lane change (MOBIL) model in traffic microsimulation.the coefficient of variance based on a normal distribution assumption of the average speed between two monitoring cameras.b

F
Sensitivity analysis to determine the effects of the intelligent driver model (IDM) and minimizing overall braking induced by lane change (MOBIL) parameters on the matching error of the vehicle longitudinal locations.

F
Sensitivity analysis to determine the effects of the IDM and MOBIL parameters on the matching error of the vehicle lane positions.
Three-axle Four-axle Five-axle Six-axle F I G U R E 1 2 Spatiotemporal trajectories of on-bridge individual vehicles obtained via HvrTS under free-flow traffic.

F
Time and weight of each vehicle when passing the bridge midspan under free-flow traffic.(a)Quasi-static deflection filtering (b) Dynamic traffic deflection filtering (c) Comparison of measured and calculated static traffic deflection F I G U R E 1 4 Spatiotemporal synchronization between calculated deflections from the HvrTS and measured deflections from the SHM system, for measurement point DL#5 under free-flow traffic.

F
Time and weight of each vehicle when passing the bridge midspan under congested traffic.F I G U R E 1 7 Longitudinal location matching error of each vehicle after final optimization under congested traffic.
Calculated deflection under right side congested traffic (d) Measured versus calculated deflections Note: Measured deflections are offset to the mean value of calculated static traffic deflections.

F
I G U R E 1 9 Structural layout and SHM systems of the continuous rigid frame girder bridge.

F
I G U R E 2 1 Time and weight of each vehicle when passing the midspan of the bridge.

TA B L E 2
Rough cost comparison of spatiotemporal information monitoring of traffic loads on the case bridge in a single traffic direction.

Equipment cost Approach Weight identification a Spatiotemporal recognition b Installation cost Social cost (traffic interference) c Weighing precision d References for the approach
Video recognition error analysis based on image coordinate transformation.
TA B L E 3

side Right side x virtual -x real (m) ER x (m) ER y (%) x virtual -x real (m) ER x (m) ER y (%) Optimization steps Max. Min. Avg. abs. Max. Min. Avg. abs.
Statistics of vehicle sequence matching errors under critical optimization steps.
TA B L E 6