Calibration and Validation of the Simulation Model Xin Zhang
Comparison with existing tools Calibration and Validation
Comparisons Comparison of vehicle simulation with VISSIM Comparison of simulation w/o pedestrians impact
Intersection layout 4 1 2 3
Pre-timed Signal Plan 10s 40s 10s 40s
Light demand (vph) From/To 1 2 3 4 1 0 200 0 40 2 200 0 40 3 40 0 0 300 4 0 40 300 0 Medium demand (vph) From/To 1 2 3 4 1 0 800 0 100 2 800 0 100 3 100 0 0 1200 4 0 100 1200 0 Heavy demand (vph) From/To 1 2 3 4 1 0 1000 0 1500 2 1000 0 1500 3 1500 0 0 1500 4 0 150 1500 0
Light Demand Light demand (vph) From/To 1 2 3 4 1 0 200 0 40 2 200 0 40 3 40 0 0 300 4 0 40 300 0
Movement Delays Delay (Low Demand) EWT EWL NST NSL Time Trans Mix Trans Mix Trans Mix Trans Mix 5 31 28 24 32 23 25 67 61 10 30 31 34 41 29 26 64 61 15 23 27 69 53 24 23 47 58 20 32 30 48 44 26 24 46 54 25 33 28 38 40 25 26 67 60 30 28 29 17 28 28 27 53 57 35 30 33 35 30 22 25 72 66 40 29 34 42 34 29 28 23 48 45 29 26 38 32 27 29 47 49 50 23 25 54 45 28 24 27 31 55 23 26 45 42 23 24 33 39 60 19 24 61 54 28 26 36 35
Medium Demand Medium demand (vph) From/To 1 2 3 4 1 0 800 0 100 2 800 0 100 3 100 0 0 1200 4 0 100 1200 0
Movement Delays Medium Demand EWT EWL NST NSL Time Trans Mix Trans Mix Trans Mix Trans Mix 5 31 34 65 74 33 32 59 65 10 31 30 57 68 32 31 79 73 15 28 35 31 41 33 30 99 86 20 30 36 60 50 32 32 50 67 25 33 27 49 48 32 33 49 54 30 31 40 78 69 33 34 88 80 35 43 32 117 85 32 31 79 70 40 32 40 49 58 31 35 73 79 45 29 38 83 86 33 29 41 51 50 45 42 98 91 33 28 61 58 55 28 30 55 72 31 34 53 57 60 33 33 49 62 31 35 59 55
Heavy Demand Heavy demand (vph) From/To 1 2 3 4 1 0 1000 0 1500 2 1000 0 1500 3 1500 0 0 1500 4 0 150 1500 0
Movement Delays High Demand EWT EWL NST NSL Time Trans Mix Trans Mix Trans Mix Trans Mix 5 33 31 53 56 38 30 127 136 10 77 78 151 134 90 87 314 298 15 88 85 83 126 141 138 428 386 20 108 126 205 164 182 179 649 652 25 221 214 305 236 212 234 798 834 30 282 275 362 309 249 268 813 897 35 366 323 448 387 292 285 961 996 40 391 381 364 421 340 330 1010 1058 45 372 394 401 443 384 376 1182 1164 50 385 403 452 465 423 438 1337 1309 55 414 425 463 487 475 497 1473 1528 60 413 434 432 504 530 513 1626 1607
Paired t-test Demand Movement p-value 95% threshold Significance EWT 0.94 2.19 Not significant Low EWL 1.13 2.19 Not significant Demand NST 0.46 2.19 Not significant NSL 0.88 2.19 Not significant EWT 0.68 2.19 Not significant Medium EWL 1.25 2.19 Not significant Demand NST 0.33 2.19 Not significant NSL 0.65 2.19 Not significant High Demand EWT 1.38 2.19 Not significant EWL 1.54 2.19 Not significant NST 1.02 2.19 Not significant NSL 1.74 2.19 Not significant
Comparisons Comparison of vehicle simulation with VISSIM Comparison of simulation w/o pedestrians impact
Campus Drive PO4 PD1 VO1 VD1 PD2 PO1 PO2 PO3 VO2 VO3
Pedestrian and Vehicle Demand Pedestrians Vehicles Origin Destination Demand PO1 PD1 200 ped/hour PD2 200 ped/hour PO2 PD1 200 ped/hour PD2 200 ped/hour PO3 PD1 200 ped/hour PD2 200 ped/hour PO4 PD1 200 ped/hour PD2 200 ped/hour VO1 VD1 300 veh/hour VO2 VD1 300 veh/hour VO3 VD1 300 veh/hour
Simulation Result Mixed-Flow Simulation Trans-modeler Pedestrians Vehicles Pedestrians Vehicles Total Throughput 927 421 0 752 Average Delay (s) 15.14 44.56 0 6.21
Comparison with existing tools Calibration and Validation
Calibration Procedure
Sensitivity Analysis Purpose: isolate parameters to understand their influence Experiment design (full-factorial or sampling) Histograms, plots and statistical test
Optimization problem Minimize RMSPE D = p P obs sim Dp Dp ( ) obs D N p p 2 Subject to: sim Dp = F(...) There are potentially several optimal solutions; The quality of these local solutions varies substantially; Existing optimization techniques can not guarantee finding the global optimal solution.
Genetic Algorithm Component: a genetic representation of the solution domain, a fitness function to evaluate the solution domain. Major steps: Initialization Selection Genetic Operators (Crossover and mutation) Termination
Laboratory test Set the parameters in the simulation model with the initial values (can be any values within reasonable range); Use the same demand pattern collected from the field (can use any other reasonable demand patterns) and run the simulation; Compute the output and treat them as the virtual observed data; Change the parameters to random initial values and perform the calibration based on the virtual observed data; Compare the calibrated parameter values with the initial set; and Compare the calibrated simulation results with the virtual observed data.
Data Collection Site Jiangnan West Road @ Jiangnan Middle Avenue Guangzhou City Guangdong Province China
20-minutes Video Calibration data: first 15-minutes Validation data: remaining 5-minutes
Simulation Input Network Demand by turning movements Free-flow moving speed Signal timings
Turning parameters Way-finding parameters Destination location Moving momentum Signal controller Conflict resolve parameters Aggression Patience Hesitation
Simulation Output Individual trajectories Individual delays Time-dependent throughput by movements
Layout Vehicle Network Pedestrian Network
Signal phasing and timing plan
Free-moving speed Walking speed Driving speed
Cumulative Counts (Pedestrian)
Cumulative Counts (Vehicle)
Delay (Pedestrian)
Delay (Vehicle)
Tunable parameter definitions Parameter name Denotation Parameter name Denotation Weight of the vehicle distance floor field veh w dist Vehicle initial aggressiveness k veh Weight of the pedestrian distance floor field ped w dist Pedestrian initial aggressiveness k ped Weight of the vehicle direction floor field veh w dir Hesitation factor k hes Weight of the pedestrian direction floor field ped w dir Vehicle patience factor veh α Weight of the vehicle signal floor field veh w sig Pedestrian patience factor ped α Weight of the pedestrian signal floor field ped w sig Vehicle cost factor veh β Pedestrian cost factor ped β
Sensitivity analysis Three levels per parameter (low, median and high) Latin Hypercube sampling (1000 combinations) Scattered plots and ANOVA statistical test
Scattered Plot
ANOVA-test results
Laboratory test Parameter Target value Initial value Calibrated value veh w dist ped w dist veh w dir ped w dir veh w sig w sig veh k k k α α β β ped ped hes veh ped veh ped 0.20 0.52 0.25 0.50 0.21 0.48 (fix) 0.10 0.10 0.10 (fix) 0.10 0.10 0.10 0.95 0.26 0.91 0.50 0.12 0.55 (fix) 0.10 0.10 0.10 0.30 0.90 0.38 0.50 0.81 0.46 0.30 0.60 0.28 0.40 0.13 0.46 0.20 0.88 0.23 (fix) 0.10 0.10 0.10
Laboratory comparison From To RMSE RMSPE Theil s Inequality Coefficient Pedestrians Vehicles a c 3.19 0.048 0.00411 a d 4.20 0.068 0.00522 c a 4.17 0.063 0.00518 c d 4.44 0.059 0.00579 d a 4.63 0.038 0.00648 d c 4.33 0.046 0.00643 1 2 4.71 0.061 0.00823 1 3 5.01 0.085 0.00569 1 4 6.22 0.086 0.00569 2 1 5.68 0.074 0.00628 3 1 6.11 0.069 0.0541
Convergence of GA on Actual Data
Where is the problem? ped w sig w ped sig _ obey w ped sig _ violate
Validation result From To RMSE RMSPE Theil s Inequality Coefficient Pedestrians Vehicles a c 3.19 0.048 0.00411 a d 4.20 0.068 0.00522 c a 4.17 0.063 0.00518 c d 4.44 0.059 0.00579 d a 4.63 0.038 0.00648 d c 4.33 0.046 0.00643 1 2 4.71 0.061 0.00823 1 3 5.01 0.085 0.00569 1 4 6.22 0.086 0.00569 2 1 5.68 0.074 0.00628 3 1 6.11 0.069 0.0541