Moving from Data Warehousing to Decision Support. Dr. Bruce Hellinga Feb. 28, PDF Free Download

Moving from Data Warehousing to Decision Support Dr. Bruce Hellinga Feb. 28, 2013

Prior to Data Warehousing Problem definition Analysis Characteristics: Limited data spreadsheet tools applicable But small sample limits confidence in results Analysis difficult to reproduce Data not (systematically) archived Ad-hoc quality control

Data Warehousing Automated data collection systems AVL/APC Farecard Data Archive (Warehouse) Characteristics: Large quantity of data Increased confidence in results, but Quality control an issue Require more robust analysis tools Analyst needs to pull data/information from warehouse to identify problems and causes

Beyond Data Warehousing Wide variety of terms in use: Decision Support System (DSS) Business Intelligence (BI) Expert System Intelligent Agent (IA) All are about automating the transformation of data to information that is useful

The Four Pillars Quality Assurance (QA) Measures Of Performance (MOP) Assisted/ Automated Diagnosis Push and Pull

Distance (km) P1: Quality Assurance Data Errors (Reality different from recorded data) 10 9 8 7 6 5 4 3 2 Route 51 - GRT Actual AINSLIE STREET TERMINAL SMART!CENTRES CAMBRIDGE HOLIDAY INN TERMINAL Scheduled CAMRIDGE CENTRE TERMINAL 1 Schedule deviation 0-21 -18-15 -12-9 -6-3 0 3 6 9 12 15 18 21 24 27 30 Time (minutes)

P1: Quality Assurance Data Anomalies Data are accurate but should not be used for calculating performance measures and/or for decision making

100% 85% Cumulative Relative Frequency All weather conditions Excluding snow storms Travel Time (minutes)

Distance (km) But it is not always easy to determine whether data are erroneous, anomalies, or valid! 40 ixpress 200; Departing 5:15pm 42 minutes CONESTOGA MALL 35 McCORMICK 30 UPTOWN WATERLOO 25 CHARLES TERMINAL 20 FAIRVIEW 15 10 SMART!CENTRES CAMBRIDGE 5 0 AINSLIE STREET TERMINAL 0 10 20 30 40 50 60 70 80 90 100 110 120 Time (minutes)

P2: Measures of Performance With rich, precise data, it is possible to compute many measures of performance, but significant care is required. Example: Schedule adherence MOP = % on-time on-time = does not depart early and does not arrive more than 3 minutes late.

Fraction of observations departing more than X seconds early 20% 18% 16% 14% 18.8% # observations (i.e. departures from time points) = 127,622 14.4% 12% 10% 8% 6% 4% 2% 10.7% 7.7% 5.9% 4.6% 0% > 0s early > 15s early > 30s early > 45s early > 60s early > 75s early

P3: Automating Diagnosis Questions: 1. What are the problems? 2. What are the (likely) causes of these problems? 3. How do we solve the problem? Data Warehouse

P4: Push and Pull An Intelligent DSS should identify problems and notify the appropriate people at the appropriate time with the appropriate information so they can make the appropriate decision Relevant information (alerts) are pushed out from the system automatically Still retain the ability for an analyst to pull information for custom queries

Leveraging the data Transit data is not only valuable for the transit operator and/or transit user!

Signalized Intersections Transit Operator Delays at signalized intersections increase operating costs and decrease quality of service Can implement All of these TSP activities or queue require jump data lanes, about but where? existing conditions at the intersections AND Inadequate we LOS don t necessitates know ahead improved of time which signal timings intersections we need data for. Signals Manager Infrastructure Planning Calibrate planning models Identify corridors approaching capacity for which capital expansion will be required

Archived AVL/APC Data Stopped delay (d) Each point is from a particular bus on a given run (e.g. route, date, and time of day) Distance (X) Signalized intersection 1

d max Stopped delay (d) Can Compute: # obs Mean Stdev Percentiles X P1 X P2 Distance (X)

GIS Data Transit routes Signalized intersections

Stopped Delay (s) Stopped Delay (Seconds) Route 200 dn (University@Phillip to University@Albert) 50 45 40 35 30 25 20 15 10 5 Total stopped delay per service trip due to traffic signal 0 16:30 16:40 16:50 17:00 17:10 17:20 17:30 17:40 17:50 18:00 Time of Day 48 44 40 36 32 28 24 20 16 12 8 4 0 0% 20% 40% 60% Relative Frequency (%)

Relative Frequency (%) 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% The stopped delays experienced by the transit vehicles are essentially the same as those experienced by general purpose vehicles! 0 10 20 30 40 50 60 70 80 90 100 Stopped Delay per service trip (Seconds) TM # service trips % trips having to stop Mean of all trips (s) Mean of trips that stop (s) Std of all trips (s) COV 85 th %ile delay (s) Thru 658 44% 7.4 16.7 9.8 1.3 20

Ranking the Route Segments on the basis of signal delays 52: DUNDAS@Easton to HESPELER/WATER@Coronation/Dundas 52: KING@River to FAIRWAY@King 11: OTTAWA@Alpine to OTTAWA@Homer Watson 51: PINEBUSH@Walmart&Home Depot to HESPELER@Eagle And Pinebush 5: ERB@Beechwood And Gateview to FISCHER-HALLMAN@Thorndale 16: HOMER WATSON@Doon South Rd And Monarch Tr to HOMER 10: DOON VILLAGE@Pioneer to HOMER WATSON@ManitouAndDoon Village 201: FISCHER-HALLMAN@McGarry to FISCHER HALLMAN@Greenbrook And 10: WILSON@Kingsway to FAIRWAY@Wilson 8: Terminal to FAIRWAY@Fairview Park Mall 15: LACKNER@Keewatin to VICTORIA@Natchez 3: OTTAWA@Alpine to OTTAWA@Homer Watson 9: NORTHFIELD@Highpoint to NORTHFIELD@Skylark 53: FRANKLIN@Clyde to FRANKLIN@Savage 23: N/A to FAIRWAY@Fairview Park Mall 110: HOMER WATSON@Doon South Rd And Monarch Tr to HOMER 64: CONCESSION@Bishop to HESPELER@Dunbar 9: N/A to NORTHFIELD@Kraus 61: Terminal to HOMER WATSON@Conestoga College 21: KING@Conestoga Mall to KING@Northfield 0 10 20 30 40 50 60 Mean Stopped Delay (s)

Route + Section Name Mean Delay 90th Percentile Delay (s) Queue (m) Proportion of Trips Experiencing Signal Delay (%) 52: DUNDAS@Easton to HESPELER/WATER@Coronation/Dundas 50.2 102.0 127.0 85% 325 52: KING@River to FAIRWAY@King 41.0 83.9 106.0 81% 385 11: OTTAWA@Alpine to OTTAWA@Homer Watson 39.1 64.0 189.0 85% 268 51: PINEBUSH@Walmart&Home Depot to HESPELER@Eagle And Pinebush 39.0 101.0 102.0 58% 280 5: ERB@Beechwood And Gateview to FISCHER-HALLMAN@Thorndale 37.8 87.0 76.0 76% 258 16: HOMER WATSON@Doon South Rd And Monarch Tr to HOMER WATSON@Conestoga College 36.6 98.0 121.0 57% 134 10: DOON VILLAGE@Pioneer to HOMER WATSON@ManitouAndDoon Village 34.4 69.4 91.0 81% 201 201: FISCHER-HALLMAN@McGarry to FISCHER HALLMAN@Greenbrook And Hwy 7And8 WB Rmp 33.4 82.0 152.0 52% 445 10: WILSON@Kingsway to FAIRWAY@Wilson 32.1 64.7 120.0 71% 258 8: Terminal to FAIRWAY@Fairview Park Mall 32.1 74.1 162.0 84% 258 15: LACKNER@Keewatin to VICTORIA@Natchez 31.7 72.7 76.0 77% 234 3: OTTAWA@Alpine to OTTAWA@Homer Watson 31.3 81.0 154.0 60% 181 9: NORTHFIELD@Highpoint to NORTHFIELD@Skylark 31.1 65.0 76.0 69% 189 53: FRANKLIN@Clyde to FRANKLIN@Savage 31.1 71.0 76.0 79% 310 23: N/A to FAIRWAY@Fairview Park Mall 30.8 75.0 131.0 65% 255 110: HOMER WATSON@Doon South Rd And Monarch Tr to HOMER WATSON@Conestoga College 30.0 87.7 107.0 56% 129 64: CONCESSION@Bishop to HESPELER@Dunbar 29.7 69.9 91.0 69% 283 9: N/A to NORTHFIELD@Kraus 29.6 69.4 76.0 75% 190 61: Terminal to HOMER WATSON@Conestoga College 29.5 69.8 91.0 64% 220 21: KING@Conestoga Mall to KING@Northfield 28.7 64.0 119.0 57% 122 Num Obs

R52Up; DUNDAS@Easton to HESPELER/WATER@Coronation/Dundas

R11IB; OTTAWA@Alpine to OTTAWA@Homer Watson 189m

Key Points Need to get the pillars right! QA; MOP; Diagnosis; Push & Pull Automating Diagnosis is not trivial but big payback Leverage the Archived AVL/APC data Distribute cost of DSS over more entities in organization.

Thank you! Email: bhellinga@uwaterloo.ca Web: www.civil.uwaterloo.ca/bhellinga www.wpti.uwaterloo.ca

Photo Sources Slide 2: http://paraelink.org/bmk3k4/bmk3k4_3.htm Slide 3: GRT buses: www.therecord.com; GPS: www.gpscentral.ca; Passenger counter: www.parvus.com Slide 6: detour sign: http://embracingthedetour.com/detours-part-ii-day-52/; snow: www.guelphmercurary.com; Octoberfest parade: www.therecord.com; temporary bus stop: www.flickr.com

Moving from Data Warehousing to Decision Support. Dr. Bruce Hellinga Feb. 28, 2013