NACTO Designing Cities Conference Project Evaluation: Tools for Measuring Success and Building Support October 29, 2015
The case for evaluation: Have a social contract with City Council, staff and community if it is a pilot, we need to collect & make public the data/evaluation and honour the outcomes. Opportunities for course correction/modification. Quantitative data can be limited in utility (esp. for pedestrian projects) may need wide-ranging efforts. Evaluation frameworks must be built into planning and implementation. Translate evaluation into criteria, guidelines, standards. Makes the case for larger application especially with $ impacts. Provides staff with opportunities to disseminate results at workshops, conferences, papers = staff development
Case Study #1: Pedestrian-only zones at two downtown universities 12 month pilot, extended for 6 months 3
The success of a pedestrian zone will depend in part on the goals of the pedestrian zone and the involved partners. Improved Pedestrian Experience o New public space/destination created o Increased pedestrian safety o Increased pedestrian usage and activity o Improved quality of experience for pedestrians Economic, Social and Community o Potential increased retail, tourism and recreational activity o Increased branding and recognition for local businesses, the neighbourhood and local partners (e.g. universities) o Increased social engagement/sense of community o Partnerships to lever resources and funding for maintenance, programming and streetscape improvements Transportation and Access o Maintaining or improving travel options for all modes in the vicinity o Continued parking options, including passenger drop-offs o Low impact on traffic circulation and access for deliveries, employees, and/or residents by vehicle, and improvements to flow and access by pedestrians, transit users, and cyclists o Fewer traffic conflicts among pedestrians and vehicles 4
Performance Tracking Devonshire before/after and after Actions: Request the partners/key stakeholders to keep track of the name, type and frequency of the events being held in the pedestrian zone. Request the partners/key stakeholders track and quantify the amount of monetary contributions and types of items invested in the pedestrian zone project. 5
Traffic Counts and Collision Data Actions: Contact City for traffic data in the proposed pedestrian zone. Collision data requests should outline specific intersections or specific sections of the road. Include nearby intersections in order to assess potential impacts to area traffic flow. Before, during and after data is best. In the absence of adequate and recent "before" data new traffic counts need to be conducted before implementing the road closure - $ implications. Willcocks St Pedestrian zone Pedestrian Traffic 6 Vehicular Traffic. St. George St.
Feedback from Key Stakeholders Successful pedestrian zones are based on strong and positive partnerships. Actions: Schedule one-on-one consultations with key stakeholders or hold a group meeting to do a presentation on the vision, goals and initial results of the project to get feedback. Document as part of the evaluation. 7
Surveys/interviews The most efficient approach is to use an online survey to collect data on the users' experiences. Surveys can gather information on the profile of the people living, working and visiting in the area, how the project has affected their activities/behaviour, the degree of their satisfaction/dissatisfaction with the project, and what they think could be improved/changed about the project. On-site surveys/interviews administered on-site by volunteers require more resources (e.g., volunteer recruitment, paper surveys, and labour to enter and analyze data). 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Walking through Type and frequency of use - UofT student and staff Biking through Attending university organized events Drinking coffee, eating and hanging out with friends Everyday A few times a week Once or twice a month A few times a year Never Reading / studying Physical Other (e.g. activities and sun bathing, sports jamming, napping ) 8
9
Visuals and On-site Information Actions: Take "before" and "after" photos. Include these in the presentations to various stakeholders as well as the evaluation and report-back. 10
Case Study #2: Scramble Intersections at 3 downtown locations 11
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Type 1 Type 2 Type 3 12
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Key Evaluation Components Traffic volumes of all modes User satisfaction Pedestrian safety and crowding Pedestrian convenience Intersection level of safety Vehicular travel times/delay Transit delays 13
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Data Inputs Auto, pedestrian and transit rider volumes; Pedestrian crossing patterns; Pedestrian violations; Illegal parking; Transit delay; User surveys; Collisions statistics; Filed complaints; Field observations; 14
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Trip Purpose of Users 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Bay & Bloor Yonge & Bloor Yonge & Dundas Other Purpose Recreation Shopping Work / School 15
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Pedestrian Corner Crowding 16
Corner Crowding - Intersection Average Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Changes in Corner Crowding 40 35 30 25 20 Conventional Signal PPP Signal 15 10 5 0 Yonge & Dundas Yonge & Bloor Bay & Bloor Yonge & Dundas Yonge & Bloor Bay & Bloor Mid-Day Peak PM Peak 17
Percentage of Crossings made Diagonally Background User Surveys Pedestrian Vehicular Transit Economic Recommendations 25% 23% Diagonal Crossings 20% 19% 19% 15% 15% 16% 12% %* = 10% 5% 0% Weekday Weekend Weekday Weekend Weekday Weekend Yonge and Dundas Yonge and Bloor Bay and Bloor * Denominator does not include pedestrian crossings during nonscramble phases 18
Aggregate Time Savings (+) or Loss (-) [Hours] Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Aggregate Time Savings 40 20 0 Yonge and Dundas Yonge and Bloor Bay and Bloor 26 24 13 16-20 -40-30 -11-60 -80-100 -64-68 -120-140 -160-145 -128 19
Agree that PPP is a good idea Agree that PPP has had a positive impact on pedestrian safety Agree that PPP has had a positive impact on pedestrian convenience Agree that PPP has had a negative impacts on traffic flow Agree that PPP is a good idea Agree that PPP has had a positive impact on pedestrian safety Agree that PPP has had a positive impact on pedestrian convenience Agree that PPP has had a negative impacts on traffic flow Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Direction Eastbound Westbound Time Period Without PPP (August 2008) AM Peak 00:31 OFF Peak 00:59 PM Peak 00:39 Whole Day 00:43 AM Peak 00:20 OFF Peak 00:33 PM Peak 00:21 Whole Day 00:25 Average Intersection Delay [mm:ss] (Change in Delay [mm:ss]) With PPP - status quo - (November 2011) 00:50 (+00:19) 03:18 (+02:19) 03:17 (+02:38) 02:28 (+01:45) 01:19 (+00:59) 01:41 (+01:08) 01:08 (+00:47) 01:22 (+00:57) With PPP - 'no parking' regulations enforced - (estimated) 00:12 (-00:19) N/A 00:30 (-00:09) N/A 00:11 (-00:09) N/A 00:16 (-00:05) N/A 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Transit rider intercepted at PPP intersection Transit rider intercepted at a transit stop up stream Yonge & Dundas Bay & Bloor 20
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations $573M in annual spending demand for office workers and students in the downtown area $14B in annual tourism spending Eaton Centre has 50 million visitors annually and 285retailers 21
Summary of Performance Yonge & Dundas Yonge & Bloor Bay & Bloor Alleviation of Corner Crowding Aggregate Pedestrian Time Saving User Support Utilization of Diagonal Crossing Aggregate Motorized Vehicle Delay x X X Collisions X X X GHG emissions X X X Fuel Consumption X X X 22
Background User Surveys Pedestrian Vehicular Transit Economic Recommendations Recommendations: Continue at Yonge & Dundas and Bloor & Yonge De-commission at Bloor & Bay Focus on corridor management of traffic disruption issues such as illegal parking Update the criteria for future use. 23
Case Study #3: Pedestrian Wayfinding Pilot 24
Evaluation Objectives Evaluate the project against its stated objectives and expected benefits Demonstrate value for investment Make the case for citywide project rollout 25
Evaluation Objectives Outline Business Case Pilot Project Evaluation Full Business Case Outline Business Case (OBC) Considers a wide range of project costs and benefits Uses Multiple Account Evaluation (MAE) Framework MAE framework adopted by Metrolinx for their projects Pilot Project Evaluation Will increase robustness of initial OBC analysis Will follow MAE framework with project-specific adjustments Full Business Case Development of a full business case Will be considered by City Council Will support discussions with potential project funders 26
Evaluation Framework MAE Evaluation Categories Transportation Environment Urban design Social and community Economic development Financial Additional City Evaluation Themes User perception/satisfaction Identification & connection of places Exploration and discovery Confidence and trust to walk 27
Methodology We Will Use the Following Tools Intercept Surveys Mystery Shoppers Accompanied Walks Online Surveys Pedestrian Counts (wayfinding elements) We Will Measure... Awareness (e.g.. on-street awareness) Change in Attitudes (e.g.. improved user perception) Change in Behavior (e.g.. increase in walking) Change in Outcomes (e.g. health benefits) INTERCEPT SURVEYS Measure attitudes & behaviour regarding walking Measure confidence to walk from a fixed point to key destinations Survey points will include locations where transitions are difficult Representative sample of passing pedestrians at pre-selected sampling points MYSTERY SHOPPER SURVEYS Unaccompanied walks between specified points and record ease of navigation and rate quality of wayfinding information. Includes post walk Focus Group ACCOMPANIED WALKS Same as mystery walker but accompanied Measures how elements are actually used PEDESTRIAN COUNTS Measure wayfinding element usage Limited applicability for TO360 due to external factors IPSOS REID'S ONLINE RESPONDERS SURVEY Can be filtered to isolate target groups 28
Methodology Next Steps Complete measurements against baseline data ( Fall 2015) Finish analysis and share with project partners ( Nov-Dec. 2015) Finalize outstanding issues ( value engineering of elements, mapping guidelines, OBC) Prepare report to Council with recommendations for City-wide roll-out based on evaluation results ( Spring 2016) 29
In conclusion: The metric for successful streets projects is people. Partners can play a critical role. Evaluation lessons lead to good policy. Partnership Program StART - Spectrum Arts, Jesse Harris 30