ADB Sri Lanka Resident Mission

Transcription

1 ADB Sri Lanka Resident Mission 1

2 Motivation Objective Theory of change Background Methodology Results Implications Way forward 2

3 ADB has funded road rehabilitation throughout the Asia Pacific region and in Sri Lanka. Yet Impact Evaluations are rare. 3

4 Objective of study: Estimate the impacts of road rehabilitation Objective of presentation: Background Methodological approach Limitations Similarities and differences between counterfactual and treatment groups 4

5 Primary school enrolment rates Secondary school enrolment rates Physical labour; financial resources Improved road conditions Improved subjective view of road quality Increased use of roads by private vehicles Increased use of roads by bicycles Increased use of roads by public vehicles (buses, trucks) Greater frequency of travel into town Reduction in transport costs (to access health services, markets, schools, transport produce, transport animals) Reduction in transport time (to access health services, markets, schools, transport produce, transport animals) Increase in household income Increase in expenditure on food Increase in number of household members employed in services Increase in number of household members employed in agriculture Increase in household assets Increase in trade Increase in salary of the household's highest income earner Increase in full-time work Increase in people undertaking employment Increase in women undertaking employment Increase in dwellings and possessions Improvements in well-being (including across women) 5

6 IE focus on a period 2 to 3 years after rehabilitation. Impacts across 6

7 Strong in-country project team support. In 2008, 78 roads were long-listed. 30 were short-listed. 7

8 In 2011, a baseline survey was undertaken. 185 HH near 8 project roads. 40 HH near 2 counterfactual roads. 8

9 Initial envisaged approach: A difference in difference IE with 2011 and 2014 data sets. Concerns: Sample size. Data sets missing. Rehabilitation of counterfactual roads. 9

10 During the 2014 fieldwork, it was revealed: 3 roads had not been rehabilitated and have no plans for rehabilitation. 8 roads had recently completed rehabilitation in had completed rehabilitation in roads completed rehabilitation in

11 11

12 What is the impact of short-term road rehabilitation? 12

13 What is the impact of short-term road rehabilitation? What is the impact of medium-term road rehabilitation? 13

14 Bias due to non-random program placement 14

15 Bias due to non-random program placement Was there a political reason why some roads were selected for rehabilitation prior to others? Do the communities surrounding the roads selected have the best chance to benefit? 15

16 Bias due to non-random program placement Was there a political reason why some roads were selected for rehabilitation prior to others? Do the communities surrounding the roads selected have the best chance to benefit? Moreover, due in part to the non-random program placement, are unobservable characteristics different between treatment and counterfactual groups? 16

17 Sample size of 3000 (2600 households and 400 enterprises) Initially considered a DID approach Separate into direct effects (households within 25 metres of the roads) and indirect effects (households in the villages surrounding the roads) 17

18 Undertake a listing of all households along the roads being surveyed (direct effect). Use 2012 census data to determine how many households were within villages surrounding roads being surveyed (indirect effect). 18

19 Survey 1300 households directly affected by roads and 1300 households indirectly affected by roads. Allocate sample size proportionally to the number of households. 19

20 To select HH surrounding roads, separate the households into blocks and randomly select a household in each block. To select HH in the villages, applied the random walk method. 20

21 Use ipad devices to collect data. Use GPS devices to record location. Consider applying cellular data to investigate changes over time. 21

22 Pilot completed in May The final survey completed under three weeks in June

23 Focus is on investigating similarities and differences between the communities surrounding treatment and counterfactual roads. Relative to counterfactual roads, on average, people surrounding treatment roads were more educated. have worked more months. worked more hours per month. more likely to live in permanent housing. travel a shorter distance and experience lower costs. use the road more. 23

24 Focus is on investigating similarities and differences between the communities surrounding treatment and counterfactual roads. Relative to counterfactual roads, on average, people surrounding treatment roads were: More educated. Have worked more months. Worked more hours per month. More likely to live in permanent housing. Travel a shorter distance to reach services and experience lower costs when reaching services. Use the road more. 24

25 Clearly people in households surrounding counterfactual roads appear worse off than those surrounding treatment roads. FE can be eliminated with at DID but will the parallel trends assumption hold? 25

26 Primary Outcome T1 C1 T2 C2 Time 1 After Intervention

27 Primary Outcome T1 C1 T2 C2 Time 1 After Intervention

28 Primary Outcome T2 T1 C1 C2 Time 1 After Intervention

29 Apply matching techniques to match households in counterfactual villages with similar households in treatment villages. But would that solve the problem? 29

30 Primary Outcome T2 T1=C1 C2 Time

31 Primary Outcome T2 T1 C1 C2 Time

32 Primary Outcome T2 T1 C1 C2 Time 1 After Intervention

33 Counterfactual roads may be rehabilitated if funds become available. Endline survey should be undertaken at least 2, if not 3, years from now. Spillover effects? 33

34 34