The Wrld Bank Human Develpment Netwrk Spanish Impact Evaluatin Fund www.wrldbank.rg/hdchiefecnmist

Evaluating Impact: Turning Prmises int Evidence Enhanced Vcatinal Educatin and Training (EVENT) Prject Dhan S. Dhami, Mahesh Dahal, Mariam Adil, Ram Kaji Khanal, Saurav Dev Bhatta, Subash Shrestha, Tirtha Khaniya Dhaka, Bangladesh Octber 9-13, 2011

1. Backgrund Technical and vcatinal educatin landscape in Nepal Skills/TVET -limited training pprtunities Wrkfrce -high LFPR -limited edu/training >300000 entrants/yr Jbs -Nepal -Abrad (> 2 mil)

2. Descriptin f Interventin Prject develpment bjective: Expand supply f skilled and emplyable labr by increasing access t quality training prgrams, and strengthening the technical and vcatinal educatin and training system in Nepal Key prject data: Prject Implementatin Perid: 2011-2015 Prject cverage: Natinal Implementing Agency: ME, GN Target beneficiaries: 16-35 year ld yuth Special fcus n: lagging regins, disadvantaged ppulatin grups

Prject Cmpnents C1. Strengthening TEVT regulatry framewrk and capacity building C2. Strengthening technical educatin (TSLC/Diplma) C4. Prj. management and M & E C3. Supprting shrt-term training 5

C3: Supprting shrt-term training and recgnitin f prir learning Shrt-term vcatinal training Recgniti n f prir learning Vucherbased financing Resultsbased financing 6

Shrt-term vcatinal training Fcus: shrt-term training in specific trades Scale: 45,000 beneficiaries trained Beneficiaries: the pr, and yuth frm lagging regins and disadvantaged sci-ecnmic grups Training prviders: private (primarily) and public institutins Emplyment fcused: payments t prviders linked t jb placement f trainees

3. Results Chain INPUTS ACTIVITIES OUTPUTS OUTCOMES LONGER-TERM OUTCOMES HIGHER ORDER GOALS Financial resurces Human resurce ( e.g., Prject Secretariat) Organizatin al capacity Training management infrmatin system (TMIS) Existing training curriculum Training infrastructure Identificatin f training areas Curriculum design, develpment and disseminatin Identificatin f selectin criteria fr training institutins and trainees Disseminatin f infrmatin t training prviders and ptential trainees Selectin f training prviders (public and private) Selectin f ptential trainees Training Mnitring Skills testing, and certificatin f trainees and prir learners Facilitating jb placement Training areas identified Curricula develped /revised in specific traits Infrmatin disseminated thrugh varius media (e.g. Radi/TV annuncements, brchures) Institutins selected Candidates selected Trainees infrmatin registered in TMIS Beneficiaries trained/ training cmpleted sft-skills training cmpleted fr selected curses Beneficiaries tested and certified Beneficiaries paced in jbs Reprts Increased emplyment Increased incme Enhancement f nncgnitive skills Enhanced access t training fr lagging regins and marginalized ppulatin grups including wmen Enhanced quality f life Enhanced recgnitin f skills test certificates (NSTB) Peace and stability Reduced deviant behavir amng yuth Increased ecnmic grwth

4. Primary Research Questins What is the impact f shrt term training n the emplyability, incme and skill levels f trainees, especially fr lagging regins, wmen and marginalized grups? What is the impact f sft-skills training n emplyability? What is the impact f certificatin f prir learning n emplyment? What is the impact f infrmatin campaign n access t and uptake f training prgrams? What is the impact f the vucher prgram n effectiveness and efficiency f training prviders and n the satisfactin f trainees?

5. Outcme Indicatrs What is the impact f shrt term training n emplyability, incme and skill levels f trainees? % f trainees emplyed within 3 t 6 mnths f cmpletin f training %f emplyed trainees wh cntinue t be emplyed after 9 mnths Average mnthly earnings f trainees after 3 t 6 mnths f cmpleting training % f trained beneficiaries wh are cnsidered well trained by emplyers What is the impact f sft-skills training sub-cmpnent n Emplyability? % f trained beneficiaries wh perfrm adequately in sft-skills test cnducted by evaluatrs % f trained beneficiaries wh emplyers cnsider adequate in terms f sft-skills

5. Outcme Indicatrs What is the impact f shrt term training n emplyability, incme and skill levels f trainees? % f trainees emplyed within 3 t 6 mnths f cmpletin f training %f emplyed trainees wh cntinue t be emplyed after 9 mnths Average mnthly earnings f trainees after 3 t 6 mnths f cmpleting training % f trained beneficiaries wh are cnsidered well trained by emplyers What is the impact f sft-skills training sub-cmpnent n Emplyability? % f trained beneficiaries wh perfrm adequately in sft-skills test cnducted by evaluatrs % f trained beneficiaries wh emplyers cnsider adequate in terms f sft-skills What is the impact f certificatin f prir learning n emplyment and incme? % f certified wrkers emplyed within 3 t 6 mnths f cmpletin certificatin? Average mnthly earnings f certified wrkers after 3 t 6 mnths f cmpleting training What is the impact f infrmatin campaign n access t training prgrams? % f training applicants frm lagging regins and marginalized grups % f trainees frm lagging regins and marginalized grups

6. Identificatin Strategy/ Methd Key interests: Impact f training as a whle Impact f including sft skills in traning curriculum Implicatins fr design: tw levels f assignment Assignment f ptential trainees Assignment f sft skills curriculum N sft skills Sft skills added N training Training Grp I Grp II ----- Grp III

6. Identificatin Strategy/ Methd Assignment f trainees: Randm assignment (ptin 1) Eligibility criteria used t screen applicants Fr each training event, training prvider ensures there are at least 50% mre eligible applicants than places Lttery system used t select beneficiaries Randmized experimental design Assignment f trainees: Scring system(ptin 2) Eligibility criteria used t screen applicants Fr each training event, training prvider ensures there are at least 50% mre eligible applicants than places Selectin criteria used t cmpute applicant scre Training prvider uses a cutff scre t assign applicant t training prgram(treatment) r reject applicant (cntrl) RD design

6. Identificatin Strategy/ Methd Assignment f sft skills curriculum: randm Fcus n sme specific trades where sft skills might be mst relevant Within these trades Curriculum f sme training events include sft skills Other training events d nt Beneficiaries nt aware which training events include sft skills and which d nt

6. Identificatin Strategy/ Methd Analysis: Impact f training: utcme f Grp (II+III)-Grp I Impact f sft skills: utcme f Grp III-Grp II Impact f training w/ sft skills: Grp II Grp I Impact f training w/ sft skills: Grp III Grp I N sft skills Sft skills added N training Training Grp I Grp II ----- Grp III

7. Sample and Data Tw dimensins (beneficiaries and training events) Sampling f individual beneficiaries If RD design used, frm each training event, take XX% f applicants frm each side f the cutff scre If randmized experimental design used, take randm sample frm each grup Sampling f institutins/training events fr sft skills Pwer cmputatins will be made later t determine required sample size f training events Randm assignment f sft skills t these events grup II training+ n sft skills; grup III training+sft skills Baseline and endline data cllected n individuals as well as training events

8. Time Frame / Wrk Plan First rund analysis starting December 2012 Fllw-up survey f trainees Octber 2012 Cllectin f wrker data frm training prviders after jb placement Jb placement f trainees (by July 2012) Baseline survey f next rund f training Feb2012 Training Nvember 2012-January 2012 Baseline survey f eligible trainees Nvember 2012 Preparatin f list f eligible trainees Nvember 2012 Applicatins cllected frm ptential trainees Nv 2012 Cmpletin f survey instruments fr baseline-sept 2012 Randm assignment f institutins/trades fr sft skills training Sept 2012 Selectin f training institutins Aug 2012

9. Surces f Financing Sme funds frm prject? Trust Funds Spanish Impact Evaluatin Fund????