Respondent Incentves n a Mult-Mode Panel Survey: Cumulatve Effects on Nonresponse and Bas Annette Jäckle and Peter Lynn 1 ISER Workng Paper 2007-01 1The vews epressed are those of the authors and not necessarly those of the Department for Educaton and Sklls.
Acknowledgement: We would lke to tnk Ian Noble at the Department for Educaton and Sklls for facltatng the data and commentng on an earler verson of the paper, Tm Tr and Rory Ftzgerald for patent and helpful response to data queres, Noah Uhrg, Heather Laure for comments and Mark Bryan for advce on the calculaton of nteractons n nonlnear models. Readers wshng to cte ths document are asked to use the followng form of words: Jäckle, Annette and Lynn, Peter (January 2007) Respondent Incentves n a Mult-Mode Panel Survey: Cumulatve Effects on Nonresponse and Bas, ISER Workng Paper 2007-01. Colchester: Unversty of Esse. The on-lne verson of ths workng paper can be found at http://www.ser.esse.ac.uk/pubs/workpaps/ The Insttute for Socal and Economc Research (ISER) specalses n the producton and analyss of longtudnal data. ISER ncorporates MISOC (the ESRC Research Centre on Mcro-socal Cnge), an nternatonal centre for research nto the lfecourse, and ULSC (the ESRC UK Longtudnal Studes Centre), a natonal resource centre to promote longtudnal surveys and longtudnal research. The support of both the Economc and Socal Research Councl (ESRC) and the Unversty of Esse s gratefully acknowledged. The work reported n ths paper s part of the scentfc programme of the Insttute for Socal and Economc Research. Insttute for Socal and Economc Research, Unversty of Esse, Wvenhoe Park, Colchester. Esse CO4 3SQ UK Telephone: +44 (0) 1206 872957 Fa: +44 (0) 1206 873151 E-mal: ser@esse.ac.uk Webste: http://www.ser.esse.ac.uk January 2007 All rghts reserved. No part of ths publcaton may be reproduced, stored n a retreval system or transmtted, n any form, or by any means, mecncal, photocopyng, recordng or otherwse, wthout the pror permsson of the Communcatons Manager, Insttute for Socal and Economc Research.
ABSTRACT Respondent ncentves are ncreasngly used as a measure of combatng fallng response rates and resultng rsks of nonresponse bas. Nonresponse n panel surveys s partcularly problematc, snce even low wave-on-wave nonresponse rates can lead to substantal cumulatve losses, and f nonresponse s dfferental, may lead to ncreasng bas across waves. Although the effects of ncentves ve been studed etensvely n cross-sectonal contets, lttle s known about cumulatve effects across waves of a panel. We provde new evdence about the effects of contnued ncentve payments on attrton, bas and tem nonresponse, usng data from a large scale, mult wave, med mode ncentve eperment on a UK government panel survey of young people. In ths study, ncentves sgnfcantly reduced attrton, far outweghng negatve effects on tem nonresponse n terms of the amount of nformaton collected by the survey per ssued case. Incentves d proportonate effects across a range of respondent cracterstcs and as a result dd not reduce attrton bas. The effects of ncentves were larger for uncondtonal tn condtonal ncentves and larger n postal tn telephone mode. Across waves, the effects on attrton decreased somewt, although the effects on tem nonresponse and the lack of effect on bas remaned constant. The effects of ncentves at later waves appeared to be ndependent of ncentve treatments and mode of data collecton at earler waves. Keywords: attrton, tem nonresponse, mal survey, CATI, uncondtonal ncentve, condtonal ncentve
1 Introducton Surveys around the world are facng declnng response rates, and wth ths, ncreasng rsks of nonresponse bas f nonrespondents systematcally dffer from respondents. For panel surveys ths s partcularly problematc, snce even low nonresponse rates at each wave can lead to large cumulatve losses. If nonresponse s dfferental, bas could ncrease wth the duraton of the panel. In order to boost partcpaton, survey organsatons ncreasngly offer respondent ncentves. Ths paper provdes new evdence on the cumulatve effects of ncentves on attrton, attrton bas and tem nonresponse, usng data from a large scale, mult wave, med mode ncentve eperment on a UK government panel survey of young people. The effects of ncentves ve been studed n many settngs: monetary ncentves ncrease response more tn gfts or lotteres (Church 1993; Snger et al. 1999); uncondtonal ncentves (gven at the tme of the survey request) ncrease response more tn condtonal ncentves (promsed n return for partcpaton) (Church 1993; Goyder 1994; Hopkns and Gullckson 1992; Snger et al. 1999); response ncreases wth the value of the ncentve (Armstrong 1975; Church 1993; Fo, Crask, and Km 1988; Hopkns and Gullckson 1992; Rodgers 2002; Yu and Cooper 1983); ncentves ve larger effects n studes wth low response rates and larger effects n postal tn ntervewer admnstered surveys (Snger et al. 1999). Most evdence of the effect of ncentves n dfferent modes, however, stems from comparsons of separate studes and fals to control for dfferences n other measures affectng response. As a result, dfferences n the effects of ncentves are not necessarly genune mode effects. The study by Ryu, Couper and Marans (2006) s an ecepton. The authors compared the effects of monetary ncentves and gfts n a med mode postal and face-to-face survey. Ther study dd not, however, nclude a no-ncentve condton and so dd not allow an evaluaton of the magntude of ncentve effects across modes. We compared the effects of ncentves n a med postal and computer asssted telephone ntervewng (CATI) survey and, n postal mode, also eamned the effects of condtonal and uncondtonal ncentves. Research on the effects of ncentves s focused on response rates and lttle s known about the effects on bas, the ultmate reason for concern about low response. Incentve studes are mostly lmted to studyng effects on bas n sample composton and some studes ve found tt ncentves do dsproportonately ncrease partcpaton of respondents typcally under-represented, for eample, those wth low educaton (Snger, Van Hoewyk, 1
and Maher 2000), poor (James 1997), black or poor (Mack et al. 1998), of black or Indan mnorty ethnc groups, lvng n larger households or households wth dependent chldren, aged 0-20, or sngle (Stratford, Smmonds, and Ncolaas 2003). Bases n sample composton are however not necessarly correlated wth bases n mportant survey estmates, especally snce typcally only a ndful of compostonal cracterstcs are studed. Ultmately, studes of nonresponse bas are lmted by the lack of nformaton about nonrespondents, a lmtaton, whch can be overcome to some etent by panel studes, where nformaton about nonrespondents s avalable from waves pror to the dropout. We estmated the etent of bas due to attrton n terms of sample composton and survey varables and the effectveness of ncentves at reducng bas, eplotng the nformaton on attrters avalable from the panel. Addtonally, lttle s known about the effects of ncentves over waves of a panel survey. In a revew of the use of ncentves n longtudnal studes, Laure and Lynn (forthcomng) concluded tt, gven the cost mplcatons of cngng ncentve condtons, there was surprsngly lttle evdence about the longer term effects of such cnges to gude survey practtoners. Quotng an nternal memorandum of the US Census Bureau, Ward et al. wrote tt a revew of the well-known longtudnal studes (Downs 1999) found tt all non- Census Bureau studes used a monetary ncentve durng each wave, but there d been no scentfc tests to determne the effectveness of the ncentves (Ward et al. 2001:2). If attrton leads to drop-out of the least co-operatve, the sample mght ncreasngly be composed of commtted respondents who are less responsve to ncentves, because they are suffcently motvated to partcpate even wthout the ncentve (Laure and Lynn forthcomng). In ths case, ncentves may ve decreasng margnal effects on response rates over the lfe of the panel. By the same token, ncentves may ve ncreasng effects on attrton bas, f they ve dsproportonate effect on sample members who would otherwse be more lkely to drop out. Although some ncentve studes ve been carred out n the contet of panels, they mostly only covered one wave, or eamned the effect of cnges n ncentve treatment from one wave to the net. Martn, Abreu and Wnters (2001) and Ward et al. (2001), for eample, studed the effects of ncentves on converson rates of prevous wave nonrespondents; Rodgers (2002) and Laure (2006) eamned the effects of cnges n ncentve values n a panel. The only studes whch eamned cumulatve effects of ncentves over more tn two waves appear to be those by James (1997), Mack et al. (1998) and Laure and Lynn (forthcomng), who reported tt the postve effect of an ncentve pad early n a panel perssted for several waves even wthout repeated ncentve payment. These studes, however, only eamned the effect of an ncentve pad n a sngle wave and dd not eamne 2
the cumulatve effects of ncentves offered over successve waves. We eamned the cumulatve effects of contnued ncentve payments across three waves spannng a tme frame of three years, as well as the effects of cnges from telephone to postal mode and from condtonal to uncondtonal ncentve treatment. Fnally, there s conflctng evdence about the effects of ncentves on data qualty. Although concern s frequently voced tt ncentves may lead to lower data qualty, by margnally motvatng respondents who would otherwse ve dropped out of the study but are not suffcently motvated to respond dlgently, estng studes ve ether found tt ncentves lead to mproved respondent effort and less tem nonresponse (James and Bolsten 1990; Mack et al. 1998; Snger et al. 2000), or ve found no relatonshp (Berk et al. 1987; Davern et al. 2003; Goyder 1994; Shettle and Mooney 1999; Snger et al. 1999; Tesl, Roe, and Vayda 2005; Tzamouran and Lynn 1999; Wllmack et al. 1995). Item nonresponse s potentally crtcal, because analysts typcally only use cases wth complete data. Ths leads to losses n effcency due to reductons n sample szes, and smlar to unt nonresponse, can lead to based estmates and nvald nference f tem nonrespondents are not a random subset of the sample (Mason, Lesser, and Traugott 2002). Problems of tem nonresponse ncrease for multvarate analyss, f the patterns of mssngness vary across tems, and for analyss of cnge, whch n addton depends on complete nformaton at dfferent ponts n tme. Snce ncentves may affect both unt and tem nonresponse, t s then not clear wt ther net effect may be on repeated measures derved from a panel study. We eamned the effect of ncentves on tem nonresponse rates and calculated ther net effect on attrton and tem response. 2 Hypotheses tested The outcomes measured for ths analyss were the attrton rate, tem NR rate and attrton bas. Attrton was an absorbng state, snce the survey dd not re-ssue nonrespondents at later waves. Item NR was measured as the number of non-branched tems mssng, ether due to refusals or don t know answers. Attrton bas was measured n terms of socodemographc cracterstcs and wave 1 survey measures. These three measures were used to test the followng: H1: Effects of ncentves on attrton, tem NR and attrton bas. In prevous studes ncentves ve generally ncreased response rates, be t because norms of socal ecnge oblge the respondent to return a favour (norm of recprocty, Gouldner 1960) or because the ncentve substtutes for a lack of motvaton to 3
partcpate for other reasons, such as cvc duty or topc nterest (leverage-salence theory, Groves, Snger, and Cornng 2000). Incentves may n addton motvate respondents to provde better qualty responses, reducng tem NR. At the same tme ncentves may cnge the sample composton to nclude more respondents who are not dlgent about answerng the survey questons, and as a result ncrease tem NR. Fnally, ncentves may ve dfferental effects on attrton across sample members. Those wth a hgh propensty to partcpate n the survey wthout the ncentve may be less susceptble to ncentves, whle those more lkely to drop out of the survey may be more susceptble. As a result, ncentves may reduce attrton bas. Null hypothess H1: Incentves ve no effect on attrton, tem NR or attrton bas. H2: Effects of ncentves across waves. Incentves may ve decreasng effect on unt response across waves, f attrton leads to dropout of the least motvated sample members and the remanng members are suffcently motvated to partcpate for other reasons and hence less susceptble to ncentves (Laure and Lynn forthcomng). At the same tme, ncentves may ve ncreasng effect on attrton bas, f bas ncreases across waves and ncentves dsproportonately retan those n the sample who are most lkely to drop out. Null hypothess H2: The effects of ncentves do not cnge across waves. H3: Effects of uncondtonal and condtonal ncentves n a panel contet. Prevous studes, on cross-sectonal surveys, suggest tt uncondtonal ncentves ve larger effects on unt NR, possbly because the pre-payment sgnals tt the survey organsaton trusts the sample member wll partcpate, renforcng the norm of recprocty. Whether the dfferent ncentve condtons ve dfferent effects on tem NR s not clear. Null hypothess H3: Uncondtonal and condtonal ncentves ve comparable effects n a panel contet. H4: Effects of ncentves n postal and telephone mode. Comparsons of prevous studes suggest tt ncentves ve a larger effect n postal mode, possbly because n telephone mode the ntervewer already functons as an eternal motvator to ncrease both unt and tem response (Snger et al. 1999) and the scope for addtonal mprovements s smaller. Null hypothess H4: Incentves ve comparable effects n postal and telephone mode. H5: Effects of cnges over waves n mode or ncentve treatment. 4
Compared to sample members allocated to the same mode and treatment across waves, those who were allocated to dfferent treatments or dfferent modes may dffer n ther eperences of prevous survey waves and ther epectatons about future waves. As a result, the effect of ncentves may not only be condtonal on the treatment and mode at the current wave, but may be nfluenced by the treatment n prevous waves. Null hypothess H5: Cnges n mode or ncentve treatment over waves do not ve lastng effects. H6: Effects of ncentves across ablty levels. Sample members wth low educaton levels are typcally more lkely to drop out of surveys. If ncentves reduce attrton bas, they should therefore dsproportonately reduce attrton among lower achevers. Low ablty respondents may at the same tme be more lkely to provde ncomplete responses, f they fnd the task of completng the postal questonnare more dffcult. Ths may ncrease tem NR dsproportonately among ths group. Null hypothess H6: Incentves ve comparable effects across ablty levels. 3 Study Desgn The Youth Cohort Study of England and Wales (YCS) nvestgates transtons from compulsory educaton to further or hgher educaton or the labour market and typcally samples cohorts of 16 to 17 year-olds every two years, who are surveyed on several occasons at annual ntervals. The ncentves eperment was embedded n waves 2, 3 and 4 of cohort 10. The survey s managed and funded by the Department for Educaton and Sklls, who jontly desgned the ncentve eperment wth the Natonal Centre for Socal Research, the survey contractors for waves 2 and 3 of YCS Cohort 10. 3.1 The Survey The populaton studed n the YCS cohort 10 conssted of pupls n England and Wales who d reached mnmum school leavng age of 16 n the 1998/1999 school year (Russell and Phelps 2001), tt s, a one year age cohort of pupls born between 1-9-1982 and 31-8-1983. A 10% random sample was drawn from the regsters of schools (ecludng specal schools and schools wth fewer tn 20 pupls of tt age) n 1999, by askng schools to provde the names and addresses of pupls born on the 5th, 15th and 25th of every month. From the resultng fle of 31,424 names and addresses a systematc random sample of 25,000 pupls was drawn. The frst wave of the survey took place a year later n sprng 2000, the second at 5
the end of 2000, the thrd n sprng 2002 and the fourth n sprng 2003. Nonrespondents were not ssued n subsequent waves and as a result, attrton was monotonc. Wave 1 was a postal survey wth telephone follow-up of nonrespondents after 4 malngs (ntal questonnare malng and three remnders). Based on reported eamnaton results, wave 1 respondents were classfed as ether hgher achevers (f they d obtaned 5 GCSEs at grades A* to C 1 ) or lower achevers otherwse. At wave 2 roughly one thrd of ssued sample members were randomly selected for addtonal questons on partcular topcs and assgned to computer asssted telephone ntervewng (CATI). In addton to the core questonnare, telephone respondents were admnstered a module on decsons about enterng hgher educaton (for hgher achevers) or on educatonal and employment aspratons (for lower achevers). The remanng sample members were admnstered the core questonnares by post. At wave 3 all lower achevers receved the core mal questonnare, although the telephone module was stll carred for hgher achevers. At wave 4 all respondents were assgned to the core postal survey. Fgure 1 llustrates the allocaton to modes and ncentves. < Fgure 1 > The core questonnare was shorter at wave 2 tn wave 1, but remaned manly uncnged for the three epermental waves. Telephone respondents were asked the core questons before the addtonal modules. The core questonnare was the same as the postal questonnare, although some tems were adapted for admnstraton over the telephone. 2 The average telephone ntervew took around 20 mnutes. 3.2 The Incentves Eperment Facng growng concerns over declnng response rates, an eperment was ntroduced n the second wave of cohort 10, to study the effect of ncentve payments on response rates and nonresponse bas. A proporton of wave 1 respondents on both the postal and telephone 1 The General Certfcate of Secondary Educaton (GCSE) s by far the most common type of qualfcaton ganed by pupls aged 14-16 n England and Wales. GCSEs can be taken n a wde varety of subjects. Grades awarded are A*, A, B, C, D, E, F, G and unclassfed. Obtanng fve or more GCSEs at grades A* to C s referred to as achevng level 2 at GCSE and s often an entry requrement for A level courses, the most common type of qualfcaton sought by pupls aged 16-18. The proporton of pupls achevng level 2 s publshed annually for schools and for educaton authortes by the Department for Educaton and Sklls as one of several key performance ndcators. 2 The questonnares and techncal reports are avalable va the UK Data Archve n the appendces of the YCS User Gude at http://www.data-archve.ac.uk/fndngdata/sndescrpton.asp?sn=4571&key=ycs. 6
surveys were sent a 5 voucher, whle the control groups receved no ncentve. Addtonally, n the postal survey the ncentves were ether uncondtonal (the ncentve was sent wth the ntal malng) or condtonal (the voucher was promsed n the orgnal malng, but only sent on recept of a completed questonnare). At waves 3 and 4, all ncentves were pad uncondtonally. 3.3 Allocaton of Respondents to Modes and Incentve Treatments At wave 2, wave 1 respondents were randomly assgned to ether telephone or postal mode. 3 The allocaton of ncentve treatments was however done at the school level (randomsed cluster assgnment by mode). Each school represented n the sample was allocated to one telephone treatment (control or ncentve) and ndependently allocated to one, potentally dfferent, postal treatment (control, uncondtonal ncentve or condtonal ncentve), so tt all sample members from the same school approached n the same mode receved the same ncentve treatment. The 4712 wave 1 lower achever respondents were stratfed by seral number wthn school wthn Government Offce Regon and alternately allocated to telephone and postal treatments. We ve ecluded from the analyss 627 cases for whch there was no vald telephone number on fle, as those amongst ths group who d been allocated to telephone mode were approached by post. Consequently, analyss of lower achevers s restrcted to 2097 approached by telephone and 1988 approached by post. A smlar procedure was carred out for hgher achevers, ecept tt a larger proporton was allocated to postal treatment. There were 8909 wave 1 hgher achever respondents of whch 751 d no vald telephone number and are ecluded from the analyss. After these eclusons there are 2922 hgher achevers allocated to telephone mode and 5236 allocated to postal mode. For the allocaton of schools to ncentve treatment groups, the schools contanng telephone sample members were stratfed accordng to the rato of lower to hgher achevers n the sample and randomly assgned to ncentve treatments wthn strata. 4 The procedure 3 Ths ecluded 64 wave 1 respondents who were used for the wave 2 plot and not ssued at any of the epermental waves. 4 In the telephone strata the proporton of schools assgned to ncentves was 1/2 f the rato of lower to hgher achevers n the sample was 2; 1/3 for 0.5 rato < 2 and 1/4 for all remander schools. In the postal strata the proportons were 2/3 f the rato was 2; 1/3 for 0 rato < 0.5 and 1/6 for all other schools. 7
was repeated for the allocaton of schools n the postal treatment groups, where those selected for ncentve treatment were randomly splt nto a condtonal and an uncondtonal treatment group. All estmates of sgnfcance presented n ths tet account for the clustered samplng desgn of the ncentve eperment. Table 1 shows the ssued sample szes at each wave for the dfferent treatment and mode combnatons, ecludng cases of known nelgblty who d ether moved abroad or ded (N=13 at wave 2; N=3 at wave 3). Inelgble cases at wave 4 are not dentfed n the data, but the number s lkely to be small. The analyss also ecludes wave 1 respondents for whom no telephone number was known at the tme of the allocaton to modes for wave 2, as descrbed above, and 117 hgher achevers assgned to telephone mode at wave 2, who responded by post and were subsequently allocated to postal mode. < Table 1 > Table 1 also documents the observed wave-on-wave and cumulatve response rates (AAPOR RR1). The rates are shown by achevement level and sequental mode/ncentve combnaton. Wave-on-wave response rates for the hgher achever sample allocated to telephone control at wave 2 and moved to postal control at wave 4 (Col 1) were, for eample, 76.82%, 69.13% and 72.21%. The ssued numbers of cases declned from 2,075 to 1,101 across the three waves, because nonrespondents were not ssued n subsequent waves. 4 Outcome measures and methods The analyss s based on the sample of wave 1 respondents, snce the epermental treatment allocatons used nformaton collected n the frst wave and the correspondng cracterstcs of wave 1 nonrespondents are unknown. The response rate at wave 1 (AAPOR RR1) was 54.80%, ecludng 5 cases of known nelgblty (Russell and Phelps 2001). Ths secton descrbes the outcome measures and methods used to evaluate the hypotheses about the effects of ncentves. 4.1 Attrton To test the effect of ncentves on attrton, we estmated the probablty of attrton as a functon of the epermental desgn varables (telephone mode, uncondtonal ncentves, condtonal ncentves, lower achevers) and ther nteractons. For each of the three epermental waves (t=2,3,4), we estmated a separate probt model of the probablty of attrton, n each case usng the wave 1 respondent sample as the base: Pr( attrton t ) = F( β 8t 0t + β tel + β unc + β cond + β la + β tel * unc + β la * tel + β la * unc + 1t β la * cond 9t 2t 3t + β la * tel * unc + ε ) 4t 5t 6t 7t (1) 8
where F s the probt lnk functon. The estmated coeffcents and standard errors from ths model were then used to calculate predcted probabltes of attrton under dfferent treatment condtons and to test for dfferences due to ncentves. 4.2 Item nonresponse To test the effect of ncentves on tem NR, we estmated count models of the number of tems mssng, usng all non-routed tems from the core questonnares n waves 2 (N=44), 3 (N=48) and 4 (N=46), where don t know was counted as a mssng value. We used the same specfcaton of the predctors as for model (1) above, to estmate separate negatve bnomal regresson models for each of the three epermental waves, condtonal on response to the gven wave. Overdsperson meant tt Posson models dd not ft the data: the P-value of the Lkelhood Rato test of equal mean and varance was 0.0000 for all three waves. The estmated coeffcents and standard errors from these models were used to calculate predcted tem NR under dfferent treatment condtons and to test for dfferences due to ncentves. 4.3 Attrton bas To test the effect of ncentves on attrton bas, we estmated the probablty of attrton usng model (1) and ncludng wave 1 respondent cracterstcs and ther nteractons wth the epermental desgn varables as predctors. We estmated separate probt models for attrton at each of the epermental waves (t=2,3,4) and for each cracterstc, agan usng the wave 1 respondent sample as the base: Pr( attrton t ) = F( β 8t 0t + β tel + β unc + β cond + β la + β tel * unc + β la * tel + β la * unc + 1t β la * cond + β la * tel * unc 9t 2t 3t + β w1cr + β w1cr * tel 10 4t 5t 11 6t +... + β la * tel * unc 19 7t + ε ) (2) where β11to β19 are the coeffcents for the nteractons of the cracterstc wth the desgn varables. The coeffcent for the respondent cracterstc, β 10, provdes nformaton about the drecton, magntude and, n combnaton wth ts standard error, the sgnfcance of attrton bas. The nteracton of the cracterstc and the ncentve ndcators provde nformaton about the cnge n attrton bas due to ncentves. 5 5 The sgnfcance of all nteractons presented n ths tet was calculated followng recommendatons for nonlnear models by Norton, Wang and A (2004) usng the command predctnl n Stata verson 9. 9
The cracterstcs tested were sample composton varables (gender, school type, eam results), current actvty (full-tme educaton, employment or nether n educaton, employment nor tranng; unemployment; studyng for vocatonal or academc qualfcatons), household composton (lvng wth parent, partner) and a set of atttudnal questons about employment and tranng. The wordng of all questons s documented n Table 6. The cracterstcs chosen were those for whch respondents and nonrespondents could be epected to dffer, based on prevous studes of nonresponse n the YCS and other surveys and on nonresponse theores (Groves and Couper 1998; Lynn et al. 1994). 4.4 Reported results Snce coeffcents from non-lnear models cannot be nterpreted substantvely (Long 1997), we report predcted values based on the model estmates, rather tn coeffcents. Unless stated otherwse, the results are for the hgher achever group. To convey a sense of the magntude of dfferences n outcomes across treatments, we report transformatons of the predcted values, comparng each treatment wth the comparson group, the hgher achever postal control. 5 Attrton, tem NR and attrton bas n the control groups As a background to the evaluaton of the effects of ncentves, ths secton documents the etent of attrton, tem NR and attrton bas n the control groups, hghlghtng dfferences across waves, achevement levels and modes. Throughout the dscusson the hgher achever postal control group s the reference category, wth whch all other treatments are compared. 5.1 Attrton The predcted cumulatve attrton rate among hgher achevers allocated to the postal control group, ncreased from 21.77% n wave 2 to 56.53% n wave 4 (Table 2, Col 1). For lower achevers (Col 2), attrton rates n the postal control group were 61% hgher at wave 2, but ths dfference decreased across waves to 29% at wave 4. The dfference by achevement level was nonetheless sgnfcant n all three waves (P-value of β 4 = 0.0000 for t=2,3,4). In telephone mode (Col 3), attrton rates n the control group were not sgnfcantly dfferent at wave 2, but 9% hgher at wave 3 (P-value of β 5 =0.0034 for t=3). Ths s contrary to fndngs from other studes, where nonresponse s generally lower n telephone mode due to the role of the ntervewer n persuadng respondents to take part n the survey. One possble reason for fndng the opposte n ths study, s tt postal nonrespondents were followed up by 10
telephone, whle telephone nonrespondents were not followed up. Secondly, the burden of the wave 2 survey (measured by the ntervew length) was hgher for the telephone respondents due to the addtonal modules, possbly leadng to hgher nonresponse at wave 3 tn among the postal sample. The predcted cumulatve response rates, whch were the base for the calculaton of percentage dfferences across treatment groups, are documented n the frst three columns of Table 5. < Table 2 > 5.2 Item nonresponse The predcted number of mssng tems n the hgher achever postal control group was 2.89 at wave 2, fallng to 1.75 at wave 4 (Table 3, Col 1). For lower achevers (Col 2), the epected count for the control group was 21% hgher at wave 2, wth the gap ncreasng to 45% at wave 4. The dfferences by achevement level were sgnfcant n all three waves (P 0.0001 for β 4, t=2,3,4). For telephone mode (Col 3), the epected count was 4% lower at wave 2 and 12% lower at wave 3 (P=0.0000 for β 5, t=2,3). The predcted tem NR counts, used as the base for the calculatons presented n Table 3, are documented n columns 4 to 6 of Table 5. < Table 3 > 5.3 Attrton bas Nonresponse n the hgher achever postal control group was dfferental for all of the domans tested (Table 4). The respondent samples sgnfcantly overrepresented those lvng wth ther parents, n full-tme educaton or studyng for academc qualfcatons. Predcted attrton rates for those n full-tme educaton n the hgher achever postal control group, for eample, were 14% lower tn for those not n full-tme educaton at wave 2, wth the dfference ncreasng to 17% by wave 4 (P=0.0000 for β 10, t=2,3,4). At the same tme, the respondent samples under-represented males, those n modern schools, wth low or no eam results, who thought employers dd not gve young people the rght tranng and tt makng plans for the future was a waste of tme, those n full-tme employment, who d eperenced unemployment or were studyng for vocatonal qualfcatons. Bas was partcularly strong for those wthout any or very low eam qualfcatons, who were around 50% more lkely to ve attrted from the sample by waves 3 and 4, compared to sample members wth better qualfcatons. Smlarly, those n full-tme employment were 17% more lkely tn those not n employment to drop out at wave 2, wth the dfference ncreasng to 22% by wave 4. 11
< Table 4 > Includng background nformaton used by the YCS for weghtng (gender, school type, eam results and regon) n the models dd not affect the bas for any of the cracterstcs (n each wave and for each tem, the P-value>0.05 from Wald tests of the equalty of β 10 estmated wth and wthout background cracterstcs; not reported), ecept for bas for low qualfcatons (1 to 4 eams wth grades D to G), for whch bas was somewt reduced when the background nformaton was ncluded. The etent of attrton bas was mostly stable across waves. In the hgher achever postal control sample, the under-representaton of males sgnfcantly ncreased from waves 2 to 4 (P-value from a Wald test of the equalty of β 10 across the two waves = 0.0295; not reported). For some of the other cracterstcs, the bas sgnfcantly decreased across waves. NR bas assocated wth attendng a modern school fell between waves 3 and 4 and bas assocated wth not vng any qualfcatons fell between waves 2 and 3 and agan between waves 3 and 4. For lower achevers there were few dfferences n the etent of attrton bas (not reported). Bas by gender, tt s the dfference n predcted nonresponse rates between males and females, was 12% less tn for hgher achevers at wave 4 (P-value of the nteracton between achevement level and gender was 0.0425 for t=4), and bas by full-tme employment was 4% less at wave 2 (P-value = 0.0269 for t=2); bas accordng to atttudes on tranng provded by employers was 9% hgher at wave 2 (P-value = 0.0056); bas accordng to whether studyng for academc or vocatonal qualfcatons was hgher at wave 2 (22% and 13%), 6% lower and 1% hgher at wave 3, and lower at wave 4 (81% and 92%). Attrton bas n telephone mode was no dfferent from postal mode, ecept for dfferental NR by gender: the bas was 7% less at wave 2, 2% less at wave 3 and 1% more at wave 4 (P-value of the nteracton between telephone mode and gender was 0.002 for t=2,3,4). 6 Evaluaton of hypotheses The evdence dscussed here s summarsed n Table 2 (effects of ncentves on attrton), Table 3 (effects on tem NR), Table 4 (effects on attrton bas) and Table 5 (net effect on unt and tem NR). 12
H1: Effects of ncentves on attrton rate, attrton bas and tem NR Incentves reduced attrton and ncreased tem NR, but dd not mpact on attrton bas. Uncondtonal ncentves reduced cumulatve attrton n the postal hgher achever sample (Table 2, Col 4) by 38% (correspondng to an 8 percentage pont dfference) at wave 2, 32% at wave 3 and 30% at wave 4 (P-value of β 2 =0.0000 for t=2,3,4). At the same tme, the ncentve ncreased tem NR by 10% at wave 2 and 17% at wave 3 (P-value of β 0. 2 05 for t=2,3), but d no effect at wave 4 (Table 3, Col 4). The dfference across waves was however not sgnfcant (see H2). Incentves d a proportonate effect on attrton across all respondent cracterstcs tested and therefore dd not reduce attrton bas: the P-value of the nteracton of uncondtonal ncentves and respondent cracterstcs was >0.05 for all cracterstcs and waves (not reported). The ecepton was the proporton of pupls n modern schools who were under-represented n all three waves. (Modern schools were the smallest category, representng only 2.8% of the wave 1 respondent sample.) Uncondtonal ncentves reduced ths bas by 60%, 47% and 78% at waves 2, 3 and 4 (P-values of the nteracton of ncentves and modern school 0.01 for t=2,3,4). < Table 5 > Snce ncentves d both postve and negatve effects on response, Table 5 documents the net effect on the amount of nformaton collected n the survey. The benefts of ncentves n terms of unt NR clearly outweghed the cost n terms of tem NR. For each sample persons ssued at wave 2, the predcted unt and tem response rates for the postal hgher achever sample, for eample, mpled tt by wave 4, 40% more vald tems were collected wth uncondtonal ncentves compared to the control group. For lower achevers, 50% more nformaton was collected wth ncentves. Ths s, however, a crude measure of the net effect of ncentves, snce n a multvarate analyss or for analyses of cnge, dfferent patterns of mssngness across tems or across waves may lead to large numbers of cases beng dropped by parwse deleton. H2: Effects of ncentves across waves The effect on attrton decreased somewt across waves, whle the effects on tem NR and attrton bas were constant. Incentves reduced attrton by -38% at wave 2, -32% at wave 3 and -30% at wave 4 (Table 2, Col 4). The effects were comparable at waves 2 and 3, but sgnfcantly dfferent between waves 2 and 4 and between waves 3 and 4 (P-value from a 13
Wald test of the equalty of β 2 across waves was 0.05). Although the relatve effect of ncentves decreased,, the absolute effect ncreased across waves (-17 percentage ponts at wave 4, compared to -8 and -14 at waves 2 and 3, see Table 5). The effect of ncentves on tem NR was not sgnfcantly dfferent across waves (P-value of equalty of β2 across waves was > 0.05), although the predcted numbers of mssng tems fell across waves. Smlarly, the effects of ncentves on attrton bas dd not dffer across waves. H3: Condtonal compared to uncondtonal ncentves Uncondtonal ncentves d more effect on attrton tn condtonal ncentves, but comparable effects on tem NR and attrton bas. For hgher achevers, the condtonal ncentves used at wave 2, were only lf as effectve at reducng attrton as uncondtonal ncentves (Table 2, Col 5) and the dfference between the two condtons was sgnfcant (Pvalue from a Wald test of the equalty of β 2 and β 3 was 0.0142). At the same tme, condtonal ncentves ncreased tem NR by 38% more tn uncondtonal ncentves (Table 3, Col 5), but the dfference was not sgnfcant. Condtonal ncentves somewt reduced attrton bas for a sngle cracterstc: sample members n the control group studyng for vocatonal qualfcatons at wave 1 were 6.8% more lkely to drop out tn those not studyng for vocatonal qualfcatons. Wth condtonal ncentves the dfference was 6.4% (P-value of the nteracton of condtonal ncentves wth ths cracterstc was 0.05 for t=2). H4: Dfferental effects by mode Incentves d more effect on attrton and tem NR n postal tn telephone mode, but no effect on attrton bas n ether mode. In telephone mode, uncondtonal ncentves d less tn lf the effect on attrton they d n postal mode for the hgher achever group (Table 2, Col 8). The dfference was sgnfcant at wave 3 (P-value of the nteracton between telephone mode and uncondtonal ncentves was 0.0057) but not at wave 2. At wave 3, ncentves ncreased tem NR 5% less n telephone mode tn n postal mode (P-value of the nteracton was 0.0442), but the dfference at wave 2 was not sgnfcant. The lack of effect of uncondtonal ncentves on attrton was no dfferent across the two modes. H5: Effects of cnges n mode or ncentve treatment Cngng the ncentve condton or mode dd not ve lastng effects. Cngng the treatment from condtonal to uncondtonal ncentves d no lastng effect on ether attrton 14
or tem NR (P> 0.05 from Wald tests of the equalty of β 2 and β 3 for t=3,4) and the effects after the cnge n treatment were comparable to those for the sample allocated to uncondtonal ncentves from the start (Tables 2 and 3, Col 5). Cngng the survey mode from telephone to postal dd not ve a lastng effect on attrton or tem NR ether (P=value of the nteracton for telephone mode and uncondtonal ncentves > 0.05 at t=4) and the effects after the cnge n mode were no dfferent from the effects for the sample allocated to postal uncondtonal ncentves from the start (Tables 2 and 3, Col 8). H6: Dfferental effects by ablty level The effects of ncentves were comparable across achevement levels. Although there were dfferences n the proportonal effects of uncondtonal and condtonal ncentves on attrton and tem NR by achevement level, these were not sgnfcant (Cols 6 and 7 n Tables 2 and 3 report the P-values of the nteractons of achevement level wth each of the ncentve treatments), snce the absolute effects were comparable. Uncondtonal ncentves, for eample, reduced attrton at wave 2 by 8 percentage ponts among hgher achevers and 10 percentage ponts among lower achevers. However, snce the level of nonresponse n the control group was 61% hgher for the lower achever group, the comparable absolute effect mpled a smaller proportonal effect of only 76% of the effect for hgher achevers. Smlarly, the dfference between modes was not dfferental by achevement (Tables 2 and 3, Col 9 report the P-values of the nteracton between achevement level, uncondtonal ncentves and telephone mode) and the lack of effect on attrton bas was no dfferent for lower achevers (not reported). 7 Summary and dscusson Ths study s provded new evdence on the effects of contnued ncentve payments n a mult-mode panel study. We tested the effects of ncentves on attrton, tem NR and attrton bas and whether these effects cnged across waves. We also tested whether condtonal and uncondtonal ncentves d comparable effects, whether ncentve effects were dfferental across modes and ablty levels, and whether cnges n the ncentve treatment or mode d lastng mpact on the effect of ncentves n subsequent waves. The fndngs showed tt uncondtonal ncentves sgnfcantly reduced attrton and, although they also ncreased tem NR, the net effect on the amount of nformaton collected by the survey was postve. Incentves d proportonate effects across a range of respondent cracterstcs and as a result dd not mpact on attrton bas n terms of those cracterstcs. 15
Item NR was reduced more by uncondtonal tn condtonal ncentves, and more n postal tn n telephone mode. Attrton bas was not affected by ncentves, regardless of ncentve treatment or mode. Across waves, ncentves d somewt decreasng effect on attrton, but comparable effects on tem NR. The lack of effect on attrton bas was also a constant across waves. Cnges n ncentve treatment from condtonal to uncondtonal, and n mode from telephone to postal, dd not affect outcomes at later waves. The fndngs mply tt respondent ncentves are an effectve means of mantanng sample szes of a panel and ensurng ts value n terms of effcency of estmaton and feasblty of subgroup analyses. Among lower achevers, fully 50% more nformaton was collected durng the three epermental waves, n terms of the number of vald tems per case ssued at the start. Incentves were safe, n the sense tt ncreased response rates dd not nadvertently ncrease NR bas n terms of observed cracterstcs. Cnges n ncentve treatment dd not ve lastng effect; however, n ths study the only cnge mplemented was an mprovement for the respondent, from condtonal to uncondtonal ncentves. Epectatons formed on the bass of prevous ncentve treatments may well mean tt cnges ve lastng effect, f the cnge reduces the value of the ncentve n the eyes of the respondent (see, Snger, Van Hoewyk, and Maher 1998). Incentves d no effect on attrton bas. We could however not evaluate the effect on bas of NR at wave 1. Ideally, we would compare both the magntude of bas due to NR at wave 1 and due to subsequent attrton, and the effects of ncentves on both. It s lkely tt NR at wave 1 s more detrmental n terms of bas tn later attrton, especally n studes such as the present one wth low ntal response rates. In ths case, the effect of ncentves on bas at wave 1 s potentally more mportant tn any effect on bas caused by attrton. In addton, the dscusson of the effects of ncentves on attrton bas s focused entrely on observed cracterstcs and although ncentves dd not ve dfferental effects n terms of these, they may nonetheless ve dfferental effects n terms of unobserved factors. If ths were the case, the use of respondent ncentves could ntroduce sample selecton bas n multvarate estmates, f the unobservables determnng the responsveness to ncentves are correlated wth outcomes measured by the survey (Kennedy 2003). For eample, f responsveness to ncentves depends on tme preferences for money and ths factor also determnes the decson to leave further educaton and work nstead, then models of the determnants of educatonal outcomes wll lead to based estmates. Fnally, there was lttle evdence tt the respondent sample became less senstve to ncentves across waves as potentally less commtted sample members dropped out. Ths 16
fndng s consstent wth Laure (2006), who reported tt an ncrease n the value of an ncentve n the Brtsh Household Panel Survey sgnfcantly ncreased response, even after 14 waves of the panel, wth already hgh annual response rates of around 95% each year. Snce prevous studes ve found tt the effects of one-off ncentves can carry over across waves (James 1997; Laure and Lynn forthcomng; Mack et al. 1998), a formal test of margnal effects of ncentves would however requre comparsons wth a treatment group only offered an ncentve at the frst wave. References Armstrong, J. Scott. 1975. "Monetary Incentves n Mal Surveys." Publc Opnon Quarterly 39:111-116. Berk, Marc L., Nancy A. Mathowetz, Edward P. Ward, and Andrew A. Whte. 1987. "The Effect of Prepad and Promsed Incentves: Results of a Controlled Eperment." Journal of Offcal Statstcs 3:449-457. Church, Allan H. 1993. "Estmatng the Effect of Incentves on Mal Survey Response Rates: A Meta-Analyss." Publc Opnon Quarterly 57:62-79. Davern, Mcel, Todd H. Rockwood, Randy Sherrod, and Stephen Campbell. 2003. "Prepad Monetary Incentves and Data Qualty n Face-to-Face Intervews: Data from the 1996 Survey of Income and Program Partcpaton Incentve Eperment." Publc Opnon Quarterly 67:139-147. Downs, Barbara. 1999. "Incentve Use n Panel Surveys." Internal Census Bureau Memorandum. Fo, Rcrd J., Melvn R. Crask, and Jonghoon Km. 1988. "Mal Survey Response Rate: A Meta-Analyss of Selected Technques for Inducng Response." Publc Opnon Quarterly 52:467-491. Gouldner, Alvn. 1960. "The Norm of Recprocty: A Prelmnary Statement." Amercan Socologcal Revew 25:161 78. Goyder, John. 1994. "An Eperment wth Cash Incentves on a Personal Intervew Survey." Journal of the Market Research Socety 36:360-366. Groves, Robert M. and Mck P. Couper. 1998. Nonresponse n Household Intervew Surveys. New York: John Wley. Groves, Robert, Eleanor Snger, and Amy Cornng. 2000. "Leverage-Salence Theory of Survey Partcpaton: Descrpton and an Illustraton." Publc Opnon Quarterly 64:299-308. Hopkns, Kenneth D. and Arlen R. Gullckson. 1992. "Response Rates n Survey Research: A Meta-Analyss of the Effects of Monetary Gratutes." Journal of Epermental Educaton 61:52-62. James, Jeannne M. and Rcrd Bolsten. 1990. "The Effect of Monetary Incentves and Follow-Up Malngs on the Response Rate and Response Qualty n Mal Surveys." Publc Opnon Quarterly 54:346-361. James, Tracy L. 1997. "Results of Wave 1 Incentve Eperment n the 1996 Survey of Income and Program Partcpaton." Pp. 834-839 n Proceedngs of the Survey Research Methods Secton, Amercan Statstcal Assocaton. Aleandra, VA: Amercan Statstcal Assocaton. Kennedy, Peter. 2003. A Gude to Econometrcs. Oford: Blackwell. 17
Laure, Heather. 2006. "The Effect of Increasng Fnancal Incentves n a Panel Survey: an Eperment on the Brtsh Household Panel Survey, Wave 14." Colchester: Unpublshed Workng Paper. Unversty of Esse. Laure, Heather and Peter Lynn. forthcomng. "The Use of Respondent Incentves on Longtudnal Surveys." n Methodology of Longtudnal Surveys, edted by P. Lynn. Colchester: Unversty of Esse. Long, J.S. 1997. Regresson Models for Categorcal and Lmted Dependent Varables. Thousand Oaks, CA: Sage. Lynn, Peter, Susan Purdon, Barry Hedges, and Ian McAleese 1994 'England and Wales Youth Cohort Study: An Assessment of Alternatve Weghtng Strateges.' Employment Department Research Seres YCS, No. 30, Sheffeld: Employment Department. Mack, Stephen, Vck Huggns, Donald Keathley, and Mahd Sundukch. 1998. "Do Monetary Incentves Improve Response Rates n the Survey of Income and Programme Partcpaton?" Pp. 529-534 n Proceedngs of the Survey Research Methods Secton, Amercan Statstcal Assocaton. Aleandra, VA: Amercan Statstcal Assocaton. Martn, Elzabeth, Dense Abreu, and Frankln Wnters. 2001. "Money and Motve: Effects of Incentves on Panel Attrton n the Survey of Income and Program Partcpaton." Journal of Offcal Statstcs 17:267-284. Mason, Robert, Vrgna Lesser, and Mcel W. Traugott. 2002. "Effect of Item Nonresponse on Nonresponse Error and Inference." Pp. 149-161 n Survey Nonresponse, edted by R. M. Groves, D. A. Dllman, J. L. Eltnge, and R. J. A. Lttle. New York: John Wley. Menard, Scott. 2002. Longtudnal research. Thousand Oaks: Sage. Rodgers, Wllard. 2002. "Sze of Incentve Effects n a Longtudnal Study." Pp. 2930-2935 n Proceedngs of the Survey Research Methods Secton, Amercan Statstcal Assocaton. Aleandra, VA: Amercan Statstcal Assocaton. Russell, Nel and Andrew Phelps. 2001. "Youth Cohort Study Cohort 10 Sweep 1: Techncal Report." London: Taylor Nelson Sofres. Ryu, Erca, Mck P. Couper, and Robert W. Marans. 2006. "Survey Incentves: Cash vs. In- Knd; Face-to-Face vs. Mal; Response Rate vs. Nonresponse Error." Internatonal Journal of Publc Opnon Research 18:89-106. Sala, Emanuela and Peter Lynn 2004 'The Effect of Dependent Intervewng on Measures of Employment Cracterstcs', ISER Workng Paper, No. 2004-26, Colchester: Unversty of Esse. http://www.ser.esse.ac.uk/pubs/workpaps/pdf/2004-26.pdf Shettle, Carolyn and Geraldne Mooney. 1999. "Monetary Incentves n U.S. Government Surveys." Journal of Offcal Statstcs 15:231-250. Snger, Eleanor, John Van Hoewyk, Nancy Gebler, Trvellore Raghunatn, and Katherne McGonagle. 1999. "The Effect of Incentves on Response Rates n Intervewer- Medated Surveys." Journal of Offcal Statstcs 15:217-230. Snger, Eleanor, John Van Hoewyk, and Mary P. Maher. 1998. "Does the Payment of Incentves Create Epectaton Effects?" Publc Opnon Quarterly 62:152-164.. 2000. "Eperments wth Incentves n Telephone Surveys." Publc Opnon Quarterly 64:171-188. Stratford, Nna, Nadne Smmonds, and Gerry Ncolaas. 2003. "Natonal Travel Survey 2002: Report on Incentves Eperment." London: Natonal Centre for Socal Research. Tesl, Maro F., Bran Roe, and Mke Vayda. 2005. "Incentve Effects on Response Rates, Data Qualty, and Survey Admnstraton Costs." Internatonal Journal of Publc Opnon Research 18:364-373. 18
Tzamouran, Panagota and Peter Lynn 1999 'The Effect of Monetary Incentves on Data Qualty - Results from the Brtsh Socal Atttudes Survey 1998 Eperment', CREST Workng Paper, No. 73, Oford: Unversty of Oford. Ward, R. Kay, Scott Boggess, Kandasamy Selvavel, and Mcel F. McMahon. 2001. "The Use of Targeted Incentves to Reluctant Respondents on Response Rates and Data Qualty." n Proceedngs of the Survey Research Methods Secton, Amercan Statstcal Assocaton. Aleandra, VA: Amercan Statstcal Assocaton. Wllmack, Dane K., Howard Schuman, Beth-Ellen Pennell, and James M. Lepkowsk. 1995. "Effects of a Prepad Nonmonetary Incentve on Response Rates and Response Qualty n a Face-to-Face Survey." Publc Opnon Quarterly 59:78-92. Yu, Jule and Harrs Cooper. 1983. "A Quanttatve Revew of Research Desgn Effects on Response Rates to Questonnares." Journal of Marketng Research 20:36-44. Fgure 1: Epermental desgn Postal core questonnare. Reported eam results used to classfy respondents for wave 2 Wave 1 allocaton. Hgher Achevers Lower Achevers Wave 2 T T u P P u P c T T u P P u P c Wave 3 T T u P P u -- -- -- P P u -- Wave 4 -- -- P P u -- -- -- P P u -- Questonnare Core + Hgher educaton Core Core + Educaton and employment Core Notes: T=telephone, P=postal, =control, u=uncondtonal ncentve, c=condtonal ncentve. Arrows ndcate cnges n ncentve treatment or mode allocaton between waves. 19
Table 1: Condtonal and cumulatve response rates Hgher Achevers Lower Achevers Wave Response Rate % T T P T u T u P u P P P P u P u P u P c P u P u T P P T u P u P u P P P P u P u P u P c P u P u 2 Condtonal 76.82 80.91 78.23 86.45 82.32 65.21 70.41 64.93 75.00 71.35 (Issued N) (2,075) (728) (3,262) (1,004) (967) (1,282) (811) (807) (608) (569) 3 Condtonal 69.13 73.17 73.07 81.91 81.36 59.09 70.93 63.36 71.93 70.20 (Issued N) (1,594) (589) (2,551) (868) (794) (836) (571) (524) (456) (406) Cumulatve 53.11 59.20 57.16 70.82 66.94 38.53 49.94 41.14 53.95 50.09 4 Condtonal 72.21 85.61 76.11 85.65 86.82 63.16 74.26 65.36 75.30 81.34 (Issued N) (1,101) (431) (1,863) (711) (645) (494) (404) (332) (328) (284) Cumulatve 38.31 50.69 43.48 60.66 58.03 24.34 36.99 26.89 40.63 40.60 Notes: AAPOR Response Rate 1. Treatment groups are dentfed by T=telephone, P=postal, =control, u=uncondtonal ncentve, c=condtonal ncentve. T T P for eample, refers to the sample allocated to telephone control at waves 2 and 3 and to postal control at wave 4. Condtonal response rates are condtonal on response at the prevous wave. The base s the number of ssued cases, whch ecludes prevous wave nonrespondents and nelgble cases. Cumulatve response rates are the percentage of wave 1 respondents remanng n the respondent sample. The base s the wave 2 number of ssued cases, ecludng three hgher achevers nelgble at wave 3 (1 P P P and 2 P c P u P u ). Table 2: Effect of ncentves on attrton rates Control groups Incentves Incentves by ablty Incentves by mode and ablty (1) (2) (3) (4) (5) (6) (7) (8) (9) la la la la la la la P P T P Pu P ( Pc P ) / P (4) ( Pc P ) / P ( T u T ) / T ( Tu T ) / T P P P (4) (4) ( P P ) / P (4) ( T T ) / T P Wave c u 2 21.77 0.6112 0.0650-0.3777 0.4966 0.7602 0.9763 0.4669 0.8471 (P-Value) (0.0000) (0.2268) (0.0000) (0.0142) (0.5085) (0.4332) (0.0556) (0.6810) 3 42.86 0.3734 0.0941-0.3191 0.7066 0.6820 0.6743 0.4074 1.4275 (P-Value) (0.0000) (0.0034) (0.0000) (0.0592) (0.7834) (0.8287) (0.0057) (0.0861) 4 56.53 0.2933-0.3040 0.8402 0.6179 0.7340 0.6597 0.8338 (P-Value) (0.0000) (0.0000) (0.2244) (0.2535) (0.8177) (0.0911) (0.9265) Notes: P=postal, T=telephone, =control, u=uncondtonal ncentve, c=condtonal ncentve, =hgher achevers, la=lower achevers. Column (1) shows the predcted attrton rate for the postal control hgher achever sample. The remanng columns show proportonate cnge n predcted rates. P-values of columns 2 4 represent standard errors of the man effects n the probt model; column 5 represents P-values from a Wald test of the equalty of the coeffcents for condtonal and uncondtonal ncentves; columns 7 9 represent P-values for the relevant nteractons calculated usng predctnl n Stata verson 9, accordng to Norton, Wang, and A (2004). la