Methodology for ACT WorkKeys as a Predictor of Worker Productivity

Transcription

1 Methodology for ACT WorkKeys as a Predctor of Worker Productvty The analyss examned the predctve potental of ACT WorkKeys wth regard to two elements. The frst s tme to employment. People takng WorkKeys were ether completng schoolng or actvely n the job market and lookng for employment. The tme to employment was measured as the number of weeks from the date of the test untl the ndvdual appeared on a wage record. The observaton count for ths was 2,44. Not everyone takng the exam had a wage record, as some ndvduals may have never found work, retred or moved out of state. Wages were also of nterest. Ideally, f ths were a longtudnal dataset, an assessment could have been made examnng the score on the test versus wage growth over a fve- or 1-year horzon. Assumng wages as a proxy for performance, a relatonshp between WorkKeys scores and on the job performance would be properly ascertaned. Unfortunately, such a longtudnal seres was not avalable so two dfferent wage measures were used. The frst, wage after test, reflects the average wage collected n the second and thrd quarters from the date of ntal employment (ths compensates for md-quarter hrng or the potental nfluence of sgnng bonuses). The observaton count for ths was 265,229. However, ths varable may be detrmentally nfluenced by selecton bas. Hghly productve employees lkely score well on the assessment and fnd a more hghly pad poston n reemployment. The hgh wage on reemployment mght be a functon of the exam, but t mght also be an ndcaton of pror wage. To accommodate ths potental, an addtonal wage dfference varable was created. It s based on the dfference between wages after reemployment wth those before the test (usng a smlar collecton method). The observaton count for ths was 64,642. The analyss addtonally looked at ndvduals age 19 and older and those between ages 16 and 18. Most ndvduals took WorkKeys durng a phase of unemployment and as a result of applyng for jobs wth specfc employers. Ths self-selecton has the potental to bas results. However, some school dstrcts provded these assessments to students pror to graduaton from hgh school and the student pool would be less lkely to suffer from self-selecton. The results were run aganst both groups to determne f any detectable dfference could be ascertaned. A standard ordnary least squares (OLS) model was utlzed to assess the data. As the tests were taken n such short tme proxmty, the potental nfluence of tme was dscounted n ths prelmnary study. Tme to Employment = β + β1(gender ) + β2(age) + βk, ( X k, ) + βn, ( Yn, ) + ε Wage Dfference Where = ndvdual k= β1(gender ) + β2(age ) + βk, ( X k, ) + βn, ( Yn, ) k= 3 9 = β + ε 8 17

2 X k = Tests: Readng for Informaton, Locatng Informaton, Observaton, Teamwork, Appled Mathematcs, Appled Technology Y n = Hghest Educatonal Attanment: 8 th Grade or Less, Grades 9-11, H.S. Graduate or Equvalent, 1 Yr College, 2 Yr College or Assocates, 3 Yr College, Bachelor s, Master s, Doctorate The results provded below ndcate no statstcal nfluence on the choce of test taken wth tme to employment or wage. The next set of tests examned the nfluence of score on the tests wth regard to tme to employment and wages, whle controllng for the addtonal varables. These tests were run ndependently on the young cohort and older cohort. Tme to Employment Wage After Test Wage Dfference Where = β + β1(gender ) + β2(age ) + β3(level ) + βn, ( Yn, ) + ε 4 12 = + β1(gender ) + β2(age) + β3(level) + βn, ( Yn, ) 4 12 = + β1(gender ) + β2(age) + β3(level ) + βn, ( Y n, ) 4 β + ε β + ε Level: Level score ndvdual acheved on the WorkKeys Assessment Test The results of the analyss ndcate lttle dfference between younger and older groups. Ether the nfluence of selecton bas as a result of mandated employer partcpaton by applcants s not large enough to be detected or the method of splttng age groups to solate the bas was nsuffcent. 12

3 Tables Observaton Count Wage After Test Test Total above to 18 Readng for Informaton Locatng Informaton Observaton Teamwork Appled Mathematcs Appled Technology Observaton Count Tme (Weeks) to Employment Test Total above to 18 Readng for Informaton Locatng Informaton Observaton Teamwork Appled Mathematcs Appled Technology Observaton Count Wage Dfference Test Total above to 18 Readng for Informaton Locatng Informaton Observaton Teamwork Appled Mathematcs Appled Technology

4 # of Obs # of Obs F(2,26528) F(19,2567) Prob>F. Prob>F. R-Sqr Adj..411 R-Sqr Adj..73 Varable Coef Std Err t p Varable Coef Std Err t p Readng for Informaton Readng for Informaton Locatng Informaton Locatng Informaton Observaton Observaton Teamwork Teamwork Appled Mathematcs Appled Mathematcs Appled Technology Appled Technology Lstenng and Wrtng Lstenng and Wrtng Locatng Informaton Locatng Informaton (omtted) Busness Wrtng (omtted) Busness Wrtng Workplace Observaton Workplace Observaton (omtted) Gender (M) ** Gender (M) Age at Test ** Age at Test ** 8th Grade of less ** 8th Grade of less ** Grades ** Grades H.S. Grad. or Equv ** H.S. Grad. or Equv ** 1 yr. College/Tech./Voc ** 1 yr. College/Tech./Voc ** 2 yrs. College or Assoc ** 2 yrs. College or Assoc ** 3 yrs. College/Tech./Voc ** 3 yrs. College/Tech./Voc ** Bachelor's Degree ** Bachelor's Degree ** Master's Degree ** Master's Degree ** Doctorate ** Doctorate ** Intercept Intercept Statstcal Sgnfcance: 1% **, 5% * Statstcal Sgnfcance: 1% **, 5% *

5 Test: Readng for Informaton Test: Readng for Informaton Test: Locatng Informaton Test: Locatng Informaton Test: Observaton Test: Observaton Tme to Employment Tme to Employment Tme to Employment Tme to Employment Tme to Employment Tme to Employment # of Obs # of Obs # of Obs # of Obs # of Obs # of Obs. 91 F(12,34656) F(1,3754) 4.54 F(12,53467) F(1,428) 4.25 F(12,41554) 12.9 F(1,89) 2.81 Prob>F. Prob>F. Prob>F. Prob>F. Prob>F. Prob>F.19 R-Sqr Adj..63 R-Sqr Adj..119 R-Sqr Adj..38 R-Sqr Adj..75 R-Sqr Adj..34 R-Sqr Adj..198 Varable Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Level Score ** ** ** Gender (M) ** Age at Test ** ** ** ** th Grade of less Grades ** H.S. Grad. or Equv ** ** 1 yr. College/Tech./Voc yrs. College or Assoc yrs. College/Tech./Voc Bachelor's Degree Master's Degree (omtted) (omtted) (omtted) Doctorate (omtted) (omtted) (omtted) Intercept Statstcal Sgnfcance: 1% **, 5% * Test: Teamwork Test: Teamwork Test: Appled Mathematcs Test: Appled Mathematcs Test: Appled Technology Test: Appled Technology Tme to Employment Tme to Employment Tme to Employment Tme to Employment Tme to Employment Tme to Employment # of Obs # of Obs. 325 # of Obs # of Obs. 399 # of Obs # of Obs. 176 F(12,13473) 6.51 F(9,315) 3.11 F(12,3753) F(1,3979) 4.39 F(12,5838) 2.47 F(9,166) 1.41 Prob>F. Prob>F.14 Prob>F. Prob>F. Prob>F.32 Prob>F.186 R-Sqr Adj..49 R-Sqr Adj..552 R-Sqr Adj..6 R-Sqr Adj..84 R-Sqr Adj..3 R-Sqr Adj..28 Varable Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Level Score ** Gender (M) Age at Test ** ** ** ** th Grade of less Grades H.S. Grad. or Equv yr. College/Tech./Voc yrs. College or Assoc yrs. College/Tech./Voc (omtted) Bachelor's Degree (omtted) Master's Degree (omtted) (omtted) (omtted) Doctorate (omtted) (omtted) (omtted) Intercept Statstcal Sgnfcance: 1% **, 5% *

6 Test: Readng for Informaton Test: Readng for Informaton Test: Locatng Informaton Test: Locatng Informaton Test: Observaton Test: Observaton Wage After Test Wage After Test Wage After Test Wage After Test Wage After Test Wage After Test # of Obs # of Obs # of Obs # of Obs # of Obs # of Obs F(12,4627) F(1,5151) 7.38 F(12,7469) F(1,581) F(12,5432) F(1,1138) 1.17 Prob>F. Prob>F. Prob>F. Prob>F. Prob>F. Prob>F. R-Sqr Adj..362 R-Sqr Adj R-Sqr Adj..265 R-Sqr Adj..117 R-Sqr Adj..19 R-Sqr Adj..74 Varable Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Level Score ** ** ** Gender (M) ** ** ** ** ** ** Age at Test ** ** ** ** ** th Grade of less ** ** ** ** ** Grades ** ** ** ** ** ** H.S. Grad. or Equv ** ** ** ** ** ** 1 yr. College/Tech./Voc ** ** ** ** ** ** 2 yrs. College or Assoc ** ** ** ** ** ** 3 yrs. College/Tech./Voc ** ** ** ** Bachelor's Degree ** ** ** Master's Degree ** (omtted) ** (omtted) ** (omtted) Doctorate ** (omtted) ** (omtted) * (omtted) Intercept Statstcal Sgnfcance: 1% **, 5% * Test: Teamwork Test: Teamwork Test: Appled Mathematcs Test: Appled Mathematcs Test: Appled Technology Test: Appled Technology Wage After Test Wage After Test Wage After Test Wage After Test Wage After Test Wage After Test # of Obs # of Obs. 424 # of Obs # of Obs # of Obs # of Obs. 223 F(12,1814) F(9,414) 3.44 F(12,4991) F(1,5547) F(12,7261) F(9,213) 3.62 Prob>F. Prob>F.4 Prob>F. Prob>F. Prob>F. Prob>F.3 R-Sqr Adj..428 R-Sqr Adj..494 R-Sqr Adj..216 R-Sqr Adj R-Sqr Adj..223 R-Sqr Adj..96 Varable Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Level Score ** ** ** ** Gender (M) ** ** ** ** Age at Test ** ** ** th Grade of less ** ** ** ** Grades ** ** ** H.S. Grad. or Equv ** ** ** ** ** 1 yr. College/Tech./Voc ** ** ** ** yrs. College or Assoc ** ** ** ** yrs. College/Tech./Voc ** (omtted) ** ** ** Bachelor's Degree ** ** ** (omtted) Master's Degree ** (omtted) ** (omtted) (omtted) Doctorate (omtted) ** (omtted) (omtted) Intercept Statstcal Sgnfcance: 1% **, 5% *

7 Test: Readng for Informaton Test: Readng for Informaton Test: Locatng Informaton Test: Locatng Informaton Test: Observaton Test: Observaton Wage Dfference Wage Dfference Wage Dfference Wage Dfference Wage Dfference Wage Dfference # of Obs # of Obs # of Obs # of Obs # of Obs # of Obs F(12,4627) F(1,5151) F(12,7469) F(1,581) F(12,5432) 12.2 F(1,1138) 5.61 Prob>F. Prob>F. Prob>F. Prob>F. Prob>F. Prob>F. R-Sqr Adj..557 R-Sqr Adj..597 R-Sqr Adj..342 R-Sqr Adj..623 R-Sqr Adj..218 R-Sqr Adj..386 Varable Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Level Score ** * * Gender (M) ** ** ** ** ** ** Age at Test ** ** ** th Grade of less ** ** ** ** ** * Grades ** * ** * ** * H.S. Grad. or Equv ** ** ** ** ** ** 1 yr. College/Tech./Voc ** ** ** ** ** ** 2 yrs. College or Assoc ** ** ** ** ** yrs. College/Tech./Voc ** ** ** ** ** ** Bachelor's Degree ** ** ** Master's Degree ** omtted ** omtted ** omtted Doctorate ** omtted ** omtted * omtted Intercept Statstcal Sgnfcance: 1% **, 5% * Test: Teamwork Test: Teamwork Test: Appled Mathematcs Test: Appled Mathematcs Test: Appled Technology Test: Appled Technology Wage Dfference Wage Dfference Wage Dfference Wage Dfference Wage Dfference Wage Dfference # of Obs # of Obs. 424 # of Obs # of Obs # of Obs # of Obs. 223 F(12,1814) F(9,414) 2.3 F(12,4991) F(1,5547) F(12,7261) 1.4 F(9,213) 3.1 Prob>F. Prob>F.348 Prob>F. Prob>F. Prob>F. Prob>F.16 R-Sqr Adj..362 R-Sqr Adj..215 R-Sqr Adj..354 R-Sqr Adj..647 R-Sqr Adj..147 R-Sqr Adj..783 Varable Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Coef Std Err t p Level Score ** Gender (M) ** ** ** ** ** Age at Test ** ** ** * 8th Grade of less ** ** ** Grades ** ** * * H.S. Grad. or Equv ** ** ** * ** 1 yr. College/Tech./Voc ** ** ** * * 2 yrs. College or Assoc ** ** ** yrs. College/Tech./Voc omtted ** ** Bachelor's Degree ** ** * omtted Master's Degree * omtted ** omtted omtted Doctorate omtted ** omtted omtted Intercept Statstcal Sgnfcance: 1% **, 5% *