Psychometric Properties of the Level of Knowledge Use Survey (LOKUS) Tool

Transcription

1 Psychomtric Proprtis of th Lvl of Knowldg Us Survy (LOKUS) Tool (Shrya Tlang, Machiko Tomita, Vathsala Ston) Prsntr: Vathsala I. Ston Univrsity at Buffalo/Cntr on Knowldg Translation for Tchnology Transfr NARRTC Annual Mting, Apr. 27, 2011

2 Ovrall Contxt: Knowldg Translation (KT) (CIHR, 2009; Sudsawad, 2007). Knowldg (Evidnc) Practic Impact on bnficiaris Spcific Contxt: Tchnology basd Rsarch - Sub-optimal lvl of dmonstratd impact from R&D invstmnt KT4TT Cntr: Background Dvlop KT modls, mthods & mtrics for tchnology basd R&D Conducting nd-of-grant KT intrvntions in 3 tchnology aras Augmntativ and Altrnativ Communication (AAC), Environmntal Accss & Whld Mobility. LOKUS masurs Knowldg Us as indicator of intrvntion ffctivnss.

3 Slctd AAC Study (nw knowldg): End-of-grant; NIDRR fundd; Innovativ Intrvntion stratgy: Background (contd.) KT Intrvntion Projct in AAC Contxtualizd Knowldg Packag (CKP) + Training (wbinars) + Tchnical Assistanc Effct on 6 typs of Knowldg Usrs (stakholdrs): Manufacturrs; Clinicians; Transition Brokrs; Rsarchrs; Policy makrs; Consumrs with disabilitis

4 1. Lvl of Knowldg Us Survy (LOKUS) tool sks to: idntify th lvl of us (and th corrsponding catgoris) of nw knowldg gnratd by tchnology-basd rsarch (attaind) by stakholdrs (usrs) of that knowldg. 2. Dvlopd by th Knowldg Translation on Tchnology Transfr (KT4TT) Cntr Basd on Hall t al (2006) 3. Psychomtric study of LOKUS conductd by Shrya Tlang (2011).

5 Dvlopmnt of LOKUS: Contnt Validation Tool basd on Hall s (2006) framwork - Lvls and Catgoris - ducational stting. Exprt tsting of itms 3 KT scholars and 4 Tchnology Transfr (TT) xprts. Tool showd 100% Fac Validity. Tool altrd: addd/changd lvls, liminatd irrlvant catgoris; closr to KT4TT contxt. Wb vrsion pilot tstd by 6 individual stakholdrs. Rsultd in LOKUS, subjct of this study.

6 Framwork for Itm distribution in LOKUS (Basd on Halls t al, 2006) CATEGORIES 0 - NON-AWARENESS Bing Awar Gtting Information Sharing Assssing Planning Implmnting 1- AWARENESS: 2 ORIENTATION: 3 PREPARATION: x x x x x x x x x x LEVELS 4 - INITIAL USE: 5 - ROUTINE USE: x x x x x x x 6 EXPANSION: 7 COLLABORATION: 8 INTEGRATION: 9 MODIFICATION: x x x x x x x x x x x x x x x x x x x x

7 Rliability Tst-Rtst Focus of th Psychomtric Study Altrnat Assssmnt mthod (Wb basd Vs. Papr-and- Pncil mthod) Rsponsivnss to chang Ability to dtct changs in knowldg us ovr tim; Examination of Dvlopmntal natur of lvls and catgoris.

8 R R Wb basd group Paprandpncil Group Study (NK) in LOKUS R s a r c h D s i g n T1 (Baslin) T2 (at 1 wk) Intrvntion (CKP) T3 (at 4 wks) A O O X O B O O O C O O O A O O X O B O O O C O O O

9 Mthod: Participants Rprsnt on of th 5 stakholdr typs in th KT4TT Cntr intrvntion projct th Clinicians. Inclusion: Collg Studnts / faculty mmbrs from allid halth disciplins (Occupational thrapy, nursing..) and clinicians xprincd in AAC; 18 yars or oldr; Exclusion: participation in similar survy/focus group within past 6 months. Sampl Siz: Basd on powr analysis. larg ffct siz d =.94 (Colbrt, 1977); N = 64 (ndd); 72 rcruitd, pr-scrnd and randomly assignd to two groups; Final N = 69 (35, papr-and-pncil, 34 wb basd).

10 Mthod: Intrvntion LOKUS itms askd about Nw Knowldg from 3 NIDRR fundd publishd studis in AAC - Study A, Study B and Study C. To idntify Rsponsivnss to changs, a simulatd Intrvntion condition was introducd btwn T2 and T3 by providing a CKP on Study A (tratmnt) only to both groups. and no CKP on th othr two studis (controls).

11 Mthod: Data Analysis Focus Data Analysis Equivalnc of Dmographics on ag, Indpndnt t-tsts and chi-squar tsts. participant groups (WB and P&P) gndr, marital status, highst ducational lvl, xprinc in AAC Tst-Rtst Rliability LOKUS scors at T1 and T2, on lvls and T1 - T2 comparison using ICC (3,1) Compar Altrnat Assssmnt mthods Rsponsivnss to chang Dvlopmntal natur of lvls catgoris. LOKUS scors at T1 and T3, on lvls and catgoris. LOKUS scors at T1 and T3, on lvls. Participant changs in lvl status. Mann-Whitny U-tsts, for diffrnc btwn groups rgarding (1) chang (dichotomous) in lvls and (2) numbr that changd thir lvl status. Idntify significant chang (i.., # of lvls changd >Std Error of Masurmnt); Compar Studis A, B and C. Dvlopmntal, if majority of changs mov up a lvl.

12 Rsults: Dmographics Thr was no significant diffrnc btwn th two groups rgarding: Ag ( t =.757; p =.452); AAC xprinc (t =.880; p =.382); Gndr (chi sq = 1.292; p =.256); Marital status (chi sq = 4.872; p =.301) and Education (chi sq = 1.003; p =.793)

13 StudyA Corr. =1.0) StudyB (Corr.=1.0 ) Study C (Corr.=1.0 ) Lvls t - R t s t R l i a b i l i t y Numbr who maintaind Lvls at T2 Catgoris Non Awarnss 50 n/a n/a n/a n/a n/a n/a 1-Awarnss 12 n/a n/a n/a n/a n/a n/a 2-Orintation n/a 7-Collaboration Non Awarnss 49 n/a n/a n/a n/a n/a n/a 1-Awarnss 7 n/a n/a n/a n/a n/a n/a 2-Orintation n/a 3-Prparation n/a 4- Initial Us 1 n/a 2 n/a n/a 7-Collaboration Non Awarnss 58 n/a n/a n/a n/a n/a n/a 1-Awarnss 8 n/a n/a n/a n/a n/a n/a 2-Orintation n/a Not: Idntical rsponss at T1 & T2

14 Study (NK) Study A Study B Study C Chang from T1 to T3 p o n s i v n s s t o C h a n g T1 Man T3 Man Man Diff. SD SEM f % Total P &P Wb Total P &P Wb Total P &P Wb

15 Study (NK) Study A Study B Study C Not: * Significant; p < 0.05 n c b t w n t h M t h Chang btwn T1 and T3 f % Changd/Not Total P &P Wb Total P &P Wb Total P &P Wb Diffrnc btwn Mthods (p-lvl) Changd Numbr of lvls 0.038*

16 m o v d Lvls at T1 Numbr u who movd up lvls at T3 1 2p Tot 0- Non Awarnss (n=97) l Awarnss (n=45) Orintation (n=41) - -v Prparation (n=18) Initial Us (n=3) - -l Routin Us (n=1) - -s Expansion (n=1) - - a Collaboration (n=0) - - t Intgration (n=1) - - T

17 Summary and Conclusions LOKUS dmonstratd: Good Fac Validity. Excllnt Tst-Rtst Rliability for both lvls & catgoris. Good rsponsivnss to dtct chang rgarding us of Nw Knowldg. Dvlopmntal natur for lowr lvls. LOKUS is a usabl tool at last for AAC fild. Longitudinal study ndd to xtnd conclusion for highr lvls and for catgoris.

18 Ky Rfrncs 1. CIHR. About knowldg translation. Rtrivd Octobr 25, 2009, from 2. Hall, G.E., Dirksn, D.J., and Gorg, A.A. (2006). Masuring Implmntation in Schools: Lvls of Us. Austin, TX: Southwst Educational Dvlopmnt Laboratory (SEDL). 3. Lan, J.P., Ston, V.I., Baur, S. M., Lahy, J.A., and Tomita, M.R. (2008). Cntr on Knowldg Translation for Tchnology Transfr. Proposal submittd to National Institut for Disability and Rhabilitation Rsarch (NIDRR) s Disability and Rhabilitation Rsarch (DRRP) Program (84.133A-7). 4. Sudsawad, P Knowldg Translation: Introduction to Modls, Stratgis, and Masurs. Austin: Southwst Educational Dvlopmnt Laboratory, National Cntr for th Dissmination of Disability Rsarch. (p.4; 21-22) 5. Tlang, S.R. Establishing Psychomtric Proprtis of th Lvl of Knowldg Us Survy (LOKUS) Qustionnair for Knowldg Translation for Tchnology Transfr. (Unpublishd Mastrs Thsis). Univrsity at Buffalo, Stat Univrsity of Nw York.

19 Acknowldgmnt This is a prsntation basd on Shrya Tlang s (2011) work for hr mastrs thsis which was partially supportd by th KT4TT Cntr undr funding by th National Institut on Disability and Rhabilitation Rsarch of th U.S. Dpartmnt of Education, undr grant numbr H133E Th opinions containd in this prsntation ar thos of th grant and do not ncssarily rflct thos of th U.S. Dpartmnt of Education.

20 Thank you!! Contact: Qustions?

21 Appndix Study A Light, J. and Dragr, K. (2007). AAC tchnologis for young childrn with complx communication nds. Stat of th scinc and futur dirctions. Augmntativ & Altrnativ Communication, 23 (3), Study B Light, J., McNaughton, D., Wyr, M. & Karg, L.(2008). Evidnc-basd litracy instruction for individuals who rquiraugmntativ and Altrnativ Communication: A cas study of a studnt with multipl disabilitis. Smin Spch Lang, 29 (2), Study C Quach, W.(2007). Facilitating childrn s larning of Augmntativ and Altrnativ Communication systms. Rtrivd from Proqust Digital Dissrtations. (AAT ).

22 LOKUS and th LoU Scal: Diffrncs Purpos Contxt Framwork Obsrvation Mthod LoU Scal To masur Us of Innovations Educational stting - part of Concrns Basd Adoption Modl (CBAM) 8 Lvls linkd by dcision points; 7 Catgoris in ach lvl Systmatic intrviws LOKUS To masur Us of Innovations Broadr, social stting focus on knowldg from tchnology basd rsarch 10 Lvls, with 3-6 Catgoris undr lvls Wb basd, branchd itms liciting slf rportd rsponss - larg scal survy capability