Filter Bank Feature Combination (FBFC) approach for Brain-Computer Interface

Transcription

1 Proceedngs of Internatonal Jont Conference on Neural Networks San Jose Calforna USA July 31 August Flter Bank Feature Combnaton (FBFC) approach for Bran-Computer Interface Zheng Yang Chn Ka Keng Ang Cunta Guan Chuanchu Wang Hahong Zhang Abstract he Flter Bank Common Spatal Pattern (FB) algorthm constructs and selects subject-specfc dscrmnatve features from a flter bank of spataltemporal flters n a motor magery bran-computer nterface (MI-BCI). However nformaton from other types of features could be etracted and combned wth features to enhance the classfcaton performance. Hence ths paper proposes a Flter Bank Feature Combnaton (FBFC) approach and nvestgates the use of and Phase Lock Value (PLV) features where the latter measures the phase synchronzaton between the EEG electrodes. he performance of the FBFC usng and PLV features s evaluated on four-class motor mageres from the publcly avalable BCI Competton IV Dataset IIa. he epermental results showed that the proposed FBFC usng and PLV features yelded a sgnfcant mprovement n cross-valdaton accuraces on the tranng data (p=0.008) and better sesson-to-sesson transfer accuraces to the evaluaton data compared to the use of features usng the FB algorthm. hs motvates the research of FBFC usng a battery of other features that could possbly beneft EEG-based BCIs and mult-modal BCI systems. I I. INRODUCION n the use of an electroencephalogram (EEG)-based Motor Imagery Bran-Computer Interface (MI-BCI) the subject performs the magnaton of movement from the frst-person perspectve wthout actually eecutng t [1]. As llustrated n the human homunculus [2] dfferent body parts have a spatally ordered layout n the prmary corte. Hence the magnaton of dfferent body part movements such as the hands feet or tongue nduces spatal changes n the EEG. o dscern these spatal changes n the EEG to the types of motor magery acton varous sgnal processng and machne learnng algorthms have been proposed. Such methods etract useful nformaton from the EEG as feature vectors for eample band power estmates [3] autoregressve (AR) models [4] Phase Lock Value (PLV) [5] and Common Spatal Pattern () [6]. he PLV feature quantfes the phase synchronzaton between the EEG electrodes and results suggest that t contans useful nformaton for dscernng the types of motor magery acton [7]. he algorthm computes spatal flters that mamze the varance hs work was supported by the Scence and Engneerng Research Councl of A*SAR (Agency for Scence echnology and Research) Sngapore. Z. Y. Chn K. K. Ang C. Guan C. Wang and H. Zhang are wth Insttute for Infocomm Research Agency for Scence echnology and Research (A*SAR) 1 Fusonopols Way #21-01 Connes (South ower) Sngapore (emal: {zychn kkang ctguan hhzhang ccwang}@2r.a-star.edu.sg). between two condtons such as left-hand and rght-hand motor magery. However as the effectveness of the algorthm depends on subject-specfc temporal flterng parameters the Flter Bank Common Spatal Pattern (FB) algorthm was proposed to address ths ssue [8]. he FB algorthm autonomously performs the selecton of key temporal-spatal dscrmnatve features that are specfc to a subject. hs algorthm yelded the best classfcaton performance relatve to the other submssons n the four-class motor magery data from the BCI Competton IV [9] Dataset IIa [10]. Several feature combnaton approaches have been eplored to mprove the performance of MI-BCI n the lterature. In [11] the gamma band power estmates feature were combned wth the slow cortcal potentals (SCP) feature to yeld better performance than the use of only the SCP feature. In [12] [13] the Autoregressve (AR) feature feature and movement related potental (MRP) feature were combned usng varous strateges and yelded mproved performance compared to the use of the ndvdual features. In [5] [14] the PLV feature and the band power estmates feature were combned and also yelded mproved performance compared to the use of the ndvdual features. hus the feature combnaton approach enhances the MI-BCI classfcaton performance. However the FB algorthm employs the flter bank approach to etract only features. Hence ths paper proposes a Flter Bank Feature Combnaton (FBFC) approach and nvestgates the use of the features and the PLV features. he FBFC approach employs a four-stage process: Frst band-pass flterng usng a flter bank to etract frequency components of the EEG; Second feature etracton to etract dfferent types of EEG features; hrd feature combnaton to select the most nformatve features from each type of feature usng a mutual nformaton crteron and to perform feature transformaton; Fnally classfcaton s performed on the transformed feature vectors. he performance of the proposed FBFC employng the features and the PLV features s nvestgated and compared wth the FB algorthm employng only the feature on the four-class sngle tral motor magery data from the publcly avalable BCI Competton IV dataset IIa [10]. II. FILER BANK COMMON SPAIAL PAERN (FB) he FB algorthm [8] comprses four stages that /11/$ IEEE 1352

2 perform an autonomous selecton of subject-specfc temporal-spatal dscrmnatve EEG characterstcs for twoclass MI-BCI shown n Fg. 1. Sngle tral EEG data 4 to 8Hz 8 to 12Hz 36 to 40Hz Frequency Flterng Spatal Flterng MIBIF4 Feature Selecton NBPW Classfcaton Predcted Motor Imagery Acton Fg. 1: Archtecture of the Flter Bank Common Spatal Pattern (FB) algorthm for two-class motor magery EEG data. MIBIF4 and NBPW represent the Mutual Informaton Best Indvdual Feature and the Naïve Bayes Parzen Wndow classfer respectvely. A. Band-pass flterng he frst stage employs 9 band-pass flters that decompose the EEG nto ts respectve frequency components from 4-8Hz 8-12Hz 36-40Hz. Varous confguratons of the flter bank are as effectve but these band-pass frequency ranges are employed as they cover the range of 4-40Hz and encompasses the alpha/mu and beta bands. hese frequency bands have been shown to ehbt Event-Related Desynchronzaton / Synchronzaton (ERD/ERS) effects durng motor magery [13] [15] [16]. B. Spatal flterng and feature etracton he second stage performs spatal flterng usng the algorthm [6] by applyng a lnear transformaton on the EEG Z = WE (1) b b b c t where Eb = È Î e e e Œ b 1 b2 bc denotes the th sngletral EEG from the b th band-pass flter; È Î e t = e (1) e (2)... e ( t ) Œ denotes the fltered b l b l EEG sgnal from the l th c t EEG channel; Z b Œ denotes c c E b after spatal flterng; W b Œ denotes the projecton matr for the b th band-pass flter; c s the number of channels; τ s the number of EEG tme samples per channel; and denotes the transpose operator. he features from Z b are then gven by where ( ( ) / tr È ) vb = log dag W b Eb Eb W b Î W b Eb Eb W b (2) 2m v b Œ ; b W represents the frst m and the last m columns of whch mamze the dfferences n the varances between 2 classes of motor magery acton; dag( ) returns the dagonal elements of a square matr; tr[ ] returns the sum of the dagonal elements n the square matr. Hence the FB feature vector for the th tral s represented as v = È Î v1 v2 v9 (3) where v Œ (9*2m). he FB feature vectors from the tranng data are gven as = È Î 1 2 nt nt ( 9*2m) V Œ ; [ ] ( ) V v v v (4) 9*2m where v = v1 v2... v9*2m Œ denotes the feature vector from the th tral n the tranng data; = 1 n t ; n t denotes the total number of trals n the tranng data. C. Mutual nformaton- based feature selecton he thrd stage performs feature selecton of the etracted features usng the Mutual Informaton-based Best Indvdual Features (MIBIF) algorthm [17] on the tranng data. hs algorthm selects the best k features that results n the hghest estmate of mutual nformaton wth the class labels. he correspondng features whch come n pars wth the selected k features are also selected. Based on the study n [8] k = 4 s used. Denotng the set of features and the true class labels from the tranng data F = È Î f 1 f 2... f 9*2m = V where n t 1 f q Œ s the q th column vector of V the mutual nformaton between feature ω={12} s gven by where ( ) fq wth the class label class I ( q; w) = H( w) - H( w q ) H w and H ( w fq ) f f (5) denotes the entropy and condtonal entropy respectvely. he detals on the computaton of these two functons are covered n [17]. After performng feature selecton on V the tranng data n d wth selected features s denoted as X Œ where d ranges from 4 to 8. Hence the FB feature vector for the th tral after feature selecton s performed s represented as csp = È Î csp1 csp2 csp d (6) d where csp Œ. D. Classfcaton he fourth stage performs classfcaton usng the Naïve Bayes Parzen Wndow (NBPW) Classfer [17] and the classfcaton rule for two-class motor magery s gven as yˆ = arg ma p w (7) j w = 12 ( csp j) where y ˆ j denotes the predcted label of the j th evaluaton tral p w the class ω={12}. csp j ; ( csp j ) csp denotes the posteror probablty of III. PHASE LOCK VALUE (PLV) FEAURE EXRACION he Phase Lock Value (PLV) s a measure of the synchronzaton n phase between two tme sgnals [5] [14]. It ranges from 0 to 1 where 0 represents no phase synchronzaton and 1 represents perfect phase synchronzaton. he PLV feature s computed as follows: After band-pass flterng a common average reference (CAR) spatal flter [18] s appled on the EEG electrodes. 1353

3 the Hlbert transform of the EEG sgnal of the l th EEG channel from the b th band-pass flter e s computed ( l ) e 1 e () t = PV d l p Ú - t - l (8) where PV denotes the Cauchy prncpal value. he nstantaneous phase s then computed as follows. e ( t) F () arctan bl bl t = e t (9) he PLV between two sgnals at two channels denoted as channel 1 and channel 2 s gven as t t = 1 () ( j{ 1 ( t) 2 () t }) t 1 PLV =  ep Fb -Fb (10) where t represents the current tme sample and τ represents the total tme samples. he PLV s averaged over the tme samples n each sngle tral. Hence the PLV feature vector for the th tral s represented as u = È Î u1 u2 u9 (11) ( n ) ; n f = c (c-1)/2 represents the number of 9* f where u Œ PLV features over all channel pars per frequency band; and c s the number of channels. Smlarly feature selecton usng the MIBIF algorthm s performed on the etracted PLV features. Hence the PLV feature vector for the th tral after feature selecton s performed s represented as plv = È Î plv1 plv2 plv k (12) k where plv Œ. IV. PROPOSED FILER BANK FEAURE COMBINAION (FBFC) he proposed Flter Bank Feature Combnaton (FBFC) approach combnes the features and the PLV features. It employs a four-stage process to: frst band-pass flter the EEG usng a Flter Bank; second etract and PLV features; thrd perform feature combnaton usng feature selecton and feature transformaton; and fnally classfy the transformed feature vector as shown n Fg. 2. A. Flter Bank and Feature Etracton he EEG data s frst band-pass fltered nto nne frequency components. After band-pass flterng features and PLV features are etracted from the frequency components as descrbed n Secton II and Secton III. B. Feature Combnaton After feature etracton feature selecton s performed on each type of the etracted features n the tranng data. he MIBIF algorthm selects the best k features from each type of features that results n the hghest estmate of mutual nformaton wth the class labels. Based on the results n [8] k = 4 s used for the selecton of features. k = 4 s also arbtrarly chosen for the selecton of PLV features n ths study. Sngle tral EEG data 4 to 8Hz 8 to 12Hz 36 to 40Hz Flter Bank 4 to 8Hz 8 to 12Hz 36 to 40Hz Flter Bank PLV PLV PLV PLV Feature Etracton Feature Etracton MIBIF4 + MIBIF4 FLD Feature Combnaton NBPW Classfcaton Predcted Motor Imagery Acton Fg. 2: Archtecture of the Flter Bank Feature Combnaton (FBFC) approach to combne nformaton from dfferent types of EEG features. In ths study features and PLV features are combned for two-class motor magery EEG data. MIBIF4 FLD and NBPW represent the Mutual Informaton Best Indvdual Feature the Fsher Lnear Dscrmnant and the Naïve Bayes Parzen Wndow classfer respectvely. For mult-class motor magery the one-versus-rest (OVR) approach s employed where classfers that dscrmnate one class aganst the other classes are constructed. he predcted motor magery acton depends on the mamum posteror probablty output from the component classfers. Hence the concatenated feature vector for the th tral s represented as = È Îcsp plv = È Î1 2 ( d + k) (13) where d ranges from 4 to 8 n ths paper as eplaned n Secton II.C. Feature transformaton s performed on the concatenated feature vector to reduce the feature dmenson. he FBFC employs the Fsher Lnear Dscrmnant (FLD) [19] on the concatenated feature vector to form a one-dmensonal feature vector for the th tral g = w fld (14) 1 ( d k) where w + fld Œ s the projecton vector; and w fld mamzes the fsher crteron a rato of between-class to wthn-class varance. C. Classfcaton and One-Versus-Rest (OVR) approach he FBFC employs the NBPW classfer to classfy the transformed feature vector of the j th evaluaton tral. yˆ = arg ma p w g (15) j w = 12 ( j) In mult-class MI-BCI the FBFC adopts the One-Versus- Rest (OVR) approach where classfers for each class of motor magery versus all the other classes are constructed. For a four-class MI-BCI four OVR classfers are requred. Hence the classfcaton rule of the NBPW classfer s thus etended from equaton (15) to yˆ j = arg ma povr w g j w (16) where povr ( g j w ) w = 1234 ( ) w s the probablty of classfyng the j th evaluaton tral between the motor magery class ω and class ω = { }\ ω; \ denotes the set theoretc dfference operaton; and g j w represents the transformed feature vector for the ω th OVR classfer. 1354

4 V. EXPERIMENAL RESULS he FBFC approach was evaluated on the four-class sngle-tral motor magery data from the BCI Competton IV dataset IIa [10] where one tranng sesson and one evaluaton sesson of EEG data from nne subjects are provded. Each sesson comprsed of 288 sngle trals wth an equal dstrbuton of left hand rght hand foot and tongue motor magery. Fg. 3 shows how each tral of motor magery s conducted. At the start of each tral a faton cross s dsplayed on the computer screen for 2s. Subsequently a vsual cue nstructs the subject to perform left-hand rghthand foot or tongue motor magery for 4s followed by a break perod of varable length before the net tral. o tran the algorthm the segment of 0.5s to 2.5s of EEG data after the onset of the vsual cue was used. More detals of the protocol are avalable n [10]. ranng Phase Evaluaton Phase test_tme_segment -4 to 4 Faton Cross 2s wndow 2s wndow 2s wndow tran_tme_segment 0.5 to 2.5 Vsual Cue Motor Imagery 2s wndow Break (varable length) Classfcaton Output ( ) 0 Fg. 3: he eperment protocol for a sngle tral of motor magery n the four-class motor magery data from the BCI Competton IV Dataset IIa. o tran the varous algorthms under study the tme segment tran_tme_segment was used. he performance of the algorthms were evaluated on the entre segment of the sngle tral EEG data n test_tme_segment usng sldng tme wndows of length tran_tme_segment All 22 channels of EEG data were used to etract features. he choce of m for the algorthm n equaton (2) was set to 2. hs s because a greater choce of m dd not sgnfcantly mprove classfcaton accuracy [6]. Only 10 out of 22 channels of EEG data as shown n Fg. 4 were used to etract PLV features. hs s to reduce the amount of processng requred to etract the features. If all 22 channels of EEG data were used nstead and there would be c (c-1)/2 = 231 features per frequency band makng that a total of = 2079 features nstead. Fg. 4: PLV features were etracted from the 10 EEG channels whch have been shaded n ths electrode map. A. Performance Measure he competton performance measure used was the mamum kappa value κ to be consstent wth the performance measure employed durng the BCI Competton IV. he kappa value s computed from the BIOSIG toolbo p0 - pe k = (17) 1- p where p 0 denotes the classfcaton accuracy and p e s the chance epected agreement. Classfcaton accuracy by chance and perfect classfcaton would have a kappa value of 0 and 1 respectvely [4]. he algorthm was evaluated on the entre sngle-tral EEG from the onset of the faton cross usng a sldng wndow of 2s. In ths study only the data from the same subject s used to evaluate the performance of the algorthm. hs s carred out n two parts. In the frst part 10 runs of 10-fold ( fold) cross-valdaton s performed on tranng data. In each run the EEG data etracted for all the 288 trals are randomly splt nto 10 equal portons of whch 9 portons are used as tranng data and the remanng porton as valdaton data. he mamum kappa value over 10-folds s noted. hs process s then repeated for 10 runs by randomzng the manner n whch the 288 trals are dvded nto 10 portons. he cross-valdaton result of the subject s then computed from the averaged kappa value of all 10 runs. In the second part a sesson-to-sesson transfer from the tranng data to the ndependent evaluaton data s performed. he algorthm uses the EEG data from the frst tranng sesson for tranng. he results of evaluatng the algorthm on the EEG data from the second evaluaton sesson are then presented. B. Classfcaton Results he fold cross-valdatons results on the tranng data are shown n terms of mean valdaton kappa value n able I. he FBFC approach that employs both features and PLV features outperforms the FB algorthm that employs the features only and the PLV algorthm that employs the PLV features only. Statstcal analyss usng the pared t-test between the FBFC and the FB algorthm showed that the former performs relatvely better than the other (p-value = 0.008). he sesson-to-sesson transfer performance of the FBFC approach on the evaluaton data n terms of kappa values s shown n able II. he results of the 2nd and 3rd placed submssons n the BCI Competton IV for ths dataset are also lsted and detals of ther methods can be found n [9]. Although not statstcally sgnfcant the FBFC approach also outperforms the FB algorthm n averaged kappa value over all nne subjects. e 1355

5 ABLE I KAPPA VALUE RESULS FROM FOLD CROSS-VALIDAIONS ON HE RAINING DAA OF HE BCI COMPEIION DAASE IIA USING HE PROPOSED FILER BANK FEAURE COMBINAION (FBFC) USING FILER BANK COMMON SPAIAL PAERN (FB) AND PHASE LOCK VALUE (PLV). CLASSIFICAION ACCURACY BY CHANCE AND PERFEC CLASSIFICAION WOULD HAVE A KAPPA VALUE OF 0 AND 1 RESPECIVELY SUBJEC FBFC FB PLV AVG ABLE II SESSION-O-SESSION RANSFER PERFORMANCE IN ERMS OF KAPPA VALUE ON HE EVALUAION DAA OF HE BCI COMPEIION IV DAASE IIA USING HE PROPOSED FBFC AND FB APPROACHES. RESULS FROM HE 2ND AND 3RD PLACED SUBMISSION HAVE ALSO BEEN INCLUDED. EVALUAION SUBJEC FBFC FB 2ND 3RD AVG VI. DISCUSSION AND CONCLUSION Feature combnaton s an etensve area of research wth applcatons n dfferent areas [20]. Epermental results n studes [5] [11] [12] [14] [21] showed that feature combnaton yelded mproved classfcaton performance for Bran-Computer Interfaces (BCIs). hus ths paper proposed a Flter Bank Feature Combnaton (FBFC) approach to nvestgate the use of the Common Spatal Pattern () feature and the Phase Lock Value (PLV) features. he performance of the proposed FBFC s compared wth the Flter Bank Common Spatal Pattern (FB) algorthm that used only features. he results on the four-class motor magery data from the BCI Competton IV Dataset IIa showed that the proposed FBFC approach that combnes the and the PLV features outperformed the FB algorthm that used only the features n terms of crossvaldaton accuracy on the tranng data and sesson-tosesson transfer on the evaluaton data. Snce there are a varety of features that can be etracted from bran actvty the challenge s to nvestgate the effectveness of usng the proposed FBFC approach usng varous types of features. hs could be appled n hybrd BCIs [22] or mult-modal BCIs where smultaneous measurements of bran actvty such as Near Infra-red Spectroscopy (NIRS) and EEG are avalable. hs could also be appled when features are etracted from other physologcal sgnals such as the ECG [23]concurrently measured wth EEG n hybrd BCIs. Hence the results from the proposed FBFC motvates further nvestgaton to use other types of features as well as other types of feature combnaton technques to mprove the classfcaton performance. ACKNOWLEDGMEN he authors would lke to thank the organzers [9] and the dataset provders of the BCI Competton IV dataset IIa [10] REFERENCES [1] H. H. Ehrsson S. Geyer and E. Nato "Imagery of Voluntary Movement of Fngers oes and ongue Actvates Correspondng Body-Part-Specfc Motor Representatons" J. Neurophysol. vol. 90 pp [2] W. Penfeld and. Rasmussen he Cerebral Corte of Man. New York N.Y: he Macmllan Company [3] G. Pfurtscheller and C. Neuper "Motor magery and drect brancomputer communcaton" Proc. IEEE vol. 89 pp [4] A. Schlogl F. Lee H. Bschof and G. Pfurtscheller "Characterzaton of four-class motor magery EEG data for the BCI-competton 2005" J Neural Eng p. L [5] E. Gysels and P. Celka "Phase synchronzaton for the recognton of mental tasks n a bran-computer nterface" IEEE rans. Neural Syst. Rehabl. Eng. vol. 12 pp [6] H. Ramoser J. Muller-Gerkng and G. Pfurtscheller "Optmal spatal flterng of sngle tral EEG durng magned hand movement" IEEE rans Rehabl Eng vol. 8 pp [7] L. Song E. Gysels and E. Gordon "Phase synchrony rate for the recognton of motor magery n bran-computer nterface" n Proc. Proc. Advances Neural Inf. Processng Systems (NIPS 05) 2006 pp [8] K. K. Ang Z. Y. Chn H. Zhang and C. Guan "Flter Bank Common Spatal Pattern (FB) n Bran-Computer Interface" n Proc. IEEE Int. Jont Conf. Neural Netw. Hong Kong 2008 pp [9] B. Blankertz "BCI Competton IV" [10] C. Brunner M. Naeem R. Leeb B. Gramann and G. Pfurtscheller "Spatal flterng and selecton of optmzed components n four class motor magery EEG data usng ndependent components analyss" Pattern Recogn. Lett. vol. 28 pp [11] B. D. Mensh J. Werfel and H. S. Seung "BCI competton 2003-data set Ia: combnng gamma-band power wth slow cortcal potentals to mprove sngle-tral classfcaton of electroencephalographc sgnals" IEEE rans. Bomed. Eng. vol. 51 pp [12] G. Dornhege B. Blankertz G. Curo and K.-R. Müller "Combnng Features for BCI" n Proc. Advances Neural Inf. Processng Systems (NIPS 02) 2003 pp [13] G. Dornhege B. Blankertz G. Curo and K. R. Muller "Boostng bt rates n nonnvasve EEG sngle-tral classfcatons by feature combnaton and multclass paradgms" IEEE rans. Bomed. Eng. vol. 51 pp [14] C. Brunner R. Scherer B. Gramann G. Supp and G. Pfurtscheller "Onlne Control of a Bran-Computer Interface Usng Phase Synchronzaton" IEEE rans. Bomed. Eng. vol. 53 pp [15] B. Blankertz R. omoka S. Lemm M. Kawanabe and K. R. Müller "Optmzng Spatal flters for Robust EEG Sngle-ral Analyss" IEEE Sgnal Process. Mag. vol. 25 pp [16] G. Pfurtscheller C. Brunner A. Schlögl and F. H. Lopes da Slva "Mu rhythm (de)synchronzaton and EEG sngle-tral classfcaton of dfferent motor magery tasks" NeuroImage vol. 31 pp [17] K. K. Ang and C. Quek "Rough set-based neuro-fuzzy system" n 1356

6 Proc. IEEE Int. Jont Conf. Neural Netw pp [18] D. J. McFarland L. M. McCane S. V. Davd and J. R. Wolpaw "Spatal flter selecton for EEG-based communcaton" Electroencephalogr Cln Neurophysol. vol. 103 pp Sep [19] R. O. Duda P. E. Hart and D. G. Stork Pattern Classfcaton Second Edton ed.: John Wley & Sons Inc [20] J. Yang J.-y. Yang D. Zhang and J.-f. Lu "Feature fuson: parallel strategy vs. seral strategy" Pattern Recognton vol. 36 pp [21] G. Dornhege B. Blankertz G. Curo and K.-R. Müller "Increase nformaton transfer rates n BCI by etenson to mult-class" n Proc. Advances n Neural Informaton Processng Systems 2004 pp [22] G. Pfurtscheller B. Z. Allson G. Bauernfend C. Brunner. Sols Escalante R. Scherer et al. "he hybrd BCI" Fronters n Neuroscence vol. 5 p Aprl [23] G. Pfurtscheller R. Leeb D. Fredman and M. Slater "Centrally controlled heart rate changes durng mental practce n mmersve vrtual envronment: A case study wth a tetraplegc" Int. J. Psychophysol. vol. 68 pp