Projection Based Data Depth Procedure with Application in Discriminant Analysis

Transcription

1 Projectio Based Data Depth Procedure with Applicatio i Discrimiat Aalysis Muthukrisha R 1, Vadivel M, Ramkumar N 3 Departmet of Statistics, Bharathiar Uiversity, Coimbatore, Tamil Nadu, Idia 1,,3 muthukrisha1970@gmail.com 1,chiastatttp@gmail.com,ramu@gmail.com 3 Abstract- Projectio depth ad its associated estimators, amely, Stahel-Dooho (S-D) estimator, Projectio Trimmed Mea (PTM), Projectio depth Cotours (PC) ad Projectio Media (PM) have bee studied i bivariate data. A attempt has bee made to compute projectio depth ad its associated estimators by usig the pair of locatio ad scale estimator (Mea, Stadard Deviatio (SD)), (Media, Media Absolute Deviatio (MAD)), ad (Media, Q ).The efficiecy of these estimators is carried out by computig average misclassificatio error i discrimiat aalysis by usig the projectio depth based Stahel-Doohoestimator uder real ad simulatig eviromet. The study cocluded that (Media, MAD) ad (Media, Q ) based projectio depth estimators performs well whe compared with (Mea, SD). Idex Terms- Projectio depth ad its associated estimators; Robust discrimiatio aalysis. 1. INTRODUCTION Data depth is a cocept which plays a importat role i may otable fields of statistics,amely; data exploratio, orderig, asymptotic distributios ad robust estimatio(liu et al. 1999). The essece of the depth fuctio i multivariate aalyses is to measure degree of cetrality of a poit relative to a data set or to a probability distributio. May robust procedures have bee developed to compute the data depth. The data depth based approach has bee received much attetio ow-a-days. Numerous depth otatios have bee proposed durig the last few decades, amely,half space depth (Tukey 1975), simplicial depth (Liu 1990), regressio depth (Rousseeuw ad Hubert 1999) ad projectio depth (Liu 199; Zuo ad Serflig 000; Zuo 003). The Projectio Depth(PD) is very favorable to the robust statistics whe compared with the other depth otatios.it is due to the reaso that all the desirable properties of the geeral statistical depth fuctio defied i Zuo ad Serflig (000), amely, affie ivariace, maximality at ceter, mootoicity relative to deepest poit, ad vaishig at ifiity are satisfied by the PD. The mai objective of this paper is to estimate the associated estimators such as Stahel- Doohoestimator, projectio trimmed mea, projectio depth cotours ad projectio media for bivariate data based o various pair of projectio depth procedures.further, the performace of the pairs has bee studied uder various levels of cotamiatios with the help of Stahel- Doohoestimator, by computig average misclassificatio probabilities i the cotext of robust liear discrimiat aalysis i Hubert ad Va Driesse (014). The rest of the paper is orgaized as follows. Sectio describes the methodology of projectio depth ad its associated estimators. Sectio 3 discussesrobust liear discrimiat aalysis. Sectio 4 examies the performace ad critically compares the three pairs of projectio depth procedures. Sectio 5 presets results obtaied i real ad simulatio studyi the cotext of robust discrimiate aalysis.the paper eds with coclusio i the last sectio.. PROJECTION DEPTH AND ITS ASSOCIATED ESTIMATORS Zuo (003) itroduced a Projectio-based depth fuctios, which has the highest breakdow poit amog all the existig affie equivariat multivariate locatio estimators ad associated medias. It ca iduce a lot of favorable estimators, such as Stahel-Doohoestimator ad depth weighted meas for multivariate data (Zuo et al. 004; Zuo 006).Further, Zuo (006) studied multidimesioal trimmig based o projectio depth. Exact computatio of bivariate projectio depth ad Stahel- Dooho estimator, with a proper choice of, are formulated ad studied by Zuo ad Lai (011). Liu ad Zuo (014) studied computatioal aspects of projectio depth ad its associated estimators.the brief descriptio of theory of projectio depth is as follows. Let µ (.) ad σ (.)be uivariate locatio ad scale measures, respectively. The the outlyigess of 84

2 a poit x R with respect to the distributio fuctiof of Xdefiedby (Liu 199, Zuo 003). 1 PD(, 1 O( where, O( (1) sup Qu, F, u 1 u T x u where, Q u, F / ad Fu is the F distributio ofu T x. If T u x ( Fu) Fu 0, the defie Q u, F 0, which deotes the projectio of xoto the uit vector u. Note that the most popular outlyig fuctio has the robust choice of µ adσ be themedia ad MAD. Here, the pair (med, Q ), where med ad Q is cosidered as locatio ad scale estimator of (µ(, σ() for a give sample X = {X 1, X,,X } from X.Let F be the correspodig distributio, the the projectio depth ad its associated estimators deped o the robust choice of (Med, Q ), Q(u,X ) i (1) with respect to u. Q X sup Qu, X u 1, () The outlyig fuctio is defied as F T x Med T, u u X X Q T u X Q u (3) Whereu T deotes the projectio ofxoto the uit vector u ad u T X T T T u X 1, u X,..., u X. LetX (1) X () X () deote the order statistics correspodig to the uivariate radom variablesz. Med ( X ) Q X 1 / X /, X d ; i j, X i X j u, ( k) h wheredis a costat factor ad k / 4, h 1 is roughly half the umber of observatios. That is, is the iterpoit distaces ofk th order statistics. The mai fuctio of the projectio depth is to be resposible for a ceter-outward orderig for the bivariate data. Based o this orderig, oe ca make the projectio depth cotours, which ca provide us with a bivariate data of the quatile of a uderlyig distributio (Hali et al. 010). It s defied as PR (, F ) x P R : PD(, (4) where 0 * sup PD( with α th xr P Projectio depth Regio (DR).The the correspodig α th Projectio Depth Cotour ca be distict as the boudary ofpdr (α,uder some coditios (Zuo 003) is give by PC(, x P R : PD(. (5) The iermost depth cotour, which is a sigleto i may situatios, is the Projectio depth Media (PM) of Zuo (003) PM ( F ) PC * (, F ). Based o the projectio depth regiopr (α,,oe ca defie theα th Projectiodepth Trimmed Mea (PTM), (Zuo (006)) as PTM (, PR x, F w w 1 PR(, F ) 1 PD x, F Fdx PD( F F( dx)), (6) wherew 1 (.)is a suitable (boud) weight fuctio o [0, 1]. PTM is highly robustess ad efficiecy α=0 ad the famous degeerates PTM ito the Stahel-Dooho locatio estimators (Stahel 1981; Dooho ad Gasko 199), i.e.theprojectio Weighted Mea (PWM) ad Projectio Weighted Scatter (PWS) PWM PWS F x w1 PD F F dx w,, 1 PD x F F dx (7) x PWMF xpwm F w PD F F dx PD F F dx F w (8) wherepwm( ad PWS(is the aforemetioed Stahel-Dooho locatio ad scatter estimator,w (.) deotes the weight fuctio o [0, 1] based o projectio depth outlyig fuctio (µ(,q ()as respectively.note that the projectio depth ad its, 85

3 associated estimators such asptm (, PWM (adpws (to be well defied, certai mootoy coditios are required as follows: w i PD( Fdx 0, i x wi PDx F F dx,, i 1,. with a fiite sample,..., X X 1, X X from X adf be the correspodig empirical distributio of F based ox. By simply replacig F byf i projectio depth ad its related estimators ca obtai their sample versio. 3. ROBUST DISCRIMINANT ANALYSIS Let p be the variable with observatios that are sampled froml differet populatios π 1,,π l. The discrimiat aalysis settigis i the membership of each observatio with respect to the populatios, i.e., the data poits itolgroups with 1,,, l l observatios. Trivially, j. Therefore, the j 1 the observatios by x ; j 1,..., l; i 1,...,. ij Based o the iitial estimates µ j,0 ad S j,0 are computed for each observatio x ij of group jad its (prelimiary) robust distace is give by RD 0 ij The assig weight 1 to x i if 0 RD ij.,0.975 The reweighted projectio depth estimator for group j is the obtaied as the mediapwm j ad the scatter matrixpws j of those observatios of group j with weight 1. It is show that this reweightig step icreases the fiite-sample efficiecy of the projectio depth estimator cosiderably, whereas the breakdow value remais the same. These robust estimates of locatio ad scatter ow allow us to flag the outliers i the data, ad to obtai more robust estimates of the membership probabilities. First compute the robust distace for each observatio x ij from group j, RDij t 1. t 1. xij xij Oe ca cosidera observatio x ij is a outlier if ad oly if RDij j,0 p,0.975 Further, the projectio depth estimatespwm j ad PWS j are obtaied for each group, ad the the S j xij. j,0 j idividual covariace matrixes are pooled together for further computatio. Let j deote the umber of o-outliers i group j, ad l j, the the robustly estimate the j 1 membership probabilities as R j Pj. Note that the usual estimates implicitly assume that all the observatios have bee correctly assiged to their group. It is however also possible that typographical or other error has occurred whe the group umbers were recorded. The observatios that are accidetly put i the wrog group will the probably show up as outliers i that group, ad so they will ot ifluece the estimates of the membership probabilities. Of course, if oe is sure that this kid of error is ot preset i the data, oe ca still use the relative frequecies based o all the observatios. 4. RESULTS AND DISCUSSION 4.1. Simulatio (Computig Projectio Depth values) A simulatio study is performed to compare the efficiecy of the various otios of projectio depth procedures. To illustrate this 5 sample poits are simulated from multivariate ormal distributio with the mea vector µ= (1,1) ad the covariace matrix I The obtaied fiite umber of optimal directio vectors uder the exact projectio depth values of the sample poits with respect to the data cloud χ is reported i Table 1 ad is give i Appedix. For the sake of compariso, it is also computed the approximate projectio depth values based o radom directio vectors. It is observed from the table, the exact projectio depth values are almost greater tha the radom projectio depth values by cosiderig all the pairs. Further it is oted that the exact projectio depth values is greater tha the radom projectio depth values produced by the pair (Mea, SD). It is cocluded that the pairs (Media, MAD) ad (Media, Q ) produces similar depth values uder exact ad radom projectios. PWM j, 0 PWS j, 0 xij PWM j, 0 The projectio depth size plots uder exact ad radom projectios are displayed i the figure 1.It is oted that the size of the plotted poits is icreases whe the depth values icreases. That is, the plotted poitsare i bigger size whe the depth value is large. Agai, thedepth cetral poits are largerelative to those o the skirts. This is a cofirmatio that the 86

4 projectio depth provides a ceter-outward orderig for the give data cloud. Mea,SD Media, MAD Media, Q Exact projectio s Radom projectio s Figure 1Projectio Depth-Size Plots 4.. Simulatio (Computig Projectio-based depth ad its associated estimators) I order to compare the projectio-based depth ad its associated estimators, 100 datapoits were geerated from the ormal distributio with the 1 0 mea vector µ=(1, 1) ad covariace matrix. 0 1 Further, the locatio ad scale estimates for the geerated data is computed which are as follows:µ = (1.171,1.039),Med = (1.1476,1.0706) ad which are mea, media ad covariace respectively. The estimated Projectio based Media, Weighted Media ad Trimmed Media uder the three pairs (Mea, SD), (Media, MAD) ad (Media, Q ) are summarized i table ad 3. Further, the study was exteded with cotamiatio. The data geerated with µ = (-4, -4), = 4I P, ad the level of cotamiatio 5%, 10% ad 15% were cosidered, ad the the same experimet was performed. For the cotamiated data, the computed locatio ad scatter values mea, media ad covariace are µ = (-0.593, -0.70), med = ( ,-0.386) ad respectively.the estimated Projectio based Media, Weighted Media ad Trimmed Media uder the three pairs (Mea, SD), (Media, MAD) ad (Media, Q ) are also summarized i table ad 3. Error TableEstimated Projectio Depth Locatio Estimators (with/without cotamiatio) Estimat Projectio Depth Procedures ors (Mea, SD) (Media, MAD) (Media, Q ) PM (1.171, 1.039) (1.0807, 1.074) (1.0830, ) PWM (1.136, ) (1.140, ) (1.136, ) PTM (1.1463, 1.050) (1.139, ) (1.1463, 1.050) PM (0.908,0.6899) (1.0198,0.8114) (1.011,0.8138) PWM (1.091,0.8099) (1.0458,0.8360) (1.0501,0.8346) PTM (1.0893,0.8661) (1.1041,0.8980) (1.106,0.8844) PM (1.171,1.039) (1.085,1.0593) (1.0830,1.0661) PWM (1.1359,1.0366) (1.13,1.041) (1.135,1.0457) PTM (1.1499,1.043) (1.161,1.0388) (1.1464,1.0519) PM (0.3484,0.410) (0.759,0.588) (0.8110,0.6337) PWM (0.6903,0.4843) (0.798,0.585) (0.8904,0.660) PTM (0.8037,0.5591) (1.0345,0.7809) (1.1385,0.8555) Table3Estimated Projectio depth weighted Scatter Estimators (with/withoutcotamiatio) Error Projectio Depth Procedures (Mea, SD) (Media, MAD) (Media, Q ) It is observed from the above tables, the estimated locatio ad scatter values are close to the

5 actual value uder the three pair of estimators whe there is o cotamiatio. Further, it is oted that the pair (Media, Q ) ca tolerate certai amout of cotamiatio, specifically, oe ca see that the cotamiatio level is 15%, the results get affected uder the pairs (Mea, SD) ad (Media, MAD) but ot i the case of (Media, Q ). It is cocluded that, the impact of the outliers o(media, Q) are very limited.the estimated locatio poits uder the three pairs alog with data poits with/without cotamiatios are displayed i the form of scatter plots i Figure.. Figures reveal that the ordiary mea is placed outside the bulk of the data poits by a few outliers; while other projectio depth based locatio estimators are positioed amog the majority of the data. (Mea,SD) (Media, MAD) (Media, Q ) e=0.00 e=0.05 e=0.10 e=0.15 Figure Projectio Depth-Size Plots (PWM, PWS) 88

6 The locatio ad scatter estimators cofirm high robustess of projectio depth ad its associated estimators (Zuo 003, 006).It is worthy to ote that, durig the computatio of PWM, PTM ad PWS; weight fuctiosw i (.), i =1,,used here as suggested by Zuo ad Cui (005). Further the projectio depth cotours applied to various projectio-based depth procedures uder various level of cotamiatios are display i figure 3 ad is give i Appedix. The cotours idicate a similarity i the structures of the projectio depth procedures (Media, MAD) ad (Media, Q ) which are both ulike the results of the procedure (Mea, SD). 5. APPLICATIONS IN DISCRIMINANT ANALYSIS 5.1. Real data This sectio presets the performace of projectio depth based SDE i robust liear discrimiat aalysis by computig misclassificatio probabilities with three pairs of locatio ad scatter approach. It is cosidered a data set with two groups (Johso ad Wicher (009)). The data descriptio is as follows: Two differet groups: π 1 is ridigmover owers ad π is without ridig movers to idetify the best sales prospects. The owers or o-owers o the basis ofthe variables x 1 (icome), x (lot size), radom sampleofsize 1 (=1) curret owers ad (=1) curret o-owers respectively. Discrimiat aalysis for these two groups is performed ad computed misclassificatio probabilities uder various projectio depths based approaches ad is give i table 4. Table 4Computed misclassificatio probabilities uder various projectio depth Procedures Misclassificatio Probabilities π 1 π Average (Mea, SD) (Media, MAD) (Media, Q ) The estimated average misclassificatio probabilities are almost same except the procedure(media, MAD). 5.. Simulatio study This sectio presets the results obtaied uder various projectio depth based approaches uder simulatig eviromet with/without cotamiatios (Locatio ad Scale). I this cotext, two groups (g=) with two variables (p=) are cosidered to simulate the data. The data were geerated uder theormal distributio which hasthe covariace matrices 1 =I P ad = I P with meas µ 1 = (1, 1) ad µ = (5, 5)uder sample sizes of 50 ad 100. The various levels of cotamiatios such as 5%, 10%, 15%, 0%, 5%, 30%, 35% ad 40% were cosidered i all cases. The obtaied results with the cotamiatio levels 0%, 5%, 10% ad 15% are same ad the results based o the remaiig cotamiatios are displayed i the table 5. Table5Computed misclassificatio probabilities uder various projectio depths with cotamiatios 1 = =50 1 = =100 Eror (Mea, SD) (Media, MAD) (Media, Q ) (Mea, SD) (Media, MAD) (Media, Q ) O comparig the average probability of misclassificatio values i the above table, it is evidet that the procedures (Media, MAD) ad (Media, Q ) produces less whe compared with (Mea,SD). Also, it is observed that whe sample size icreases the misclassificatio probabilities decreased uder all the procedures. It is cocluded that the procedure (Media, Q ) performs better tha the other two procedures. It shows that it is superior to the other two procedures whe the level of cotamiatio icreases. 6. CONCLUSION Locatio ad scatter estimator play vital role i almost all statistical data aalyses. The covetioal estimates, sample mea vector ad covariace matrix are very sesitive whe the outlyig observatios i the data. I order to obtai the reliable locatio ad scatter estimate, data depth approaches attract the researchers ow-a-days. This paper proposes a projectio based data depth approach to compute locatio ad scatter estimate, amely (Media, Q ). Further the superiority of the proposed estimator has bee studied uder real ad simulatio by applyig it i discirmiat aalysis by computig the misclassificatio probabilities with various other projectio based depth approaches (Mea, SD) ad 89

7 (Media, MAD). The simulatio study shows that the projectio depth based o the mea ad stadard deviatio fails to produce reliable results whe compared with the other projectio depth procedures. It is oted that (Media, MAD) ad (Media, Q ) performs well over the (Mea, SD). The study cocluded that the proposed projectio depth procedure (Media,Q )shows that its superiority over the other procedures (Mea, SD) ad (Media, MAD), i the cotext of tolerace level of cotamiatios ad misclassificatio rate. Ackowledgemet This research work was fuded by the Rajiv Gadhi Natioal Fellowship (RGN programme, UGC, New Delhi ad carried out at Departmet of Statistics, Bharathiar Uiversity, Tamil Nadu ad Idia. REFERENCES 1. Dooho, D. L., ad Gasko, M. (199). Breakdow properties of locatio estimates based o halfspace depth ad projected outlyigess.. The Aals of Statistics. 0, Hali, M., Paidaveie, D., ad Sima, M. (010). Multivariate quatiles ad multiple-output regressio quatiles: From L1 optimizatio to halfspace depth. The Aals of Statistics, 38(), Hu, Y., Li, Q., Wag, Y., ad Wu, Y. (01). Rayleigh projectio depth. Computatioal Statistics, 7, Hubert, M. ad Va Driesse, K. (004). Fast ad Robust Discrimiat Aalysis. Computaioal Statistics & Data Aalysis, 45,301, Johso, R.A., ad Wicher, D.W (009). Applied multivariate aalysis, 5 th ed. Pretice Hall, Eglewood Cliffs, New Jersey. 6. Liu, R.Y. (1990). O a otio of data depth based o radom simplices. The Aals of Statistics, 18, Liu, R.Y. (199). Data depth ad multivariate rak test. I: L1- Statistical Aalysis ad Related Methods, North-Hollad, Amsterdam. 8. Liu, R.Y., Parelius, J.M., ad Sigh, K. (1999). Multivariate aalysis by data depth: Descriptive Statistics, Graphics ad Iferece. The Aals of Statistics, 7, Liu, X., ad Zuo, Y. (014). Computig projectio depth ad its associated estimators. Stat.Comput., 4, Liu, X.H., Zuo, Y.J., Wag, Z.Z. (011). Exactly computig bivariate projectio depth cotours ad media. Preprit. 11. Rousseeuw, P.J., ad Hubert, M. (1999). Regressio depth. Joural of the America Statistical Associatio, 94, Stahel, W.A. (1981). Breakdow of covariace estimators. Research Report 31, Tukey, J.W. (1975). Mathematics ad the picturig of data. I: Proceedigs of the Iteratioal Cogress of Mathematicias, pp Caadia Mathematical Cogress, Motreal. 14. Zuo, Y. (003). Projectio-based depth fuctios ad associated medias, The Aals of Statistics, 31, Zuo, Y., ad Serflig, R. (000). Geeral otios of statistical depth fuctio. The Aals of Statistics, 8, Zuo, Y.J. (006). Multidimesioal trimmig based o projectio depth. The Aals of Statistics, 34, Zuo, Y.J., ad Lai, S.Y. (011). Exact computatio of bivariate projectio depth ad Stahel-Dooho estimator. Computatioal Statistics & Data Aalysis, 55(3), Zuo, Y.J., ad Cui, H.J. (005). Depth weighted scatter estimators. The Aals of Statistics, 33, Zuo, Y.J., Cui, H.J., He, X.M. (004). O the Stahel-Dooho estimators ad depth weighted meas for multivariate data. The Aals of Statistics, 3(1), Appedix A. Id ex Table 1 Computed Exact ad Radom Projectio Depth Values (Mea, SD) ExPDV RdPDV (50000) (Med (Med (Med (Mea (Med ia, ia, ia,, ia, MAD MAD Q ) ) SD) Q ) )

8 (Mea,SD) (Media, MAD) (Media, Q )

9 Figure 3Projectio Depth Cotour Plots uder various procedures with level of cotamiatios 83