Hyper-Geometric Distribution Model to Estimate the Number of Residual Software Faults

Transcription

1 HyperGeoeric Disribuion Model o Esie he Nuber of Residul ofwre uls Yoshihiro TOHMA, Ryond JACOBY, Yukihis MRATA, nd Moriki YAMAMOTO Depren of Copuer cience Tokyo Insiue of Technology MeguroKu, Tokyo Absrc This pper presens he pplicbiliy of he HyperGeoeric Disribuion odel for esiing he nuber of residul sofwre fuls o rel esnddebug d of differen ypes. The relionship of our odel o hose proposed erlier hs been de cler. Nuericl resuls of he esiion by our odel nd he ohers re copred. Inroducion In sofwre developen process, sofwre is designed, coded, esnddebugged, nd finlly relesed o service. or progr of considerble size, he esnddebug is crried ou in generl by es workers independen of he progrers who hve wrien nd debugged preliinry he progr. In spie of he progrers confidence h he progr is copleely debugged, ore fuls re usully found by he es workers in he esnddebug sge. However, since no one knows how ny fuls re residen in given progr he beginning of he esnddebug sge, he es workers re unble o decide definiely when he esnddebug coplees nd he progr cn be pssed o service. Thus, he esiion of he nuber of residul sofwre fuls is n iporn issue no only for giving he es workers n ide bou he ie of he relese of he esnddebugged progr, bu lso o evlue indirecly is dependbiliy. ince he definiion of sofwre dependbiliy iself is sill conroversil, he evluion of sofwre dependbiliy is hrd o ke. However, he esiion of he nuber of residul sofwre fuls fer he esnddebug sge y hopefully give us quliive esure of sofwre dependbiliy. If we cn esie he nuber of fuls iniilly residen in given progr he beginning of he esnddebug sge (hese ful will be referred o s iniilfuls), we cn lso know he esied nuber of fuls he end of he esnddebug sge by observing he nuber of fuls deeced nd reoved (fixed) so fr. Hence, he proble of finding n esied nuber of residul sofwre fuls reduces o esie he nuber of iniil fuls of given progr. o fr he ehods bsed on Goperz curve, he logisic curve, nd NonHoogeneous Poisson Process (NHPP) [I] hve been proposed. However, hey see o lck enough subsniion why hey should be used. Noing he sensiiviy of fuls o es insnces, we de he disincion beween he deecion nd he sensiizion of fuls he pplicion of es insnces, nd proposed hypergeoeric disribuion odel o esie he nuber of iniil fuls []. The one of key conceps of our odel is he sensiiviy fcor w(i) of fuls o es insnces, which is esure o represen how ny fuls will nifes heselves s errors upon he p plicion of es insnce l(i). w(i) cn be regrded in differen wys, depending on he definiion of es insnces of rel esnddebug d, he wy of pplying (i), he nure of (i), nd he wy of couning fuls s well. In [], however, we ssocied w(i) only wih he nuber of es workers involved in (i). The firs i of his pper is o show vrious wys of inerpreing w(i) nd ccordingly, he pplicbiliy of our odel o rel esnddebug d of differen ypes. econdly, he relionship of our odel o hose proposed erlier will be rgued. I will be shown h soe of he cn be inroduced fro our odel by ssuing w(i) ppropriely. Nuericl resuls of he esiion by our odel nd ohers will be copred. Bsic Concep nd Terinology Le us firs review briefly he bsic concep of our odel o esie he nuber of sofwre fuls iniilly residen in given progr. As described previously, he progr hs been debugged by he progrers who wroe i. Afer his preliinry debugging, i will be pssed o es workers for furher rigorous esnddebugging. We rgue he deecion nd reovl (fixing) of fuls his esnddebug sge only. Th is, we ssue progr sill hs iniil fuls when he esnddebugging by he es workers begins. A se of iniil fuls will be denoed by. We cll se of es insnces he es. Tes insnces will be denoed by (i), i =,,... in ccordnce wih he order of p plying he. In soe cses, es insnces y be clssified ino es clsses TC(j), j =,,... by he nure of es insnces. The order of pplying es clsses s well s es insnces of es clss will be decided rher rbirrily. When clssified, es insnce will be expressed s (i,j), which ens i is he ih es insnce of TC(j). Wih hese TC(j) s, heir doins on he progr cn be defined [], bu will no be rgued in his pper. We define he response funcion of ful f(k) E s follows. If f(k) nifess iself s error(s) upon he pplicion of (i), nd r(i,k) =.(I, k) =, oherwise. () When r(i,k) = l,f(k) is sid sensed by (i). Then, w(i) =I {f(k) E I r(i,k) = ) I () where I X I is he crdinliy of se X. Noe here h soe of w(i) fuls y lso be nifesed by es insnces oher hn (i). Now, we ssue he following. ///$. IEEE

2 . uls which nifes heselves upon he pplicion of es insnce will be reoved (fixed), before he nex es insnce is pplied.. During he reovl of deeced fuls, no fuls will be insered newly. Thus, he dependbiliy of he progr grows long wih he progress of he esnddebugging. (We know here is criicis gins his ssupion. We re presenly opiisic, hoping h his ssupion could be reoved in he fuure.). w(i) fuls sensed by (i) re hose ken rndoly ou of iniil fuls. (In reliy, he cully sensed fuls re definie for (i). However, we followed his ssupion, since he order of pplying es insnces is rher rbirry nd furher, i is very difficul o ke ino ccoun he definie relionship beween w(i) fuls nd (i).) When he firs es insnce () is pplied, he nuber of deeced (nd reoved) fuls is, of course, ). When () is pplied, however, he nuber of newly deeced fuls is no necessrily w(), becuse soe of hese ) fuls y hve been deeced nd reoved by (). Considering he pplicion of (i), le C(i ) be he cuulive nuber of fuls newly deeced by (), (),..., (i ) nd N(i) be he nnber of fuls newly deeced by (i). C() is defined o be. Then, soe of w(i) fuls sensed by (i) y be hose h re lredy couned in C(i I), nd he reining of w(i) fuls ccouns for he newly deeced fuls. Therefore, he probbiliy h N(i) = z is given by Prob(zl, C(i ),w(i)) = f \ where z,,, = in{w(i), C(i )). The firs er in he nueror of Eq.() represens he nuber of cses where z fuls ou of u(:) fuls belong o hose of C(i ) fuls sill residen in he progr, while w(i) z fuls re hose lredy deeced (nd reoved) nd ccordingly us be he ebers of C(: ) fuls. The second er of he nueror is jus he nuber of such cses. The denoinor is he nuber of cses o ke w(i) fuls ou of fuls. The probbilisic disribuion of his Eq. () is clled hypergeoeric disribuion. The expeced vlue of N(i), denoed by N(i), is [] By he definiion, N(i) = { C(i ) :) In he sequel, he esie for preer P will be denoed by E[P]. Then, we use he esie for C(i ) such s k=l i () E[C(i I)] = N(k) () k=l nd subsiue i for C(i ) in Eq. (). Th is, :) = E[C(i I)]{] } + w(i) nd he coefficiens of w(i) ( see ecion ) re unknown preers o be esied. Presenly, we hve no ye obined wy of esiing hese preers nlyiclly. Therefore, looking he rel esnddebug d, we ssue heir vlues enively, nd clcule E[C(i)] for i =,,... for king he coprison wih cully observed C( i). These preer vlues will be chnged exhusively wih heir respecive sll increens nd/or decreens o find heir opiu vlues, which give he iniu vlue of he following E. El = IC(i) E[C(i)]l n.,= where n is he ol nuber of es insnces. El is esure of he difference beween C(i) s nd E[C(i)] s. We regrd he opil vlue of nd he coefficiens hus deerined o be heir esies. Inerpreion nd ensiiviy cor As cn be seen in ler exples, esnddebug d re ofen ghered on dy by dy bsis. In such cses, we cn regrd he es operion execued in dy o be es insnce. In his conex, he nuber of fuls sensed in dy is jus sensiiviy fcor. Hence, he lrger he nuber of es workers involved in (i), he ore fuls will possibly be sensed by (i). iilrly, longer copuer ie for execuing (i) will conribue o ge evenully lrger w(i). If we check ore es ies in (i), we shll probbly find ore fuls, geing lrger w(:). urher, he skill of es workers y iprove when he es proceeds. Or, i y hppen h only fuls such s hose ore difficul o be deeced rein in ler es insnces. Therefore, w(i) y chnge wih he progress of i. Thus, we cn ke such fcors ino considerion for w(i) s. he nuber of es workers,. he chnge of he ese of es,. he execuion ie of copuer, nd. he nuber of es ies. Exples of he pplicions of our odel, which ke he bove fcors ino considerion. will be shown below.. Nuber of Tes Workers As shown in Tble, he nuber of es workers re recorded dy by dy. Therefore, we ssued funcion for w(i) s w(i) = * {eser(i)}p () where eser(:) is he represenion of he nuber of es workers involved in (i). Why we considered exponen p is h even if we increse he nuber of es workers wo ies, w(i) will no necessrily increse wo ies. In his for of w(i), preers o be esied re nd p s well s. The pplicions of our odel wih w(i) of Eq. () re lredy considered in []. or coprive purpose, pr of he resul is shown in ig.. () I

3 I hy i s 'uls * ' ' ' :uuled uls.. ' ' ' Tble : D 'luber Workers * ' * ' *.. s Ill uls Cuuled uls s Nuber of Tes Workers!erer(i) ' + ' ' ' * *. ' * I* + ' ' '. ' *. ' * '... '. ' ' ' ' ' ' ' ' ' *..... f. Y OBERVED ETIWTED. El = u(le[c(i)l w(i) = s ( eser(i)*cp ) Debug Dys igure : Esiions for Eq.(lO) Tes Clss. Chnge of Ese of Tes I ws observed h E[p] ws round. Therefore, we ke p of vlue in he sequel. In conrs, we ke he chnge of he ese of es ino considerion of w(i). Then, we ssue w(i) = eser(i)(li + bl) () or w(i) = eser(i)(i + bzi + c) () In Eq.(ll), we regrd he ese of es o chnge linerly long wih he progress of es insnces. The second er of he forul is o represen his effec. I is lso likely h he chnge of he ese of es will sure s skeched in ig.. This surion is odeled by he qudric expression in Eq.( ). igure : Ese of Tes The resul of he pplicion of our odel wih M( i) of Eq.( ) o he esnddebug d of Tble is shown in ig.. As seen in he figure, he consecuive es insnces re clssified ino five (consecuive) es clsses due o he nure of es insnces [Ill. The coefficiens in Eq.(ll) s well s were esied in ech es clss, respecively. We lso ried o use Eq.(), bu here ws lile difference beween boh cses. Therefore, Eq.(ll) is beer for sipliciy. Tble shows he esies for preers.

4 Tble : D Tes Tes Tes E[] E[] Clss Insnces Descripion E[bl] The esie of El] for he h es clss in Tble is very unusul. We suppose h his rher surprising vlue is due o he sll nuber of es insnces in his clss I.. Tble : Esies for Preers.. i OBERVED ETIMATED. El = u(!e[c(i)lc(i)!)/n w(i) = eser(i) * ( l*i + bl ) i Tes Clss E(l Elbll El I/.., L Debug Dys igure : Esiions for Eq.(), ive Tes Clsses. f... Execuion Tie of Copuer In he d of Tble, debug ies insed of he nuber of es workers were recorded dy by dy. Therefore, we regrded he es operion in dy o be es insnce s before, ssociing w(i) wih he debug ie. We ke here he for of w(i) such s 'w(:) = h(i)(li + bl) (). I OBERVED. IO. ETIMATED. EI = u(!elc(i)lc(i)!)/n w(i) = debug ie(i) * ( l*i + bl E[l] =.. Elbll =. E [l =. where h(i) is he represenion of he debug ie. In his cse,. I/ L only he liner chnge of he ese of es ws considered Debug Dys The coprison of C(i) nd E[C(I)] is shown in ig.. The esied preers re surized in Tble (). igure : Esiions for Eq.() T

5 ~ We observe he irregulr increse of debug ie he h es insnce nd he shrp bend of C(i) curve he h es insnce. We specule h his shows he chnge of he nure of es insnces fer he h es insnce. Therefore, we clssified es insnces ino wo clsses of he h es insnces nd of he h o he ls one. The esiion ccording o his clssificion is shown in ig., where he esie for ws obined in ech es clss. The finess beween C(i) nd E[C(i)] looks beer hn h of ig.. The esied preers re found in Tble (b). Tes Clss Tes Insnces N. c Eh] =. E[] E[ol] E[bl] Dys i uls Cuuled uls o Tble : D Ies R uls ~ Cuuled uls l zo Nuber of Te Ies I(i) l. c..... I Tes Clss I. i=l... Tes Clss. i=... / OBERVED E[I =... pi;. ETIMATED. EI = ~(!EIC(~)IC(~)!)/n w(i) = debug ie(i) * ( l*i + bl ) Tes Clss E[ll EIbll EIl L. Debug Dys igure : Esiions for Eq.(), Two Tes Clsses. Nuber of Tes Ies The d of Tble shows dy by dy he nuber of es ies s well s he nuber of newly deeced fuls. urher, i ws known h he es insnces were clssified ino wo clsses due o he nure of es insnces. The one of clsses consiss of s es insnces, while he noher includes he h es insnces. w(i) of he for s w(i) = I(i)(i + bl) () is ssued, where I(:) represens he nuber of es ies (:). The resul of he esiion is surized in ig. nd Tble. Tble : Esies for Preers Tes Tes Tes clss Insnces DescriDion Module Tes Cobinion Tes irs Tes Clss El] =. / El E[ E[hI Eh] =. econd Tes Clss. OBERVED ETIMATED. EI = u(!e[c(i)lc(i)!)/n w(i) = es ies(i) * ( l*i + bl Tes Clss E[ll E[bll E[] Debug Dys igure : Esiions for Eq.()

6 Relionship o oher Models Here le us consider he relionship of our odel o hose proposed erlier. Eq.() cn be rewrien s E[C(i)] E[C(i l)] = AE[C(i l)] = w(i){l E[C(i )) This is difference equion of E[C(i)] for he uni increen of i. Trnsforing i ino differenil equion of coninuous funcion of i, nd ignoring he difference beween i nd i, we ge deic(i)l = w(i){l E[C(~)]) di () Therefore, he generl expression of E[C(i)] is. sing he Weibull disribuion, hkush nd Kishid proposed odel for he growh curve of he cuulive nuber of deeced fuls [] such s E[c(~)I = ( e(;)') () (he noion hs been chnged in ccordnce wih our sybols). The bove equion is he se s Eq.() wih he following w(i). w(j) = bjb () vb E[C(~)] = ( e.jo'w(i)di) (). If we ssue w(i) o be consn w, Eq.() reduces o This expression is he se s h of he en vlue funcion in Goel nd Okuoo's odel []. nd bin heir odel correspond o nd in Eq.(), respecively. Thus, our odel (Eq.()) is n exension o Goel nd Okuoo's odel, considering he vriion of b es insnces.. If we consider he chnge of he ese of es nd pu w(i) o be w(i) = w(i + b) () Eq.( ) reduces o In order o know he shpe of he curve of Eq.(), le us norlize he equion s follows. ~[~(i)/] = E[C'(~)] = e.{~+(~~)') () igure : hpe of Eq.() where Plos of E[C'(z)] for vrious k's re illusred in ig.. I is ineresing o noe h he curve of E[C'(z)] for soe k is shped s in Ohb e l's odel [].. If we regrd w(i) of he Weibull for s w(i) = y/bib'ebib () o be he quniive expression of es effor, Eq.() is copleely he se s h of he en vlue funcion proposed by Yd []. ince he funcion of Eq.() is of differen for ccording o b, w(i) is considered here o chnge wih he progress of: in ore generl wy hn he liner chnge s described in previous equions. Coprison of Esiions Coprive esiions were de for severl d, using our s well s oher (inly NHPP) odels.. Exple ig. shows he resul of he esiion for he d given by Goel []. ince only he nuber of newly deeced fuls ws recorded here, we ssued Eq.( ). The esies obined for w,, nd b re Tble : Resuls of Exple I. GoelOkuoo Model.

7 I I OBERVED... I ETIMTED. El = u(:e[c(i)lc(i)!)/n w(i) = w e ( * i + b ) E[w] =., E[].. E[bl =. E[l =. GoelOkuo NHPP odel () =. * ( exp(. )., Debug Hours.. El = u(le[c(i)lc(i) I)/n w(i) =w* ( * i + b ) E[wl =.. Er =. E [bl =.. Eh] =. Ohb. delyed shped Growh Model g()=.{(+. )exp(o. ) L.OO..... Debug Dys igure : Esiions for D Given by Goel [] igure : Esiions for D Given by Ohb []. Exple ig. is he resul of he esiion for he d given by Ohb []. Eq.() ws lso used in his cse. The esies for he preers re x.. Model Hyper. Geo. Disrib. Delyed shped Model Acul Observed E[I E[ Ebl WI Collecing d for his sofwre projec for ore hn hree yers, he cul vlue of is repored o be. The NHPP odel kes he esiion essenilly by observing he nuber of deeced fuls long wih he elpsed ie. In he coprison o our odel, he esied cuulive nuber of fuls hus obined were ploed in MCOIdnce wih he wy of he represenion of es insnces in ig... Exple Ohb lso gve noher d [], for which he esiion ws de. The resul is shown in ig.. ince. b he execuion ie ws recorded in his d, we ssued Weeks Eq.(), where h(i) ccouns for he execuion ie. The esies for he preers re given in Tble. igure : Esiions for D Given by Ohb [].... / El = u(!e[c(i)lc(il!)/n w(i) = h(i) * (* i + b E [] =., E [bl =. E [l =. Ohb. Goel Okuoo NHPP Model () =.* (Iexp (.) Ohb. inflecion shped Model h() =.*(exp(o.o)/ (+. XP (.) ) I

8 Model E[MI Hyper. Geo. Disrib.. Inflecion shped Model. GoelOkuoo Model. Acul Observed %I E[bI.. Afer long run, he ol nuber of iniil fuls cully observed ws repored o be.. Exple inlly we show he esiion for he d given by Dle [lo]. The resul is illusred in ig.. We used Eq.() here. Tble : Resuls of Exple Model II E[I II EkJl I E[I I I Hyper. Geo. Disrib... I. I. JelinskiMornd Model )I. x... I. Conclusion. We hve shown h our odel cn be well pplied o differen ypes of esnddebug d for king he esiion of he nuber of iniil sofwre fuls. As he exples show, he finess of he esied growh curves of our odel o rel d looks sisfcory. We hve lso de cler he relionship of our odel o hose proposed erlier. oe of he cn be expressed s specil cse of our odel. The shped chrcerisic of he growh curve cn lso be inroduced fro our odel. Currenly, we re conducing furher sudy on he deerinion of esies for preers by ens of nlyicl wy. Acknowledgens This work ws suppored in pr by he Minisry of Educion of Jpn under he Grn of cienific Reserch. We lso express our hnks o copnies which kindly provided us wih rel esnddebug d. References. Yd nd. Oski: Nonhoogeneous Error Deecion Re Models for ofwre Relibiliy Growh, ochsic Models in Relibiliy Theory ed. by. Oski nd Y. Hoy, pp., pringerverlg, Berlin,. Y. Toh, K. Tokung,. Ngse, nd Y. Mur: rucurl Approch o he Esiion of he Nuber of Residul ofwre uls Bsed on he HyperGeoeric Disribuion, IEEE Trns. ofwre Engineering, Vol., No., pp., Mrch. T. C. ry: Probbiliy nd is Engineering ses (nd Ed.), pp., Vn Nosrnd Co. Ld.,. A. L. Goel nd K. Okuoo: TieDependen Error Deecion Re Model for ofwre Relibiliy nd Oher Perfornce Mesures, IEEE Trns. Rel., R,, pp., Augus. M. Ohb,. Yd, K. Tked, nd. Oski: shped ofwre Growh Curve: How good is i?, Proc. COMP AC, pp., Chicgo,.. Yd: A TesEffor Dependen ofwre Relibiliy Growh Model nd is Applicions, T, IECE Technicl Repor, Vo., No., pp., ebrury (in Jpnese).. K. nkushi nd Y. Kishid: Esiion of Bug Curve El = u(le[c(i)lc(i)!)/n sing Experienl Regression Anlysis, hrp Technicl P,,,,,, w(i) = w * ( * i + b ) Bullein (hrp Technicl Gih), No., pp., (in EIwl =.. E[l =. Jpnese). E[b] =.. E[l =. Ari. L. Goel: ofwre Relibiliy Models: Assupions, Esied Growh Curve by Dle Liiions, nd Applicbiliy, IEEE Trns. ofwre En Jelinskikrnd &del. E [l=. gineering, Vol.E, No., pp., Deceber.... Debug.. Misuru Ohb: ofwre Relibiliy Anlysis Models, Inervls.... IBM Jour. Res. Dev., Vo., No., pp., July. igure : Esiions for D Given by Dle [lo] [IO] Hrris Dle: Approch o ofwre Relibiliy Predicion, Proc. Ann. Relibiliy nd Mininbiliy yposiu, pp.,. [Ill Y. Toh,. Ngse, nd Y. Mur: rucurl Approch o ofwre Relibiliy Growh Model bsed on he Hyper Geoeric Disribuion, IECE Technicl Repor, T, pp., Noveber.