THE performance disparity between processor speed and the

Transcription

1 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X Matix Stipe Cache-Based Contiguity Tansfom fo Fagmented Wites in RAID-5 Sung Hoon Baek, Student Membe, IEEE, and Kyu Ho Pak, Membe, IEEE, Abstact Given that contiguous eads and ites beteen a cache and a disk outpefom fagmented eads and ites, fagmented eads and ites ae focefully tansfomed into contiguous eads and ites via a poposed matix stipe cachebased contiguity tansfom (M-CT) method, hich employs a ule of consistency fo data integity at the block level, and a ule of pefomance that ensues no pefomance degadation. M- CT pefoms fo eads and ites, both of hich ae poduced by ite equests fom a host, as a ite equest fom a host employs eads fo paity-update and ites to disks in a RAID- 5 aay. M-CT is compatible ith existing disk technologies. The poposed implementation in a Linux kenel delives a peak thoughput that is. times highe than a case ithout M-CT on epesentative okloads. The esults demonstate that M- CT is extemely simple to implement, has lo ovehead, and is ideally suited fo RAID contolles not only fo andom ites but also fo sequential ites in vaious ealistic scenaios. Index Tems Stoage Management, Paallel I/O, RAID I. INTRODUCTION THE pefomance dispaity beteen pocesso speed and the disk tansfe ate can be compensated fo via the disk paallelism capability of disk aays. Patteson and Chen [] descibed six types of disk aays and temed them RAID- though RAID-6. RAID-6 potects against double disk failues using P+Q edundancy, and equies six disk accesses in ode to pefom a small ite opeation. RAID-5 equies fou disk accesses fo a small ite opeation and potects against single-disk failue. RAID-5 aays ae idely used due to thei easonable ovehead and modeate eliability. RAID-5 equies fou disk accesses to update a data block; to to ead old data and paity, and to to ite ne data and paity. To educe the ovehead of small ites in a RAID-5 aay, Menon posited the fast ite pocess that utilizes non-volatile stoage (NVS) as a ite cache in a disk aay contolle []. The NVS is commonly built using battey-backed volatile RAM. A block eceived fom a host system is initially itten to the NVS in a disk aay contolle and the disk aay contolle sends a completion message to the host system at that time. Both the data and the paity blocks on the disks can then be itten (destaged) asynchonously at a late time, thus hiding the ite latency of the disk. A. Destage Due to the asynchonous natue of the ite cache, the contents of the ite cache may be destaged in a desied ode to educe the aveage seek time o impove the peak ite thoughput of the The authos ae ith the Division of Electical Engineeing, Koea Advanced Institute of Science and Technology, 5 Gahangno Yuseong-gu, Daejeon, Koea, shbaek@coe.kaist.ac.k, kpak@ee.kaist.ac.k system. Some disk schedules (see Section I-E.), least ecently itten (LRW) [] and ise odeing fo ites (WOW) [4], focus on the question of the ode in hich the ites ae destaged fom the NVS. In paticula, WOW exploits both tempoal locality and spatial locality by consideing both data ecency and disk seek latency. Anothe poblem in the use of the ite cache is to detemine hen to destage data fom the cache to the disk to guaantee continued high esponse time fo ites. In contast, if ne data that ee itten to the cache ae destaged too aggessively, it is not then possible to fully exploit the benefits of ite caching. Vama et al. [5] intoduced linea theshold (LT) scheduling, hich adaptively vaies the ate of destages based on the instantaneous occupancy of the ite cache. A simple fom of scheduling is knon as high/lo mak (HLM) [6] scheduling, hich enables destages to the disk hen the cache occupancy dops belo a lo mak, and disables destages hen it inceases ove a high mak. Gill and Modha [4] intoduced a destage schedule combining high/lo mak scheduling ith linea theshold scheduling. B. Ho to Destage Fagmented Wites The destage stats by using a decision algoithm such as LT o HLM, that detemines hen to destage. When the stat time of the destage is decided, it is then necessay to decide hich block o stipe should be destaged by using a destage algoithm such as LRW and WOW, that detemines hat to destage. Ou ok contends ith hat comes afte the selecting of a block o stipe that is to be destaged. The focus hee is on ho to efficiently destage fagmented ites fom the NVS to the disk in a RAID contolle. This ok is compatible ith the existing destage algoithms in tems of hat to destage and hen to destage it. Fagmented ites ae poduced in a numbe of cases, as follos: ) Random ites ith destage: All andom ites fom the host can be conveted into fagmented ites to the disk by destage algoithms that e-ode ite-equests in an inceasing block index ode. CAN [7], CLOCK [8], and WOW destages dity blocks to disks only in an inceasing block index ode. WOW is a vaiant of CLOCK ith a stipe cache that is managed in tems of stipe goups. If a destage algoithm that does not destage blocks in an ascending ode manages thei cache in tems of stipe goups, all ites ithin the stipe ae sequentially contiguous o sequentially discontiguous. Section III-A descibes the stipe cache in detail. ) Multiple concuent ites: In the local stoage of a seve and in stoage seves such as netok attached stoage [9], multiple uses ead o ite concuently. Especially if multiple uses ceate a lage amount of files simultaneously, these files may be inteleaved ith each othe. Thus, fagmented ites ae poduced due to the in-place update of the files. Digital Object Indentifie.9/TC /$5. 7 IEEE

2 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X Nomalized tunaound time ST7454LC (Seagate 5pm SI) STAS (Seagate 7pm SATA) 6V6E (Maxto 7pm SATA) WD5JD (WestenDigital 7pm SATA) simulation the sequentially contiguous ite Aveage distance [block] t t t t 4 t 5t 6 t 7 t 8 time data on the bus disk head position??? iting the data to the media tansfeing the data fom the host completion message Missing point that the disk head misses the block due to a busy opeation ith the host Fig.. An example that exhibits the unexpected disk otation. stat (a) The nomalized tunaound time of the ite vesus the aveage stide distance ith an exponential distibution fo vaious disks. ite sequence B B B B B4 B5 B6 B7 B8 B9 B B B B B4 B5 B6 B7 B8 B9 B B B B B4 B5 B B B5 B6 sequentially contiguous ite sequentially discontiguous ite ith an exponential distibution B B stide distance B7 B9 B (b) A sequentially contiguous ite and a sequentially discontiguous ite ith an exponential distibution. Fig.. Wites to the disk pefom in an inceasing block index ode. The distibution of the stide distance is exponential. The ite stats at the same peset block and stops hen it eaches the same peset destination. Each tunaound time is nomalized ith the sequentially contiguous ite, of hich the nomalized value is one. ) Aged filesystems: On eal filesystems, files ae divided into pieces scatteed aound the disk, thus sequential ites at the filesystem level may appea as fagmented ites at the block level. Fagmentation occus natually hen e use a disk fequently, ceating, deleting, and modifying files. 4) Sequential ites of multiple small files ith pe-allocation: When a ne disk data block is allocated, filesystems such as ext and ext intenally pe-allocate a small numbe of disk blocks adjacent to the just allocated block, in ode to expand the files in the futue []. Hence, thee exist pe-allocated, unused blocks beteen files, hich cause fagmented ites hen e copy a diectoy that contains small files. As the size of the files deceases, the potion of fagmentation inceases. C. Motivation of Ou Wok It is evident that a sequentially discontiguous (fagmented) ite to a disk is sloe and employs a longe seek distance compaed to a sequentially contiguous ite if the to types of ites contain the same amount of data. Hoeve, e found anothe chaacteistic about a fagmented ite. By efeing to the expeimental esults shon in Fig., e discoveed that a sequentially contiguous ite outpefoms a sequentially discontiguous ite even if the to types of ites employ the equivalent seek distance ith diffeent amounts of data as shon in Fig. (b). The equivalent seek distance indicates that identical stat and end positions ae applied, and signifies that the imum stide distance does not exceed a theshold value. The stide distance is defined by the numbe of blocks beteen to discontiguous I/Os. We pesume that the esult shon in Fig. is induced by the folloing pocess: afte finishing a block ite, a host sends the next data to the disk fo the subsequent discontiguous block in the same disk tack as the pevious ite. If sending the subsequent B end B5 data tansfe equies moe time than the otational latency that is equied fo the disk head to each to the subsequent block, then the disk head misses the block and equies anothe evolution. It is impotant to educe not only the seek time but also unexpected disk otation. Fo example,the evolution time of the disk ST7454LC is 4 ms, hich is as long as the aveage iteseek time (i.e. 4. ms). The okloads used in the expeiment of Fig. ae sequentially discontiguous ites, the stide distance of hich has an exponential distibution as shon in Fig. (b), hee the imum stide distance is limited up to blocks, and the block size is set to 4 KiB (KiB is the abbeviation of kibibyte, as defined by the Intenational Electotechnical Commission. KiB efes to bytes, instead kb efes to bytes. []). Fig. (a) shos the nomalized tunaound time of a ite fo vaious disks by vaying the aveage stide distance ith an exponential distibution. Wite stats at the same peset block and stops hen it eaches the same peset destination. Each tunaound time of the ites shon in Fig. (a) is nomalized ith that of the sequentially contiguous ite. The nomalized tunaound time of the contiguous ite is. All of the nomalized tunaound times of the fagmented ites shon in Fig. (a) ae alays highe than. Theefoe a sequentially contiguous ite outpefoms sequentially discontiguous ites ith an equivalent seek distance. Sequentially discontiguous ites ith a small stide distance significantly degade disk pefomance. In othe ods, fagmented ites of fine ganulaity give ise to a much highe pobability of the additional disk otation. Some disks also sho such a chaacteistic fo eads as shon in Fig. 9. To obtain quantitative esults of the unexpected disk otation, e implemented a benchmak as a kenel dive of Linux (vesion.6) in ode to eliminate the effects of cache, pefetching, synchonous I/O, and disk scheduling. The benchmak uses the function geneic make equest() of the Linux kenel fo a disk inteface to avoid cache and pefetching. The benchmak poduces fom to 64 outstanding I/Os, pocesses all I/Os asynchonously, and peiodically calls the function unplug() evey position of blocks to mitigate the unexpected aiting time caused by the anticipatoy disk schedule [], hich is the default disk schedule of Linux. Figue illustates an example hee unexpected disk otations occu. In Fig., the host tansfes sequentially discontiguous data (blocks,, 5, 7, 9,, and so on) to the disk in a command queuing fashion hee the host queues multiple commands to the disk. In Fig., the host tansfes blocks,, 5, and 7 to the disk at the fist time (t ), and the disk buffes the blocks in its cache and seeks the block (t t ). Afte iting blocks and to the magnetic media, the disk sends completion messages fo

3 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X blocks and (t,t 4 ). The host tansfes the subsequent blocks 9 and (t 5 t 7 ) afte it eceives the completion messages. While the disk eceives the subsequent blocks 9 and fom the host, the disk head misses the media location that coesponds to the queued block 5 (t 6, missing point). In ode to ite the missed block to the media, the disk must fully otate (t 6 t 8 ). In this scenaio, missing points occu evey to itten blocks if the host sends to data blocks simultaneously in ode to sequentially ite odd blocks. The fequency of the missing point depends on the cache policy of the disk, the stide distance, the command queuing policy of the host, the intenal/extenal tansfe ate of the disk, and atypical opeation of the disk. The folloing equation povides an analytical model of this poblem. t d = t c + t N, () hee t d and t c ae the tunaound time of the discontiguous ite and the contiguous ite, espectively. t is one evolution time of the disk. N is the numbe of occuences of unexpected disk otation, hich occus if the sum of the data tansfe time and the computational ovehead equies moe time than the otational latency that coesponds to the inteval beteen to discontiguous blocks. The tunaound time t c consists of only the tansfe time ithout any ovehead. The sum of seek time and tansfe time of t d equals t c due to the equivalent distance, but t d equies additional otations (t N ). The simulation esult fo this analytical model ith pefomance paametes of ST7454LC is shon in Fig.. The simulation esult is simila to the expeimental esults. The ite-back policy of the disk aggegates itten data into its on-boad cache and tansfes the data fom the on-boad cache into the disk media at a late time, theeby mitigating the missing points shon in Fig.. Hoeve, the ite-back policy is uneliable against poe-failue. Commecial disks, shon in Fig., idely use ite-on-aival [] capability, hich begins iting sectos to the disk media as soon as they ae tansfeed to the on-boad cache, theeby educing otational latency. The closely-discontiguous ead can be enhanced by a pefetching scheme that eads ahead up to the end of the tack o cylinde containing the last secto of the ead equest []. Hoeve, some disks, shon in Fig. 9(a) and 9(d), do not use this pefetching scheme in ode to educe esponse time. D. Ou Contibutions Thee ae a lage numbe of algoithms fo fagmentation eduction at the filesystem level, o spatial locality and tempoal locality of the ite cache at the block level. Hoeve, in contast to the pupose of taditional schemes, ou poposal, knon as the matix stipe cache-based contiguity tansfom (M-CT) as explained in Section III, mitigates the unnecessay otational latency caused by fagmented ites at the block level. M- CT is compatible ith existing algoithms such as destage, disk scheduling, pefetching, ead cache management, and filesystem; it coopeates ith the existing algoithms fo bette pefomance as it solves an entiely novel poblem. We make seveal contibutions to the design and implementation of a lo ovehead, self-tuning algoithm fo an actual RAID system that exploits a sophisticated edundancy scheme such as the distibuted-paity and P+Q edundancy of RAID-5 and RAID-6 []. To efficiently destage fagmented ites, M- CT employs a stipe cache management scheme and a contiguity tansfom scheme. By using a stipe cache that is managed in tems of stipe goups, M convets all eads (fo paityupdate) and ites into sequentially contiguous o sequentially discontiguous eads and ites ithin the stipe at the destage. CT then tansfoms discontiguous eads and ites into contiguous eads and ites by inseting additional eads and ites. Such tansfom is pefomed by a poposed ule fo consistency, theeby not affecting the filesystem and not changing the data and block locations. Futhemoe, M-CT does not involve pefomance degadation by using ou poposed ule fo pefomance in a selftuning fashion. We implemented a kenel module in Linux vesion.6 as a RAID-5 dive ith M-CT, hich significantly outpefoms the MD (Multi-Device) dive that is a taditional RAID-5 dive of Linux. Ou expeiments ae pefomed on andom ites, concuent sequential ites using existing benchmaks, atificially aged filesystems, and vaious types of ealistic sequential copies.in summay, M is a pactical algoithm that does not conflict ith existing algoithms and enhances a RAID contolle to pocess moe fagmented I/Os and sequential I/Os. E. Pio Woks fo the Fagmented I/O To impove andom o fagmented I/O, hich is the leading cause of pefomance degadation, eseaches have investigated a vaiety of ne technologies that ae elated to filesystems, disk scheduling, pefetching, ead cache management, RAID, destage, and etc. Hoeve, ou scheme is independent of and compatible ith the pio technologies. This section biefly descibes these technologies. ) Filesystems: Fagmented I/Os can be poduced by a highly utilized filesystem, in hich files ae fagmented into multiple segments [4]. Sequential access to a fagmented file causes fagmented I/Os at the block level. A log-stuctued filesystem (LFS) [5] alays sequentially ites data ithout in-place updates. A de-fagmented filesystem (DFS) [6] dynamically elocates the fagmented data of a file. Hoeve, such filesystems involve gabage collection and degadation of ead pefomance. ) Disk Scheduling: The disk schedule is esponsible fo dynamically e-odeing the pending equests in a block device s equest queue into a dispatching ode that esults in minimal seeks o instantaneous access time fo andom I/Os. Such algoithms include FCFS [7], SSTF [8], AN [8], CAN (it is also knon as C-LOOK) [7], VAN [9], FAN [7], SPTF [], GSTP []. In addition, disk schedules ae also esponsible fo load balancing, quality of sevice, and accumulating pending equests. The complete fainess queuing, the ealiest deadline fist schedule, and the anticipatoy schedule [] ae adopted to Linux fo these puposes. ) Pefetching: By speculatively pefetching o pestaging pages even befoe they ae equested, multiple sequential eads may be made of a single ead equest, thus the pefetching educes a deep ead latency []. Fagmented eads ithin pefetched pages lead to cache hits, theeby impoving ead pefomance by an ode of magnitude. One of the latest and most outstanding investigations of pefetching involves sequential pefetching in adaptive eplacement cache (SARC) []. SARC combines and adaptively balances both the cache and pefetch.

4 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 4 Paity block goup Disk Disk Disk Disk Disk 4 Stipe Stip Stip Stip Stip Block Block Block Block Paity Stip Paity, Paity, Paity, Paity, Stip Block 4 Block 5 Block 6 Block 7 Stip Block 6 Block 7 Block 8 Block 9 Block 8 Block 9 Block Block Block Block Block Block Block Block Block 4 Block 5 Block 4 Block 5 Block 6 Block 7 Paity Stip Paity, Paity, Paity, Paity, Stipe Stip Stip Stip Block 8 Block 9 Block Block Stipe Paity Stip Stip Stip Stip Fig. 4. block () ite a ne data fo block 4 ne data at the cache memoy (4) ite the ne data to block 4 () block 4 (t) () dual XOR fo the old data and the ne data () old data at a tempoay memoy () XOR fo the old paity (x) old paity at a memoy = () ead the old data of block 4 to a tempoay memoy (t) block 8 block () ead the old paity () A ead-modify-ite cycle in a RAID-5 aay. () XOR destination (_) ne paity paity () (4) ite the ne paity () Paity block goup... mnemonics to conveniently descibe ou ok. Fig.. The data oganization and teminologies of a RAID-5 aay. TABLE I OPERATION MNEMONICS 4) Read Cache Management: Fagmented o andom eads can be mitigated by a ead cache. A lage numbe of cache eplacement algoithms have been ell studied. Examples ae LRU, CLOCK, LRU-, LRFU, Q, LIRS, FBR, MQ, CAR [4], ARC [5], and DULO [6]. Most of the ead cache algoithms focus on the hit atio by statically o adaptively combining ecency and fequency. Meanhile, DULO gives andomly-placed blocks highe duability than sequentially-accessed blocks in the ead cache, theeby impoving fagmented o andom eads. 5) RAID: A lage numbe of eseaches have investigated RAID technologies. Among these studies, AutoRAID [7], AFRAID [8], and DMP [9] focus on the small ite that also poduces a fagmented ite. To impove the esponse time elated to the small ite that poduces to eads and to ites in RAID-5, AutoRAID ites data to the mioed stoage class in a log-stuctued fashion, and then migates the data to the RAID-5 stoage class at a late time. AFRAID does not ait fo the paity to be updated; it updates the paity hen disks ae idle, theeby impoving the instantaneous esponse time, hoeve it loses guaanteed eliability. DMP exploits moe than one (R) numbe of paity stips pe stipe fo single-disk-failue toleance, theeby enabling R numbe of ites to acquie the stipe lock simultaneously. II. BACKGROUND OF RAID-5 RAID-5 employs a coase-gained stiped paity technique so that small equests can benefit fom the concuency duing the sevicing of multiple equests. The RAID Advisoy Boad [] descibes RAID-5 using the teminology of both stip and stipe in moe detail, as shon in Fig.. A RAID-5 aay is oganized by stipes, each of hich consists of a paity stip and data stips. Each stip compises a set of blocks that ae patitioned by disks. The paity block goup (PBG ) is defined by the goup of blocks that ae located at the same offset of the membe disks. The paity block in the paity stip stoes the esult of the bitise exclusive OR (XOR) of the data blocks that ae in the same PBG. The paity stip compises the paity blocks in the stipe. A. Reconstuct-ite and Read-modify-ite All actions of RAID-5 can be categoized by six opeations:, t,, x,, and that ae descibed in Table I. We use these Many people confuse stipe ith PBG, as thee is no idely-used teminology fo PBG. Mnemonic t x Desciption Read the block fom the disk to the block cache memoy Read the block fom the disk to a tempoay memoy Wite the block cache memoy to the disk XOR the block cache memoy XOR the block cache memoy and the tempoay memoy The destination of the XOR esult To update one block ith ne data, it is necessay to () ead all othe blocks of the PBG to hich the updated block belongs, unless it is cached; () XOR all data blocks; and () ite the paity block and the ne block. This opeation equies (N d c) eads and (d +) ites, both of hich compise (N c) I/Os, hee N is the numbe of disks, c is the numbe of cached clean blocks, and d is the numbe of dity blocks to be updated. This pocess is knon as a econstuct-ite cycle. When d = N, it is unnecessay to ead any block; this case is knon as a full-paity-block-goup-ite. A ead-modify-ite cycle can be used to educe the numbe of I/Os hen N c >(+d), as the econstuct-ite cycle equies (N c) I/Os hile the ead-modify-ite cycle equies ( + d) I/Os. This pocess does the folloing: () it copies the ne data to the cache memoy; () it eads the old paity block () and eads the old block to a tempoay memoy (t) simultaneously; () it XORs the ne block ith the old block (), and XORs the esult ith the old paity block (x) to geneate the ne paity block ( ); and (4) it ites the ne block () and ites the ne paity block () simultaneously, as shon in Fig. 4. The eadmodify-ite cycle equies ( + d) eads and ( + d) ites, both of hich compise ( + d) I/Os. Fo vaious cases of cache status, Fig. 5 shos opeations fo PBGs to be destaged by the mnemonics. If to blocks ae cached (clean) and anothe block is itten (dity) as shon in Case of Fig. 5, e choose the econstuct-ite cycle to destage the PBG, as N c<( + d), hee N =5, c =, and d =. Hence it is necessay to ead the empty block (), XOR all data blocks to update the paity block (x), and ite the dity block and the ne paity (). Theefoe, the clean blocks only involve x, the dity block equies x, the empty block equies x, and the paity block equies. If all blocks ae dity as in Case 4, The teminology full-stipe-ite is usually used instead of it because thee is no geneal teminology fo PBG.

5 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 5 Fig. 5. Case : Case : Case : Case 4: Case 5: Case 6: Paity block goup (PBG) Paity block goup (PBG) Paity block goup (PBG) Paity block goup 5 (PBG5) B B4 B8 B B B4 B8 B P E D E E C C E D D E D E D D D D D D cache status of a PBG t x x x x x x x x x x x x _ D/F C D x x x x _ E/F D D t tt opeations fo destage D: dity C: clean E: empty F: faulty Destage opeations of RAID-5 in vaious cases of PBG Block : dity Block : dity LRW List Head Block 5 Block 9: dity Block Paity, Block 4 Block 8 Block Paity, Block Block 7 Block : dity Block 5 Paity, Paity, Block 7: dity Block Block 5: dity Block 9 (a) A LRW list in the Paity Block Goup Cache (the taditional method) Block : dity Stipe Cache Block : dity unit Block Block Stipe Cache unit Paity, Paity, Paity, Paity, Block 4 Block 5 Block 6 Block 7 Block 6 Block 7: dity Block 8 Block 9 LRW List Head Block 8 Block Block 9: dity Block Block Block 4 Block : dity Block 5 Block Block Block Block Block 4 Block 5: dity Block 6 Block 7 (b) A LRW list in the Stipe Cache Paity, Paity, Paity, Paity, Block 8 Block 9 Block Block Fig. 6. The LRW list examples fo the conventional paity block goup and the stipe cache hen the sequence of block numbes fo ites is 7,, 9,,, 5 ae managed in tems of paity block goups (PBG) (see Section II fo PBG). Fo example, the softae RAID-5 of the Multi-device (MD) of Linux manages the LRW cache in tems of PBG. Fig. 6(b) shos an example of the LRW list of a stipe cache () fo RAID-5. Each unit coesponds to each stipe, hich is the management unit fo cache eplacement and fo destages. Figue 6 shos the LRW list examples hen the sequence of the block numbes fo ites is 7,, 9,,, 5. In a PBGC that is destaged by LRW, the LRW list head points to the least ecently itten PBG. The blocks to be destaged ae selected fom the head of the LRW list. When a block in a PBG becomes dity, the updated block and all of the othe blocks that belong to the same PBG move to the end of the LRW list. Fo example, because block 5, hich belongs to PBG5, is the most ecently itten block, PBG5 is in the end of the LRW list even though the block 7, hich belongs to the PBG5, is the least ecently itten block. Similaly, in an that is destaged by LRW, the head of the LRW list points to the least ecently itten stipe cache unit. When a block in a stipe cache unit becomes dity, the updated block and all of the othe blocks that belong to the same stipe cache unit move to the end of the LRW list. Theefoe, exploits spatial locality by coalescing ites that ae in the stipe. When a stipe cache unit destages, all blocks in the stipe ae destaged sequentially, theeby conveting all types of ites into sequentially contiguous o sequentially discontiguous ites ithin the stipe. also allos a bette exploitation of spatial locality by coalescing ites togethe. Disk schedules and destage algoithms that detemine hat to destage also save disk head seeks. Hoeve, M-CT can coexist ith such schemes and helps such schemes to exploit much spatial locality. WOW also manages the cache in tems of stipe goups to educe the soting ovehead of CLOCK and to exploit spatial locality. Hoeve M-CT teats the stipe cache unit as a matix and enhances the pefomance of fagmented ites using the CT scheme. it is necessay to XOR all data blocks ithout a ead, and ite all blocks and the paity. In othe ods, all data blocks and the paity block equie x and, espectively. Case and 6 in Fig. 5 sho the cases that use the ead-modify-ite cycle. III. MATRIX STRIPE CACHE-BASED CONTIGUITY TRANSFORM We no descibe ou scheme fo ho to efficiently destage fagmented ites, namely, matix stipe cache-based contiguity tansfom (M-CT). Most of the ites fom the host can be conveted into sequentially contiguous o sequentially discontiguous (fagmented) ites to the disk duing the destage by a stipe cache () that is managed in tems of stipe goups. M is a stipe cache managed by the x-matix (explained in Section III-B) fo the contiguity tansfom (CT) of fagmented ites. CT tansfoms fagmented I/Os into a contiguous I/O by inseting unnecessay I/Os into the discontiguous egion using the ules fo consistency and pefomance. A. Stipe Cache Figue 6(a) shos the taditional paity block goup cache (PBGC) of RAID-5. Taditional cache eplacements and destages B. Contiguity Tansfom If a disk head misses the subsequent fagmented block due to the missing point of Fig., it equies a full disk spin (see Section I-C). To alleviate this poblem, e popose a contiguity tansfom (CT) pocess at the destage. CT tansfoms the discontiguous ites into the contiguous ites by inseting additional ites in beteen the discontiguous ites. CT is also applied to the ead opeations that occu in the econstuct-ite and eadmodify-ite cycles. Fig. 7 shos an example of the CT pocess in a RAID-5 aay hen a stipe is destaged. Hoeve CT is applicable to the othe types of RAID, such as RAID-6. Figue 7(a) shos an example of the block status in a stipe, hee D denotes a dity block in hich ne data is in the cache but not yet updated to a disk, C denotes a clean block in hich consistent data ith the disk is in the cache, and E denotes an empty block in hich valid data is not in the cache. Let u be the numbe of blocks pe stip, and let v be the numbe of disks consisting of a RAID-5 aay. The cache status of the blocks in a stipe that is shon in Fig. 7(a) can be epesented by the folloing u (v ) matix:

6 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 6 The cache status of blocks in a stipe basic x-matix, M ead XOR ite stipe S S S S Stip Stip Stip Stip D E E E Block Block8 Block6Block4 E C E D Block Block9 Block7Block5 E E E E Block Block Block8Block6 D D D D Block Block Block9Block7 D C D D Block4 Block BlockBlock8 E D E E Block5 Block BlockBlock9 D E E E Block6 Block4 BlockBlock E E D E Block7 Block5 BlockBlock (a) S S S S P Stip Stip Stip Stip Paity t m m m m4 m5 x x x x _ m m m m4 m5 m m m m4 m5 x x x x _ m4 m4 m4 m44 m45 x x x x _ m5 m6 t t m7 m7 m 8 m 8 m5 m5 m54 m55 m6 m6 m64 m65 m7 m74 m75 t m 8 (b) contiguity tansfom m 84 m 85 taditional path S t t S S S P t t S S S S P x x x x x x x x x x x x _ (c) destage opeations by the basic x-matix S S S S P x-matix M afte ead contiguity tansfom x-matix M afte ite contiguity tansfom ead XOR ite S S S S P S S S S P S S S S P S S S S P S S S S P t t t x x x x tx x tx x tx x tx x t t _ x x x x tx tx tx x tx x tx x t t _ t t t t t t x x x x x x x x x x x x _ t t t (d) contiguity tansfom (e) destage opeations by the x-matix ith contiguity tansfom Fig. 7. M-based contiguity tansfom: When a RAID dive destages a stipe, it uses the econstuct-ite cycle o the ead-modify-ite cycle fo each o to geneate the basic x-matix. It is possible to successfully destage the stipe by executing eads, XORs, and ites based on the basic x-matix (c). Hoeve the poposed scheme insets the contiguity tansfom pocess (d) beteen the geneation of the basic x-matix (b) and the actual execution of the x opeations (e). Z =[z ij ] u (v ) = 6 4 D E E E E C E D E E E E D D D D D C D D E D E E D E E E E E D E. () 7 5 Befoe the actual execution of the ead, XOR, and ite fo all blocks in a stipe, it is necessay to detemine hich blocks should be ead, ho the paity blocks should be made, and hich blocks should be itten, by geneating a basic x-matix, as shon in Fig. 7(b). By choosing one of the econstuct-ite cycle and the ead-modify-ite cycle shon in Fig. 5 fo each o of the matix Z, e detemines the basic x-matix, M, hose element, m ij, is a subset of {t,, x,,,}. The basic x-matix epesents all opeations that destage all blocks in the stipe. We can expess the basic x-matix shon in Fig. 7(b) as the folloing equation: M =[m ij ] u v = {t,, } {} {} {} {, x,, } {, x} {x} {, x} {x, } {, } {} {} {} {} {} {x, } {x, } {x, } {x, } {, } {x, } {x} {x, } {x, } {, }. 6 {} {t,, } {} {} {, x,, } {t,, } {} {} {} {, x,, } {} {} {t,, } {} {, x,, } It is possible to successfully destage the stipe by executing eads, XORs, and ites that ae based on the basic x-matix as shon in Fig. 7(c). Hoeve, the poposed scheme insets the contiguity tansfom pocess beteen the geneation of the basic x-matix and the actual execution of the x opeations as shon in Fig. 7(d). The contiguity tansfom consists of a ead contiguity tansfom and a ite contiguity tansfom. The ead contiguity tansfom must pecede the ite contiguity tansfom to incease the possibility of the ite contiguity tansfom. Section III-C descibes the easons in detail. The ead contiguity tansfom insets ead opeations, hich appea as the bold-faced in Fig. 7(d) and 7(e), beteen to discontiguous eads fo each column in the basic x-matix. Fo ()

7 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 7 the example in Fig. 7(b), thee exists the ead opeation in m, {,x}; the ead opeation t belongs to m 7, {t,,}. Hoeve, thee is no ead opeation beteen the to elements m and m 7. Hence, o t is inseted into all of the elements beteen m and m 7. The ead opeations, and t, must be popely chosen fo consistency. The choice of and t is explained in Section III-C. The left matix of Fig. 7(d) shos the x-matix afte applying the ead contiguity tansfom. If e fomally descibe the above constaints, i and j of the x element m ij, to hich e can add the ead opeation, must satisfy at least the folloing pedicate: i j a b[ (a<i<b) ((m aj {, t}) ) ((m bj {, t}) ) (4) ( h h[((m hj {, t}) ) (a <h<b)]) ]. The x element m ij, to hich the ead opeation can be added, is in the discontiguous egion beteen to discontiguous blocks, m aj and m bj, both of hich include o t. In the discontiguous egion, thee exists no element (m hj ) that includes o t. In the case of the stip shon in Fig. 7(b), it is possible to add o t beteen m and m 7 by pedicate (4) ith j =, a =, and b =7. The ite contiguity tansfom is pocessed in the same manne as the ead contiguity tansfom. Fo example, in the basic xmatix, M, shon in Fig. 7(b), thee is the ite opeation,, in the to sepaated elements, m and m 4 but thee is no in m and m, to hich, theefoe, is added. It is no feasible to tansfom to discontiguous ites into a single ite command. The indices i and j of the x element m ij, to hich the ite opeation can be added, must satisfy at least the folloing pedicate that is simila to pedicate (4): i j a b[ (a<i<b) ( m aj ) ( m bj ) (5) ( h h[( m hj ) (a <h<b)]) ]. In this ay, to sepaated equests can be meged into a single equest, theeby educing the numbe of fagmented equests that sho pooe pefomance than the contiguous I/O that employs the same stat and end block. This section descibes the contiguity tansfom and the seveal ules equied fo consistency and pefomance. The folloing sections pesent the ules. C. Rules fo Consistency We must distinguish and t fo data consistency in the ead contiguity tansfom. If a cache status z ij is dity, eading old data fom the disk to the cache though spoils the latest data in the cache. Hence, t should be used to peseve the neest data in the cache. t eads data to a tempoay memoy that ill be eleased afte the destage. If the block status is not dity, is used fo a cache hit in the futue. Fo example, t is added to m 5 in Fig. 7, as z 5 is dity (D). Hoeve is used fo m 6,asz 6 is not dity. This pocess can be fomally itten as the folloing equation: ( mij {t} if z ij = D, m ij (6) m ij {} otheise. We add t to m ij if the cache status z ij is dity, otheise is added to m ij. If e implement a M dive that a physically contiguous memoy is allocated to each stip, a ead goup that is composed only of can be made fo a single equest ithout a scatte-gathe list, by hich a single pocedue call sequentially eads data fom a single data steam to multiple buffes. A ead goup that contains t may be meged into a single equest using a scatte-gathe list. In contast to the ead contiguity tansfom, the ite contiguity tansfom can not be pocessed in the case that at least one cache block, hich is empty (E) and contains no, exists in the discontiguous egion that is located beteen the to sepaated blocks. By, the empty cache block becomes clean (C) befoe the ite execution to the disk. It is possible, theefoe, to add into an x element containing, although the coesponding block cache is empty. Theefoe, the ead contiguity tansfom must pecede the ite contiguity tansfom. In othe ods, the ite contiguity tansfom beteen m aj and m bj must satisfy the folloing pedicate: l l[ (a<l<b) (z lj = E) ( / m lj )]. (7) Fo all m lj that ae in the discontiguous egion beteen m aj and m bj, thee must be no m lj such that its cache status, z lj, is empty and it does not contain. Fo example, m 4 and m 44 in Fig. 7(b) fom a discontiguous ite opeation but m 4 of the final x-matix shon in Fig. 7(d) does not contain and z 4 is empty. Hence it is impossible to make a ite contiguity tansfom in this case. In the othe case, blocks beteen m and m 7, z and z 6 ae empty, and m and m 6 of the basic x-matix do not contain. Hoeve, m and m 6 ill contain afte the ead contiguity tansfom (see Fig. 7(d)), theeby containing the latest data in the cache befoe the ite execution and enabling the ite contiguity tansfom of m,m 4 and m 5,m 7. The ead contiguity tansfoms of m 6, m 7, and m 5 also enable the ite contiguity tansfoms. D. Rules fo Pefomance To guaantee the pefomance of CT ithout degadation, e intoduce a teminology, stide distance, hich is defined by the numbe of blocks beteen to discontiguous ites. In othe ods, if to discontiguous ites ae located at block position a and b and if a ite opeation beteen a and b does not exist, then the stide distance is (b a). In addition, the stide distance fo a ead is applied in the same ay. The ite contiguity tansfom is esticted by automatically detemined paametes, S and S except, that ae shon in Fig. 9. S is the theshold value of the stide distance S in ode to limit a ite contiguity tansfom pocess. A ite contiguity tansfom is pemitted hen its stide distance, S, is less than S and not included in any excluded egions that ae detemined by S except (= {S except,,s except,,...}). Fig. 9(d) Stide distance 4 I/O sequence B B B B B4 B5 B6 B7 B8 B9 B B B B B4 B5 B B B4 B6 B8 B B B6 B9 B B4 B8 Fig. 8. The stide benchmak geneates the okload in stide patten by vaying the stide distance. All okloads employ the same stat and end positions. The stide distance of denotes the contiguous I/O. When the stide distance is i, afte one block is ead o itten, the consecutive (i ) blocks ae skipped and this cycle epeats. B B B B B4 B5

8 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 8 nomalized tunaound time S S ead ite stide distance, S [blocks] nomalized tunaound time S S ead ite stide distance, S [blocks] (a) ST7454LC (Seagate 5pm, SI) (b) STAS (Seagate 7pm, SATA) nomalized tunaound time S S ead ite stide distance, S [blocks] nomalized tunaound time S = [ S, S ] S except except S S,, s,, e except except except,,, s,, e except except = { S, } S S ead ite stide distance, S [blocks] (c) 6V6E (Maxto 7pm, SATA) (d) WD5JD (WestenDigital 7pm, SATA) Fig. 9. S, S, S except, and the nomalized tunaound time vesus the stide distance fo vaious disks. A fagmented I/O hose nomalized tunaound time is belo one is faste than the contiguous I/O. shos an excluded egion, S except,, that anges fom S except,,s to S except,,e. If thee exists a second excluded egion, it can be expessed as S except,. Fo the ead contiguity tansfom, S and S except ae applied in the same manne. S, S except, S, and S except ae obtained fom a membe disk of a disk aay by a stide benchmak, hich is automatically pefoms hen an administato ceates the disk aay. The obtained paametes ae saved in a non-volatile stoage, on hich the othe infomation of the disk aay is also stoed. Hence, the stide benchmak that obtains the paametes pefoms only once. It is possible to etieve the paametes fom the non-volatile stoage ithout executing the stide benchmak any moe. To find the statistical and elative costs of discontiguous I/Os ove the contiguous I/O shon in Fig. 9, the stide benchmak geneates the okload in a stide patten by vaying the stide distance as shon in Fig. 8. The stide distance S of indicates the contiguous I/O. If S is i, afte one block is ead o itten, the next consecutive (i ) blocks ae skipped, and this cycle epeats. In this ay, the stide benchmak epeats such a patten to statistically evaluate a fagmented I/O ith a specific stide distance. All okloads stat at the same souce block and stop at the same destination block. Figue 9 displays the paametes and the nomalized tunaound time vesus the stide distance fo vaious disks. A fagmented I/O is sloe than the contiguous I/O if its nomalized tunaound time is lage than. In all of the disks shon in Fig. 9, a fagmented I/O such that its stide distance S is less than a theshold shos degaded pefomance. The theshold value can be the imum stide distance, S fo ite o S fo ead. Theefoe the ite contiguity tansfom pefoms hen its S is less than S. S is chosen as the smallest S such that the nomalized tunaound time is less than. fo all S.. is used instead of. because it is necessay conside the toleance of the benchmak esults and the pefomance degadation caused by additional I/Os that occupy additional bus esouces such as the SI bus and the memoy bus. The theshold value of. as obtained by some empiical esults, hence it may not be optimal is also detemined in the same ay. This selection method of S and S is validated though an expeiment, hich is descibed in Section IV-F. We can fomally descibe the pefomance ules fo the ead contiguity tansfom, hose stide distance is denoted by S(= b a), as the folloing pedicate: to apply the value to all systems. S [(S<S ) ( k k[s except,k,s S S except,k,e ]) ]. (8) S must be less than S and not be included in [S except fo all k. Similaly, the ite contiguity tansfom must satisfy the folloing pedicate: [(S<S ) ( k k[s except,k,s,k,s,sexcept,k,e ] S S except,k,e ]) ]. (9) In summay, the total ule fo the ead contiguity tansfom is constucted by aggegating (6), (4), and (8). The total ule fo the ite contiguity tansfom is constucted by aggegating (5), (7), and (9). Some disks, as shon in Fig. 9(d), may have excluded egions, S except o S except. CT does not pefom if S is included in such excluded egions because it is impossible to achieve a gain though CT. Unlike S, some disks have a S value of, as shon in Figs. 9(b) and 9(c).

9 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 9, S except ) fo (j ; j v; j++) { // fo each column p ; fo (i ; i u; i++) { // fo each o 4 if (m ij {, t}) ) { 5 S i p; ReadContiguityTansfom(M, Z, S 6 if (p and S<S and (S <S except,k,s o S>S except,k,e 7 fo (h p +; h<i; h++) { 8 if (j = v) m hj m hj {}; 9 else if (z hj = D) m hj m hj {t}; else m hj m hj {}; } } p i; 4 }}} WiteContiguityTansfom(M, Z, S, S except ) 6 fo (j ; j v; j++) { // fo each column 7 p ; 8 fo (i ; i u; i++) { // fo each o 9 if ( m ij ) { S i p; if (p and S<S, fo all k)) { and (S <S except,k,s o S>S except,k,e, fo all k)) { fo (h p +; h<i; h++) { if (j = v) { 4 if ( / m hj ) goto Next; 5 } else if ( / m hj and z hj = E) goto Next; 6 } 7 fo (h p +; h<i; h++) 8 m hj m hj {}; 9 } Next: p i; }}} Fig.. The pocedues of CT If S is, thee is no ite contiguity tansfom that is assisted by the ead contiguity tansfom (see Section III-C). All eads shon in Fig. 9 ae neve belo one. In othe ods, the thoughput may be lightly affected even if e do not obey the ule of the nomalized tunaound time of. to detemine S Theefoe, if e aggessively incease S ithout obeying the. theshold value of. in ode to incease the possibility of the ite contiguity tansfom, e may achieve bette pefomance. So e plan to eseach a bette scheme of an aggessive tansfom. E. Pocedue Figue shos the pseudo code that epesents the pocedues of CT. Lines 4 and esolve to discontiguous eads, m pj and m ij. Line 6 detemines that the condition of the stide distance satisfies the pefomance ule. If the condition is tue, lines 7- add o t into all elements beteen m pj and m ij by the consistency ule. If column j is the paity column, is added ithout checking the cache status (line 8) because the cache fo paity blocks is not employed. In the ite contiguity tansfom, if the condition of the stipe distance satisfies the pefomance ule (line ), lines -6 detemine that it is possible to pefom the ite contiguity tansfom beteen m pj and m ij by the consistency ule (Eq. (7)). The discontiguous ite egion is skipped by jumping to line if thee is at least one element that does not satisfy the consistency ule. If column j is the paity column (line ), e do not check the cache status (line 4). Finally, is added to all elements beteen m pj and m ij (lines 7-8). F. Pefomance Issues The poposed scheme causes a negligible amount of pefomance degadation because CT equies vey inexpensive computational poe that investigates hich blocks ae applicable to CT. The ovehead of CT is that additional eads and ites consume additional bus esouces, such as those of the SI bus and memoy bus. Hoeve, fagmented ites significantly deteioate ite pefomance, theeby aely satuating those buses. Moeove, ou expeimental esults sho that CT enhances pefomance to a much geate extent than the degadation that is caused by the additional bus occupation. The amount of pefomance enhancement depends on the amount of the fagmented ites and on the distibution of the stide distance. The poposed CT scheme employs no gain fo the puely contiguous ite, but affects vaious types of ites that include () concuent sequential ites, () sequential ites on an aged filesystem, () sequential ites of multiple small files, (4) andom ites, and so on. The benefit fom M-CT depends on the intenal chaacteistics of the disk dives and on the potion of the closely discontiguous ites that depend on the ite cache size and the okspace, to hich equested I/O locations ae bounded in tems of stoage capacity. The aveage stide distance inceases as the okspace inceases, and the aveage stide distance deceases as the ite cache size inceases. CT has geate benefits fo shote distibutions of the stide distance. CT pefoms fo ites and eads fo paity-update but not ead equests fom the host. A ead equest cannot be delayed to the disk unlike a ite equest. Hence a ead equest must immediately espond to the host ith data fom the disk. Hoeve CT equies multiple pending I/Os that fom as sequentially discontiguous. Theefoe, it is impossible to apply CT to the ead equests. M is a stipe cache managed by the x-matix fo the contiguity tansfom (CT) of fagmented ites. By coalescing I/Os togethe, allos fo a bette exploitation of spatial locality than paity block goup cache (PBGC). simplifies cache management and contol, theeby enhancing bulky o sequential I/Os. CT depends highly on the stip size of because CT pefoms ithin the stip. The imum stide distance fo CT is esticted by the stip size if the numbe of blocks pe stip is less than S o S. G. Implementation The RAID can be implemented eithe using a dedicated computing device, called hadae RAID, o using a host pocesso, called softae RAID. Teigland [] discussed seveal softae RAIDs that include MutliDevice of Linux, Veitas Volume Manage, Sun Solstice DiskSuite, and FeeBSD Vinum. In Linux.6, e have implemented a softae RAID-5 ith the M-based CT (M-CT) scheme as a block device dive. To be pecise, this is the M dive, hich includes a stipe pefetching scheme [], a configuation tool fo administatos, and a status notification inteface using the /poc filesystem. Fo complete implementation of the M dive, e exploit seveal design techniques as follos:

10 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X ) When and What to Destage: An impotant decision fo destage is When to Destage. We chose a destage schedule as the ell-knon schedule, the high/lo mak schedule [6]. Destages to disks ae disabled hen the cache occupancy is belo a lo mak (85%). If the cache occupancy is at o above a high mak (95%), then the destage is continued. Bette destage schedules such as the linea theshold (LT) schedule [5], vaiants of LT [], and a combination of LT and HLM [4] may be consideed in the design, but destage schedules ae not in the scope of this pape. Upon stating the destage, e must detemine hich block o stipe is selected to be destaged. Fo a fai compaison ith the existing RAID-5 dive of Linux, the simplest scheme, LRW, as used in ode to decide hat to destage. ) Lock: A lock management of ou implementation fo paity access pefoms in tems of stipe goups, theeby esulting in a coase ganulaity compaed to a lock that is managed in tems of paity block goups. Hoeve ou lock management is simple, theeby poviding easy implementation and efficient pefomance fo sequential o bulky I/Os. The coase-gained lock is somehat acceptable, except fo a ite lock hile eading. The ead lock can ente into the citical section that as peviously ganted ith anothe ead lock. The ead lock can not be ganted if the section is locked ith a ite mode. Hoeve, the ganted duation of the ite lock is vey shot because the ite fom the host to the NVS pefoms meely by copying equested data to the NVS. Destaging a stipe acquies the ite lock, hich delays ead equests and ite equests fo the stipe until the destage of the stipe finishes. Hoeve, the stipe selected fo destage may be the least fequently o least ecently used stipe, thus ead o ite equests fo the stipe, hich is unde destage, ae aely claimed. ) Memoy allocation: To altenative memoy allocation schemes can exist fo. The fist is that cache memoies ae only assigned to blocks that contain valid data, theeby inceasing memoy utilization. The second is that a physically-contiguous memoy of the stipe size is assigned to the stipe cache unit, hich, although, holds only a single clean o dity block. The latte scheme deceases memoy utilization fo idely-spead andom small I/Os; hoeve, it can educe the numbe of memoy allocation pocesses by an ode of magnitude, and is efficient fo sequential o bulky I/Os. We adopted the latte allocation scheme. IV. EVALUATION A. Expeimental Set-up This section descibes a system that e built to measue the pefomance of M-CT and, and to compae them ith the existing RAID-5 dive of Linux, MultiDevice (MD), the cache of hich is managed in tems of PBG. In the souce code of MD, the stipe head stuctue epesents the PBG. Hsieh et al. [4] evaluated the softae RAID of Linux, MD, hich has a compaable pefomance as a hadae RAID fo most of test cases that ae pefomed by them. The system in the expeiments uses dual. GHz Xeon pocessos, GB of main memoy, to Adaptec SI host bus adaptes, and five ST7454LC disks, each of hich has a speed of 5 pm and a 74 GB capacity. A Linux kenel (vesion.6.) uns on this machine hosting benchmak pogams, the ext filesystem, and the poposed M dive o the MD dive as a RAID-5 dive. Apat fom the page cache of Linux, both Thoughput [MB/s] PBGC M-CT okspace size [GB] Fig.. A thoughput compaison fo andom ite okloads by vaying the okspace size using Tiobench. As the okspace size inceases, the gain of M-CT ove deceases but neve become negative. the MD dive and the M dive have thei on cache memoy of 5 MiB. The block size is set to 4 KiB. All though the figues and tables of this pape, PBGC (paity block goup cache, Fig. 6(a)) denotes the PBGC scheme using the MD dive, (stipe cache, Fig. 6(b)) hen using the M dive ithout CT, and M-CT hen using the M dive ith CT. B. Random Wite Wokload We use a andom ite okload using Tiobench [5], hich geneates multiple theads that unifomly ite 4 KiB pages in thei on file. In Fig., e compae PBGC,, and M- CT using andom ite okloads ith a fixed numbe of ites (6k) on a RAID-5 aay by vaying the size of the okspace. A GB file is allocated fo each thead, and e vay the numbe of theads. Hence the okspace inceases as the numbe of theads inceases. At the loest okspace size of GB, M-CT outpefoms by 7%, and M-CT outpefoms PBGC by %. With a okspace size of 8 GB, M-CT outpefoms by 8%, and outpefoms PBGC by %. As the okspace size inceases, the gain of M-CT ove deceases but ill not be negative because the inceased okspace size ith the fixed cache size causes fo the aveage stide distance to incease and CT only pefoms hen a stide distance is less than S o S. Theoetically, if the okspace size is infinite, the thoughput of M-CT is nealy identical to that of because thee is no CT ith an infinite stide distance. Figue shos vaious metics such as thoughput, aveage latency, imum latency, and thoughput ove CPU load by vaying the numbe of ites fom a host ith a fixed okspace size of.5 GB using Tiobench. In Fig. (a), the incease in the thoughput is satuated as the numbe of I/Os inceases. Meanhile, too small a numbe of ites cannot consume the entie cache befoe the destaging of the cache and deceases the numbe of dity blocks in a stipe, theeby inceasing the aveage stide distance and educing the tansfom oppotunity. With the smallest numbe of ites (4k), the thoughput gain of M- CT ove is 5%. Hoeve, ith the lagest numbe of ites (56k), M-CT outpefoms by 4%. shos slightly less thoughput than PBGC due to the memoy allocation scheme of ou implementation but not itself. The memoy allocation of a full stipe size povides less memoy utilization than the MD dive.

11 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X Thoughput [MB/s] PBGC M-CT Aveage latency [ms] PBGC M-CT Numbe of ite opeations [k] Numbe of ite opeations [k] (a) Thoughput (b) Aveage latency Maximum latency [ms] PBGC M-CT Thoughput/Load PBGC M-CT Numbe of ite opeations [k] Numbe of ite opeations [k] (c) Maximum latency (d) Thoughput / CPU load Fig.. Vaious metics by vaying the numbe of I/Os using Tiobench Figue (b) shos the aveage latency in this expeiment. The aveage latency shos a simila tend to the thoughput. When the numbe of ites is 56k, M-CT delives the imum latency that is 6% less than. PBGC delives seveely pooe imum latency than ; nealy 56 times ose. This significant diffeence in the imum latency may depend on the diffeent implementation techniques of the MD dive fo PBGC and the M dive. It is necessay to measue the amount of CPU load degadation by the additional I/Os caused by CT. Thoughput ove CPU load may be an adequate metic fo this measuement. Accoding to Fig. (d) that shos this metics, it becomes clea that PBGC,, and M-CT have simila values of thoughput ove CPU load. Theefoe, CT aely has a CPU load degadation. C. Concuent Sequential Wite To geneate concuent sequential I/Os, e used the benchmak DBench [6], hich poduces the local filesystem load. It does all the same I/O calls that the smbd seve in Samba [7] ould poduce hen confonted ith a Netbench [8] un. DBench geneates multiple theads, each of hich poduces vaious I/O pattens. Hoeve, a lage potion of its ites fom sequential o bulky accesses. Figue compaes PBGC,, M-CT, and a hadaebased RAID using DBench. The hadae-based RAID is an Intel SRCU4X [9] ith 5MB memoy and ite-back/cached-io capability. The benchmaks used in this pape, except fo DBench, cannot faily compae SRCU4X ith the softae RAID dives because thee is no inteface to fully flush the cache of SRCU4X. This can only be accomplished by shutting don its device dive. Hoeve, DBench excludes the esults of the cleanup phase and thus it is unnecessay to fully flush the cache. Thoughput [MB/s] Thoughput [MB/s] Fig PBGC M-CT SRCU4X The numbe of clients (a) Vaying the numbe of clients PBGC M-CT SRCU4X stip size [KiB] (b) Vaying the stip size A compaison of PBGC,, and M-CT using DBench

12 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X Thoughput [MB/s] Thoughput [MB/s] Fig M-CT Aveage file size [KiB] (a) Vaying the aveage file size Cycle of deletion-ceation (b) Aging cycle M-CT A compaison of and M-CT on a synthetically aged filesystem We vay the numbe of clients fom to 56 in Fig. (a). When the numbe of clients is, thee is no gain by CT because most of the ites that each client poduces ae sequential o bulky. Hoeve, M-CT significantly outpefoms, ith the exception of a single thead and the ange that the pefomance is satuated. In the best case, M-CT outpefoms by %, and PBGC by 59%. Though this expeiment, it becomes clea that concuent sequential ites at the filesystem level may poduce fagmented I/Os at the block level. In Fig. (b), e vay the stip size fom 8 to 56 ith 8 clients. When the stip size is 8, thee is no discontiguous egion in a stip because one 8 KiB stip consists of only to 4 KiB blocks. As the stip size inceases, the gain by CT inceases. In the best case, M-CT outpefoms PBGC by 8%. The thoughput of deceases as the stip size inceases as a esult of the memoy allocation scheme of ou implementation. Hoeve, significantly outpefoms PBGC in all of the cases shon in Fig.. D. Aged Filesystem In actual filesystems, files ae divided into pieces scatteed aound the disk, thus sequential ites at the filesystem level may appea as fagmented ites at the block level. We made a benchmak to atificially age a filesystem. Up to 9% of the stoage capacity, the benchmak sequentially ites files, the sizes of hich have a nomal distibution ith a mean value μ and standad deviation μ/4. The benchmak deletes % of the files, and then ceates such files up to 9% of the stoage capacity. To incease the fagmentation of the filesystem, the deletion and ceation cycles ee epeated seveal times. This expeiment uses an ext [4] filesystem of GB. As shon in Fig. 4(a), the gain by M-CT in an aged filesystem TABLE II EXPERIMENTAL RESULTS OF BULKY I/OS Benchmaks PBGC (MD) M-CT video self-copy.4 s.6 s.6 s music self-copy.6 s.5 s.5 s sc self-copy 9. s 5. s.5 s mkfs (ext) 4. s 4. s 4. s sequential ead (fs) 9 MB/s 5 MB/s 5 MB/s sequential ite (fs) 66 MB/s 89 MB/s 89 MB/s bulky andom ead (fs) 57.5 MB/s 67.4 MB/s 67.4 MB/s bulky andom ite (fs) 58.8 MB/s 68. MB/s 68. MB/s sequential ead (block) 6 MB/s 8 MB/s 8 MB/s sequential ite (block) 68 MB/s 89 MB/s 89 MB/s bulky andom ead (block) 8 MB/s 85 MB/s 85 MB/s bulky andom ite (block) 8 MB/s 94 MB/s 94 MB/s is evaluated by epeating the pocess of deletion and ceation 5 times. Fo an aveage file size of 6 KiB, M-CT is. times faste than. Fo an aveage file size of 48 KiB, M- CT outpefoms by 6%. The smalle the file, the bette the thoughput. The aveage file size of the /us folde in a Linux filesystem is appoximately KiB. The size of photo files anges fom MiB fom MiB. Theefoe the expeimental esults help infe the enhancement of M-CT in tems of idely used files. Figue 4(b) shos the tend of the gain of M-CT on the aging amount of a filesystem ith an aveage file size of 6 KiB. Fo the tentieth aging cycle ove the fist aging cycle, is degaded by 86% but M-CT is degaded by only 7%. E. Sequential o Bulky I/O To investigate the effect of sequential o bulky I/Os on M- CT, e pefomed vaious expeiments, as shon in Table II, hee self-copy denotes the iting of files, that ae ead fom a stoage device, to a diffeent diectoy on the same stoage device. The video, music, and sc benchmaks ee pefomed by a single file of. GB, 54 mp files of 68 MB, and 568 Linux kenel souce and object files of 458 MB. We also pefomed sequential-andom ead-ite tests both at the filesystem level using Tiobench and at the block level ithout a filesystem, hee the equest size and the bounday of the andom I/O ae aligned in the stipe so that thee is no contiguity tansfom. The stipe size as set to 5 KiB in this expeiment. M-CT shos the same thoughput as at video selfcopy, music self-copy, and sequential/andom ead/ite, hile significantly outpefoms PBGC in all of the cases, as shon in Table II because exploits moe spatial locality than PBGC. delives 45% bette thoughput than PBGC fo video selfcopy, and 7% highe thoughput than PBGC fo sc self-copy. In addition, the M dive significantly outpefoms the MD dive. In paticula, the best incease in speed of 7% as achieved fo the sc self-copy benchmak. Theefoe, M-CT employs no disadvantage fo sequential o bulky I/Os, and delives a moe efficient pefomance than PBGC fo sequential o bulky andom I/Os. M-CT outpefoms by % fo the sc self-copy benchmak, hich sequentially ites a lage numbe of small files. The sc self-copy benchmak poduces fagmented ites that ae caused by the pe-allocation scheme used in the ext filesystem:

13 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X Nomalized tuaound time D vaying S and S. Fig. 9 shos S and S that ee selected by the selection method descibed in Section III-D. Fig. 9(a) ith a stide patten shos the same S and S as the fastest cases shon in Fig. 5 ith an exponential distibution. The paametes, S and S, that ae obtained by the stide benchmak by using the stide patten, ae applicable to ealistic okloads. Nomalized tunaound time Maximum stide distance fo the ite CT (a) ite D Maximum stide distance fo the ead CT (b) ead Fig. 5. By vaying S and S, CT is applied to a fagmented ead and ite ith an exponential distibution. The optimal S and S should poduce the fastest tunaound time. hen a ne disk data block is allocated, a filesystem such as ext o ext intenally pe-allocates a small numbe of disk blocks adjacent to the just allocated block in ode to expand files in the futue []. Hence, thee exists pe-allocated, unused, blocks beteen files. The block-level thoughput of the MD dive (PBGC) shos sevee degadation because the anticipatoy disk schedule [], hich is the default disk schedule of Linux.6, does not function ell ith the MD dive due to the ite-afte-ead patten. The anticipatoy disk schedule expects the ead-afte-ead patten. Ou M dive shos the full thoughput of the physical devices by epaiing the poblem that the scheduling stategy of the MD dive intefees ith the anticipatoy disk schedule, hee e neithe modified no disabled the anticipatoy disk schedule. The bulky andom ite of M-CT is bette than the bulky andom ead because the anticipatoy disk schedule significantly educes the aveage seek distance of ites. F. Expeimental Veification of S and S This section veifies the selection method of S and S that is pefomed by the epeating stide patten descibed in Section III-D. The method selects S and S as the minimum stide distance, S, ith a nomalized tunaound time of less than.. Figue 5 shos the nomalized tunaound time of the disk ST7454LC, hich is shon in Fig. 9(a). The block inteval beteen sequentially discontiguous blocks is exponentially distibuted, hee e use the mean value of. We assume that the stipe size is 8 KiB (= 64 blocks ith the 4 KiB block size), hence thee is no stide distance that exceeds 64 blocks ithin the 8 KiB stipe. Theefoe, e limit the imum stide distance up to 64 blocks. CT as pefomed on the okload by V. CONCLUSION Conventional destage algoithms typically focus on hen and hat to destage. Hoeve, the poposed scheme, M-CT, solves ho to efficiently destage fagmented ites. It as found that a sequentially contiguous ite outpefoms a sequentially discontiguous ite, even if the to types of ites employ the equivalent seek distance. The stipe cache that manages the cache in tems of stipe goups convets all ites fom the host into sequentially contiguous o sequentially discontiguous eads fo paity-updates and ites to the disk. M-CT geneates the xmatix to destage a stipe and tansfoms to discontiguous eads o ites into a contiguous ead o ite by inseting additional eads o ites, theeby educing fine-gained fagmented eads and ites. M-CT exploits the ules fo consistency and pefomance, hich enable data to be consistent ithout filesystem dependency, data modification, and pefomance degadation. Additional eads of the ead contiguity tansfom can assist the ite contiguity tansfom by the x-matix that sepaates the ead stage fom the ite stage in tem of stipe. Theefoe M- CT is effective to disk aays ith sophisticated fault-toleant schemes such as RAID-5, RAID-6, and Hieachical RAID [4], hich equie additional eads to update edundant infomation. In such disk aays, a fagmented ite poduces a fagmented ead, theeby also obtaining the gain by the ead contiguity tansfom and the highe possibility of the ite contiguity tansfom. We have implemented the M dive fo RAID-5 as a block device dive of Linux, and compaed it ith the MD dive that is a softae RAID dive of Linux. We used vaious types of okloads that include andom ites using Tiobench, multiple concuent sequential ites using DBench, sequential ites on an atificially-aged filesystem, and sequential ites of multiple small files on an ext filesystem. M-CT delives a peak thoughput that is. times highe than the taditional scheme in a andom okload. In a setup-up fo concuent sequential I/O using DBench, M-CT delives a peak gain that is 8% highe than MD. In a sequential ite on an aged filesystem ith an aveage file size of 6 KiB, M-CT delives 8% thoughput gain. Futhemoe, in a sequential copy of small files, M-CT shos.7 times bette thoughput compaed to MD. M-CT outpefoms MD by 45% in a video file copy. In summay, M-CT is a destage algoithm that destages fagmented I/Os in a RAID contolle. M-CT is extemely simple to implement, has lo ovehead, and is ideally suited fo RAID contolles fo andom I/Os as ell as sequential I/Os. It as demonstated that the RAID-5 implementation of M-CT significantly outpefoms the existing RAID-5 dive of Linux in a vaiety of ealistic scenaios. REFERENCES [] P. M. Chen, E. K. Lee, G. A. Gibson, R. H. Katz, and D. A. Patteson, RAID: High-pefomance, eliable seconday stoage, ACM Computing Suveys, vol. 6, no., pp , June 994.

14 This aticle has been accepted fo publication in a futue issue of this jounal, but has not been fully edited. Content may change pio to final publication., VOL. X, NO. X, MONTH X 4 [] J. Menon and J. Cotney, The achitectue of a fault-toleant cached aid contolle, in th Annual Intenational Symposium on Compute Achitectue, May 99, pp [] M. Bake, S. Asami, E. Depit, J. Oustehout, and M. Seltze, Nonvolatile memoy fo fast, eliable file systems, in Intenational Confeence on Achitectual Suppot fo Pogamming Languages and Opeating System (ASPLOS), vol. 7, no. 9, 99, pp.. [4] B. S. Gill and D. S. Modha, WOW: Wise odeing fo ties - combining spatial and tempoal locality in non-volatile caches, in USENIX Confeence on File and Stoage Technologies, 5, pp [5] A. Vama and Q. Jacobson, Destage algoithms fo disk aays ith non-volatile caches, in High Pefomance Mass Stoage and Paallel I/O: Technologies and Applications. IEEE Compute Society Pess and Wiley,, pp [6] P. Bisas and K. K. Ramakishnan, Tace diven analysis of ite caching policies fo disks, in ACM SIGMETRICS Confeence on Measuement and Modeling of Compute Systems, May 99, pp.. [7] P. H. Seaman, R. A. Lind, and T. L. Wilson, An analysis of auxiliaystoage activity, IBM Systems Jounal, vol. 5, no., pp. 58 7, 966. [8] F. J. Cobatȯ, A paging expeiment ith the multics system, in In Hono of P. M. Mose. MIT Pess, 969, pp [9] G. A. Gibson and R. V. Mete, Netok attached stoage achitectue, Communications of the ACM, vol. 4, no., pp. 7 45, Nov.. [] T. Y. Ts o and S. Teedie, Planned extensions to the linux ext/ext filesystem, in USENIX Annual Technical Confeence, FREENIX tack, June, pp [] Pefixes fo binay multiples, Intenational Electotechnical Vocabulay (IEC 67-). [Online]. Available: si bytes.htm [] S. Iye and P. Duschel, Anticipatoy scheduling: A disk scheduling fameok to ovecome deceptive idleness in synchonous i/o, in USENIX Annual Technical Confeence,, pp. 7. [] B. L. Wothington, G. R. Gange, Y. N. Patt, and J. Wilkes, On-line extaction of SI disk dive paametes, The Univesity of Michigan, Tech. Rep. CSE-TR--96, 996. [4] K. A. Smith and M. I. Seltze, File system aging - inceasing the elevance of file system benchmaks, in Measuement and Modeling of Compute Systems, 997, pp.. [5] M. Rosenblum and J. K. Oustehout, The design and implementation of a log-stuctued file system, ACM Tansactions on Compute Systems, vol., no., pp. 6 5, 99. [6] W. H. Ahn, K. Kim, Y. Choi, and D. Pak, DFS: A de-fagmented file system, in th IEEE Intenational Symposium on Modeling, Analysis, and Simulation of Compute and Telecommunications Systems (MAOTS ),, pp [7] E. Coffman, L. Klimko, and B. Ryan, Analysis of scanning policies fo educing disk seek times, SIAM Jounal of Computing, vol., no., pp , 97. [8] P. J. Denning, Effects of scheduling on file memoy opeations, in AFIPS Sping Joint Compute Confeence, Ap. 967, pp. 9. [9] R. Geist and S. Daniel, A continuum of disk scheduling algoithms, ACM Tansactions on Compute Systems, vol. 5, no., pp. 77 9, Feb [] B. L. Wothington, G. R. Gange, and Y. N. Patt, Scheduling algoithms fo moden disk dives, in Poceedings of the 994 ACM SIGMETRICS Confeence on Measuement and Modeling of Compute Systems, 6 994, pp [] M. Seltze, P. Chen, and J. Oustehout, Disk scheduling evisited, in USENIX Winte Tech. Conf., 99, pp. 4. [] S. P. VandeWiel and D. J. Lilja, Data pefetch mechanisms, ACM Computing Suveys, vol., no., pp ,. [] N. Megiddo and D. S. Modha, SARC: Sequential pefetching in adaptive eplacement cache, in USENIX Annual Technical Confeence, 5, pp [4] S. Bansal and D. S. Modha, CAR: Clock ith adaptive eplacement, in USENIX Confeence on File and Stoage Technologies, Ma 4, pp. 87. [5] N. Megiddo and D. S. Modha, Outpefoming LRU ith an adaptive eplacement cache algoithm, IEEE Compute, vol. 7, no. 4, pp , Apil 4. [6] S. Jiang, X. Ding, F. Chen, E. Tan, and X. Zhang, DULO: An effective buffe cache management scheme to exploit both tempoal and spatial localities. in USENIX FAST, 5. [7] J. Wikes, R. Golding, C. Staelin, and T. Sullivan, The HP AutoRAID hieachical stoage system, ACM Tans. on Compute Systems, vol. 4, no., pp. 8 6, Feb 996. [8] S. Savage and J. Wilkes, AFRAID A fequently edundant aay of independent disks, in Poceedings of the 996 USENIX Technical Confeence, 996, pp [9] K. Yeung and T. Yum, Dynamic multiple paity (DMP) disk aay fo seial tansaction pocessing, IEEE Tansactions on Computes, vol. 5, no. 9, pp , Sept.. [] The RAIDBook: A Souce Book fo RAID Technology sixth edition. The RAID Advisoy Boad, Lino Lakes MN, 999. [] D. Teigland and H. Mauelshagen, Volume manages in linux, in USENIX Annual Technical Confeence, June, pp [] S. H. Baek and K. H. Pak, Massive stipe cache and pefetching fo massive file I/O, in IEEE Intenational Confeence on Consume Electonics, Jan 7, pp [] C. P. Young Jin Nam, An adaptive high-lo ate mak destage algoithm fo cached RAID5, in Pacific Rim Intenation Symposium on Dependable Computing (PRDC ). IEEE Compute Society Pess and Wiley,, pp [4] J. Hsieh, C. Stanton, and R. Ali, Pefomance evaluation of softae aid vs. hadae aid fo paallel vitual file system, in ICPADS : Poceedings of the 9th Intenational Confeence on Paallel and Distibuted Systems,, p. 7. [5] Tiobench - Theaded I/O bench fo Linux. [Online]. Available: [6] M. Vieia and H. Madeia, A dependability benchmak fo oltp application envionments, in Poceedings of the 9th VLDB Confeence,. [7] R. Shape, Just hat is SMB?. [Online]. Available: http: //samba.og/cifs/docs/hat-is-smb.html [8] G. Memik, W. H. Mangione-Smith, and W. Hu, NetBench: A benchmaking suite fo netok pocessos, in ICCAD,, pp [9] Intel Copoation, Intel RAID contolle SRCU4X ovevie, 6. [Online]. Available: index.htm [4] S. Teedie, Jounaling the Linux extfs filesystem, in LinuxExpo 98, 998. [4] S. H. Baek, B. W. Kim, E. J. Joung, and C. W. Pak, Reliability and pefomance of hieachical RAID ith multiple contolles, in Tentieth ACM Symposium on Pinciples of Distibuted Computing, Aug, pp Sung Hoon Baek eceived the B.S. degee in electonics engineeing fom Kyungpook National Univesity, Koea in 997 and the M.S. degee in electical engineeing fom the Koea Advanced Institute of Science and Technology (KAIST) in 999. He oked fo Electonics Telecommunication Reseach Institution (ETRI) as a engineeing staff fom 999 to 5. He is cuently pusuing a Ph.D. degee in the division of electical engineeing at KAIST. His eseach inteests include stoage systems, flash file systems, and embedded systems. Kyu Ho Pak eceived the B.S. degee in electonics engineeing fom Seoul National Univesity, Koea in 97, the M.S degee in electical engineeing fom the Koea Advanced Institute of Science and Technology (KAIST) in 975, and the D.Ing. degee in electical engineeing fom the Univesity de Pais XI, Fance in 98. He has been a pofesso in the division of electical engineeing of KAIST since 98. He as a pesident of Koea Institute of Next Geneation Computing fo the peiod of 5-6. His eseach inteests include compute achitectues, file systems, stoage systems, ubiquitous computing, and paallel pocessing. D. Pak is a membe of KISS, KITE, Koea Institute of Next Geneation Computing, IEEE, and ACM.