EC-FRM: An Erasure Coding Framework to Speed up Reads for Erasure Coded Cloud Storage Systems

Transcription

1 215 44th Intenational Confeence on Paallel Pocessing EC-FRM: An Easue Coding Famewok to Speed up Reads fo Easue Coded Cloud Stoage Systems Yingxun Fu, Jiwu Shu *, and Zhiong Shen Tsinghua National Laboatoy fo Infomation Science and Technology Depatment of Compute Science and Technology, Tsinghua Univesity Beijing, China, 184 * Coesponding Autho: shujw@tsinghua.edu.cn mooncape1986@126.com, zhiong.shen261@gmail.com Abstact With the eliability equiements inceasingly impotant, easue codes have been widely used in today s cloud stoage systems because they achieve both high eliability and low stoage ovehead. Howeve, the pefomance fo most existing easue codes can be futhe impoved on both nomal eads to use s data without device failues and degaded eads unde device failues, which ae cucial in cloud stoage systems. In this pape, we popose an easue coding famewok named EC-FRM to integate existing codes in ode to impove the ead pefomance. The constucted code ove EC-FRM named EC-FRM-Code, which keeps most of wondeful popeties of the integated code and achieves good pefomance on both nomal eads and degaded eads. We tansfom Reed-Solomon code and LRC code to EC-FRM- RS and EC-FRM-LRC espectively, and then conduct a seies of expeiments to evaluate thei ead pefomance. The esults show that EC-FRM-RS code gains 19.2% to 33.9% highe nomal ead speed and 9.1% to 9.9% highe degaded ead speed than standad Reed-Solomon code, while EC-FRM-LRC code owns 23.5% to 46.9% highe nomal ead speed and 3.3% to 12.8% highe degaded ead speed than standad LRC code. Keywods-Easue Code; Cloud Stoage System; Nomal Read; Degaded Read; I. INTRODUCTION With the data size apidly inceasing, easue codes have eceived moe and moe attentions in today s cloud stoage systems because they achieve both low stoage ovehead and high eliability [1][2]. Cloud stoage system using easue code to assue the eliability foms easue coded cloud stoage system, which tansfes the equiements of taditional easue codes due to the fact that they usually adopt append-only wite wokloads and lage block size. In easue coded cloud stoage systems, wites ae usually accumulated in the end of the files and buffeed in memoy o devices until a block is fully witten and then the blocks is easue coded (i.e., full stipe wites) [3]. Since existing easue codes usually pefom similaly on full stipe wites, the pefomance on wites is not significant. Since the GF-Complete [4] libay significantly impoves the Galois Field s aithmetic efficiency, the encoding and decoding computation pefomance between vaious codes ae no much diffeent compaed with the I/O efficiency [3]. On the othe hand, eads tun into vey cucial pefomance metic and eceive a lot of attentions because they occu vey fequently [5]. Howeve, most pevious woks put thei eyes on how to impove the degaded ead pefomance (i.e., eads unde disk failues), but ignoe a fact that nomal eads (i.e., eads when all disks ae available) usually appea moe fequently than degaded eads. By assuming the contiguous symbols ae stoed on diffeent disks [3] and the bandwidth is sufficient (e.g., in inne-entepise cloud stoage), the ead pefomance in most hoizontal codes (the codes stoe all paities in specialized disks, such as Reed-Solomon code [6] and LRC code [5]) can be futhe impoved. Fo example, a (6,2,2) LRC code has 6 data disks and 4 paity disks, when it eads to 7 data elements, thee exists at least one disk needing to affod 2 elements and thus degades the pefomance, because the ead speed is usually limited by the slowest disk to espond the eading data. Yet, if thee exist moe than 7 disks can help this ead opeation (thee exist up to 6+4 = 1 potentially contibuted disks), the pefomance will be futhe impoved. Though some methods like otating mappings fom logic disks to physical disks ae able to alleviate this poblem in some level, the ead pefomance unde these methods still has some potential impovements. On the othe hand, though many vetical codes (the codes distibuted paities among all disks) like WEAVER code [7] and X-Code [8] povide good pefomance on nomal eads, they usually cannot achieve high fault toleance and low stoage ovehead simultaneously, and usually cannot apply to abitay numbe of disks. Due to these shotcomings, taditional vetical codes ae aely used in today s easue coded cloud stoage systems [3][5]. In this pape, we popose a new easue coding famewok named EC-FRM to integate existing codes, in ode to impove the ead pefomance. EC-FRM is constucted by a seies of mathematical equations, which can integate some oiginal easue codes and edeploy thei data and paities. The integated code is named EC-FRM-Code, which not only keeps most meits of the oiginal code, but also achieves /15 $ IEEE DOI 1.119/ICPP

2 good pefomance on both nomal eads and degaded eads. The contibutions of this pape can be summaized as follows: We popose a novel easue coding famewok temed EC-FRM, which can integate existing codes like Reed- Solomon code and LRC code while keeping most meits of these codes. E.g., EC-FRM-RS code (EC- FRM ove Reed-Solomon code) povides MDS popeties and can toleate abitay numbe of disk failues, and EC-FRM-LRC code (EC-FRM ove LRC code) significantly educe the I/O accesses on degaded eads. We implement two concete EC-FRM-Code based on Jeasue 1.2 [21], and evaluate thei ead pefomance in a eal system. The expeiment esults illustate that EC-FRM-Code pefoms well on both nomal eads and degaded eads. Specifically, EC-FRM-RS gains 19.2% to 33.9% highe nomal ead speed and 9.1% to 9.9% highe degaded ead speed than standad Reed- Solomon code, while EC-FRM-LRC code owns 23.5% to 46.9% highe nomal ead speed and 3.3% to 12.8% highe degaded ead speed than standad LRC code, espectively. The est of this pape continues as follows: the next section states the backgound and elated wok. Section III discusses the poblem in existing easue codes and gives ou motivations. Section IV epesents the detailed designs of EC-FRM. Section V analyzes the popeties. We focus on expeimental evaluations in Section VI. Finally, we conclude the pape in the last section. II. BACKGROUND AND RELATED WORK A. Tems and Notations Element: Element is the fundamental unit fo easue codes, which is a chunk of data o paity infomation. Thee mainly exist two kinds of elements: data elements which contain the oiginal data infomation, and paity elements that keep the edundant infomation. In Figue 1, d, is a data element, while p, is a paity element. Stipe: Stipe is the maximal set of elements that ae dependent of each othe in tems of layout and encoding elationships [1]. Figue 1 and 2 illustate a stipe of Reed- Solomon code [6] and LRC code [5], espectively. Row: Row is an maximum set of elements that belong to the same ow of stipe. In Figue 1, all elements constitute a specific ow, because a stipe of Reed-Solomon code compises only one ow. Rotated Stipes: In ode to balance the I/O loads on each disk, some stoage systems otate the mapping fom logic disks to physical disks stipe by stipe. In this pape, we call these stipes as otated stipes. B. The Easue Codes Many easue codes have been poposed in ecent yeas. Based on the paity distibutions, easue codes can be simply classified into hoizontal codes and vetical codes. Hoizontal Codes: Hoizontal codes stoe thei paities in specialized paity disks, which can be futhe categoized into GF (2 n ) based codes and XOR-Based codes accoding to diffeent encoding ules. The GF (2 n ) based easue codes usually can toleate abitay disk failues, including Reed- Solomon code [6], Rotated Reed-Solomon code [3], LRC code [5], SD code [18], and STAIR code [19], of which Reed-Solomon code is an MDS [13] code and thus povides the optimal stoage efficiency. LRC code sacifices some stoage ate to minimize the I/O accesses on degaded eads. Rotated Reed-Solomon code utilizes the ovelapping elements to educe the I/O accesses on degaded eads. SD code and STAIR code ae new kinds of easue codes that focus on both disk failues and secto failues. Thee also exist some XOR-Based hoizontal codes, such as Cauchy Reed-Solomon code [16], RDP code [14], Libeation code [15] and STAR code [2]. Cauchy Reed-Solomon code is an impoved Reed-Solomon code and can toleate abitay disk failues. RDP code and Libeation code ae classic RAID-6 codes and can toleance just up to 2 disk failues. Vetical Codes: Vetical codes distibute paities among all disks, including X-Code [8], C-Code [24], D-Code [25], HDP code [17], HV-Code [26], and WEAVER code [7], which ae usually constucted by only XOR opeations and povide good encoding/decoding complexity and good load balancing. But unfotunately, they cannot achieve both high fault toleance and low stoage oveheads simultaneously. Fo example, X-Code and P-Code ae RAID-6 codes and just toleate up to 2 disk failues, while WEAVER code always povides no moe than 5% stoage usage atio. Moeove, vetical codes usually cannot apply to abitay numbe of disks. C. Easue Codes fo Cloud Stoage Systems Though such easue codes have been poposed in liteatue, thee ae just a few implementations appeaed in moden cloud stoage systems. In this subsection, we discuss two epesentative and widely used easue codes. Reed-Solomon Code fo Google: Reed-Solomon code is a classic easue code and can toleate abitay numbe of disk failues. The encoding of Reed-Solomon code is based on both addition opeations and multiply opeations, whee the addition is based on XOR opeations and the multiply is based on a finite field GF (2 w ) [13]. A specifical Reed- Solomon code can be epesented as a two-tuple (k, m), whee k and m epesent the numbe of data disks and paity disks, espectively. Figue 1 shows an example of (6,3) Reed-Solomon code. As it shows, a stipe of Reed-Solomon code just contains only one ow, because the layout and the encoding elationship of abitay two elements in diffeent ows ae independent of each othe. In ode to save stoage space,

3 Figue 1: An example of (6,3) Reed-Solomon code. Google uses the (6,3) Reed-Solomon code to substitute tiple-eplications [1], while Facebook implements the Reed- Solomon code in its local stoage systems as well [2]. LRC Code fo Azue: LRC code is anothe typical easue code fo educing the I/O cost on degaded eads. Thee ae two types of paities in LRC code: local paity, which is calculated by a pat of data elements in each ow, and global paity which is computed by all data elements of each ow. A specifical LRC code can be epesented by a thee-tuple (k, l, m), whee k indicates the numbe of data disks, l denotes the numbe of local paity disks, and m epesents the numbe of global paity disks. Figue 2: An example of (6,2,2) LRC code. Figue 2 shows an example of (6,2,2) LRC code. As the figue shows, the local paity elements (l, and l,1 ) ae calculated by thee data elements, while the global paity elements (m, and m,1 ) ae computed by all data elements. D. Pefomance Metics With the apid gowth of data scale, disk failues ae nomal things in today s stoage systems [1]. Among the divese failue pattens, pevious study [11] has evealed that most of ecoveies (up to 99.75%) ae tiggeed by single disk failues. Based on this obsevation, ecently studies [3][27][28] point out that thee ae two cucial pefomance metics emeged in cloud stoage system: ecovey fom single failues and degaded eads to espond uses ead equests. Huang et al. [5] futhe indicate that the pefomance on degaded eads is moe significant than that on single failue ecoveies, because moe than 9% of data cente failues ae tiggeed by the system upgading with no data lost [1]. On the othe hand, it is easy to deduce that the pefomance on eads without disk failues is anothe vey impotant metic, because it is vey fequently appeaed in cloud stoage systems [12][2]. In ode to distinguish the above two types of eads, we denote the eads without disk failues as nomal eads. On the othe hand, most cloud stoage systems adopt append-only wite stategy causing that the pefomance on wites ae not as impotant as that on eads [3]. Moeove, since GF-Complete [4] libay significant impoves the Galois Field s aithmetic efficiency, the encoding and decoding pefomance between vaious codes ae not much diffeent compaed to the I/O efficiency of stoage devices and thus isn t so cucial. III. PROBLEM STATEMENTS AND MOTIVATIONS Diffeent cloud stoage systems usually have diffeent netwok bandwidth[29][3]. In this pape, we focus on the kind of cloud stoage systems with sufficient bandwidth (E.g., inne-entepise cloud stoage systems). We now state the poblems in existing easue codes and give ou motivations. A. Poblem Statements We fist give an assumption that the contiguous elements use equests ae stoed on diffeent disks in the stoage system in ode to take maximum advantage of paallel I/O (the same assumption is also applied in [3] and be widely used in cloud stoage systems). Based on this assumption, hoizontal codes usually cannot povide satisfied nomal ead pefomance, because 1) the paity disks don t contibute to nomal eads 2) the ead speed is esticted by the access time on the slowest disk, which is usually the most loaded disk. Now, we state this poblem though a simple example. (a) In conventional stipes. (b) In otated stipes Figue 3: An example of 8-elements ead opeation in (6,2,2) LRC code. Figue 3 shows two examples of ead to 8-elements in (6,2,2) LRC code with standad stipes and otated stipes, espectively. As Figue 3(a) shows, in ode to ead the equied elements, disk 1 and disk 2 need to access two elements that fom the bottleneck, because the othe disks just need to access only one element. Howeve, if all 1 disks can contibute to this ead opeation, the most loaded disks just need to affod one element s access, thus impoving the ead pefomance. The otated stipes also cannot solve this poblem, because the paity elements in otated stipes laid on the same ow with data elements and thus affect

4 the numbe of disks contibuted to eads (As Figue 3(b) illustates). In addition, degaded eads suffe fom the simila poblem. One may ague that if the numbe of elements use equests is small (less than 6 in the case of Figue 3(a)), most of hoizontal codes don t suffe fom the ead poblems efeed above. We follow this agument as well. Howeve, pevious study in [3] mentions that use equests moe than 6 data elements is fequently appeaed in eal cloud stoage systems, because the size of some common files (like MP3 files) is usually fom a few megabytes to dozens of megabytes and the size of each elements in some stoage systems is usually seveal megabytes (E.g., 1MB). Theefoe, this obsevation inspied us that to impove the pefomance on seveal elements eads is meaningful. Moeove, though some taditional vetical codes such as X-Code and WEAVER code pefom well on nomal eads, they usually cannot simultaneously povide both high fault toleance and low stoage ovehead, and usually cannot apply to abitay numbe of disks. Because of these shotcoming, vetical codes ae aely used in eal cloud stoage systems [3]. B. Motivations Though many easue codes have been poposed in liteatue, thee only exist a few implementations in cloud stoage systems, of which Reed-Solomon code has been widely used in Google and Facebook [23][2][1] and LRC code has been used in Windows Azue [5]. Theefoe, the designed famewok should keep the meits of integated codes besides impoving ead pefomance. Based on this eason and the poblems in existing codes discussed above, we have the following motivations. Minimize I/O Accesses on The Most Loaded Disk: As efeed above, the ead pefomance is mainly esticted by the I/O accesses on the slowest disk. Since the most loaded disk is moe likely to be also the slowest disk when the block size is lage (such as disk 1 in Figue 3(a) and Figue 3(b)), ou fist motivation is to geneate easue coding schemes to minimize the I/O accesses on the most loaded disk. Design Famewoks to Integate Existing Codes While Keeping Thei Meits: Existing codes such as Reed- Solomon code and LRC code have some absobing popeties, which attact the designes to choose them in the eal cloud stoage system implementations. Theefoe, the second motivation of us is to design a famewok to integate popula codes in ode to geneate new codes that maintain the oiginal codes meits. Adapt fo Abitay Numbe of Disks: Existing vetical codes usually cannot apply to abitay numbe of disks, which estict thei usage in some stoage applications [15]. Theefoe, the designed famewok needs to satisfy the condition that, if a code befoe tansfomation can apply to abitay numbe of disks, the new code afte tansfomation must be able to apply to abitay numbe of disks as well. IV. EC-FRM DESCRIPTION A. Definitions Candidate Code: Since EC-FRM needs to integate othe easue codes to constuct new coding schemes, we define candidate code as the code which can be integated to EC- FRM. Candidate codes should satisfy the condition that each stipe contains only one ow. Fo example, the widely used codes, Reed-Solomon code and LRC code, ae candidate codes. EC-FRM-Code: We use EC-FRM-Code to denote the coding scheme constucted by EC-FRM. Specifically, we use EC-FRM-RS code to denote the EC-FRM constucted upon Reed-Solomon code, and use EC-FRM-LRC code to indicate the EC-FRM constucted upon LRC code. Fo each EC-FRM-Code, we use the same paametes as its candidate code to descibe it. E.g., if an EC-FRM-Code constuct upon (6,2,2) LRC code, we label the constucted easue coding scheme as (6,2,2) EC-FRM-LRC. Goup: In EC-FRM-Code, goup is the maximum set of data elements and paity elements that ae dependent of each othe in encoding elationships. Fo each goup, all paity elements ae calculated by this goup s data elements. B. Layout and Constuction Rules We simply denote each candidate code as a two-tuple (n,k), whee n indicates the total numbe of elements in each stipe (ow) and k pesents the numbe of data elements in each stipe (ow), in ode to descibe EC-FRM s detailed designs in mathematics. Fo example, a (6,2,2) LRC code can be denoted as a (1,6) candidate code. We now discuss how to constuct a geneic EC-FRM-Code by its candidate code. n A stipe of any EC-FRM-Code have gcd(n,k) ows and n columns, whee the mathematics notation gcd(n, k) means the geatest common diviso between n and k. In ode to claify the layout of the EC-FRM-Code, we denote the ith data element of the jth column as d i,j, denote the ith paity element of the jth column as p i,j, and define the paamete as: = gcd(n, k) The data elements of any EC-FRM-Code have been laid in the fist k ows, while the paity elements ae deployed in othe ows. All elements in EC-FRM-Code can be classified as n goups identified by G i ( i n 1). We label all data elements of G i as D i, denote all paity elements of G i as P i, and use P i,j to epesent the intesection of P i and the (j + k n k )th ( j 1) ow. Fo example, as shown in Figue 4, if G 2 = {d 1,2, d 1,3, d 1,4, d 1,5, d 1,6, d 1,7, p 3,8, p 3,9, p 4,, p 4,1 } and k =3, we can deduce that D

5 Fo example, as shown in Figue 4, we can easily find that D = {d,,d,1,,d,5 }, P,1 = {p 3,6,p 3,7 } and P,2 = {p 4,8,p 4,9 }, whee the column numbe of these elements ae {, 1, 2,, 9}. Obviously, they ae deployed in diffeent disks. On the othe hand, we can simply identify all the elements as follows. Figue 4: The layout of any (6,4) EC-FRM-Code. = {d 1,2, d 1,3, d 1,4, d 1,5, d 1,6, d 1,7 }, P 2 = {p 3,8, p 3,9, p 4,, p 4,1 }, P 2, = {p 3,8, p 3,9 }, and P 2,1 = {p 4,, p 4,1 }. Based on these definitions, EC-FRM-Code can be constucted by the following two steps. Step-1: Identify All Goups. Each goup and its coesponding elements can be identified by the following equations. D i = {d i k,<i k>n,di k+1,<i k+1>n, n n,di k+k 1 n,<i k+k 1> n } Equation (1) illustates the data elements of each goup, which means all data elements ae sequentially patitioned into n goups. Fo example, when i =, 1, D = {d,,d,1,,d,5 } and D 1 = {d,6,d,7,,d 1,,d 1,1 }. We can easily find that the data elements of D and D 1 ae sequential. P i,j = {(p k +j,<i k+k+j i+1>n, +j,<i k+k+j i>n,pk,pk +j,<i k+k+j i+ 1>n) (2) ( j n k 1) Equation (2) give the paity elements of P i,j, which contains elements due to the fact that (i k+k+j i+ 1) (i k + k + j i) +1=. P i = n k 1 j= (1) P i,j (3) G i = D i P i (4) Equation (3) and (4) give the definition of P i and G i, whee P i is composed of all P i,j and G i is meged by D i and P i. Figue 4 shows an example of any (6,4) EC-FRM-Code (we use diffeent icons to distinguish diffeent goups). As it shows, each goup contains n elements which ae deployed in exact n diffeent columns, because the column numbe of its elated D i and P i ae {< i k> n, <i k +1> n,, < i k + k 1 > n } and {< i k + k > n, < i k + k +1 > n,, < i k + n 1 > n }, espectively. 1) Sequentially patition all data elements into D i, whee each D i contains k sequential data elements. E.g., in the (6,2,2) EC-FRM-Code case shown in Figue 4, all data elements can be easily sequentially patitioned into D to D 5, whee each D i is constituted by 6 data elements. 2) Fo each D i, if the last element is d x,y, we identify its elated P i,j by P i, = {p k, p k,<y+>n}, P i,1 = {p k,<y+1>n, p k,<y+2>n, +1,<y++1>n, 1 =,<y+(n k )+2>n,, p k,<y++2>n,, p k,<y++>n},, P i, n k {p n 1,<y+(n k )+1>n, p k,<y+(n k )+>n}. E.g., as shown in Figue 4, the p k last data element of D 3 is d 2,3, then P 3, = {p 3,4, p 3,5 } and P 3,1 = {p 4,6, p 4,7 }, because = 2 and k =3. 3) Mege all the elements of each D i and P i,j ( j n k 1) into G i. Step-2: Constuct ove Each Goup. Afte Step-1, each goup (G i ) contains exact n elements distibuted in n diffeent columns, while all goups ae independent of each othe in encoding elationship. Theefoe, we can logically conside each goup as one stipe (ow) of its candidate code, and then calculate all paity elements by the constuction ules of the candidate code. C. Fault Toleance EC-FRM-Code povides the same fault toleance as its candidate code. I.e., fo a specific EC-FRM-Code, if its candidate code can toleate any f concuent disk failues, the fault toleance of this EC-FRM is exact f as well. We fist give a lemma to assist the poof. Lemma 1: Fo a given easue code, switching any two elements of the same disk doesn t affect the fault toleance. Poof: Suppose that the two elements ae d i1,j and d i2,j befoe switching, while assume they ae d i and 1,j d i afte 2,j being switched. Obviously, d i := d 1,j i 2,j and d i := d 2,j i 1,j. When some disks fail, the state of the jth disk may be eithe failed o suvived. If failed, d i and 1,j d i ae 2,j simultaneously lost befoe and afte switching, thus we can econstuct d i and 1,j d i 2,j following the method fo ecoveing d i2,j and d i1,j befoe switching espectively; if suvived, all lost elements ae able to be ecoveed following the method befoe switching. Theefoe, no matte the jth disk failed o suvived, if we can ecove it befoe switching, we can econstuct all failed elements afte switching as well. This featue assues that, switch any two elements of the same column doesn t affect the fault toleance

6 (a) Local paity constuction ules (Each paity element is calculated by the data elements with the same numbe of it. E.g., p 3,2 is calculated by d,6, d,7, and d,8 ). (b) Global paity constuction ules (Each paity element is computed by the data elements with the same lette of it. E.g., p 4,4 is computed by d,6, d,7, d,8, d,9, d 1,, and d 1,1 ). Figue 5: The constuction ules of the (6,2,2) EC-FRM-LRC code. Accoding to Lemma 1, we can easily deploy all elements of each goup in the same ow by a seies of switching two elements on the same disk, which doesn t affect the easue code s fault toleance. Afte switching, each ow of EC-FRM-Code has the same layout and encoding ules as its candidate code, thus we can deduce that EC-FRM-Code have the same fault toleance as its candidate codes as well. D. Reconstuction The econstuction pocess of any EC-FRM-Code is staight-fowad and can be biefly summaized as the following steps. 1) Stat fom the stipe level and identify the failed elements. 2) Move to the goup level, and establish decoding equations based on the decoding elationships of its candidate code. 3) Solve the decoding equations to econstuct the failed elements. Now, we can econstuct all failed elements fom disk failues following above steps. In the next subsection, we will futhe discuss how to ecove EC-FRM-Code fom disk failues though a concete case. E. Case Study: LRC code in EC-FRM We now discuss a case of EC-FRM-Code constucted by (6,2,2) LRC code. A stipe of (6,2,2) EC-FRM-LRC has 5 ows and 1 columns, whee the fist 3 ows lay data elements and the last 2 ows deploy paity elements. In ode to facilitate ou discussion, we simply lay all local paity elements in the 3th ow, and deploy all global paity in the last (4th) ow. Figue 5 shows the layout and constuction ules. 1) Constuction: As discussed above, goups in EC- FRM-Code ae independent of each othe in encoding elationship, thus we can easily calculate all paity elements based on the encoding equations of its candidate code. In this case, we denote the 1 elements of each goup as {d, d 1, d 2, d 3, d 4, d 5, l, l 1, m, m 1 }, and have the following encoding equations fom LRC code [5], l = d + d 1 + d 2 (5) l 1 = d 3 + d 4 + d 5 (6) m = a d + a 1 d 1 + a 2 d 2 + b d 3 + b 1 d 4 + b 2 d 5 (7) m 1 = a 2 d + a 2 1d 1 + a 2 2d 2 + b 2 d 3 + b 2 1d 4 + b 2 2d 5 (8) Fo example, we can easily deduce that G 1 = {d,6, d,7, d,8, d,9, d 1,, d 1,1, p 3,2, p 3,3, p 4,4, p 4,5 } (the geen icons), whee {d,6, d,7, d,8, d,9, d 1,, d 1,1 } maps {d, d 1, d 2, d 3, d 4, d 5 }, {p 3,2, p 3,3 } maps {l, l 1 }, and {p 4,4, p 4,5 } maps {m, m 1 }. Theefoe, we can calculate p 3,2 by the equation of p 3,2 = d,6 d,7 d,8 (as Figue 5(a) shows), and compute p 4,4 by the equation of p 4,4 = a d,6 + a 1 d,7 + a 2 d,8 + b d,9 + b 1 d 1, + b 2 d 1,1 (as shown in Figue 5(b)). Similaly, we can encode othe goups following this method, until all paity elements of the stipe have been computed. Figue 6: An example of econstuction fom disk 1, 2, 3 concuent failues in (6,2,2) EC-FRM-LRC

7 (a) An 8-elements nomal ead. (b) A 14-elements degaded ead. (c) Anothe 14-elements degaded ead. Figue 7: The ead featues of (6,2,2) EC-FRM-LRC code (The sta icons indicate the data elements that need to be ead, while the ound icons epesent the elements that need to be exta ead). 2) Reconstuction: We now discuss how EC-FRM-LRC addesses disk failues by analyzing an example shown in Figue 6. In this case, we can easily establish the elated decoding equations afte identifying the failed elements in each goup. Fo example, let s conside G 3 (identity by squae icons), whee the failed elements ae d 2,1 (d 3 ), d 2,2 (d 4 ), and d 2,3 (d 5 ). We can diectly establish the following decoding equations based on Equation (6)-(8). d 3 + d 4 + d 5 = l 1 (9) b d 3 + b 1 d 4 + b 2 d 5 = m + a d + a 1 d 1 + a 2 d 2 (1) b 2 d 3 + b 2 1d 4 + b 2 2d 5 = m 1 + a 2 d + a 2 1d 1 + a 2 2d 2 (11) Since the ight-hand side of above thee equations ae constants, it is easy to deduce that the coefficients matix of the unknown vaiables on the left-hand is as follows: G = b b 1 b 2 (12) b 2 b 2 1 b 2 2 Accoding to [5], (6,2,2) LRC code can be ecoveed fom any kinds of tiple disk failues, thus Det(G) and we can ecalculate the failed elements based on Equation (9)(1)(11). Simila, we can ecove the failed elements in othe goups following this method, until all lost elements ae econstucted. V. PROPERTIES In this section, we analyze some impotant popeties fo EC-FRM-Code, including ead pefomance, fault toleance, stoage ovehead, and the numbe of disks that can be applied to. The goal of ou analysis is to illustate that EC- FRM-Code not only pefoms well on both nomal eads and degaded eads, but also elaxes the estictions that existing vetical codes confont. A. The Read Popeties We stat with some ead examples ove (6,2,2) EC-FRM- LRC code, in ode to explain how I/O accesses distibution unde ead opeations. Nomal Reads: Since EC-FRM-Code sequentially deploys data elements among all disks, the code will povide good pefomance on nomal eads. Fo example, as shown in Figue 7(a), the most loaded disk in EC-FRM-LRC code just needs to ead only one element fo the 8-elements ead opeation, while it needs to ead two elements in LRC code with both standad fom and otated stipes (as shown in Figue 3(a) and 3(b)). Theefoe, EC-FRM-Code will povide bette pefomance than its candidate codes due to the fact that it povides moe evenly distibution on nomal eads. Degaded Reads: The featue on degaded eads is moe complex than that on nomal eads. E.g., let s fist obseve the Figue 7(b), whee the most loaded disk only needs to affod two elements ead accesses. Fo standad LRC code, the most loaded disk needs to ead at least 3 elements, because the total numbe of ead accesses is no less than 14 and thee ae just 6 disks (5 data disks and 1 local paity disk) contibuted to degaded eads. Howeve, things ae not always fine. When it comes to the example shown in Figue 7(c), the system also needs to ead 14 elements in total, but the most loaded disk needs to ead 3 elements, thus could not impove the pefomance. In summay, EC-FRM- Code could impove the degaded ead pefomance by moe evenly distibution, but the impoved ange will be less than that on nomal eads. B. Othe Outstanding Popeties EC-FRM-Code povides some outstanding popeties concened by cloud stoage system designes, which eliminates weakness of vetical codes mentioned in [3]. Adapte fo Abitay Numbe of Disks: Fom the constuction steps in Section IV-B, EC-FRM-Code can be constucted upon any paamete of its candidate code, and the total numbe of disks in EC-FRM-Code is the same as

8 Read Speed RS R-RS EC-FRM-RS (6,3) (8,4) (1,5) (a) The speed fo RS code (unit: MB/s). Read Speed LRC R-LRC EC-FRM-LRC (6,2,2) (8,2,3) (1,2,4) (b) The speed fo LRC code (unit: MB/s). Figue 8: The nomal ead esults fo tested codes with diffeent foms. its candidate code. Since the candidate codes ae hoizontal codes which can be applied to abitay numbe of disks, EC-FRM-Code can be applied to abitay numbe of disks as well. High Fault Toleance and Low Stoage Ovehead: As efeed in Section IV-C, EC-FRM-Code and its candidate code povide the same fault toleance. Similaly, since EC- FRM-Code is constucted by edeploying the data and paity distibution ove its candidate code, the stoage ovehead of EC-FRM-Code and its candidate code should be the same as well. Theefoe, since candidate codes like Reed-Solomon code and LRC code can be designed to toleate abitay numbe of disk failues with low stoage ovehead, EC- FRM-Code achieves both low stoage ovehead and high fault toleance as well. VI. EXPERIMENT EVALUATIONS We select Reed-Solomon code and LRC code as candidate codes to evaluate ou poposed EC-FRM, because these codes ae widely used in eal stoage systems [5][12]. We mio the codes and paametes in Table I. Fo each code, we implement the code with standad fom, the code with otated stipes, and the code with EC-FRM fom (i.e., its elated EC-FRM-Code) in a eal system based on Jeasue-1.2 libay [21], which is an open souce libay and commonly used in easue code community [22]. Table I: The tested easue codes and paaments RS Code LRC Code (6,3) (6,2,2) (8,4) (8,2,3) (1,5) (1,2,4) A. Expeiment Envionment Due to the estiction on ou expeiment platfom, we un all expeiments on a machine and a disk aay. The simila expeiment envionment has been widely used as testbed fo cloud stoage systems [3]. The tested machine has an Intel Xeon X5472 pocesso and 12 GB of RAM. The disk aay is constituted with 16 disks, whee the type of each disk is Seagate/Savvio 1K.3 and the model numbe is ST9363SS. Each disk has 3GB capability and 1 pm. The opeation system of the machine is SUSE with Linux fs B. The Expeiments on Nomal Reads We un 2 times of expeiment fo Reed-Solomon code and LRC code with standad fom, otating fom, and EC-FRM fom, espectively. Each time of expeiment we andomly geneate the stat point and the ead size, whee the stat point may be an abitay data element and the ange of ead size is 1 to 2 data elements. We ecod the eal ead speed in each time of expeiment, and calculate the aveage value as ou evaluation metic, in ode to compae the eal pefomance among the tested codes with diffeent configuations. Figue 8 shows the ead speed fo vaious codes with diffeent paametes. Let s fist conside Reed-Solomon code. As Figue 8(a) shows, compaed to standad Reed-Solomon code, EC-FRM-RS achieves 19.2% to 33.9% highe ead speed, because all disks in EC-FRM-RS contibute to ead opeations while only a pat of disks (i.e., k disks) in standad RS code help to eads. The moe numbe of contibuted disks will decease the possibility of the needed elements placed in the same disk, thus minimize the loads on the most loaded disk and impove the speed. Similaly, as shown in Figue 8(b), EC-FRM-LRC gains 23.5% to 46.9% highe ead speed than standad LRC code. On the othe hand, though otating the mappings fom logic disks to physical disks stipe by stipe could impove the ead speed in some level, the codes with otated stipes still povide much lowe speed than EC-FRM-Code. In statistics, EC-FRM-RS code achieves 17.7% to 18.1% highe ead speed than Reed-Solomon code with otated stipes, while EC-FRM-LRC achieves 19.6% to 29.3% highe speed than LRC code with otated stipes. All these esults illustate that EC-FRM-Code achieves good pefomance on nomal eads

9 Degaded Read Cost Degaded Read Speed RS R-RS EC-FRM-RS (6,3) (8,4) (1,5) (a) The degaded ead cost fo RS code. RS R-RS EC-FRM-RS (6,3) (8,4) (1,5) (c) The speed fo RS code (unit: MB/s). Degaded Read Cost Degaded Read Speed LRC R-LRC EC-FRM-LRC (6,2,2) (8,2,3) (1,2,4) (b) The degaded ead cost fo LRC code LRC R-LRC EC-FRM-LRC (6,2,2) (8,2,3) (1,2,4) (d) The speed fo LRC code (unit: MB/s). Figue 9: The degaded ead esults fo tested codes with diffeent foms. C. The Expeiments on Degaded Reads We then conduct 5 times of expeiment to evaluate the pefomance on degaded eads. Fo each time of expeiment, we andomly geneate the stat point, the eased disk, and the ead size, whee the stat point may be an abitay data element, the easue disk maybe an abitay disk, and the ange of ead size is 1 to 2 data elements. We select both degaded ead cost and degaded ead speed as evaluation metics, because the cost epesents the usage of netwok bandwidth while the speed diectly eflects the pefomance. In addition, we calculate the aveage cost the the aveage speed among the evaluations fo compaison. The Degaded Read Cost: Figue 9(a) and 9(b) show the degaded ead cost among the tested configuations. As they show, the distinctions between the diffeent foms of both Reed-Solomon code and LRC code ae vey tiny: fo Reed-Solomon code, the diffeence between vaious foms is less than.9%, while fo LRC code the diffeence is even less than.7%. On the othe hand, the degaded ead cost fo LRC code is much less than that in Reed-Solomon code, because LRC code tades the stoage ovehead fo educing the degaded ead cost. In summay, EC-FRM-RS code and EC-FRM-LRC code povide vey close degaded ead cost with othe foms of Reed-Solomon code and LRC code, espectively. The Degaded Read Speed: We show the degaded ead speed esults in Figue 9(c) and 9(d). As the figue shows, EC-FRM-RS code achieves 9.1% to 9.9% highe speed than standad Reed-Solomon code, while EC-FRM- LRC code gains 3.3% to 12.8% highe degaded ead speed than standad LRC code. Howeve, Reed-Solomon code and LRC code with otated stipes also povide good pefomance on degaded eads, because the otated stipes incease the possibility of the needed elements placed in diffeent disks. Compaed to Reed-Solomon code with otated stipes, EC-FRM-RS achieves 4.7% highe degaded ead speed when k =1, but povides.26% and 2.9% lowe degaded ead speed when k =8and k =6, espectively. Since the diffeence is small and degaded eads only appea when some disks become unavailable, the impact of such a slight degaded speed can be ignoed in cloud stoage systems. On the othe hand, EC-FRM-LRC povides 2.6%, 2.9%, and 5.7% highe speed than LRC code with otated stipes when k =6, 8, 1, espectively. In summay, EC-FRM-RS gains highe degaded ead speed than standad Reed-Solomon code, achieves a little highe speed than Reed-Solomon code with otated stipes in some cases, but povides a little slowe speed than Reed- Solomon code with otated stipes unde the othe cases; EC-FRM-LRC code gains much highe degaded ead speed than both standad LRC code and LRC code with otated stipes. All in all, combined with high ead speed on nomal eads, we can conclude that EC-FRM-Code povides high pefomance on ead opeations. VII. CONCLUSION In this pape, we popose a novel easue coding famewok named EC-FRM, in ode to integate existing codes

10 to impove the ead pefomance fo cloud stoage systems. The integated easue code named EC-FRM-Code, which is constucted by edeploying the layout of the data elements in its candidate code (maybe Reed-Solomon code o LRC code). EC-FRM-Code not only achieves good pefomance on both nomal eads and degaded eads, but also keeps most of wondeful popeties of its candidate code. E.g., EC- FRM-RS code povides the MDS popeties and can toleate abitay numbe of disk failues, while EC-FRM-LRC code significantly educe the I/O accesses on degaded eads. The expeiment esults show that EC-FRM-RS gains 19.2% to 33.9% highe nomal ead speed and 9.1% to 9.9% highe degaded ead speed than standad Reed-Solomon code, while EC-FRM-LRC code owns 23.5% to 46.9% highe nomal ead speed and 3.3% to 12.8% highe degaded ead speed than standad LRC code. ACKNOWLEDGMENT We would like to geatly appeciate the anonymous eviewes fo thei insightful comments. This wok is suppoted by the National Natual Science Foundation of China (Gant No , ), the National High Technology Reseach and Development Pogam of China (Gant No. 213AA1321), the State Key Laboatoy of High-end Seve and Stoage Technology (Gant No.214HSSA2). REFERENCES [1] D. Fod, F. Labelle, F. Popovici, M. Stokely, V. Tuong, L. Baoso, C. Gimes, and S. Quinlan. Availability in globally distibuted stoage systems. In Poceeding of USENIX OSDI 1, 21. [2] D. Bothaku, et al. HDFS RAID. Hadoop Use Goup Meeting, Novembe, 21. [3] O. Khan, R. Buns, J. Plank, and W. Piece. Rethinking easue codes fo cloud file systems: minimizing I/O fo ecovey and degaded eads. In Poceedings of USENIX FAST 12, Febuay 212. [4] J. Plank, E. Mille, K. Geenan, B. Anold, J. Bunum, A. Disney, and A. McBide. GF-Complete: A compehensive open souce libay fo galois field aithmetic. Vesion 1.. Technical Repot UT-CS , Septembe, 213. [5] C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calde, P. Gopalan, J. Li, and S. Yekhanin. Easue coding in Windows Azue Stoage. In Poceedings of USENIX ATC 12, June, 212. [6] I. Reed and G. Solomon. Polynomial codes ove cetain finite fields. Jounal of the Society fo Industial and Applied Mathematice, pages 3-34, 196. [7] J. Hafne. WEAVER codes: Highly fault toleant easue codes fo stoage systems. In Poceedings of USENIX FAST 5, Decembe, 25. [8] L. Xu, and J. Buck. X-Code: MDS aay codes with optimal encoding. IEEE Tansaction on Infomation Theoy, 45(1): , Januay [9] B. Schoede and G. Gibison. Disk failues in the ead wold: What does an MTTF of 1,, mean to you? In Poceedings of USENIX FAST 7, 27. [1] J. L. Hafne, V. Deenadhayalan, T. Kanungo, and K. Rao. Pefomance metics fo easue codes in stoage systems. Technical Repot, RJ 1321, IBM Reseach, San Jose, 24. [11] E. Pinheio, W. Webe, and L. Baoso. Failue tends in a lage disk dive population. In Poceedings of the USENIX FAST 7, Febuay 27. [12] S. Ghemawat, H. Gobioff, and S. Leung. The google file system. In Poceedings of ACM SOSP, 23. [13] F. MacWilliams and N. Sloane. The Theoy of Eo Coecting Codes. Amstedam: Noth-Holland, [14] P. Cobett, B. English, A. Goel, T. Gcanac, S.Kleiman, J. Leong, and S. Sanka. Row-Diagonal Paity fo double disk failue coection. In Poceedings of the USENIX FAST 4, San Fancisco, Mach 24. [15] J. Plank. The RAID-6 libeation codes. In Poceedings of the USENIX FAST 8, Febuay 28. [16] J. Blome, M. Kalfane, R. Kap, M. Kapinski, M. Luby, and D. Zuckeman. An XOR-based Easue-Resilient coding scheme. Technical Repot TR-95-48, Intenational Compute Science Institute, [17] C. Wu, X. He, G. Wu, S. Wan, X. Liu, Q. Cao, and C. Xie. HDP Code: A hoizontal diagonal paity code to optimize I/O load balance in RAID-6. In Poceedings of IEEE/IFTP DSN 11, Hong Kong, June 21. [18] J. Plank, M. Blaum, and J. Hafne. SD Codes: Easue Codes Designed fo How Stoage Systems Really Fail. In Poceedings of the USENIX FAST 13, Febuay 213. [19] M. Li and P. Lee. STAIR Codes: A Geneal Family of Easue Codes fo Toleating Device and Secto Failues in Pactical Stoage Systems. In Poceedings of the USENIX FAST 14, 214. [2] C. Huang and L. Xu. STAR: an efficient coding scheme fo coecting tiple stoage node failues. In Poceedings of USENIX FAST 5, 25. [21] J. Plank, S. Simmeman, and C. Schuman. Jeasue: A libay in C/C++ facilitating easue coding fo stoage applications- Vesion 1.2. Technical Repot CS-8-627, Univesity of Tennessee, August, 28. [22] J. Plank, J. Luo, C. Schuman, L. Xu and Z. W. O Hean. A Pefomance Evaluation and Examination of Open-Souce Easue Coding Libaies Fo Stoage. In Poceedings of USENIX FAST 9, Febuay, 29. [23] J. Kubiatowicz, D. Bindel, Y. Chen, S. Czewinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weathespoon, C. Wells, et al. Oceanstoe: An achitectue fo global-scale pesistent stoage. In Poceedings of ASPLOS, 2. [24] M. Li and J. Shu. On Cyclic Lowest Density MDS Aay Codes Constucted Using States. In Poceedings of IEEE ISIT 1, June, 21. [25] Y. Fu and J. Shu. D-Code: An Efficient RAID-6 Code to Optimize I/O Loads and Read Pefomance. In Poceedings of IEEE IPDPS 15, May, 215. [26] Z. Shen and J. Shu. HV-Code: An All-aound MDS Code to Impove Efficiency and Reliability of RAID-6 Systems. In Poceedings of DSN 14, 214. [27] L. Xiang, Y. Xu, J. Lui, and Q. Chang. Optimal ecovey of single disk failues in RDP code stoage systems. In Poceedings of the ACM SIGMETRICS 1, New Yok, 21. [28] Y. Fu, J. Shu, and X. Luo. A Stack-Based Single Disk Failue Recovey Scheme fo Easue Coded Stoage Systems. In Poceedings of IEEE SRDS 14, Octobe, 214. [29] J. Li, S. Yang, X. Wang, and B. Li. Tee-Stucted Data Regeneation in Distibuted Stoage Systems with Regeneating Codes. In Poceedings of IEEE INFOCOM 1, 21. [3] M. Zahaia, A. Konwinski, A. Joseph, R. Katz, and I. Stoica. Impoving MapReduce Pefomance in Heteogeneous Envionments. In Poceedings of USENIX OSDI 8,