Efficient I/O for Computational Grid Applications

Transcription

1 Efficient I/O for Computational Grid Applications Ron Oldfield hd. Thesis Defense Department of Computer cience, Dartmouth College May 5, 2003 Committee: David Kotz (chair), Thomas Cormen, Robert Gray, and David Womble Armada p.

2 Computational Grids Networks of geographically ributed heterogeneous systems and devices Data-intensive scientific applications Access large remote datasets (terabyte petabyte) Datasets often need pre/post-processing Often computationally intensive Examples Climate modeling Astronomy Computational biology High-energy physics Armada p.2

3 The Armada Framework Application deploys a graph of ributed objects (ships) Requests cause pipelined data flow through graph Graph has two inct portions: frag replacements from the data provider (describes layout of data set) from the application-programmer (pre/post-processing) client processors M filt rep storage servers from application from data provider Armada p.3

4 The Armada Framework Application deploys a graph of ributed objects (ships) Requests cause pipelined data flow through graph Graph has two inct portions: frag replacements from the data provider (describes layout of data set) from the application-programmer (pre/post-processing) client processors Requests M filt rep storage servers from application from data provider Armada p.3

5 The Armada Framework Application deploys a graph of ributed objects (ships) Requests cause pipelined data flow through graph Graph has two inct portions: frag replacements from the data provider (describes layout of data set) from the application-programmer (pre/post-processing) client processors Data (read) M filt rep storage servers from application from data provider Armada p.3

6 The Armada Framework Application deploys a graph of ributed objects (ships) Requests cause pipelined data flow through graph Graph has two inct portions: frag replacements from the data provider (describes layout of data set) from the application-programmer (pre/post-processing) client processors Data (write) M filt rep storage servers from application from data provider Armada p.3

7 Armada Armada is not a data storage system. Armada is not a parallel system. The data ments that make up a data set are stored in conventional data servers as s, databases, or the like. The Armada graph encodes most functionality provided by the I/O system: programmers interface, data layout, caching and prefetching policies, interfaces to heterogeneous data servers. Armada p.4

8 Armada can... With Armada, one can... build a graph for parallel access to a group of legacy s, present many similar data sets through a standard interface, and provide transparent access to derived virtual data either cached or calculated as needed. Armada p.5

9 Restructuring roblems with the example application: otential bottlenecks in composed graph original graph restricts placement alternatives for filter ments client processors Original graph storage servers frag bottleneck replacements M filt rep Restructured graph client processors rep M rep rep M storage servers filt filt filt filt from application from data providerbottleneck from application from data provider Armada restructures original graph to improve data flow. Armada p.6

10 lacement After restructuring:. Armada deploys ships to appropriate administrative domains to optimize data flow, then 2. domain-level resource manager decides placement of individual ships. client domain rep rep rep M M server domain filt filt filt filt G E server domain 2 Armada p.7

11 lacement After restructuring:. Armada deploys ships to appropriate administrative domains to optimize data flow, then 2. domain-level resource manager decides placement of individual ships. client domain rep rep rep M M server domain filt filt filt filt G E server domain 2 Armada p.7

12 Talk Outline Introduction Framework details hips Graph Representation Restructuring graphs to improve data flow artitioning graphs and placing ships Experiments Conclusion Armada p.8

13 hips Armada includes a rich set of extensible ship classes. Armada hips tructural Non- tructural Distribute (partition, select, copy) Merge Data rocessing Optimization Filter (>, <, =) Transform (FFT, unit conversion) Reduce (min, max, sum) ermute (sort, transpose) Cache refetch Interface Client (Matrix, Line, tring, stdio) torage (File, Query) Armada p.9

14 hips Armada includes a rich set of extensible ship classes. Armada hips tructural Non- tructural Distribute (partition, select, copy) Merge Data rocessing Optimization frag replacements Interface Filter (>, <, =) Transform (FFT, unit conversion) Reduce (min, max, sum) ermute (sort, transpose) Cache refetch Client (Matrix, Line, tring, stdio) torage (File, Query) Distribute ships partition requests or data to multiple output streams. R 2 3 Armada p.9

15 hips Armada includes a rich set of extensible ship classes. Armada hips tructural Non- tructural Distribute (partition, select, copy) frag replacements Merge Data rocessing Optimization Merge R Filter (>, <, =) Transform (FFT, unit conversion) Reduce (min, max, sum) ermute (sort, transpose) Cache refetch Interface 2 3 Client (Matrix, Line, tring, stdio) torage (File, Query) Merge ships interleave requests or data from multiple input streams. R R 2 R 3 Armada p.9

16 hips Armada includes a rich set of extensible frag replacements ship classes. Armada hips tructural Non- tructural Data Distribute (partition, select, copy) rocessing Merge Data rocessing Optimization Interface Filter (>, <, =) Transform (FFT, unit conversion) Reduce (min, max, sum) R R 2 R 3 ermute (sort, transpose) Cache refetch Client (Matrix, Line, tring, stdio) 2 torage (File, Query) 3 Data-processing ships manipulate data, either individually, or in groups as it passes through the ship. R Armada p.9

17 hips Armada includes a rich set of extensible ship classes. Armada hips tructural Non- tructural frag replacements Distribute (partition, select, copy) Optimization Merge Data rocessing Optimization Interface Filter (>, <, =) Transform (FFT, unit conversion) Reduce (min, max, sum) R R 2 R 3 ermute (sort, transpose) Cache refetch Client (Matrix, Line, tring, stdio) 2 torage (File, Query) 3 Optimization ships improve I/O performance through latency-reduction techniques like caching and prefetching. R Armada p.9

18 hips R 2 R 3 Armada includes a rich set of extensible frag replacements ship classes. Armada hips tructural Non- tructural Distribute (partition, select, copy) Merge Data rocessing Optimization Interface Interface Client-interface ships convert method calls to a set of requests for data. 2 3 Filter (>, <, =) Transform (FFT, unit conversion) R R 2 R 3 R 2 R Reduce (min, max, sum) ermute (sort, transpose) Cache refetch Client (Matrix, Line, tring, stdio) torage (File, Query) 2 3 R torage-interface ships access storage devices to process requests. R Armada p.9

19 roperties of hips roperties of ships are used by restructuring and placement algorithms assigned by the programmer encoded in the ship s definition roperties identify whether a ship is data- or request-equivalent increases or decreases data flow, is parallelizable Armada p.0

20 Request and Data Equivalent hips A sequence A is equivalent to sequence B (denoted A B) if B is a permutation of A, or if B is a set of subsequences that partition A. Examples: {, 2, 3, 4, 5} {2, 3, 5,, 4} {, 2, 3, 4, 5} {{2, 3}, {, 4, 5}} {, 2, 3, 4, 5} {{2, 3}, {, 5, 4}} In other words, order does not matter. Armada p.

21 Request and Data Equivalent hips A sequence A is equivalent to sequence B (denoted A B) if B is a permutation of A, or if B is a set of subsequences that partition A. A request-equivalent ship produces request sequence equivalent to its input. A data-equivalent ship produces data sequence equivalent to its input. Most structural ships are both request and data-equivalent. Armada p.

22 Request and Data Equivalent hips A sequence A is equivalent to sequence B (denoted A B) if B is a permutation of A, or if B is a set of subsequences frag replacements that partition A. Distribution ships partition requests or data, 2, and 3 are subsequences of R. R {, 2, 3 } R 2 3 Armada p.

23 Request and Data Equivalent hips A sequence A is equivalent to sequence B (denoted A B) if B is a permutation of A, or if B is a set of subsequences frag replacements that partition A. Merge ships interleave requests or data R, R 2, and R 3 are subsequences of. {R, R 2, R 3 } R R 2 R 3 Armada p.

24 hips that Change Data Flow Data-reducer: a ship that decreases the data flow filter compress reduce (min, max, sum) Data-increaser: a ship that increases the data flow cache decompress Armada p.2

25 arallelizable hips arallelizable: a ship that can transform into multiple ships process requests and data in parallel parallelized by swapping with structural ships parallel version produces equivalent output Types of parallelizable ships: replicatable, recursive Armada p.3

26 arallelizable hips arallelizable: a ship that can transform into multiple ships process requests and data in parallel parallelized by swapping with structural ships parallel version produces frag replacements equivalent output mentsfrag replacements Types of parallelizable ships: replicatable, recursive Right-parallelizable A A R A B 2 R B A 2 R M B A 2 Original 3 A 3 Replicated A Recursed 3 Armada p.3

27 arallelizable hips arallelizable: a ship that can transform into multiple ships process requests and data in parallel parallelized by swapping with structural ships ments parallel version produces frag replacements equivalent output frag replacements Types of parallelizable ships: replicatable, recursive Left-parallelizable R R 2 R 3 B A Original T R R 2 R 3 A A A B Replicated T R R 2 R 3 A A B M A Recursed T Armada p.3

28 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally M filt rep essors ervers lication rovider ite) M filt rep 0 client processors storage servers from application from data provider Data (write) 0 Armada p.4

29 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally M filt rep essors ervers lication rovider ite) M filt rep 0 client processors storage servers from application from data provider Data (write) 0 Armada p.4

30 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally M filt rep essors ervers lication rovider ite) M filt rep 0 client processors storage servers from application from data provider Data (write) 0 Armada p.4

31 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally M filt rep essors ervers lication rovider ite) M filt rep 0 client processors storage servers from application from data provider Data (write) 0 Armada p.4

32 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally M filt essors ervers lication rovider ite) M filt rep client processors storage servers from application from data provider Data (write) rep Armada p.4

33 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally essors ervers lication rovider ite) M filt rep client processors storage servers from application from data provider Data (write) M filt rep Armada p.4

34 Graph Representation We use a series-parallel tree (-tree) to describe the composition of an Armada graph. ents frag replacements yntactically easy to describe (we use XML) Easy to manipulate internally essors ervers lication rovider ite) M filt rep client processors storage servers from application from data provider Data (write) M filt rep Armada p.4

35 Graph Restructuring Goals: remove bottlenecks (increase parallelism) allow effective placement of ships We restructure by swapping adjacent ships in the -tree increase parallelism by swapping parallelizable ships with structural ships reduce network traffic on slow links by moving data-reducing ships toward data source, moving data-increasing ships toward data dest Armada p.5

36 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Armada p.6

37 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append B to N B C A D Armada p.6

38 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append B to N B C A D Armada p.6

39 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append C to N B C A D Armada p.6

40 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append C to N B C A D Armada p.6

41 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel frag node replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N lide C left N B C A D swap? 4. mark N clean Armada p.6

42 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append A to N B C A D Armada p.6

43 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append A to N B C A D Armada p.6

44 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel frag node replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N lide A left N B C A D swap? 4. mark N clean Armada p.6

45 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel frag node replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N lide A left N B A C D swap? 4. mark N clean Armada p.6

46 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean Append D to N A B C D Armada p.6

47 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel node frag replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean A B C D Append D to N Armada p.6

48 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel frag node replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N A B C D lide D left swap? N 4. mark N clean Armada p.6

49 The Restruct Algorithm The RETRUCT algorithm traverses the -tree (depth-first) from node N, revisiting when necessary (all series and parallel nodes are initially marked dirty).. if N is a leaf or clean (base case) (a) return 2. else if N is a parallel frag node replacements (a) RETRUCT each child of N 3. else if N is a series node (a) create a new series node (b) while N has children i. child remove leftmost child of N ii. append child to iii. LIDE child left (c) N 4. mark N clean N N A B C D Assign to N Armada p.6

50 wapping hips Conditions for swapping two series-connected ships (labeled A and B) A and B are commutative (A or B is request-equivalent and A or B is data-equivalent) swapping A and B is beneficial to the application (see next slide), and the graph resulting from a swap is an -DAG (we allow four configurations). Armada p.7

51 wapping hips Conditions for swapping two series-connected ships (labeled A and B) A and B are commutative (A or B is request-equivalent and A or B is data-equivalent) swapping A and B is beneficial to the application (see next slide), and the graph resulting from a swap is an -DAG (we allow four configurations). ents frag replacements (A) Non-structural, (B) Non-structural A B A B B A B A Armada p.7

52 wapping hips Conditions for swapping two series-connected ships (labeled A and B) A and B are commutative (A or B is request-equivalent and A or B is data-equivalent) swapping A and B is beneficial to the application (see next slide), and the graph resulting from a swap is an -DAG (we allow four configurations). ents frag replacements (A) Non-structural, (B) Distribution, arallel node A B A B B A A A B A A A ARALLELIZE right Armada p.7

53 wapping hips Conditions for swapping two series-connected ships (labeled A and B) A and B are commutative (A or B is request-equivalent and A or B is data-equivalent) swapping A and B is beneficial to the application (see next slide), and the graph resulting from a swap is an -DAG (we allow four configurations). ents frag replacements arallel node, (A) Merge, (B) Non-structural A B A B B B B A B B B A ARALLELIZE left Armada p.7

54 wapping hips Conditions for swapping two series-connected ships (labeled A and B) A and B are commutative (A or B is request-equivalent and A or B is data-equivalent) swapping A and B is beneficial to the application (see next slide), and the graph resulting from a swap is an -DAG (we allow four configurations). ents frag replacements arallel node, (A) Merge, (B) Distrib, arallel node A B A B B B B A A A B B B A A A ARALLELIZE right and left Armada p.7

55 Beneficial wap A swap is deemed beneficial if it increases parallelism, moves a data-reducing ship closer to the data source, or moves a data-increasing ship closer to data destination. Algorithm to decide a beneficial swap of adjacent ships A and B. Assign a preferred direction to each ship ( for right, for left, or 0) Merge ships prefer to go right (increase parallelism) Distribution ships prefer to go left (increase parallelism) Data-reducing ships prefer to swap toward the data source Data-increasing ships prefer to swap toward the data destination 2. return true if preferred direction of A is greater than preferred direction of B 3. else return f alse Armada p.8

56 Restructuring the Example Graph frag replacements M filt rep M filt rep Armada p.9

57 Restructuring the Example Graph frag replacements M filt rep swap swap M filt rep Armada p.9

58 Restructuring the Example Graph frag replacements M rep filt filt filt M rep filt Armada p.9

59 Restructuring the Example Graph frag replacements M rep filt swap filt swap swap filt M rep swap filt Armada p.9

60 Restructuring the Example Graph rep M filt filt filt filt frag replacements rep M filt filt filt filt Armada p.9

61 Restructuring the Example Graph rep M filt filt filt filt swap frag replacements rep M filt filt filt filt swap Armada p.9

62 Restructuring the Example Graph rep rep rep M M filt filt filt filt frag replacements rep rep rep M M filt filt filt filt Armada p.9

63 lacement Hierarchical graph partitioning. artition the ships into k sets (each set represents an administrative domain). 2. artition the ships within each domain to processors provided by domain-level schedulers. The Graph artitioning roblem Given graph G(V, E) with weighted vertices and weighted edges, partition the vertices into k sets in such a way to balance the sum of the vertices and to minimize the weights of the edge crossings between sets (N-hard [Garey et al., 976]). Armada p.20

64 artitioning an Armada Graph Chaco Graph artitioning oftware [Hendrickson and Leland, NL] Algorithm for placement of Armada ships. Construct model from -tree (a) Assign edge weights (b) Assign vertex weights 2. partition graph (using CHACO) 3. for each domain (a) request procs from domain (b) partition sub-graph G E Armada p.2

65 artitioning an Armada Graph Chaco Graph artitioning oftware [Hendrickson and Leland, NL] Algorithm for placement of Armada ships. Construct model from -tree (a) Assign edge weights (b) Assign vertex weights 2. partition graph (using CHACO) for each domain (a) request procs from domain (b) partition sub-graph G E Armada p.2

66 artitioning an Armada Graph Chaco Graph artitioning oftware [Hendrickson and Leland, NL] Algorithm for placement of Armada ships. Construct model from -tree (a) Assign edge weights (b) Assign vertex weights 2. partition graph (using CHACO) 3. for each domain (a) request procs from domain (b) partition sub-graph G E Armada p.2

67 artitioning an Armada Graph Chaco Graph artitioning oftware [Hendrickson and Leland, NL] Algorithm for placement of Armada ships. Construct model from -tree (a) Assign edge weights (b) Assign vertex weights 2. partition graph (using CHACO) 3. for each domain (a) request procs from domain client domain server domain (b) partition sub-graph G server domain 2 E Armada p.2

68 artitioning an Armada Graph Chaco Graph artitioning oftware [Hendrickson and Leland, NL] Algorithm for placement of Armada ships. Construct model from -tree (a) Assign edge weights (b) Assign vertex weights 2. partition graph (using CHACO) 3. for each domain (a) request procs from domain client domain server domain (b) partition sub-graph G server domain 2 E Armada p.2

69 Experiments Evaluate performance benefit of restructuring and placement Representative application lacement considerations File copy and permutation Third-party transfers Data permutations Number of processors required by Armada eismic processing C++ interface Recursive filter Latency effects Armada p.22

70 Representative Application Examined four configurations of the example application with a filter that removed exactly 50% of the data. ements application ata provider same host ferent hosts lan M filt rep lan2 frag replacements from application from data provider same host different hosts lan M filt rep lan2 lan3 lan3 ements application ata provider ferent hosts lan rep rep rep (a) orig M M lan2 filt filt filt filt same host frag replacements from application from data provider same host lan rep rep rep (b) orig2 lan2 M M different hosts filt filt filt filt lan3 (c) restruct lan3 (d) restruct2 Armada p.23

71 Experiment etup The area between the blobs represents the WAN each LAN connected tofrag the replacements WAN by single router each WAN link has limited capacity lan WAN lan2 lan3 Ran experiments on the Emulab Network Testbed Three LANs, each with... Five 850 MHz entium III processors 00 Mbps switched network (0.5 msec latency) WAN consisted of... Three network links with 2.0 msec latency Bandwidth ranged from 2 to 00 Mbps Armada p.24

72 Results: Effective Throughput Effective Throughput (Mbit/sec) orig orig2 restruct restruct2 WAN bandwidth 2*WAN bandwidth Total client/server WAN bandwidth (Mbit/sec) Armada p.25

73 Results: Effective Throughput orig Effective Throughput (Mbit/sec) orig orig2 restruct restruct2 WAN bandwidth 2*WAN bandwidth Total client/server WAN bandwidth (Mbit/sec) Armada p.25

74 Results: Effective Throughput orig2 Effective Throughput (Mbit/sec) orig orig2 restruct restruct2 WAN bandwidth 2*WAN bandwidth Total client/server WAN bandwidth (Mbit/sec) Armada p.25

75 Results: Effective Throughput restruct same host Effective Throughput (Mbit/sec) orig orig2 restruct restruct2 WAN bandwidth 2*WAN bandwidth different hosts Total client/server WAN bandwidth (Mbit/sec) Armada p.25

76 Results: Effective Throughput restruct2 different hosts Effective Throughput (Mbit/sec) orig orig2 restruct restruct2 WAN bandwidth 2*WAN bandwidth same host Total client/server WAN bandwidth (Mbit/sec) Armada p.25

77 File Copy and ermutation Copy ributed from lan to ributed on lan0. ements Input Output disk disk2 disk3 disk4 frag replacements 2 ments Input Output disk disk2 disk3 disk swrt dec cmp srd dec dec dec dec srd srd srd srd swrt swrt swrt swrt swrt cmp cmp cmp cmp cmp Input Output disk disk disk2 disk3 disk disk2 8 8 swrt cmp Armada p.26

78 File Copy and ermutation ementscopy ributed from lan to ributed on lan0. Input Output disk disk2 disk3 disk4 np=all 2 frag replacements ments Input Output disk disk2 disk3 disk np=all swrt dec cmp srd dec dec dec dec srd srd srd srd swrt swrt swrt swrt swrt cmp cmp cmp cmp cmp Input Output disk disk disk2 disk3 disk disk2 8 8 swrt cmp Armada p.26

79 File Copy and ermutation ementscopy ributed from lan to ributed on lan0. Input Output disk disk2 disk3 disk4 np= 2 frag replacements ments Input Output disk disk2 disk3 disk np= dec dec dec dec swrt dec cmp srd srd srd srd srd swrt swrt swrt swrt swrt cmp cmp cmp cmp cmp Input Output disk disk disk2 disk3 disk disk2 8 8 swrt cmp Armada p.26

80 File Copy and ermutation ementscopy ributed from lan to ributed on lan0. Input Output disk disk2 disk3 disk4 np=0 2 frag replacements ments Input Output disk disk2 disk3 disk np=0 swrt dec cmp srd dec dec dec dec srd srd srd srd swrt swrt swrt swrt swrt cmp cmp cmp cmp cmp Input Output disk disk disk2 disk3 disk disk2 8 8 swrt cmp Armada p.26

81 Results (effective throughput) Effective Throughput (Mbit/sec) ingle Original Restructured WAN bandwidth WAN bandwidth (Mbit/sec) Armada p.27

82 Results (different placements) Effective Throughput (Mbit/sec) Compressed np=0 Compressed np= Compressed np=all WAN bandwidth WAN bandwidth (Mbit/sec) Armada p.28

83 ost-tack eismic Imaging roperties of seismic processing Compute intensive Large (terabyte) data sets Collections of s (> K) Each contains a set of traces (recorded pressure waves) reprocessing tack co-located traces FFT time traces Distribute frequencies to compute nodes Armada p.29

84 Constructing the Armada Graph lacements ω Row Column Tower X Y Base nodes interface with Armada Connect with the data provider and describe compute node ribution // called by all nodes, //... node0 gets graph from data provider //... constructor decomposes data (3D block decomposition) TraceDataset dataset(comm, pmesh, providerurl) Armada p.30

85 Constructing the Armada Graph fft stk lacements Column Tower ω Row X Compute partition. tdst torage servers.. Y Base nodes interface with Armada smg app-specific from data-provider Connect with the data provider and describe compute node ribution // called by all nodes, //... node0 gets graph from data provider //... constructor decomposes data (3D block decomposition) TraceDataset dataset(comm, pmesh, providerurl) Armada p.30

86 Constructing the Armada Graph fft stk lacements Column Tower ω Row X Compute partition. tdst torage servers.. Y Base nodes interface with Armada smg app-specific from data-provider Append operators. // called by node0 dataset.appendop(new FFTOp()); Armada p.30

87 Constructing the Armada Graph stk lacements Column Tower ω Row X Compute partition. tdst fft torage servers.. Y Base nodes interface with Armada smg app-specific from data-provider Append operators. // called by node0 dataset.appendop(new FFTOp()); dataset.appendop(new tackop()); Armada p.30

88 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition. tdst fft stk torage servers.. Y Base nodes interface with Armada smg app-specific from data-provider Append operators. // called by node0 dataset.appendop(new FFTOp()); dataset.appendop(new tackop()); Armada p.30

89 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition. tdst fft stk torage servers.. Y Base nodes interface with Armada smg app-specific from data-provider Restructure and Deploy the Armada graph. // called by node0 dataset.open(); Armada p.30

90 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition. tdst fft stk torage servers.. Y Base nodes interface with Armada smg app-specific from data-provider Restructure and Deploy the Armada graph. // called by node0 dataset.open(); //... connect app-specific with data-provider portion Armada p.30

91 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition fft smg. fft smg fft smg fft smg tdst tdst smg smg torage servers stk stk stk stk.. Y Base nodes interface with Armada app-specific from data-provider Restructure and Deploy the Armada graph. // called by node0 dataset.open(); //... connect app-specific with data-provider portion //... restructure graph Armada p.30

92 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition fft smg. fft smg fft smg fft smg tdst tdst smg smg torage servers stk stk stk stk.. Y Base nodes interface with Armada app-specific from data-provider Restructure and Deploy the Armada graph. // called by node0 dataset.open(); //... construct entire Armada graph //... restructure graph //... assign placement Armada p.30

93 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition fft smg. fft smg fft smg fft smg tdst tdst smg smg torage servers stk stk stk stk.. Y Base nodes interface with Armada app-specific from data-provider Restructure and Deploy the Armada graph. // called by node0 dataset.open(); //... construct entire Armada graph //... restructure graph //... assign placement //... deploy Armada p.30

94 Constructing the Armada Graph lacements Column Tower ω Row X Compute partition fft smg. fft smg fft smg fft smg tdst tdst smg smg torage servers stk stk stk stk.. Y Base nodes interface with Armada app-specific from data-provider Collectively read dataset. // called by all procs int size=dataset.getlocalize(); float *data = new float[size]; dataset.read(data); // do computation... Armada p.30

95 Experiment etup frag replacements Compute partition torage servers Original. tdst fft stk.. smg app-specific from data-provider Compute partition torage servers Restructured. fft fft fft fft smg smg smg smg tdst tdst smg smg stk stk stk stk.. Armada p.3

96 Results (effective throughput) 40 Effective Throughput (Mbit/sec) WAN bandwidth Original Restructured WAN bandwidth (Mbit/sec) Armada p.32

97 Results (different latencies) 40 Effective Throughput (Mbit/sec) WAN bandwidth 5 Original lat=0ms Original lat=25ms 0 Original lat=50ms Restruct lat=0ms 5 Restruct lat=25ms Restruct lat=50ms WAN bandwidth (Mbit/sec) Armada p.33

98 Related Work arallel processing of I/O streams 2 [Messerli, 999] data-flow model with automatic parallelization DataCutter [pencer et al., 2002] component-based, analytic model to decide parallelization Armada does not force the whole application into a data-flow model Armada widens data flow for parallel clients and parallel servers Operation re-ordering to improve data flow, e.g., in databases dquob [plale et al. 2000] optimize query tree to move high-filtering portions close to data exploit well-defined properties associated with query processing Armada provides a more general approach Armada p.34

99 Future Work Other Applications fmri application (time-series analysis of brain data) Can components be reused between applications? Modifications to BENEFICIAL and COMMUTATIVE Non-greedy methods Analytic models to approximate benefit lacement incorporate domain-specific information into the partitioner (compute capacity, memory capacity, etc...) dynamic re-deployment when network conditions change Tuning for cluster computing (in addition to the grid) Armada p.35

100 ummary The Armada framework data provider can describe complex ributed data sets application describes processing required before computation data-flow model provides a latency-tolerant approach Restructuring algorithm arranges graph to provide end-to-end parallel I/O enables effective placement of data-processing components lacement domain assignments to minimize data flow. host assignments based on administrative domain policies. Experiments demonstrate good performance in multiple environments. Armada p.36

101 Efficient I/O for Computational Grid Applications Ron Oldfield Department of Computer cience, Dartmouth College dfk/armada/ upported by andia National Laboratories under contract DOE-AV684. Armada p.37