For Input-Queued Switches

Transcription

1 2007 I Internationa Symposium on Signa Processing and Information Technoogy Seectivey Weighted Muticast Scheduing Designs For Input-Queued Switches Mohammed Shoaib Department of ectrica ngineering Indian Institute of Technoogy, Madras Chennai , India. mai: shoaib.m@iitm.ac.in Abstract- The objective of this work is to propose hardware- but are more practica and hardware efficient [15], [13]. We efficient schemes for muticast scheduing in input-queued restrict our attention to packet-switched, crossbar-based, inputswitches based on the Weight Based Arbiter (WBA), motivated queued (FIFO) switches. Input-queued (IQ) switches usuay by the practica impementation of a scheduer for a 64-port queue o) sitches. Inpt-queued (IQ) sw ic ay optica crossbar switch. We demonstrate that aternating fanout- operate on fixed-size data units caed ces. A practica and and age-based weight cacuations in subsequent time sots ead efficient way to integrate muticast and unicast scheduing was to higher cock speeds and better FPGA area utiization, with proposed in [4]. performance characteristics cose to the conventiona WBA. Our FPGA sizing experiments and cock speed evauations show A. Fifo queue improvements of upto 35.25% and 47.06%, respectivey, over the WBA. In addition, atency-throughput resuts for the proposed Shown in Fig. 1 is the structure of a FIFO queue. I, 2, and variations highight the trade-offs between fairness, throughput, 3 are incoming muticast requests which are queued up in a hardware compexity and speed. FIFO fashion. The structure of request 2, shown enhanced in Fig. 1, is essentiay a bitmap of N bits, N being the number of Index Terms- Muticast Scheduing, Weight Based Agorithm switch ports, where the bit in position i indicates whether the (WBA), Input Queued Switches, High Performance Computing, corresponding muticast ce requests output port i. There are FIFO queues, outing, Switching, Poicy based networking. N such FIFO queues corresponding to the N switch inputs. I. INTODUCTION equest MC Queue Broadcast networks and muti-user appications have caused an increase in muticast traffic density over the internet, faciitated by MBone [5], [6]. Dense muticast traffic is aso 3 2 a natura resut of growing demand for network services ike audio and video distributions. To cope with the increasing muticast traffic voume, high performance routers must be abe to hande muticast traffic with minimum hardware compexity and high efficiency. The kind of growth entais new hardware chaenges and objectives in contemporary switch design. BIT MAP A number of impementations and architectures have been proposed for muticast switches [7], [8]. Queueing for muticast traffic has been widey studied and various queueing Fig. 1. Muticast FIFO Input Queue, N 8. schemes have been proposed. Virtua Output Queues (VOQs) for muticast traffic [11] avoid Head-of-Line (HOL) bock- B. Fanout ing to a arge extent but are impractica, requiring 2N -_1 queues for a NxN switch. The mutipe-queue architecture The tota number of active requests in a particuar bitmap [14], [3] uses k queues for a NxN switch, where 1 < k < constitutes the fanout of that particuar ce. As the input (2N - 1), but is unabe to achieve high performance or queues are organized in a FIFO fashion, ony the bitmaps at the run at high speeds. FIFO queues introduce HOL bocking heads of the FIFOs are considered by the muticast scheduer. Fig. 2 shows a 64 x 64 crossbar scheduer with FIFO queues. In t This work was performed whie the author was at the IBM Zurich the figure, queue Qoo has an input ce destined for the outputs eseairch% eeachlboatry Laborator- Switze--rand wizead 1,2,3,4 and hence the fanout of queue Qoo is 4 (Number of t OSMOSIS is a contract for IBM-Corning to deveop opticay-switched requested outputs is four. Hence the fanout is 4). Simiary, the interconnects for supercomputers, funded by the Department of nergy (Do) fanout of Q63 is 3 - Here fanout is used as a genera term to - Nationa Weapons Lab and The Nationa Nucear Security Administration sadfrbt h osiuinadtecriaiyo h nu (NNSA), USA sadfrbt h osiuinadtecriaiyo h nu vector /07/$ I 92

2 Q63 00MC COSSBA SCHDUL or throughput, the WBA does not provide insights into the possibe trade-offs with a possibe bargain A. Motivation and objectives The motivation for the WBA is the search for a muticast 1 2 scheduing agorithm that can be impemented in an FPGA- 32- _ specificay, in a Xiinx Virtex 1 Pro XC2VP00-6FF _ -for a 64-port switch. This requires a greater coective effort for the organization of the queued packets and provides a more Fig x64 muticast crossbar switch with a singe FIFO queue at every broader perspective of paraeism by distribution. In the WBA input. design, the definition of fairness is very rigid and uniformy same for a the inputs. This kind of rue-setting does not hep especiay when hardware efficiency is the design constraint C. Scheduer and not atency or throughput performance. A critica component of the switch is the scheduer, which When the WBA was proposed in by McKeown eta in [1], in every time sot computes a matching between the inputs the main design objectives were the search for an aternative and the outputs and configures the crossbar. Our time sot agorithm to the WBA, which is target is 51.2 ns(256-byte ces at a ine rate of 40 Gb/s 1) Simpe to impement in hardware. [10]). Known optimum muticast matching agorithms, e.g. 2) Fair in the scheduing of traffic and achieves a high the Concentrate and TATA agorithms proposed in [1], throughput. are typicay compex in terms of hardware impementation. Sti the primary objective was to propose a practica a- Hence, practicay feasibe, approximate agorithms are used. ternative to the concentrate agorithm, with reduced hardware These agorithms empoy independent seectors (aso referred compexity. to as arbiters [12]). A popuar approximate agorithm is the Weight Based Arbiter (WBA), proposed in [1]. It was shown B. Operation of WBA to be hardware impementabe and cose to Concentrate in The operation of WBA is based on assigning weights to the performance. Therefore, we focus on WBA in the rest of our input ces depending on their age and fanout at the beginning discussion...of every time sot. Once the weights are assigned, each output The main contribution of this work is an optimization of chooses the heaviest ce among the inputs subscribing to it. In the WBA agorithm. Our proposed schematic variations yied case of mutipe requests with the same weight, the scheduer sizeabe improvements in hardware area and cock speed whie breaks ties randomy. Dictated by fairness objectives, a positive achieving a performance cose to the WBA. weight is given to age whie fanout is weighted negativey to Section II discusses the WBA scheduing poicy and its maximize throughput. Thus, the oder the ce, the heavier it hardware topoogy. Section III shows FPGA synthesis resuts is and arger the fanout, the ighter it is. Basing ce grants on (area occupancy and cock speeds) and simuation resuts age and fanout resuts in a compromise between extremes of of the WBA atency-throughput performance. Section IV pure residue concentration and strict fairness. The weight wi presents the proposed aternatives to the WBA scheduing of input i is computed as poicy and Section V shows the hardware impementation resuts as we as performance simuations demonstrating the wi * fanout(hol(i)) + a * age(hol(i)), (1) sizeabe gains of the proposed schemes in comparison with the WBA. Finay, we concude in Section VI. II. TH WIGHT BASD ABIT n An agorithm that maximizes residue concentration at the S expense of fairness can starve some inputs even though it may IB achieve high throughput. If an agorithm aims to be fair, it sdestsd A A N N may not achieve the best possibe residue concentration and NIq:tI G L N D -- A WVeight thereby sacrifice some throughput. The demarcation betweend s the choice for fairness and throughput is decided based on their / T T reative importance in a particuar fied in comparison with Une / hardware compexity and performance speed. There is no room GrantFromOB Une for such fexibe decision making in the conventiona WBA TxOve =UTCS design. It is very strict in granting the input ports and foows =NCS a reguar, fixed and strict methodoogy for weight computation. If in a design, hardware efficiency is important and not fairness Fig x 64 WBA Input Bock (IB) Design Schematic. 93

3 where f and a are the weights assigned to fanout and age, respectivey. It can be shown that for an M x N switch, no ce waits at the HOL for more than M + f * N/a-I ce times [1]. By means of the weights f and a, the weight cacuation can prioritize either age or fanout. In particuar, if we assign equa weights to age and fanout, no ce waits at the HOL for more ce than M+N 1times. a bbe f u spitting, meaning that a ce can WBA empoys fanout served over the course of mutipe time sots, where in every sot a subset of the remaining fanout is served. This is the opposite of one-shot scheduing, in which every ce must be served in a singe sot. As fanout spitting achieves a significant performance benefit at a sma cost, it is usuay preferred [2], WBAeehanMps spitin,mea. [ 16]. bocks and then fed-back to them to recompute the weights for a new operation cyce. C. Topoogy inpu ac st.t reduce i nt ation cmexity ha ia the ces ahead of it have gained access ti to a of the outputs that they have requested. As discussed, the WBA has input an ipiyn output bock. There bock and an etrsicue ntewa h WBA.The incuded The neaurs arenuerousof sithe scheduer The is shown in connectvity a Figp5ea hctopoogy e ri miseobackoype of thenbradcas aenmru. weight i -----INPUT OUTPUT As the weight computation for an input ce ony depends BLOCK KBLOCK J A on oca information, it can be done at each input separatey 2+1ogN,weghtj and in parae. Aso, the weight comparisons at the outputs can be done in parae. This eads to a separation of WBA weighti into two separate sections: 'A BLOCKU PLOCKT. The input bocks (IB) which perform the weight computation.. The output bocks (OB) which perform the weight comparison and grant seection. 5. N x N WBA scheduer connection detais Connecting N Bs and Fig.OBS The impementation of the WBA BA 1is of order The pemntahn copexty compexity ofthe rdern forming an N X N WBA Scheduer. O(1). The hardware impementation is therefore simpe.[ ] - III. IMPLMNTATION SULTS GANT ABIT A. FPGA synthesis Weight[O] ~ Weight[] The motivation for this synthesis exporation is the im(ia,t(to)igpementation of a crossbar scheduer for a 64-port optica CO A switch demonstrator (caed OSMOSIS) with 40-Gb/s ports M P A s Grant Affay and a predefined scheduing cyce (time sot) of 51.2 ns for O T G I high performance computing appications [9], [10]. A major (TO in this project is to impement a 64-port scheduer chaenge (TOIB) U Weight[62] with this time sot duration in FPGAs, which were used for * Weight[63] minimizing cost and improving the fexibiity of the design. The FPGA synthesis resuts, shown in Tabe I, show that the 64-port WBA scheduer fits in the target FPGA device, neary Fig x 64 WBA Output Bock (OB) Design Schematic. saturating it (84% fu). The device used for impementation 1) The input bock (IB): Fig. 3 shows the architecture of was "xc2vpo0-6ff1704", a Xiinx Virtex-II Pro series FPGA an IB, which has to cacuate the weight for each of the input with 8 M system gates (100 K ogic cest) and 1040 User ces based on their age and fanout. The age counter is reset Os. whenever a new ce advances to the HOL, and is incremented B Performance simuations in every time sot unti the ce has been served competey. The atency-throughput simuation resuts are shown in The fanout adder determines the fanout of the HOL ce. The The mm (muticast ound obin Matching) scheme Fig.6. grants coming from the OBs are fed back to the Bs to update is a simpe muticast round robin arbiter granting the requesttheir age and fanout vaues for the next time sot. ces in a round robin fashion. This is the simpest in ing 2) The output bock (OB): For a MxN switch, the output impementation compexity but has poor atency-throughput bock has N comparators (corresponding to the N output performance. The Concentrate agorithm [1] gives the best in M an array of ports), each with M-inputs which takes performance score and the mm the worst. WBA has much In weights, each of ength (og2n + 2). every time sot, each comparator identifies the highest weight forwarded by the ower impementation compexity than Concentrate, yet has input bocks and grants the input with the highest weight deny- comparabe performance. On the other hand, WBA ceary ing a other requests. See Fig.5 and Fig.4.The grant vectors outperforms mm. \ I C N 0 coming from each of the N comparators (corresponding to the N output ports) are then rearranged into grants for the input tvirtex ogic ce = One, 4-Input LUT + One, Fip Fop + Carry Logic. One Virtex Sice = Two Virtex ogic ces. 94

4 64X64 Switch v th Ui iform-ucoreaed Arnas (Bmernu)* with ameaifrm aoeutof4 20 WBA mnm iiim 16 COcentrate - * Ore time sot 51.2 ~ Q~~~~~~~~~~~~~~~~~~V ~~~~~~~~WA.B A oncentate /7.,~~~~~~~~~~ c 'A 05 Throughput ,9 * hardware structure described in Fig. 3. The signed subtractor f Lf f T' f 'Tf from in the B subtracts the TT)fanout the age of a particuar to determine the of that This subtractor inpu ort weight port. iporuegh ot coud be potentiay substituted for seective weight estimation between age and fanout rather than using both of them in every cyce. There are 64 such subtractors at the head of the 64 Bs. Hence, subtractor optimization (repacement) coud be a substantia performance enhancer. 20 *' I A. The OCF Scheme The Odest Ce First (OCF) scheme uses weights based ony on the HOL ce age, i.e., wi = age(hol(i)). As a resut,,the subtractor at the head of the Bs is no onger needed. This scheme ceary simpifies the architecture of the WBA design, but has other repercussions on the scheduing, which are discussed in the foowing sections. Fig. 6. Latency-throughput performance of the WBA scheduer. WIGHT u PAAMT SLCTO \ C. Issues with WBA Tabe I shows the FPGA impementation resuts for the structura design of the WBA. The structura design has a cock speed of ns for a 64 x 64 switch and occupies 83.96% (Tabe.1) of the chip. When the same design was impemented the chip rapidy,^ aowing a behavioray, the WBA fis up ^ maximum impementabe switch size of ony 40-ports. This occupancy (even of the structura design) eaves itte room for additiona ogic for data seriaizing and deseriaizing (SerDes) and signa resoution circuitry to be impemented aong with the design. In Sec. IV we expore further optimizations of WBA to achieve a cock period reduction by about 15-20% and about 10% area reduction, whie maintaining a eve of performance comparabe to the basic WBA as shown in Fig. 6. IV. POPOSD WBA VAIATIONS TABL I NxN WBA SCHDUL - FPGA SYNTHSIS SULTS IN XILINX VITX-II-PO[SPD GAD-6] AG\/ eset S squt. B D WA ight s ~~~~~~~~~~~~~~~~~T s Fig. 7. T Unservedeqs TxOver Faout Grn M < Ct t { I1TICAST U=~~~~~~~~UUNICAST OCF, LFF and AF schemes - epace the signed subtractor with the simpe weight parameter seector operating during every ce cyce in the IB. B. The LFF Scheme Design enhancement exporations of the WBA scheme suggest aternate scheduing strategies to be potentia performance enhancers, where the weight estimation methodoogy is atered to choose between simper subsets of weight components. The cacuation of weights in the WBA ed to its IB having the N x The Largest Fanout First (LFF) scheme uses weights based ony on the HOL ce fanout, i.e., wi = fanout(hol(i)). Hence, the age counter and the subtractor in the IB are removed. Another advantage of OCF and LFF is that the weights are one bit ess, so the OB has a comparator instead of a more compex sign-magnitude comparator (Now we have weights of (og2n + 1) bits instead of (og2 N + 2) bits earier, where the MSB was for the sign of the weight which coud have been negetive because of the subtraction of age and fanout) The LFF design is as simpe as the OCF but has other impications. # sices C. Fairness and performance issues % sices { The main concern in the OCF and the LFF schemes is Min. cock period (ns) I Iperformance. In Sec. V we wi see that there are substantia { gains in terms of ower ogic compexity and higher cock Max. cock freq. (MHz) speeds, but the schemes are either not fair or ess performing in thoughput. LFF tends to be unfair in not serving a the queues 95

5 with equa consideration (the FIFO with the east fanout may dominate and starve the other ports at the HOL) and the OCF reduces the throughput performance of the scheduer. The issue is the trade-off between performance characteristics and hardware simpicity - The WBA was aso a resut of trade-offs and is not better performing than the concentrate agorithm but simpifies compexities to a arge extent. The above schemes are a step forward in this simpification process. D. Mixed AF Design 1) IAIF: To baance performance and compexity, we propose to mix the OCF and LFF schemes. Instead of using ("A) M Z _111 J WA W A(5O W F(75 both parameters during every time sot as in WBA, our scheme uses age and fanout separatey in subsequent time sots in an aternating fashion. We refer to this scheme as AIF, i.e., in one sot, age is used and in the next one, fanout. A. FPGA synthesis The investment, compared to OCF or LFF, is that we have The impementation resuts of the proposed schemes are to have additiona hardware to mutipex the age or fanout in specific cyces to serve as the weights to be forwarded to the shown in Tabe II, Fig. 8 and Fig. 9. OCF gives the owest OB, see Fig. 7. An acceptabe eve of fairness and throughput FPGA area occupancy (54.37%) and best cock speeds ( is achieved with ess compexities and higher speed than WBA. ns), an improvement of 35.25% in area and 47.06% in cock The signed subtractor is repaced by a simper mutipexer and speed over WBA. LFF yieds improvements of 22.55% and the sign-magnitude comparator of(og2n+2) bits in the WBA 16.14% in area and speed respectivey. The impementation is repaced by an unsigned, ordinary comparator of(og2n+1) of AIF resuts in gains of 6.22% in area and 14.28% in bits. The reduction in the number of bits is expained by the speed over WBA, see Fig. 9. In the synthesis behaviour there fact that in the bitmap coming out of the IB corresponding to are additiona routing overheads and other signa resoution the weight (age and fanout incuded), the most significant bit issues that disaow direct interpoation of the resuts from is the sign bit obtained after subtracting the fanout from the one scheme to another. TABL II age of that ce, which may ead to negative weights. Hence, the comparator in the OB is required to incude the most 64 X 64 MULTICAST SCHDUL - FPGA SYNTHSIS SULTS IN XILINX significant bit during the comparison. significant bit during the comparison.vitx-11-po[spd GAD-6]. A = AG, F = FANOUT 2) ]A3F: Fig. 10 demonstrates that the atency-throughput 64 x 64 WBA LFF OCF performance of OCF is significanty worse than LFF. ThereIAIF A3F mm Sices fore, we can expect improved performance by using fanout more often than age; for instance, in a three to one ratio. This % Sices 83.96% 65.03% 54.37% 80.03% 78.74% 72.94% Mi. cock T expoits the good atency-throughput performance of LFF, whie ensuring no port is starved indefinitey. This scheme period (ns) is referred to as 1A3F. The atency-throughput performance Max. cock and hardware utiization are very cose to WBA. freq. (MHz) V. HADWA IMPLMNTATION SULTS I XWXXeight B. Performance simuations Fig. 10 shows that OCF is very poor in performance and that 4A A the atency-throughput characteristics of LFF are practicay 3L (WB identica to WBA. Hence, a deicate baance between them is 2(22 the way to better performance. The AIF and 1A3F schemes t to I4 34j?. achieve this, having atency-throughput curves in between the OCF and LFF curves. The 3A1F, AIF, and 3A1F curves are a in between OCF and LFF. Moreover, the arger the reative frequency of using fanout as opposed to age, the...coser the curve shifts to LFF. In doing this. we incude OCF occasionay, so the scheduer does not starve any of XX xx y W A f7 the contending input ports. The WBA has strict fairness and Fig. 8. Comparative Minimum Cock periods for the designs. throughput constraints even for appications which may not require such strictness. The xayf-design is more fexibe, 45 F FAotft A, W'50 96

6 aowing appication-specific customization of weight compu- [2] J.F. Hayes,. Breaut, and M. Mehmet-Ai, "Performance anaysis of a tation. muticast switch," I Trans. Commun., vo. 39, no. 4, pp , Apr Swi.t with U.iform-JJnoreated Arrivas (Bernou) [3] S. Gupta and A. Aziz, "Muticast scheduing for switches with mutipe with a mean fanout of 4 input queues," in Proc. Hot Interconnects, pp , [4]. Schiattarea and C. Minkenberg, "Fair integreated scheduing of 20 WBA t Iw ai Iou unicast and muticast traffic in an input queued switch," in Proc. I WBA WBA {w=agfanout) (w=arget) m ICC, Istanbu, Turkey, WBA (w=agef#anout(2%)) A.ffiXhY- mnr[5] H. riksson, "MBone: The Muticast Backbone," Communications of 18 WBfA t(_ad X M 16 WBA (wvageifanout(50%)) (w=aqejfan6ut(75/)) the ACM, vo 37 (no.8) pp.54-60, August rnm [6] V. Paxson, "Growth trends in wide-area TCP connections," I Net- 14 oncentrae work, vo. 8, no. 4,pp. 8-17, Juy-Aug One tim? sot 51.2 n [7] T.T. Lee, "Non-Bocking copy networks for muticast packet switching," u 2.0iW to idot 51 2 n X (Wa=f 25/ I J. Se. Areas Commun., vo. 6,no. 5,pp , Dec v0 K=a (Wet50/) [8] J.S. Turner, "Design of a broadcast switching network," Proc. I (af75%) INFOCOM, pp , t* = anot) ^ 8 [9].P. Luijten, C. Minkenberg, B.. Hemenway, M. Sauer, and. Grzy- W. C i b i_4 g g oncentrate bowski, "Viabe opto-eectronic HPC interconnect fabrics," Proc. 6 CM/I SC2005 Conference on High-performance Networking and Computing, Nov , Seatte, WA, USA, p [10] B.. Hemenway,.. Grzybowski, C. Minkenberg,.P. Luijten, "An 2 optica packet-switched interconnect for supercomputer appications," OSA J. Opt Netw, vo. 3, no. 12, pp , Oct [11] M. Karo, M. Huchyj, and S. Morgan, "Input versus output queueing 0.1 0t2 0.3 A on a space division switch," I Trans. Commun., vo. 35, no. 12, pp. Throuthput [12] N. McKeown, "islip scheduing agorithm for input-queued switches," Fig. 10. Latency vs Throughput performance of the Age-Fanout WBA I Trans. Netw., vo. 7, no. 2, pp , Apr Schemes. [13] M. Andrews, S. Khanna, and K. Kumaran, "Integrated scheduing of unicast and muticast traffic in an input-queued switch," in Proc. I INFOCOM, pp , [14] A. Bianco,. Leonardi, F. Neri, C. Pigione, and P. Giaccone, "On the VI. CONCLUSION number of input queues to efficienty support muticast traffic in input queued switches," in Proc. I HPS, pp , Jun The WBA scheme proposed in [1] for scheduing of muti- [15] N. McKeown, "A fast switched backpane for a gigabit switched router," cast ces was synthesized on the Xiinx Virtex II Pro FPGA. Business Commun. ev., vo. 27, no. 12, The scheduer had an area occupancy of 83.96% and a [16] J.Y. Hui and T. enner, "Queueing anaysis for muticast packet switchminimum cock period of ns for the 64 x 64 switch. ing," I Trans. Commun., vo. 42, no. 2/3/4, pp , Improvement of cock speeds by 15-20% and reduction in FPGA area by at east about 10% ed us to investigate compromises between strict fairness and throughput constraints incorporated in the WBA design. The main contribution of this paper is that a hybrid weight cacuation methodoogy with reaxed constraints, which achieves substantia area reductions (up to 35.25%) and improved cock speeds (up to 47.06%) with respect to WBA. The schemes proposed use the age and fanout of the HOL ces separatey in a configurabe number of subsequent cyces, instead of using both of them in every cyce. This reduces the hardware compexity of WBA, making it more practica to impement. There is an acceptabe tradeoff between performance and compexity making it a practica scheduer choice in muticast scheduing. ACKNOWLDGMNT The author woud ike to extend specia thanks to and acknowedge the hep of Francois Abe and Cyrie Minkenberg, esearch Staff at the IBM esearch GmbH, Zurich esearch Laboratory, CH-8808, uischikon,switzerand for their assistance in simuations and hepfu suggestions in the manuscript. FNCS [1] B. Prabhakar, N. McKeown,. Ahuja, "Muticast scheduing for input queued switches," I J. Se. Areas Commun., vo. 15, no. 5, pp , June Feb