Moving an exchangepoint - What happened? Kurtis Lindqvist <kurtis@netnod.se>
Who are Netnod? Largest Internet exchangepoint in Sweden Operations in four cities Stockholm, Malmö, Gothenburg, Sundsvall Various medias in use at various locations Gigabit Ethernet, FDDI, DPT622 and DPT2.5G Sida 1
Who are Netnod? Focus have been on redundancy Switches and shared infastructure are in government owned mountain caves Each of the technologies are redundant Each of the locations are redundant Sida 2
The test Locations in Stockholm called A and B. Not enough space in A Migration to a new location needed This would take down half of the redundancy Sida 3
Netnod setup - GigE 1 Customer router A B Extreme switches FDDI Switches Sida 4
Netnod setup - GigE 2 Customer routers A B Extreme switches FDDI Switches Sida 5
Netnod setup - DPT Sida 6
and then we migrated A so what happened? Nothing really Sida 7
Government project Study of the overall stability of the Internet in Sweden DNS Exchange points Gathered a number of data BGP logs of 10 ISPs almost BGP listening on US East/west coast Load graphs Connection characteristics of 10 locations Sida 8
Traffic shifted as planned Sida 9
BGP Activity in the US - Withdrawals Sida 10
BGP Activity in the US - Advertisements Sida 11
BGP Activity - details Sida 12
Failed connection attempts Andel misslyckade uppkopp 0,70% 0,60% 0,50% 0,40% 0,30% 0,20% 0,10% 0,00% Före Under Efter Sida 13
Packetloss IP-paketförlu 5,00% 4,50% 4,00% 3,50% 3,00% 2,50% 2,00% 1,50% 1,00% 0,50% 0,00% 1 2 3 Sida 14
18,00% 16,00% 14,00% 12,00% 10,00% 8,00% 6,00% 4,00% 2,00% 0,00% Packetloss - details Paketförluster Sida 15 19.02.01 19.30.01 19.58.01 20.26.01 20.54.01 21.22.01 21.50.01 22.18.01 22.46.01 23.14.01 23.42.01 00.10.01 00.38.01 01.06.01 01.34.01 02.02.01 02.30.01 02.58.01 03.26.01 03.54.01 04.22.01 04.50.01 05.18.01 05.46.01 06.14.01 06.42.01 07.10.01 07.38.01 08.06.01 08.34.01 09.02.01 09.30.01 09.58.01 10.26.01 10.54.01 Tid Serie1
Packet delay IP-paket fördröjning (ms) 160,00 140,00 120,00 100,00 80,00 60,00 40,00 20,00 0,00 1 2 3 Sida 16
Packet delay -details Roundtriptime 4000000 3500000 3000000 2500000 2000000 1500000 1000000 500000 0 0 100 200 300 400 500 600 sekvensnumme modo-leksand djurgarden-leksand modo-brynas timra-brynas hv71-brynas malmo-brynas sodertalje-modo malmo-modo brynas-timra hv71-timra malmo-timra djurgarden-timra malmo-sodertalje brynas-hv71 timra-hv71 sodertalje-hv71 malmo-hv71 brynas-malmo modo-malmo timra-malmo hv71-malmo djurgarden-malmo leksand-djurgarden timra-djurgarden malmo-djurgarden Sida 17
TCP Throughput TCP-genomströmning (k 1600,00 1500,00 1400,00 1300,00 1200,00 1100,00 1000,00 900,00 800,00 700,00 600,00 500,00 400,00 300,00 200,00 100,00 0,00 1 2 3 Sida 18
What conclusions can we make? Exchange points are not really crucial Some ISPs actually shifted the traffic beforehand Interconnects can be made stable Layer 2 convergence will help But GigE can also be made to work but you need to be careful Sida 19
So what do we think happened? Well, the only thing that really changed was the NEXT_HOP Backup paths was known So BGP recalculation was very limited, if at all But this needs planning and luck! And not necessarily in that order Sida 20
Operational stability This proves that major topology changes can be done with little impact Or that IXPs are irrelevant Operational stability is becoming more and more important Mostly based on commercial reasons But there might be other needs as well How dependent is your country on the Internet? Sida 21
Conclusions Built right, an IXP can prove to be a very reliable tool for exchanging traffic Some of the characteristics are hard to get with private peerings Maybe there is a future for IXPs. Sida 22
? Sida 23
Contact Netnod Internet Exchange i Sverige AB Bellmansgatan 30 I SE-118 47 Stockholm Sweden Office address: Bellmansgatan 30 I Telephone: +46-8-615 85 70 Telefax: +46-8-442 09 67 E-mail: kurtis@netnod.se URL: http//www.netnod.se/ Sida 24