New Data Center-Switching-Platform in ANX04
A strong, scalable network with high capacity – that’s what we stand for with our infrastructure, especially at our data center locations within the Backbone Europe. To meet these requirements we continuously evaluate status quo of all our locations as well as the infrastructure and integrate up-to-date hardware and technology. After months of preparation we are now ready to upgrade the network infrastructure of one of our most important locations, ANX04 (DATASIX, Vienna), to a new level. Our customers therefore profit from more capacity, even more security and modern technology.
Project: New Data Center-Switching-Platform
Our core location, ANX04, in Vienna is an important part of the Backbone Europe. At this location we operate a classic, flat layer 2 infrastructure. In respect of the increasing requirements, new available technologies on the market and the ambitious growth targets of Anexia, the evaluation of a new, future-oriented data center-switching-platform has been needed. Further automation, for example in bare-metal-compute or private cloud, as well as the harmonisation of network infrastructure across all our brands are further milestones, that will follow in the future.
Anexia established a special team for this and similar projects, that exclusively deal with the evaluation, qualification and implementation of new technologies in our network. The utilization of our infrastructure is continuously monitored and compared with our capacity planning.
Selection criteria for our new platform
The selection of the right components for our data center locations is of high importance. Obvious parameters like bandwidth, port availability and life cycle play an important role. Special attention has been paid to the often underestimated, less visible technology and chip/silicon-based scaling limits. The evaluation and improvement of these constantly enables us to respond to growth at an early stage.
Consistent availability of APIs, abandonment of manufacturer proprietary technologies, effective layer 2 security (loop-protection) and easy operation are further requirements for the selection of a new platform.
These and other aspects have been transferred to a functional specification document and have been discussed and evaluated with different manufacturers – both in relation of technology and hardware selection. Two manufacturers were shortlisted. EVPN/VXLAN have been determined as technologies.
Advantages of the new platform & technology
The definition of the new hardware platform and use of EVPN/VXLAN offers not only us, but also our customers many advantages:
- Enhancement of all relevant scaling limits and upgrade to 25/100/400G
- Layer 2 security (Loop Protection) by use of EVPN/VXLAN
- Extension of the available amount of layer 2 domains (VNIs)
- Complete Active/Active-Use of the link band widths
- Automatic load balancing by use of ECMP (Equal Cost Multipath)
- Abandonment of manufacturer proprietary “black box” technologies and associated risks
- Easy maintainability thanks to reliable BGP based traffic control
- Extensive insights to the platform thanks to “Streaming Telemetry”
Test phase
Vital for every hardware and technology selection is, apart from the theoretical fulfillment of the documented requirements, a as realistic as possible field test. With that we can avoid problems effectively. Both manufacturers supplied us with the chosen hardware for an on-site test in Vienna. For the test phase we developed an extensive testing plan.
Amongst others, we tested the expected behavior in standard operation, software updates during ongoing operation and outage of single components of the new platform. Test devices for simulation of high amounts of MAC-addresses and data traffic in a dimension of multiple 100 Gbps were additionally in use.
After two weeks of intense tests the selection process was finished and the order for the new platform was made. Our employees have successfully finished the installation of the new hardware in our data center before Christmas last year and prepared the migration of customer systems.
Maintenance plan and migration
The implementation and in this case replacement of infrastructure is a highly complex process. Several thousand customers as well as hundreds of Gbps of data traffic are handled by the network infrastructure in our data center in Vienna. Additionally, thousands of VLANs, physical ports and cables are in use, that have to be taken in consideration as part of this migration.
Our network team has developed an extensive and well-prepared maintenance plan, to enable a friction-less and efficient migration to the new infrastructure. Every task within this project is done by use of the four-eyes-principle of our engineering team and in close coordination with other involved colleagues. Our customers will be informed of every maintenance in advance.
The first maintenance serves as connection between the old and new platform. Following this, the first internal systems will be migrated to the new infrastructure. As soon as this migration is accomplished, the migration of the customer systems will start. These can mostly be done without the help of the customers, thanks to the highly-available compute and storage infrastructure. Only for a small number of customers with self-managed bare-metal-systems is involvement necessary.
After this step, all layer 2 dependent customer systems will already have been switched to the new platform and can profit from its advantages. In the next step the switch of the layer 3-uplinks to a new routing infrastructure with direct connection to the new platform will be done. Afterwards the old infrastructure is taken out of service.
Prospect
After completion of the project, Anexia customers at location ANX04 profit by a completely new, technologically advanced network infrastructure. With this we lay the foundation for further automatisation and associated self-services for the Anexia Engine. In the next weeks our teams will work intensively on the realization of the described migration and will complete this task successfully, thanks to a diligent preparation. A possible impact of customer systems will be prevented or at least minimized due to the detailed planning.
Our customers will receive further details in the maintenance announcements within the next days. We thank all our customers in advance for their patience and understanding for our maintenance tasks.