Before Installing your Avance servers it is important to understand the networking requirements, additional attention should be made if your are planning to run a split-site configuration.
Table of Contents
General Network Requirements
Both of the physical network ports (one on each node) that make up a shared network on an Avance system must be in the same L2 broadcast domain, without any protocol filtering. Any and all Ethernet packets sent by one Avance node must not be obstructed or rate limited from being received by the other Avance node either by being routed or switched by any L3 network infrastructure.
Avance relies on full IPv4 and IPv6 protocol access, including IPv4 and IPv6 multicast. Any obstruction of this traffic will prevent a successful install or will compromise the availability of the Avance deployment. Avance should never generate more than 10 multicasts per node per second on any one network link.
Private Network Requirements
The first onboard Ethernet port on each node must be connected via a private network. These ports are usually labeled “GbE1”, “NIC1”, etc on the physical server, and the private network that connects the two Avance nodes is called “priv0”. The private network must have no other network hosts connected. The simplest private network consists of a single Ethernet cable (crossover or straight through) that connects the first onboard Ethernet port on each server. If a single Ethernet cable is not used for the private network, see Split-Site Considerations below for additional requirements.
10Gb Sync Network Requirements
Avance may also use 10Gb Ethernet ports as private sync networks. Network traffic for storage replication between nodes is sent over these networks if they are present. The requirements are similar to the Private network, as these networks should have no other network hosts connected other than the Avance end points. If a single Ethernet cable is not used for any of the sync networks, see Split-Site considerations below for additional requirements.
Business Network Requirements
All Ethernet ports, other than 10G Ethernet ports and the Private Network port (first onboard Ethernet port), are treated as Business Networks, which VMs can use to pass traffic. In order to ensure that Ethernet traffic flows unobstructed to and from VMs from either Avance node:
- The switch ports connected to Business Networks must not filter ARP packets including gratuitous ARP packets. Avance will send gratuitous ARP packets on behalf of guest VMs in order to prompt Ethernet switches to update their port forwarding tables to direct VM traffic to the appropriate physical Ethernet port on the appropriate Avance node.
- The switches connected to Business Networks must not enable any MAC address security features that would disable the movement of a MAC address from one business link to the matching business link on the other node. Cisco Catalyst switches, for example, support MAC address security features which must be disabled.
If these requirements are not met, or if the switch does not properly update it’s forwarding table when a VM is migrated from one Avance node to the other, the VM may experience a blackout where network traffic is not properly directed to and from the VM.
Split-Site Considerations
If the Private Network or any 10G Sync Networks pass through any networking equipment (i.e. are not connected by just an Ethernet cable), then the requirements outlined in this section must also be met.
Private Network Requirements
- Switch ports and/or fiber-to-copper converters connected to the Private Network must be set to auto negotiate Ethernet rates and must also support negotiating to both 100Mb/s and 1000Mb/s speeds. This is a common configuration error made with split-site deployments.
- Switches and/or fiber-to-copper converters connected to the Private Network must be non-routed and non-blocking with a round-trip latency that does not exceed 10ms. Calculate latency at 1ms for each 100 miles of fiber, and around 1ms for each non-routed, non-blocking switch or fiber converter.
- VLANs used to connect the Private network ports must not add any filtering on any of the network equipment between the two VLAN switch ports that are connected to the Avance unit nodes.
No Single Point of Failure
The path taken between Avance nodes in the Private Network must be physically segregated from the path taken by traffic on the Business Networks. Avance determines liveness of each node by sending heartbeat communication on the Private Network and the first Business Network (example “GbE2”, “NIC2”, etc of each physical server). If a single physical device or software service fails which causes traffic on both the Private and first Business Network to fail, the Avance nodes may split-brain. If Avance nodes split-brain, they will each start and run the guest VMs, which will corrupt the VM data (e.g. which copy is correct?) and will have two instances of the same VM communicating on the Business Networks.
next to their name, they are fully in sync, redundant.