Aviz Logo
Contact
Modern network stack built for the AI era.
Modernize your Network
Start here: Cisco, Arista, Juniper network users; Gigamon and Netscout observability users
Autonomous Agentic AI PlatformNetwork CopilotAviz Packet BrokerAviz Service NodeVirtual Aviz Service Node / vTAP
SONiC & AI Experience Hub
AI for Networks. Networks for AI.
Hamburger
Aviz Logo

Ensuring Seamless OpenStack VM Live Migration in EVPN-VXLAN Fabrics in SONiC Switches Open Network Fabric

Blog date iconAugust 26, 2026Blog category iconBlog
Ensuring Seamless OpenStack VM Live Migration in EVPN-VXLAN Fabrics in SONiC Switches Open Network Fabric

TL;DR

OpenStack live migration can be used to live migrate any running VM from one physical host to the other with very minimal perceived downtime. Seamless live migration over the new EVPNVXLANbased, cloud ready data center network requires ensuring that the network fabric has sustained Layer-2 presence , VM IPs &gateway address always routable and the fabric knows the best place to look for the VM's MAC/IP. Seamless live migration is tightly coordinated between compute(Nova),v-switch (OVS/OVN implemented as Neutron agents),physical EVPNfabric and allows the packets to be tunneled from the old host to new host without even single frame loss.SONiC is rapidly emerging as the world leader among all open source NOPs(Network Operating Systems) in the DC and Enterprise space. In this paper, we delve into building up a scaled EVPN VXLAN network with the help of SONiC and achieve live VM migration with the aid of OpenStack .

Sequence diagram of VM live migration steps across an EVPN-VXLAN fabric

Introduction

OpenStack Live migration live migration running virtual machine from the compute node on the physical host to other compute node on the physical host with minimize VM interruption. It must to ensure the same layer-2 connectivity between the running VM, the IP address and gateway reachable for VM and the correct location of the MAC/IP address for VM is always known to the network after Live VM migration is successfully completed in current EVPN-VXLAN type data center network. It needed high integration between Compute Layer(Nova), Virtual Network Layer(Neutron, OVS/OVN) and physical EVPN fabric, so that traffic should to be seamlessly move from source to target VM and no packet is dropped and VM interruption. SONiC has leading the way the Open Source Network Operating System for both Data Center and Enterprise network, This document describes how to use SONiC OpenStack deployment large scale network fabric, and carry on the live VM migration between different compute nodes.

Why Does Live Migration Matter in EVPN-VXLAN Fabrics?

OpenStack Live Migration of a virtual machine’s memory, state, and process running on one hypervisor to another, without application interruption or, with very low interruption to any users of the VM. On a mega data center that has deployed an EVPN-VXLAN technology to provide Layer 2 & Layer 3 overlays, Live Migration imposes hard requirements on MAC/IP mobility, gateway, uniformity and convergence of control planes. To ensure application transparency, the SONiC switches that are on top of the EVPN overlay have to seamlessly continue forwarding traffic to the correct VM’s Location while maintaining its IP, MAC Address & Default GateWay.

This requires tight integration between:

  • Compute orchestration (Nova)
  • Virtual networking (Neutron with OVS/OVN)
  • The physical network fabric (EVPN-VXLAN leaf-spine architecture)

This document outlines the conditions and behaviors required to ensure deterministic, loss-free VM mobility in such environments.

What Are the Preconditions for Seamless Migration in EVPN-VXLAN Fabrics?

Consistent EVI / VNI Membership

  • Source and destination compute nodes must connect to SONiC leaf switches advertising the same EVPN Instance (EVI) and VXLAN Network Identifier (VNI).
  • VLAN-to-VNI mappings must be consistent across all SONiC Top-of-Rack (ToR) leaf switches.

This guarantees that the VM remains within the same Layer-2 broadcast domain after migration.

Anycast Gateway Consistency

  • All the SONiC Leaf switches should advertise same gateway ip address and MAC address for tenant subnet (Anycast gateway).
  • In such a case this will not cause Layer 3 disruption. The VM should not notice any change to its default gateway.

Host-Independent Layer-2 Reachability

  • All compute-facing SONiC leaf switches must participate in the same EVPN instance.
  • VM MAC addresses must be learned and advertised as EVPN Type-2 (MAC/IP) routes.
  • During migration, the MAC/IP binding must move immediately from the source leaf to the destination leaf.

VTEP and MTU Consistency

  • All compute-connected SONiC leaf switches must support VXLAN encapsulation and decapsulation.
  • The underlay MTU must accommodate VXLAN overhead to prevent fragmentation or packet loss.

What Happens During VM Live Migration in an EVPN-VXLAN Fabric?

Sequence diagram of VM live migration steps across an EVPN-VXLAN fabric

Step 1: Nova Initiates Migration
Nova triggers live migration. Memory pages copy happens from source compute to destination compute beforehand. Neutron configures the destination port with tap or vhost-user port and OVS flows.

Step 2: VM Interface Activation on the Destination Host
We establish a new virtual port on the destination compute node. OVS/OVN associates this new VIF with the intended tenant VNI. The destination leaf switch discovers the VM MAC on the new server-facing port.

Step 3: EVPN Type-2 Route Update
The destination leaf initiates a MAC Mobility Event BGP EVPN Type-2 update is advertised to the fabric:MAC New Leaf New VTEP IP withdrawing old MAC entry Modern BGP optimizations result in <100ms convergence.

Step 4: Gratuitous ARP / Neighbor Discovery Refresh
Neutron or OVS transmits a Gratuitous ARP (IPv4) or an Unsolicited Neighbor Advertisement (IPv6).

This refreshes the ARP/ND tables across other VMs, firewalls, load balancers, and network appliances
preventing stale MAC-to-IP mappings.

Step 5: Traffic Redirection to the New VTEP
VXLAN traffic is sent to the remote leaf with VTEP to get to the VM. There is no change in the topology from the VM perspective.

Step 6: Source Compute Cleanup
Nova deassociates the VM interfaces from the source compute. EVPN revokes the Type-2 routes that have been expired. Nova finishes the workflow.

How Does OpenStack Live Migration Map to VMware vMotion?

VMware vMotion OpenStack Equivalent
vMotion Nova Live Migration
Storage vMotion Block Migration
vCenter OpenStack Controller
ESXi KVM

This mapping helps teams familiar with VMware environments reason about OpenStack migration behavior using known operational constructs.

How Does FTAS (Fabric Test Automation Suite) Validate OpenStack Live Migration?

Aviz Networks Fabric Test Automation Suite (FTAS) “The automated, containerized, vendor-agnostic solution for validating your SONiC NOS network fabric prior to production.” We validate your Data Center/Edge network readines with FTAS with capabilities covering functionality, scale, day-2 operations and even chaos scenarios such as link flapping etc. “FTAS is in-use by large Retail, Data Center and Enterprise Customers to validate Automated OpenStack Live Migration in their existing SONiC network fabric – proving it Production ready”

FTAS Automation Test Case-01: Basic Live Migration (Shared Storage)

Objective: Validate live migration with negligible downtime using shared storage.
Expected Result: Migration completes successfully Zero or minimal packet loss VM remains reachable throughout

FTAS Automation Test Case-02: Live Migration Under CPU and Memory Load

Objective: Validate stability under heavy workload.
Expected Result: Migration completes without VM crash or reboot Temporary latency increase acceptable Sessions remain intact

FTAS Automation Test Case-03: Network Continuity During Migration

Objective: Ensure uninterrupted Layer-2 and Layer-3 connectivity.
Expected Result: IP & MAC addresses did not change ARP / ND refresh was noted No TCP session re-set was noted

FTAS Automation Test Case-04: Concurrent Live Migrations

Objective: Validate behavior at scale.
Expected Result: All migrations succeed Control plane stable EVPN fabric manage MAC mobility correctly

FTAS Automation Test Case-05: Migration Failure Handling

Objective: Confirm graceful failback after migrating fail.
Expected Result: “Abort” a migration gracefully Guest continues to live on the source node “No data loss, no split-brain.”

FTAS Automation Test Case-06: Fabric-Level EVPN/VXLAN Validation

Objective: Verify EVPN control plane behavior upon VM mobility.
Expected Result: MAC/IP entry withdraws to new VTEP Old route withdrawn in a timely fashion No black-holeing of traffic Mobility timer configured as in the fabric.

Conclusion

In short, OpenStack live migrations of live VMs in an EVPN/VXLAN-based technology is less a compute and virtualization space than a "systems integration" arena where the network at the DCs and other DC sites is considered a highly predictable and resilient platform. What has been shown in this document: EVPN-VXLAN Layer 2 routing at scale and production, fully designed in a highly operational, open SONIC infrastructure, delivers an up-time IP/MAC VM mobility at EVPN-VXLAN Type 2 routing options with, for example, a set of Anycast Gateways, a stable VNI mapped to ESI configuration, etc which, given all required control-plane services, can now ensure live VM migration in a fault-intolerant environment and high throughput (speed of the operation), etc. In as demonstrated in earlier parts of this blog. The Openstack live migrations, through tested, high capacity and experienced ecosystem including the tested FTAS services all operating within a tested and qualified Openstack network fabric, can fully operate near wire speed with the provided network fabric functionality delivering, at best guess and similar experience found running a highly performant and fault resilient Vmotion across thousands of ESXi servers on highly scalable NX-OS network to allow business operation across all simulated and real failure scenarios on the network to resume.

Frequently Asked Questions

What is OpenStack live migration and why does it matter in EVPN-VXLAN fabrics?

OpenStack live migration moves a running virtual machine from one physical compute node to another with minimal interruption. In EVPN-VXLAN fabrics, this requires tight coordination between Nova, Neutron, and the physical SONiC-based network to ensure MAC/IP mobility, gateway consistency, and zero packet loss.

What network preconditions must be met before performing a live migration?

Source and destination compute nodes must connect to SONiC leaf switches that share the same EVI and VNI, with consistent VLAN-to-VNI mappings across all ToR switches. All leaf switches must also advertise the same anycast gateway IP and MAC address to prevent Layer 3 disruption during migration.

How does the EVPN fabric update its forwarding tables when a VM moves?

When the destination SONiC leaf detects the VM's MAC address on a new port, it advertises a BGP EVPN Type-2 MAC Mobility update to the fabric, withdrawing the old MAC entry. Modern BGP optimizations allow this control-plane convergence to complete in under 100 milliseconds.

What role does Gratuitous ARP play during live migration?

After the VM interface is activated on the destination host, Neutron or OVS sends a Gratuitous ARP (IPv4) or Unsolicited Neighbor Advertisement (IPv6) to refresh ARP and ND tables across other VMs, firewalls, and load balancers, preventing stale MAC-to-IP mappings.

Does SONiC support multicast-based BUM traffic replication for VXLAN?

No, SONiC does not support multicast-based BUM replication for VXLAN. Ingress-replication mode must be used for handling Broadcast, Unknown unicast, and Multicast traffic across the fabric.

How does FTAS help validate live migration readiness?

Aviz Networks' Fabric Test Automation Suite (FTAS) is a containerized, vendor-agnostic tool that automates validation of OpenStack live migration across SONiC fabrics. It covers test cases including basic migration, migration under load, network continuity, concurrent migrations, failure handling, and EVPN control-plane verification.

What happens if a live migration fails partway through?

FTAS test cases confirm that a failed migration can be aborted gracefully, with the VM continuing to run on the source compute node. No data loss or split-brain condition should occur when the fabric and compute layers are correctly configured.

How does OpenStack live migration compare to VMware vMotion?

Nova Live Migration is the OpenStack equivalent of vMotion, with KVM replacing ESXi and the OpenStack Controller replacing vCenter. Teams familiar with VMware environments can use these mappings to reason about OpenStack migration behavior using known operational constructs.

  

If you are currently running (or planning to) run an EVPN-VXLAN based Openstack, ensure you are not impact production traffic and validate your fabric performs correctly under live migration conditions. Aviz Networks can help you use FTAS to test MAC mobility, gateway persistence, and migration traffic for your SONiC fabric before your migration on the critical path.  

  

    Learn More   

Share:FacebookTwitterLinkedIn
Tags:NetworkingONES