My setup consists of the following:
The Architectural Design of the end Solution:
NSX-V Site:
Low level design on the NSX-V Site:
Based on the above,
Internet/Non-SD-WAN traffic path will be as follows
VM1–>DLR–>ESG–> Inernet/VLAN
SD-WAN traffic path :
VM1–>DLR–>VCE–> Internet. (Note that the VCE could have multiple ISP Links or MPLS links that will leverage the Dynamic Multipath Protocol Optimization known as DMPO).
VCE will build tunnels to a VMware VeloCloud hosted Gateways (VCGs) and to the Orchestrator (VCO). VeloCloud Gateways will be the VCEs distributed control plane and hence VCEs will learn about all other branches routes via the updates those VCGs send over. (refer to Image 1 to help you understand the path).
Now that we are done configuring the San Jose site, lets go and Configure the San Francisco NSX-T data center.
NSX-T Site:
A new Tier-0 uplink will be connected to an NSX-T Geneve Logical Switch. This Transit logical switch will also be connected to one of the VCE’s interfaces as Downlink.
On the NSX-T Tier-0 and VCE, we will build an e-BGP neighborship via the transit Logical Switch created. VCE will hence know about the routes being advertised from the Tier-0.
Note in NSX-T, Tier-1 Auto plumb all routes towards the Tier-0.
Now that the VCE knows about the San Francisco Routes, it will advertise them to the VCG that is again hosted somewhere on the internet by VMware VeloCloud.
Low level design on the NSX-T Site:
Internet/Non-SD-WAN traffic path will be as follows:
VM1–>Tier-1–>Tier-0–> Inernet/VLAN
SD-WAN traffic path :
VM1–>Tier1–>Tier-0–> VCE–> Internet.
Note that the VCE could have multiple ISP Links or MPLS links that will leverage the Dynamic Multipath Protocol Optimization known as DMPO.
VCE will build tunnels to a VMware VeloCloud hosted Gateways (VCGs) and to the Orchestrator (VCO). Gateways will be the VCEs control plane and hence VCEs will learn about all other branches routes via the VCG.
Now San Jose and San Francisco workloads know how to reach each other via SD-WAN.
Summary
The magic of SD-WAN is that we can add “n” number of sites with or without NSX and connect them via L3 seamlessly. For instance, I can connect 50 branches to those 2 DCs by deploying a VCE on each branch.
We can also use the DMPO technology to improve the Quality of Service of the traffic destined to branches. Business policies can also be enforced using the VCE.