Friday, January 27, 2017

Cross vCenter NSX

This blog post provides following about Cross vCenter NSX

  1. Overview
  2. Use Cases
  3. How Cross vCenter NSX works
  4. Sample Topology
  5. Installation and Configuration
  6. Validation


Pre-NSX 6.2, although NSX provides the flexibility, agility, efficiency and other benefits of network virtualization, the logical networking and security was constrained to the boundaries of one vCenter domain

The Cross-VC NSX feature introduced in VMware NSX 6.2, allows for NSX networking and security support across multiple vCenters. Logical switches (LS), distributed logical routers (DLR) and distributed firewall (DFW) can now be deployed across multiple vCenter domains.

These Cross-VC NSX objects are called Universal objects. The universal objects are similar to local objects such as distributed logical switches, routers, and firewall except they have universal scope, meaning they can span multiple vCenter instances.

With Cross-VC NSX functionality, in addition to the prior local-site scope objects, users can implement universal objects such as Universal Logical Switches, Universal Logical Routers, and Universal DFW across a multi-vCenter environment that can be within a single data center site or across multiple data center sites.

Use Cases

Cross-VC NSX enables following key use cases .  For detailed information please refer the Cross VC NSX Design Guide (Link provided in the References section)

  • Work Load Mobility
  • Resource Pooling
  • Unified Logical Networking and Security Policy
  • Disaster Recovery

How Cross vCenter NSX works

In a cross-vCenter NSX environment, you can have multiple vCenter Servers, each of which must be paired with its own NSX Manager. One NSX Manager is assigned the role of primary NSX Manager, and the others are assigned the role of secondary NSX Manager.

The primary NSX Manager is used to deploy a universal controller cluster that provides the control plane for the cross-vCenter NSX environment. The secondary NSX Managers do not have their own controller clusters.
The primary NSX Manager can create universal objects, such as universal logical switches. These objects are synchronized to the secondary NSX Managers by the NSX Universal Synchronization Service. You can view these objects from the secondary NSX Managers, but you cannot edit them there. You must use the primary NSX Manager to manage universal objects. The primary NSX Manager can be used to configure any of the secondary NSX Managers in the environment.

On both primary and secondary NSX Managers, you can create objects that are local to that specific vCenter NSX environment, such as logical switches, and logical (distributed) routers. They will exist only within the vCenter NSX environment in which they were created. They will not be visible on the other NSX Managers in the cross-vCenter NSX environment.

Sample Topology

Installation and Configuration

Key here is that you can start of deploying NSX Managers as standalone and later assign them specific roles i.e. Primary or Secondary
In this blog I will only include details which are specific to Cross VC NSX. I have written detailed blog for standard standalone NSX Setup. You can refer the same

Also in my setup I have both vCenters in Linked Mode. It is not mandatory to have vCenter’s in linked mode however it gives us option to manage it from single pane of glass and linked mode has its own benefits/use cases which can be researched


Assign Primary NSX Manger Role

Navigate to Networking and Security via Site A (PROD) vCenter

Click Actions and Select “Assign Primary Role “

NSX Universal Synchronization Service will change status to Running in PROD NSX Manager

Add Secondary Manager

Click Actions and Select “Add Secondary Manager”
It will prompt you for credentials. Please ensure to specify “admin” credentials and Accept the Certificate

You can observe the Role Column added under NSX Managers

Configuring Primary NSX Manager

Following steps needs to be configured on Primary NSX Manager

Adding Universal Controller(s)

You can add controller(s) while it is in standalone role and then later update the controller state after the roles have been assigned so that it replicates Secondary NSX Manager

Segment ID for Primary NSX Manager

Navigate to Segment ID under “Logical Network Preparation”

Notice the Universal Segment ID section

Universal Transport Zone on the Primary NSX Manager

In my lab I only have “Single Cluster” which I will be using hence i will creating single Universal Transport Zone. In Production, you may have multiple Transport Zones and you can have multiple combination of Universal and Local Transport Zones
Please ensure to Check Box where it says "Mark this object for Universal Synchronization"

Universal Logical Switch on the Primary NSX Manager

We need to ensure Logical Switch we create we have to ensure we connect to Universal Transport Zone we created earlier

I will be creating 2 Universal Logical Switches

  1. Universal Transit LS ( DLR will be attached to ESGs which will be deployed 1 per Site)
  2. Universal App LS (VMs will be attached to this)

Universal Logical (Distributed) Router on the Primary NSX Manager

Now we will deploy UDLR and connect it to the Universal Switch we created earlier.
Navigate to NSX Edges section and click “+”  If you have deployed NSX Edge on Standalone NSX Manager you would have only seen 2 options
Edge Services Gateway and Logical (Distributed) Router.
After you have enabled Primary for the NSX Manager you can now deploy Universal Logical (Distributed) Router  
One key item which I want to highlight is “Enable Local Egress”
Local Egress can be enable on UDLR at the time of deployment only hence it is key to Check or Uncheck as per your design.  You can read more about Local Egress

Rest of the steps is pretty much same as standard DLR

Once it is successfully deployed you can see the Universal Distributed Logical Router in both Primary and Secondary NSX Managers

Configuring Secondary NSX Manager

Following steps needs to be configured on Secondary NSX Manager

Segment ID for Secondary NSX Manager

Navigate to Segment ID under “Logical Network Preparation”

This would be local Segment IDs for local Site and standard configuration. You will already see the Universal Segment IDs on the screen for Secondary Manager

Universal Transport Zone on the Secondary NSX Manager

Universal Transport Zone will already be prepopulated on Secondary NSX Manager however we will need to add Clusters specify to Site B (DR)
If you notice the option to "Mark Universal Transport Zone as Universal Synchronization" is already greyed out as this is Secondary NSX Manager
As soon as you add the Clusters it will automatically create the Port Group for Universal Logical Switch we already created on Primary NSX Manager

With this we finished configuration specific to Cross VC NSX .


I performed following steps to perform validation to ensure Cross VC NSX Networking works as per Sample Topology mentioned earlier
  • Created Universal Logical Switch named “ Universal App LS”
  • Added “Internal” interface in Universal Distributed Logical Router and connected to “Universal App LS” with IP as
  • In Site A(PROD)  - Deployed Linux VM with IP & Gateway and attached it to “Universal App LS”
  • In Site B (DR)  - Deployed Linux VM with IP & Gateway and attached it to “Universal App LS”
  • Verified following from PROD Linux VM
  • Able to ping Gateway (UDLR) IP i.e.
  • Able to ping DR Linux VM i.e.
  • Verified following from DR Linux VM
  • Able to ping Gateway (UDLR) IP i.e.
  • Able to ping PROD Linux VM i.e.

As the both VMs are part of same Universal Logical Switch they are able to reach each other via standard VXLAN to VXLAN communication
Traffic gets encapsulated in VXLAN headers and travels between sites via VTEPs


Following are the references I used to setup my infrastructure and write this blog. You can refer them for more information which might be specific to your design


Sunday, January 8, 2017

VMware NSX Setup Script


It contains Power CLI Scripts(16) to quickly setup VMware NSX Lab to perform testing for basic Testing It takes input from JSON file and performs following tasks  

  1. NSX Manager Deployment
  2. Register VC in NSX Manager
  3. Assign NSX License
  4. Prepare ESXi Host(s) in Cluster
  5. Create IP Pool for Controller & VTEP
  6. Deploy Controller ( As it is lab currently deploying only 1 , Can we use same script to deploy multiple if required)
  7. Add/Create Segment
  8. Configure VXLAN
  9. Create Transport Zone
  10. Create Transit Logical Switch to interconnect DLR and Edge
  11. Deploy DLR with uplink from Transit Logical Switch
  12. Deploy ESG with Uplink from VLAN backed PG and Internal NIC from Transit Logical Switch
  13. Configure Default Gateways on DLR and Edge
  14. Configure OSPF in DLR
  15. Configure  OSPF in Edge


It can be downloaded from GitHub

Usage Instructions

  1. Create a folder called "Scripts" under C:\ for windows ( I haven't had chance to test it in Linux) and save all the scripts along with JSON file
  2. Modify the JSON file as per your environment
  3. Open Power CLI and navigate to the C:\Scripts and execute "nsxsetup.ps1"


  1. Management VC – VC where NSX Manager will be deployed. It could be same as Compute VC
  2. Compute VC -   VC which will be integrated with NSX
  3. VDS – Distributed Switch on Compute VC along with Port Group with MTU 1600
  4. Datastore(s) – Datastores where all the VMs will be deployed i.e. NSX Manager , NSX Controller, NSX DLR & NSX Edge. Details needs to be updated in JSON file
  5. IP Details – All the IP details for all the VMs deployed. Details needs to be updated in JSON file

Demo Link

I have recorded Demo for the Script . ### No Audio. Visual Updates Only ###

Friday, January 6, 2017

VCSA 6.5 CLI Installation

You can use the CLI installer to perform a silent deployment of a vCenter Server Appliance or Platform Services Controller appliance on an ESXi host or vCenter Server instance.

Reference Documentation

Before we being we need to ensure we do following

  1. Download and Mount the vCenter Server Appliance Installer
  2. Prepare Your JSON Configuration File for CLI Deployment

Prepare JSON File

Browse to the template directory and choose your appropriate json file. I  will be deploying embedded VCSA on VC in my lab. Update the file with your environment specific details. Following is my JSON file


Mount the ISO as CD-ROM and Browse the vcsa-deploy.exe setup file
Run the deployment command
vcsa-deploy install --accept-eula --acknowledge-ceip optional_arguments path_to_the_json_file
You might get warnings for the certificate thumb print  which you need to Accept
You could provide additional argument so that you don’t have this warning. I wanted to demonstrate hence I ran with minimal arguments

Once you Accept and Continue it will deploy the VCSA Appliance

Once it is deployed it will Power ON the VM and start installing & configuring services

It should take about 20-25 mins to finish the setup post which you should be able to access it via web client

You should be able to access vCenter it via Web Client.

Caution - As of vSphere 6.5 Windows Client is not Supported