Objective 9.4 – Troubleshoot Common Connectivity Issues

Knowledge

• Review netcap logs for control plane connectivity issues
  • I believe this is a typo – should be netcpa logs
    ssh into the NSX controller and run:

    show log cloudnet/cloudnet_java-vnet-controller.log

    ssh into the ESX host and run:

    tail –f /var/log/netcpa.log

  • If you want to troubleshoot your User World Agent, you can increase the netcpa log level like this:
    Start by stopping the daemon

    # /etc/init.d/netcpad stop

    Enable write permisions on netcpa’s config file:

    # chmod +wt /etc/vmware/netcpa/netcpa.xml

    Increase log level:

    # vi /etc/vmware/netcpa/netcpa.xml

    Change the XML’s /config/log/level value to “verbose”, save and restart netcpad

    # /etc/init.d/netcpad start

• Verify VXLAN, VTEP, MAC, and ARP mapping tables
  • VXLAN

    # show control-cluster vnet vxlan vni <vni>

    # esxcli network vswitch dvs vmware vxlan network list --vds-name=<VDS_NAME>
    
    VXLAN ID  Multicast IP               Control Plane                        Controller Connection  Port Count  MAC Entry Count  ARP Entry Count  MTEP Count
    --------  -------------------------  -----------------------------------  ---------------------  ----------  ---------------  ---------------  ----------
        5000  N/A (headend replication)  Enabled (multicast proxy,ARP proxy)  192.168.110.202 (up)            1                1                0           0
        5004  N/A (headend replication)  Enabled (multicast proxy,ARP proxy)  192.168.110.203 (up)            1                0                0           0

  • VTEP

    # show control-cluster vnet vxlan vtep-table <vni>

    # esxcli network vswitch dvs vmware vxlan network vtep list --vds-name=<VDS_NAME> --vxlan-id=<VXLAN_ID>

    # show control-cluster logical-switches vtep-records <ESXi_MGT_IP>
    
    VNI      IP              Segment         MAC               Connection-ID
    5001     192.168.150.51  192.168.150.0   00:50:56:60:6a:3a 2

  • MAC

    # show control-cluster vnet vxlan mac-table <vni>

    # esxcli network vswitch dvs vmware vxlan network mac list –-vds-name=<VDS_NAME> --vxlan-id=<VXLAN_ID>

    # show control-cluster logical-switches mac-records <ESXi_MGT_IP>

  • ARP

    # show control-cluster vnet vxlan vtep-table <vni>

    # esxcli network vswitch dvs vmware vxlan network vtep list --vds-name=<VDS_NAME> --vxlan-id=<VXLAN_ID>
    

• List VNI configuration
  • From the NSX Controller CLI
    • List

      # show control-cluster vnet vxlan vni –l

    • Display

      # show control-cluster vnet vxlan vni <vni>

• View VXLAN connection tables and statistics
  • Connection tables

    # show control-cluster vnet vxlan connection-table <vni>

  • Statistics

    # show control-cluster vnet vxlan vni-stats <vni>

• Perform VTEP connectivity tests
  • At a logical switch level on the monitoring tab, use unicast or broadcast test to verify VTEP connectivity
    

Tools

• NSX Administration Guide
• NSX Command Line Interface Reference Guide
• vSphere Command-Line Interface Concepts and Examples
• NSX Controller CLI
• vSphere Web Client
• ESXi Host CLI
• esxcli
• http://www.slideshare.net/fullscreen/VMworld/vmworld-2013-operational-best-practices-for-nsx-in-vmware-environments
• And a big thanks to http://www.yet.org/2014/09/nsxv-troubleshooting/ again for this section.