C U S T O M E R (https://access.redhat.

com/)
P O R TA L


3 9/18/19 17:15:00 UTC: The ongoing downloads issue continues to be our first priority.
We are working 24x7 with our vendor on a long-term solution. Updates will be 
communicated on status.redhat.com (https://status.redhat.com/).

Service IP or Pod IP not reachable using OpenShift

SDN on top of VMware NSX VXLAN
$ SOLUTION VERIFIED - Updated November 30 2018 at 7:15 PM - English ()

Environment
OpenShift Container Platform 3.x
OpenShift SDN over NSX SDN
VMware with using NSX SDN version 6.2.3 or later
VMware with using NSX SDN version 6.2.2 and older and migrating to NSX SDN 6.2.3 or later

Issue
Service IPs do not seem to be accessible from some nodes in the cluster
For example, when we try to deploy an application, an image fails to download from the
internal registry with a "no route to host" error
Unable to communicate to or from pod IPs between container nodes and infrastructure nodes,
where one node is on VM and a different node is bare-metal/physical host.
Can we change the SDN port 4789?

Resolution

3.10 and earlier

As the OpenShift SDN VXLAN port is fixed and cannot be configured, a work around to the issue
would be to modify the VXLAN port in the NSX SDN, from 4789 to the legacy value of 8472.
This only impacts using OpenShift SDN with NSX SDN version 6.2.3 and later, where VMware
C U S T O M E R (https://access.redhat.com/)
changed the standard VXLAN
P O R TA Lport from 8472 to 4789. 

Customers currently using and older version of NSX SDN (version 6.2.2 and earlier) will not be
impacted by this issue since the default port is 8472.

Customers migrating from an older version of NSX SDN to version 6.2.3 and later, will encounter this
issue, and should configure the standard VXLAN port accordingly.

3.11+
Starting in 3.11, it is possible to change the vxlan port in OpenShift as per the documentation
(https://docs.openshift.com/container-platform/latest/install_config/configuring_sdn.html#config-
changing-vxlan-port-for-cluster-network)

Root Cause
When configuring OpenShift with OpenShift SDN using VMware NSX SDN, where the OpenShift
SDN is overlaid on top of NSX SDN, both SDNs will use the standard VXLAN port of 4789, per the
latest VXLAN RFC (https://tools.ietf.org/html/rfc7348), resulting in message packets being
dropped.

Diagnostic Steps
NSX admins should also check East/West rules TCP vs UDP settings on 4789.
East/West refers to NSX policies allowing/blocking traffic between nodes.

Product(s) Red Hat OpenShift Container Platform (/taxonomy/products/openshift)

Category Troubleshoot (/category/troubleshoot)

Tags networking (/tags/networking) openshift (/tags/openshift)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions
that Red Hat engineers have created while supporting our customers. To give you the knowledge
you need the instant it becomes available, these articles may be presented in a raw and unedited
form.
 C U S T O M E R (https://access.redhat.com/)

People who viewedPthis solution
O R TA L also viewed

Is NSX-T SDN supported on OpenShift?

Solution - Dec 17, 2018

Third party SDN plugin for OpenShift fails to install due to package conflicts with atomic-
openshift-sdn-ovs

Solution - Mar 20, 2018

Unable to change default vxlan port number in OpenStack

Solution - Jul 7, 2016

Comments

Partial service (https://status.redhat.com)

Privacy Statement (http://www.redhat.com/en/about/privacy-

policy)
Customer Portal Terms of Use
(https://access.redhat.com/help/terms/)
All Policies and Guidelines
(http://www.redhat.com/en/about/all-policies-guidelines)
Copyright © 2019 Red Hat, Inc.