Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

Share 9

$ cd /tmp/
$ curl -L https://github.com/vmware/govmomi/releases/download/v0.20.0/govc_linux_amd64.gz | gunzip >
/usr/local/bin/govc
$ curl -O https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.1/latest/rhcos-4.1.0-x86_64-
vmware.ova
$ chmod +x /usr/local/bin/govc

1 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

$ export GOVC_URL='vcsa.example.com'
$ export GOVC_USERNAME='administrator@vsphere.local'
$ export GOVC_PASSWORD='password'
$ export GOVC_NETWORK='VM Network'
$ export GOVC_DATASTORE='vmware-datastore'
$ export GOVC_INSECURE=1 # If the host above uses a self-signed cert
$ govc import.ova -name=rhcos-latest ./rhcos-410.8.20190516.0-vmware.ova

$ORIGIN apps.upi.example.com.
* A 10.x.y.38
* A 10.x.y.39
* A 10.x.y.40
$ORIGIN upi.example.com.
_etcd-server-ssl._tcp SRV 0 10 2380 etcd-0
_etcd-server-ssl._tcp SRV 0 10 2380 etcd-1
_etcd-server-ssl._tcp SRV 0 10 2380 etcd-2
bootstrap-0 A 10.x.y.34
control-plane-0 A 10.x.y.35
control-plane-1 A 10.x.y.36
control-plane-2 A 10.x.y.37
api A 10.x.y.34
api A 10.x.y.35
api A 10.x.y.36
api A 10.x.y.37
api-int A 10.x.y.34
api-int A 10.x.y.35
api-int A 10.x.y.36
api-int A 10.x.y.37
etcd-0 A 10.x.y.35
etcd-1 A 10.x.y.36
etcd-2 A 10.x.y.37
compute-0 A 10.x.y.38
compute-1 A 10.x.y.39
compute-2 A 10.x.y.40

2 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

$ cd /tmp/
$ curl -O https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-install-linux-
4.1.7.tar.gz
$ curl -O https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-client-linux-
4.1.7.tar.gz
$ tar xzvf openshift-install-linux-4.1.7.tar.gz
$ tar xzvf openshift-client-linux-4.1.7.tar.gz
$ mv oc kubectl openshift-install /usr/local/bin/
$ mkdir ~/vsphere/
$ vi ~/vsphere/install-config.yaml

apiVersion: v1
baseDomain: example.com
metadata:
name: upi
networking:
machineCIDR: "10.x.y.0/23"
platform:
vsphere:
vCenter: vcsa67.example.com
username: "administrator@vsphere.local"
password: "password"
datacenter: dc
defaultDatastore: vmware-datastore
pullsecret: ‘omitted’
sshKey: ‘omitted’

$ openshift-install --dir ~/vsphere create ignition-configs

$ git clone -b release-4.1 https://github.com/openshift/installer

3 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

$ vi openshift/installer/upi/vsphere/terraform.tfvars

bootstrap_ip = "10.x.y.34"
control_plane_ips = ["10.x.y.35","10.x.y.36","10.x.y.37"]
compute_ips = ["10.x.y.38","10.x.y.39","10.x.y.40"]
machine_cidr = "10.x.y.0/23"
cluster_id = "upi"
cluster_domain = "upi.example.com"
base_domain = "example.com"
vsphere_server = "vcsa67.example.com"
vsphere_user = "administrator@vsphere.local"
vsphere_password = "password"
vsphere_cluster = "cluster"
vsphere_datacenter = "dc"
vsphere_datastore = "vmware-datastore”
vm_template = "rhcos-latest"

// Due to size limitations for the OVA use a URL for the bootstrap ignition config file
bootstrap_ignition_url = "https://gist.githubusercontent.com/test/12345/raw/12345/bootstrap.ign"

// Ignition config for the control plane machines. Copy the contents of the master.ign generated by the installer.
control_plane_ignition = <<END_OF_MASTER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.upi.example.com:22623/config/master","verification":{}}]},"securit
y":{"tls":{"certificateAuthori
..omitted..
END_OF_MASTER_IGNITION

// Ignition config for the compute machines. Copy the contents of the worker.ign generated by the installer.
compute_ignition = <<END_OF_WORKER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.upi.example.com:22623/config/worker","verification":{}}]},"securit
y":{"tls":{"certificateAuthori
..omitted..
END_OF_WORKER_IGNITION

$ vi openshift/installer/upi/vsphere/machine/ignition.tf

gw = "10.x.y.254"

$ vi openshift/installer/upi/vsphere/machine/ignition.tf

DNS1=10.x.y.2

4 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

$ vi openshift/installer/upi/vsphere/main.tf

/*
module "dns" {
source = "./route53"

base_domain = "${var.base_domain}"
cluster_domain = "${var.cluster_domain}"
bootstrap_count = "${var.bootstrap_complete ? 0 : 1}"
bootstrap_ips = ["${module.bootstrap.ip_addresses}"]
control_plane_count = "${var.control_plane_count}"
control_plane_ips = ["${module.control_plane.ip_addresses}"]
compute_count = "${var.compute_count}"
compute_ips = ["${module.compute.ip_addresses}"]
}
*/

$ cd openshift/installer/upi/vsphere/
$ terraform init

Initializing modules...
- module.folder
Getting source "./folder"
- module.resource_pool
- module.bootstrap
- module.control_plane
- module.compute

Initializing provider plugins...

..omitted..

$ terraform apply -auto-approve

..omitted..
module.control_plane.vsphere_virtual_machine.vm[0]: Creation complete after 40s (ID: 4203f331-984c-8103-83a9-20d760fbf605)
module.bootstrap.vsphere_virtual_machine.vm: Creation complete after 40s (ID: 4203d5ab-d5e1-4ca7-eece-66aeb4a52b7d)

Apply complete! Resources: 9 added, 0 changed, 0 destroyed.

5 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

$ openshift-install --dir ~/vsphere wait-for bootstrap-complete

INFO Waiting up to 30m0s for the Kubernetes API at https://api.upi.example.com:6443...

INFO API v1.14.0+ef1f86e up
INFO Waiting up to 30m0s for bootstrapping to complete...
INFO It is now safe to remove the bootstrap resources

$ terraform apply -auto-approve -var 'bootstrap_complete=true'

..omitted..
module.bootstrap.vsphere_virtual_machine.vm: Destruction complete after 7s
Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

$ cp ~/vsphere/auth/kubeconfig ~/.kube/config
$ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":

6 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

{"emptyDir":{}}}}'

$ openshift-install --dir ~/vsphere wait-for install-complete

INFO Waiting up to 30m0s for the cluster at https://api.upi.example.com:6443 to initialize...

INFO Waiting up to 10m0s for the openshift-console route to be created...
INFO Install complete!
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/go/src/github.com/openshift/
installer/bin/auth/kubeconfig'
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.upi.e2e.bos.redhat.com
INFO Login to the console with user: kubeadmin, password: my-kube-password

$ terraform destroy -auto-approve

7 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

8 of 9 11/5/2019, 1:53 PM
Deploying a User Provisioned Infrastructure environment for OpenShift 4.1 on vSphere – Red Hat O... https://blog.openshift.com/deploying-a-user-provisioned-infrastructure-environment-for-openshift-4-...

9 of 9 11/5/2019, 1:53 PM