{"id":231,"date":"2018-12-08T01:54:47","date_gmt":"2018-12-08T01:54:47","guid":{"rendered":"http:\/\/cephswift.com\/?page_id=231"},"modified":"2019-07-26T17:32:18","modified_gmt":"2019-07-26T17:32:18","slug":"setup_centos_kubernetes_docker_cluster_tutorial","status":"publish","type":"page","link":"https:\/\/cephswift.com\/index.php\/setup_centos_kubernetes_docker_cluster_tutorial\/","title":{"rendered":"setup_centos_kubernetes_docker_cluster_tutorial"},"content":{"rendered":"<p><strong>Pre-Steps,\u00a0 install 3 &#8211; 5 VM or physical nodes with Centos7.X,\u00a0 make \/etc\/hosts file on each of them !<\/strong><\/p>\n<p><strong>Step1:\u00a0 Run my setup below to config systems, setup docker and kubernetes repository, then install on each k8 notes:<\/strong><\/p>\n<p><strong># cat\u00a0install_docker_k8.sh<\/strong><br \/>\n#! \/usr\/bin\/bash<br \/>\n#<br \/>\n# Disable SELinux<br \/>\nsetenforce 0<br \/>\nsed -i &#8211;follow-symlinks &#8216;s\/SELINUX=enforcing\/SELINUX=disabled\/g&#8217; \/etc\/sysconfig\/selinux<\/p>\n<p># Enable br_netfilter_kernel_module<br \/>\nmodprobe br_netfilter<br \/>\necho &#8216;1&#8217; &gt; \/proc\/sys\/net\/bridge\/bridge-nf-call-iptables<\/p>\n<p># Disable SWAP<br \/>\nswapoff -a<br \/>\ncp \/etc\/fstab \/etc\/fstab.orig.BK ### backup this file before change<br \/>\nsed -i &#8216;s\/^\\\/dev\\\/mapper\\\/centos-swap\/#\\\/dev\\\/mapper\\\/centos-swap\/g&#8217; \/etc\/fstab<br \/>\ncat \/etc\/fstab ### verify if the swap line get comment out<br \/>\nsleep 15<\/p>\n<p># update systeam and install Docker-CE from docker repository<br \/>\nyum update -y<br \/>\nyum install -y yum-utils device-mapper-persistent-data lvm2<br \/>\nyum-config-manager &#8211;add-repo https:\/\/download.docker.com\/linux\/centos\/docker-ce.repo<br \/>\nyum install -y docker-ce<\/p>\n<p># setup kubernetes repository and install k8<br \/>\ncat &lt;&lt;EOF &gt; \/etc\/yum.repos.d\/kubernetes.repo<br \/>\n[kubernetes]<br \/>\nname=Kubernetes<br \/>\nbaseurl=https:\/\/packages.cloud.google.com\/yum\/repos\/kubernetes-el7-x86_64<br \/>\nenabled=1<br \/>\ngpgcheck=1<br \/>\nrepo_gpgcheck=1<br \/>\ngpgkey=https:\/\/packages.cloud.google.com\/yum\/doc\/yum-key.gpg<br \/>\nhttps:\/\/packages.cloud.google.com\/yum\/doc\/rpm-package-key.gpg<br \/>\nEOF<\/p>\n<p>#cat \/etc\/yum.repos.d\/kubernetes.repo ## this line is to verify the repo contains<br \/>\nyum install -y kubelet kubeadm kubectl<\/p>\n<p><strong># .\/install_docker_k8.sh<\/strong><\/p>\n<p><strong># Step2:\u00a0 \u00a0reboot each node after install<\/strong><\/p>\n<p><strong># Step3:\u00a0 \u00a0Start docker and kubernetes services on each node:<\/strong><\/p>\n<p># cat start_docker_k8_service.sh<br \/>\nsystemctl start docker &amp;&amp; systemctl enable docker<br \/>\nsystemctl start kubelet &amp;&amp; systemctl enable kubelet<\/p>\n<p>systemctl status docker<br \/>\nsystemctl status kubelet<\/p>\n<p># .\/start_docker_k8_service.sh<\/p>\n<p># The process above will give output as below:<\/p>\n<p>Created symlink from \/etc\/systemd\/system\/multi-user.target.wants\/docker.service to \/usr\/lib\/systemd\/system\/docker.service.<br \/>\nCreated symlink from \/etc\/systemd\/system\/multi-user.target.wants\/kubelet.service to \/etc\/systemd\/system\/kubelet.service.<br \/>\n\u25cf docker.service &#8211; Docker Application Container Engine<br \/>\nLoaded: loaded (\/usr\/lib\/systemd\/system\/docker.service; enabled; vendor preset: disabled)<br \/>\nActive: active (running) since Fri 2018-12-07 20:26:32 EST; 121ms ago<br \/>\nDocs: https:\/\/docs.docker.com<br \/>\nMain PID: 3824 (dockerd)<br \/>\nCGroup: \/system.slice\/docker.service<br \/>\n\u251c\u25003824 \/usr\/bin\/dockerd -H unix:\/\/<br \/>\n\u2514\u25003847 containerd &#8211;config \/var\/run\/docker\/containerd\/containerd.toml &#8211;log-level info<\/p>\n<p>&#8230;&#8230;<\/p>\n<p>Hint: Some lines were ellipsized, use -l to show in full.<br \/>\n\u25cf kubelet.service &#8211; kubelet: The Kubernetes Node Agent<br \/>\nLoaded: loaded (\/etc\/systemd\/system\/kubelet.service; enabled; vendor preset: disabled)<br \/>\nDrop-In: \/etc\/systemd\/system\/kubelet.service.d<br \/>\n\u2514\u250010-kubeadm.conf<br \/>\nActive: active (running) since Fri 2018-12-07 20:26:32 EST; 76ms ago<br \/>\nDocs: https:\/\/kubernetes.io\/docs\/<br \/>\nMain PID: 4018 (kubelet)<br \/>\nCGroup: \/system.slice\/kubelet.service<br \/>\n\u2514\u25004018 \/usr\/bin\/kubelet &#8211;bootstrap-kubeconfig=\/etc\/kubernetes\/bootstrap-kubelet.conf &#8211;kubeconfig=\/etc\/kubernetes\/kubelet.conf &#8211;config=\/va&#8230;<\/p>\n<p>Dec 07 20:26:32 centos_k8_node4 systemd[1]: Started kubelet: The Kubernetes Node Agent.<\/p>\n<p><strong># Step 4,\u00a0 Change the cgroup-driver<\/strong><\/p>\n<pre class=\"command\"><code spellcheck=\"false\">docker info | grep -i cgroup<\/code><\/pre>\n<pre class=\"command\"><code spellcheck=\"false\">sed -i 's\/cgroup-driver=systemd\/cgroup-driver=cgroupfs\/g' \/etc\/systemd\/system\/kubelet.service.d\/10-kubeadm.conf\r\n\r\n<\/code><\/pre>\n<p>[root@centos_k8_master ~]# docker info | grep -i cgroup<br \/>\nWARNING: bridge-nf-call-iptables is disabled<br \/>\nWARNING: bridge-nf-call-ip6tables is disabled<br \/>\nCgroup Driver: cgroupfs<br \/>\n[root@centos_k8_master ~]# sed -i &#8216;s\/cgroup-driver=systemd\/cgroup-driver=cgroupfs\/g&#8217; \/etc\/systemd\/system\/kubelet.service.d\/10-kubeadm.conf<br \/>\n[root@centos_k8_master ~]# docker info | grep -i cgroup<br \/>\nWARNING: bridge-nf-call-iptables is disabled<br \/>\nWARNING: bridge-nf-call-ip6tables is disabled<br \/>\nCgroup Driver: cgroupfs<br \/>\n[root@centos_k8_master ~]# systemctl daemon-reload<br \/>\n[root@centos_k8_master ~]# systemctl restart kubelet<br \/>\n[root@centos_k8_master ~]# systemctl status kubelet<br \/>\n\u25cf kubelet.service &#8211; kubelet: The Kubernetes Node Agent<br \/>\nLoaded: loaded (\/etc\/systemd\/system\/kubelet.service; enabled; vendor preset: disabled)<br \/>\nDrop-In: \/etc\/systemd\/system\/kubelet.service.d<br \/>\n\u2514\u250010-kubeadm.conf<br \/>\nActive: activating (auto-restart) (Result: exit-code) since Fri 2018-12-07 20:37:30 EST; 8s ago<br \/>\nDocs: https:\/\/kubernetes.io\/docs\/<br \/>\nProcess: 4889 ExecStart=\/usr\/bin\/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)<br \/>\nMain PID: 4889 (code=exited, status=255)<\/p>\n<p>Dec 07 20:37:30 centos_k8_master systemd[1]: Unit kubelet.service entered failed state.<br \/>\nDec 07 20:37:30 centos_k8_master systemd[1]: kubelet.service failed.<\/p>\n<p>[root@centos_k8_master ~]# docker info | grep -i cgroup<br \/>\nCgroup Driver: cgroupfs<br \/>\nWARNING: bridge-nf-call-iptables is disabled<br \/>\nWARNING: bridge-nf-call-ip6tables is disabl<\/p>\n<p><strong>Step 5:\u00a0 Kubernetes Cluster Initialization<\/strong><\/p>\n<p># cat config_k8_master.sh<br \/>\n# setup firewall to allow k8 process<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=6443\/tcp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=10250\/tcp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=6443\/udp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=10250\/udp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;list-ports<br \/>\nfirewall-cmd &#8211;reload<br \/>\nfirewall-cmd &#8211;get-services<\/p>\n<p>## setup k8 iptables to allow net.bridge traffic<br \/>\ncat &lt;&lt;EOF &gt; \/etc\/sysctl.d\/k8s.conf<br \/>\nnet.bridge.bridge-nf-call-ip6tables = 1<br \/>\nnet.bridge.bridge-nf-call-iptables = 1<br \/>\nEOF<\/p>\n<p># create k8 cluster from &#8220;kubeadm init&#8221;<br \/>\necho &#8220;kubeadm init&#8221; &gt; k8_init.log<br \/>\n#kubeadm init | tee -a k8_init.log<br \/>\nkubeadm init &#8211;apiserver-advertise-address=192.168.57.59 &#8211;pod-network-cidr=10.244.0.0\/16 | tee -a k8_init.log<\/p>\n<p># deploy the flannel network to the kubernetes cluster using the kubectl command<br \/>\nmkdir -p $HOME\/.kube<br \/>\nsudo cp -i \/etc\/kubernetes\/admin.conf $HOME\/.kube\/config<br \/>\nsudo chown $(id -u):$(id -g) $HOME\/.kube\/config<br \/>\nkubectl apply -f https:\/\/raw.githubusercontent.com\/coreos\/flannel\/master\/Documentation\/kube-flannel.yml<\/p>\n<p># run the script above:<\/p>\n<p># .\/config_k8_master.sh<\/p>\n<p>[root@centos-k8-node4 ~]# cat k8_init.log<br \/>\nkubeadm init<br \/>\n[init] Using Kubernetes version: v1.13.1<br \/>\n[preflight] Running pre-flight checks<br \/>\n[preflight] Pulling images required for setting up a Kubernetes cluster<br \/>\n[preflight] This might take a minute or two, depending on the speed of your internet connection<br \/>\n[preflight] You can also perform this action in beforehand using &#8216;kubeadm config images pull&#8217;<br \/>\n[kubelet-start] Writing kubelet environment file with flags to file &#8220;\/var\/lib\/kubelet\/kubeadm-flags.env&#8221;<br \/>\n[kubelet-start] Writing kubelet configuration to file &#8220;\/var\/lib\/kubelet\/config.yaml&#8221;<br \/>\n[kubelet-start] Activating the kubelet service<br \/>\n[certs] Using certificateDir folder &#8220;\/etc\/kubernetes\/pki&#8221;<br \/>\n[certs] Generating &#8220;etcd\/ca&#8221; certificate and key<br \/>\n[certs] Generating &#8220;etcd\/healthcheck-client&#8221; certificate and key<br \/>\n[certs] Generating &#8220;apiserver-etcd-client&#8221; certificate and key<br \/>\n[certs] Generating &#8220;etcd\/server&#8221; certificate and key<br \/>\n[certs] etcd\/server serving cert is signed for DNS names [centos-k8-node4 localhost] and IPs [192.168.57.59 127.0.0.1 ::1]<br \/>\n[certs] Generating &#8220;etcd\/peer&#8221; certificate and key<br \/>\n[certs] etcd\/peer serving cert is signed for DNS names [centos-k8-node4 localhost] and IPs [192.168.57.59 127.0.0.1 ::1]<br \/>\n[certs] Generating &#8220;ca&#8221; certificate and key<br \/>\n[certs] Generating &#8220;apiserver&#8221; certificate and key<br \/>\n[certs] apiserver serving cert is signed for DNS names [centos-k8-node4 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.57.59]<br \/>\n[certs] Generating &#8220;apiserver-kubelet-client&#8221; certificate and key<br \/>\n[certs] Generating &#8220;front-proxy-ca&#8221; certificate and key<br \/>\n[certs] Generating &#8220;front-proxy-client&#8221; certificate and key<br \/>\n[certs] Generating &#8220;sa&#8221; key and public key<br \/>\n[kubeconfig] Using kubeconfig folder &#8220;\/etc\/kubernetes&#8221;<br \/>\n[kubeconfig] Writing &#8220;admin.conf&#8221; kubeconfig file<br \/>\n[kubeconfig] Writing &#8220;kubelet.conf&#8221; kubeconfig file<br \/>\n[kubeconfig] Writing &#8220;controller-manager.conf&#8221; kubeconfig file<br \/>\n[kubeconfig] Writing &#8220;scheduler.conf&#8221; kubeconfig file<br \/>\n[control-plane] Using manifest folder &#8220;\/etc\/kubernetes\/manifests&#8221;<br \/>\n[control-plane] Creating static Pod manifest for &#8220;kube-apiserver&#8221;<br \/>\n[control-plane] Creating static Pod manifest for &#8220;kube-controller-manager&#8221;<br \/>\n[control-plane] Creating static Pod manifest for &#8220;kube-scheduler&#8221;<br \/>\n[etcd] Creating static Pod manifest for local etcd in &#8220;\/etc\/kubernetes\/manifests&#8221;<br \/>\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory &#8220;\/etc\/kubernetes\/manifests&#8221;. This can take up to 4m0s<br \/>\n[apiclient] All control plane components are healthy after 17.506171 seconds<br \/>\n[uploadconfig] storing the configuration used in ConfigMap &#8220;kubeadm-config&#8221; in the &#8220;kube-system&#8221; Namespace<br \/>\n[kubelet] Creating a ConfigMap &#8220;kubelet-config-1.13&#8221; in namespace kube-system with the configuration for the kubelets in the cluster<br \/>\n[patchnode] Uploading the CRI Socket information &#8220;\/var\/run\/dockershim.sock&#8221; to the Node API object &#8220;centos-k8-node4&#8221; as an annotation<br \/>\n[mark-control-plane] Marking the node centos-k8-node4 as control-plane by adding the label &#8220;node-role.kubernetes.io\/master=&#8221;&#8221;<br \/>\n[mark-control-plane] Marking the node centos-k8-node4 as control-plane by adding the taints [node-role.kubernetes.io\/master:NoSchedule]<br \/>\n[bootstrap-token] Using token: 94q0sq.1erns4x5ail242dv<br \/>\n[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles<br \/>\n[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials<br \/>\n[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token<br \/>\n[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster<br \/>\n[bootstraptoken] creating the &#8220;cluster-info&#8221; ConfigMap in the &#8220;kube-public&#8221; namespace<br \/>\n[addons] Applied essential addon: CoreDNS<br \/>\n[addons] Applied essential addon: kube-proxy<\/p>\n<p>Your Kubernetes master has initialized successfully!<\/p>\n<p>To start using your cluster, you need to run the following as a regular user:<\/p>\n<p>mkdir -p $HOME\/.kube<br \/>\nsudo cp -i \/etc\/kubernetes\/admin.conf $HOME\/.kube\/config<br \/>\nsudo chown $(id -u):$(id -g) $HOME\/.kube\/config<\/p>\n<p>You should now deploy a pod network to the cluster.<br \/>\nRun &#8220;kubectl apply -f [podnetwork].yaml&#8221; with one of the options listed at:<br \/>\nhttps:\/\/kubernetes.io\/docs\/concepts\/cluster-administration\/addons\/<\/p>\n<p>You can now join any number of machines by running the following on each node<br \/>\nas root:<\/p>\n<p>kubeadm join 192.168.57.59:6443 &#8211;token 94q0sq.1erns4x5ail242dv &#8211;discovery-token-ca-cert-hash sha256:8ab6c17b87d60e4fc5eaad8739e557c4788ab8e753b79ccc47bfb2ab5ed61dc6<\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>mkdir -p $HOME\/.kube<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>sudo cp -i \/etc\/kubernetes\/admin.conf $HOME\/.kube\/config<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">cp: overwrite \u2018\/root\/.kube\/config\u2019? y<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>sudo chown $(id -u):$(id -g) $HOME\/.kube\/config<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl apply -f https:\/\/raw.githubusercontent.com\/coreos\/flannel\/master\/Documentation\/kube-flannel.yml<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">clusterrole.rbac.authorization.k8s.io\/flannel created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">clusterrolebinding.rbac.authorization.k8s.io\/flannel created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">serviceaccount\/flannel created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">configmap\/kube-flannel-cfg created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">daemonset.extensions\/kube-flannel-ds-amd64 created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">daemonset.extensions\/kube-flannel-ds-arm64 created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">daemonset.extensions\/kube-flannel-ds-arm created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">daemonset.extensions\/kube-flannel-ds-ppc64le created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">daemonset.extensions\/kube-flannel-ds-s390x created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>systemctl restart docker<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>systemctl restart kubelet<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>systemctl status docker kubelet<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl get nodes<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NAME STATUS ROLES AGE VERSION<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node4 Ready master 12m v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl get pods &#8211;all-namespaces<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NAMESPACE NAME READY STATUS RESTARTS AGE<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system coredns-86c58d9df4-4nkc6 1\/1 Running 1 12m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system coredns-86c58d9df4-q8bdp 1\/1 Running 1 12m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system etcd-centos-k8-node4 1\/1 Running 1 12m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-apiserver-centos-k8-node4 1\/1 Running 1 11m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-controller-manager-centos-k8-node4 1\/1 Running 2 11m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-flannel-ds-amd64-fb427 1\/1 Running 2 12m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-proxy-8pct6 1\/1 Running 2 12m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-scheduler-centos-k8-node4 1\/1 Running 1 11m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #21409a;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\"><b># add 2 more node to the cluster and verify:<\/b><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\"># cat config_k8_slave.sh\u00a0 ## run this script from k8 slave node<\/span><\/span><\/span><\/p>\n<p>cat\u00a0config_k8_slave.sh<br \/>\n# setup firewall to allow k8 process<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=6443\/tcp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=10250\/tcp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=6443\/udp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;add-port=10250\/udp<br \/>\nfirewall-cmd &#8211;zone=public &#8211;permanent &#8211;list-ports<br \/>\nfirewall-cmd &#8211;reload<br \/>\nfirewall-cmd &#8211;get-services<\/p>\n<p># setup k8 iptables to allow net.bridge traffic<br \/>\ncat &lt;&lt;EOF &gt; \/etc\/sysctl.d\/k8s.conf<br \/>\nnet.bridge.bridge-nf-call-ip6tables = 1<br \/>\nnet.bridge.bridge-nf-call-iptables = 1<br \/>\nEOF<\/p>\n<p>echo &#8220;1&#8221; &gt; \/proc\/sys\/net\/bridge\/bridge-nf-call-iptables<\/p>\n<h2>run above script on each slave node:<\/h2>\n<p>.\/config_k8_slave.sh<\/p>\n<p>Then add each slave node to master with command:<\/p>\n<p>kubeadm join 192.168.57.59:6443 &#8211;token 94q0sq.1erns4x5ail242dv &#8211;discovery-token-ca-cert-hash sha256:8ab6c17b87d60e4fc5eaad8739e557c4788ab8e753b79ccc47bfb2ab5ed61dc6<\/p>\n<p>## verify<\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl get nodes<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NAME STATUS ROLES AGE VERSION<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node4 Ready master 47m v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node5 Ready &lt;none&gt; 4m41s v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node6 Ready &lt;none&gt; 27s v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl get pods &#8211;all-namespaces<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NAMESPACE NAME READY STATUS RESTARTS AGE<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system coredns-86c58d9df4-4nkc6 1\/1 Running 1 47m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system coredns-86c58d9df4-q8bdp 1\/1 Running 1 47m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system etcd-centos-k8-node4 1\/1 Running 1 46m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-apiserver-centos-k8-node4 1\/1 Running 1 46m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-controller-manager-centos-k8-node4 1\/1 Running 2 46m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-flannel-ds-amd64-4w6kh 1\/1 Running 0 5m30s<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-flannel-ds-amd64-fb427 1\/1 Running 2 46m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-flannel-ds-amd64-tp6df 1\/1 Running 0 76s<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-proxy-8pct6 1\/1 Running 2 47m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-proxy-djqp5 config_k8_slave.sh 1\/1 Running 0 76s<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-proxy-v4rwl 1\/1 Running 0 5m30s<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kube-system kube-scheduler-centos-k8-node4 1\/1 Running 1 46m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl create deployment nginx &#8211;image=nginx<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">deployment.apps\/nginx created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl describe deployment nginx<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Name: nginx<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Namespace: default<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">CreationTimestamp: Fri, 14 Dec 2018 18:20:18 -0500<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Labels: app=nginx<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Annotations: deployment.kubernetes.io\/revision: 1<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Selector: app=nginx<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">StrategyType: RollingUpdate<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">MinReadySeconds: 0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">RollingUpdateStrategy: 25% max unavailable, 25% max surge<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Pod Template:<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Labels: app=nginx<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Containers:<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">nginx:<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Image: nginx<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Port: &lt;none&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Host Port: &lt;none&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Environment: &lt;none&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Mounts: &lt;none&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Volumes: &lt;none&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Conditions:<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Type Status Reason<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&#8212;- &#8212;&#8212; &#8212;&#8212;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Available True MinimumReplicasAvailable<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Progressing True NewReplicaSetAvailable<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">OldReplicaSets: &lt;none&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NewReplicaSet: nginx-5c7588df (1\/1 replicas created)<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Events:<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Type Reason Age From Message<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&#8212;- &#8212;&#8212; &#8212;- &#8212;- &#8212;&#8212;-<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Normal ScalingReplicaSet 15s deployment-controller Scaled up replica set nginx-5c7588df to 1<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl create service nodeport nginx &#8211;tcp=80:80<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">service\/nginx created<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl get nodes<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NAME STATUS ROLES AGE VERSION<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node4 Ready master 50m v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node5 Ready &lt;none&gt; 7m38s v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">centos-k8-node6 Ready &lt;none&gt; 3m24s v1.13.0<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>kubectl get svc<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">kubernetes ClusterIP 10.96.0.1 &lt;none&gt; 443\/TCP 50m<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">nginx NodePort 10.109.14.203 &lt;none&gt; 80:32543\/TCP 44s<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <span style=\"color: #21409a;\"><b>curl centos-k8-node6:32543<\/b><\/span><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;!DOCTYPE html&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;html&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;head&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;title&gt;Welcome to nginx!&lt;\/title&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;style&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">body {<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">width: 35em;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">margin: 0 auto;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">font-family: Tahoma, Verdana, Arial, sans-serif;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"> <span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">}<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;\/style&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;\/head&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;body&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;h1&gt;Welcome to nginx!&lt;\/h1&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;p&gt;If you see this page, the nginx web server is successfully installed and<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">working. Further configuration is required.&lt;\/p&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;p&gt;For online documentation and support please refer to<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;a href=&#8221;http:\/\/nginx.org\/&#8221;&gt;nginx.org&lt;\/a&gt;.&lt;br\/&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">Commercial support is available at<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;a href=&#8221;http:\/\/nginx.com\/&#8221;&gt;nginx.com&lt;\/a&gt;.&lt;\/p&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;p&gt;&lt;em&gt;Thank you for using nginx.&lt;\/em&gt;&lt;\/p&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;\/body&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">&lt;\/html&gt;<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #333333;\"><span style=\"font-family: courier new, courier, monospace;\"><span style=\"font-size: small;\">[root@centos-k8-node4 ~]# <\/span><\/span><\/span><\/p>\n<p>&nbsp;<\/p>\n<p>refer:<\/p>\n<p>https:\/\/www.howtoforge.com\/tutorial\/centos-kubernetes-docker-cluster\/<\/p>\n<pre class=\"command\"><code spellcheck=\"false\"><\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Pre-Steps,\u00a0 install 3 &#8211; 5 VM or physical nodes with Centos7.X,\u00a0 make \/etc\/hosts file on each of them ! Step1:\u00a0 Run my setup below to config systems, setup docker and kubernetes repository, then install on each k8 notes: # cat\u00a0install_docker_k8.sh #! \/usr\/bin\/bash # # Disable SELinux setenforce 0 sed -i &#8211;follow-symlinks &#8216;s\/SELINUX=enforcing\/SELINUX=disabled\/g&#8217; \/etc\/sysconfig\/selinux # Enable [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/pages\/231"}],"collection":[{"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/comments?post=231"}],"version-history":[{"count":4,"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/pages\/231\/revisions"}],"predecessor-version":[{"id":235,"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/pages\/231\/revisions\/235"}],"wp:attachment":[{"href":"https:\/\/cephswift.com\/index.php\/wp-json\/wp\/v2\/media?parent=231"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}