阿里云上購(gòu)買(mǎi)ECS 3臺(tái):
? 其中1臺(tái)ECS: ?8核cpu, 內(nèi)存16G, 系統(tǒng)盤(pán)80G, 作為openshift master以及企業(yè)dockerregistry
? 其中2臺(tái)ECS：2核cpu, 內(nèi)存8G, 系統(tǒng)盤(pán)30G、數(shù)據(jù)盤(pán)40G, 作為openshift node
【注】以下安裝全部用root用戶(hù)，root用戶(hù)應(yīng)有ssh遠(yuǎn)程登錄權(quán)限
1.node節(jié)點(diǎn)安裝
? #設(shè)置主機(jī)名
?hostnamectl set-hostname node1.honsen.com
?hostnamectl set-hostname node2.honsen.com
? #安裝依賴(lài)包
? yum install wget git net-tools bind-utils iptables-services bridge-utilsbash-completion kexec-tools sos psacct
? yum update
? #啟用&啟動(dòng)Docker服務(wù)
?systemctl enable docker; systemctl start docker
? #修改docker配置
? vi/etc/sysconfig/docker
? 首行Options追加上--insecure-registry172.30.0.0/16 --registry-mirror=https://docker.mirrors.ustc.edu.cn
? #重啟docker
?systemctl restart docker
? #啟用&啟動(dòng)網(wǎng)絡(luò)管理器
?systemctl enable NetworkManager; systemctl start NetworkManager
? #停止&禁用防火墻
?systemctl stop firewalld ; systemctl disable firewalld
? #Ansible和系統(tǒng)自帶的urllib3有沖突, 卸載之: Error unpacking rpm package python-urllib3-1.10.2-3.el7.noarch
? pip uninstall urllib3
? #以下設(shè)置Docker中配置overlay/overlay2存儲(chǔ)驅(qū)動(dòng)
? #檢查overlay驅(qū)動(dòng)是否已加載
??lsmod | grep overlay
? #以下設(shè)置node節(jié)點(diǎn)的docker數(shù)據(jù)存儲(chǔ)區(qū)
??see https://docs.openshift.org/latest/install_config/install/host_preparation.html#configuring-docker-storage
? #查看磁盤(pán)情況
?fdisk -l
? #修改docker存儲(chǔ)配置
? vi /etc/sysconfig/docker-storage-setup
???STORAGE_DRIVER=overlay2
???DEVS=/dev/vdb
???VG=docker-vg
? #執(zhí)行docker存儲(chǔ)配置
?docker-storage-setup
? 【注】執(zhí)行這一步后會(huì)創(chuàng)建邏輯分區(qū)/dev/vdb1
?fdisk -l?
? #格式化數(shù)據(jù)盤(pán)為xfs
?mkfs.xfs -f /dev/vdb1
? #停止docker
?systemctl stop docker
? rm-rf /var/lib/docker/*
?#mount數(shù)據(jù)盤(pán)到/var/lib/docker
?mount -t xfs /dev/vdb1 /var/lib/docker
? #檢查mount是否成功
? df -Th /var/lib/docker
? #配置重啟時(shí)自動(dòng)mount
? vi /etc/fstab
? 加上一行：
???/dev/vdb1?????/var/lib/docker????? xfs???? defaults???0 0
? #重啟docker??????
?systemctl restart docker
? #查看docker配置信息
?docker info
2.master安裝
? #設(shè)置主機(jī)名
?hostnamectl set-hostname master.honsen.com
? #設(shè)置本地域名解析
?echo "172.18.113.134 master.honsen.com" >> /etc/hosts
?echo "172.18.113.136 node1.honsen.com" >> /etc/hosts
?echo "172.18.113.135 node2.honsen.com" >> /etc/hosts
? #安裝依賴(lài)包
? yum install -y docker wget git net-tools bind-utils iptables-services bridge-utilsbash-completion
? #啟用&啟動(dòng) Docker服務(wù)
?systemctl enable docker; systemctl start docker
? #修改docker配置
? vi/etc/sysconfig/docker
? 首行Options追加上--insecure-registry172.30.0.0/16 --registry-mirror=https://docker.mirrors.ustc.edu.cn
? #重啟docker
?systemctl restart docker
? #啟用&啟動(dòng)網(wǎng)絡(luò)管理器
?systemctl enable NetworkManager; systemctl start NetworkManager
? #停止&禁用防火墻
?systemctl stop firewalld ; systemctl disable firewalld
? #Ansible和系統(tǒng)自帶的urllib3有沖突, 卸載之: Error unpacking rpm package python-urllib3-1.10.2-3.el7.noarch
? pip uninstall urllib3
? #以下修改master的docker存儲(chǔ)配置
? vi /etc/sysconfig/docker-storage?
?DOCKER_STORAGE_OPTIONS=--storage-driver devicemapper --storage-optdm.fs=xfs
? #重啟docker
?systemctl stop docker
? rm -rf /var/lib/docker/*
?systemctl restart docker?
? #查看docker配置信息
?docker info?
? #安裝, 啟用&啟動(dòng)etcd分布式數(shù)據(jù)庫(kù)
? yum -y install etcd
?systemctl enable etcd; systemctl start etcd
? #下載安裝EPEL
? yum -y installhttps://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
? #設(shè)置epelenable=0
? sed -i -e "s/^enabled=1/enabled=0/" /etc/yum.repos.d/epel.repo
? #安裝ansible
? yum -y --enablerepo=epel install ansible pyOpenSSL
? #生成秘鑰, 實(shí)現(xiàn)ssh免密碼連接
?ssh-keygen -f ~/.ssh/id_rsa -N ''
? #復(fù)制秘鑰到集群中的所有節(jié)點(diǎn),實(shí)現(xiàn)無(wú)密碼訪(fǎng)問(wèn)
? for host in master.honsen.com node1.honsen.com node2.honsen.com; do ssh-copy-id -i~/.ssh/id_rsa.pub -p root@$host;? done
? #設(shè)置ssh端口, 例如52113
?? vi ~/.ssh/config
??host master.honsen.com
??port 52113?
??host node1.honsen.com
??port 52113
??host node2.honsen.com
??port 52113
? #改變~/.ssh/config的權(quán)限
?chmod 600 ~/.ssh/config
? #測(cè)試ssh免密碼連接
? ssh node1.honsen.com
?[honsen@node1 ~]$exit
? #以下下載安裝openshift-ansible
?wget https://github.com/openshift/openshift-ansible/archive/openshift-ansible-3.7.31-1.tar.gz
? tar zxvf openshift-ansible-3.7.31-1.tar.gz
? vi /etc/ansible/hosts
??????????????????????????? [OSEv3:children]
??????????????????????????? masters
??????????????????????????? nodes
??????????????????????????? etcd
??????????????????????????? #Set variables common for all OSEv3 hosts
??????????????????????????? [OSEv3:vars]
??????????????????????????? #SSH user, this user should allow ssh based auth without requiring a password
??????????????????????????? ansible_ssh_user=root? #ssh用戶(hù)名
??????????????????????????? ansible_ssh_port=52113?? #ssh服務(wù)端口
??????????????????????????? openshift_deployment_type=origin
??????????????????????????? openshift_release=3.7.1
??????????????????????????? #如果CPU內(nèi)存滿(mǎn)足條件, 可以注釋掉openshift_disable_check
??????????????????????????? #Master節(jié)點(diǎn)要求 2 CPU核心, 16G內(nèi)存, 40G磁盤(pán)
??????????????????????????? #?? Node節(jié)點(diǎn)要求 1 CPU核心,? 8G內(nèi)存, 20G磁盤(pán)
??????????????????????????? #openshift_disable_check=disk_availability,docker_storage,memory_availability,docker_image_availability
??????????????????????????? #uncomment the following to enable htpasswd authentication; defaults toDenyAllPasswordIdentityProvider
??????????????????? openshift_master_identity_providers=[{'name':'htpasswd_auth','login':'true','challenge':'true','kind':'HTPasswdPasswordIdentityProvider','filename':'/etc/origin/master/htpasswd'}]
??????????????????????????? #host group for masters
??????????????????????????? [masters]
??????????????????????????? master.honsen.com
??????????????????????????? #host group for nodes, includes region info
??????????????????????????? [nodes]
??????????????????????????? master.honsen.com
??????????????????????????? node1.honsen.comopenshift_node_labels="{'region': 'primary', 'zone': 'default'}"
??????????????????????????? node2.honsen.comopenshift_node_labels="{'region': 'infra', 'zone': 'default'}"
??????????????????????????? [etcd]
??????????????????????????? master.honsen.com
???????? #開(kāi)始安裝openshift集群(用ssh用戶(hù))
???ansible-playbook ~/openshift-ansible-openshift-ansible-3.7.31-1/playbooks/byo/config.yml????????
3.成功安裝后，檢查安裝結(jié)果
#基本檢查??????
???????? oc get nodes --show-labels?????
?????????????????? NAME??????????????? STATUS???????????????????? AGE?????? VERSION???????????? LABELS
?????????????????? master.honsen.com?? Ready,SchedulingDisabled?? 1h???????v1.7.6+a08f5eeb62??beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=master.honsen.com,openshift-infra=apiserver
?????????????????? node1.honsen.com??? Ready?????????????????????1h??????? v1.7.6+a08f5eeb62??beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node1.honsen.com,region=primary,zone=default
?????????????????? node2.honsen.com??? Ready????????????????????? 1h??????? v1.7.6+a08f5eeb62?? beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=node2.honsen.com,region=infra,zone=default
***安裝過(guò)程中常見(jiàn)問(wèn)題***
1) 多次執(zhí)行ansible-playbook后，出現(xiàn)ssh error：
????ssh: connect to host node1.honsen.com port 52113: No route to host\r\n
???????? #檢查防火墻狀態(tài)
???????? systemctl status firewalld??????
???????? ●firewalld.service
???????? ?? Loaded: masked (/dev/null; bad)? //masked表明防火墻被鎖住了！！！
???????? ?? Active: inactive (dead)
???????? #以下是解決辦法??
???????? #修改并重啟iptabless
???????? vi /etc/sysconfig/iptables
?????????????????? 將
?????????????????? -AINPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
?????????????????? 替換為
?????????????????? -AINPUT -p tcp -m state --state NEW -m tcp --dport 52113 -j ACCEPT
???????? service iptables restart??
2)Failure summary:
???????? ?Hosts:???master.honsen.com, node1.honsen.com, node2.honsen.com
????Play:???? OpenShift Health Checks
????Task:???? Run health checks(install) - EL
????Message:? One or more checksfailed
????Details:?
?????????????? check"package_version":
?????????????? Some required package(s) areavailable at a version
?????????????? that is higher than requested
???????????????? docker-1.13.1
?????????????? This will prevent installing theversion you requested.
?????????????? Please check your enabledrepositories or adjust openshift_release.?????
?? #解決辦法
?? cd /roles/openshift_health_checker/openshift_checks
?? vi package_version.py
?????????????????? openshift_to_docker_version= {
???????????????????????????????????? (3,1): "1.8",
???????????????????????????????????? (3,2): "1.10",
???????????????????????????????????? (3,3): "1.10",
???????????????????????????????????? (3,4): "1.12",
???????????????????????????????????? (3,5): "1.12",
???????????????????????????????????? (3,6): "1.12",
???????????????????????????????????? (3,7): ["1.12", "1.13.1"], # <== here add the dockerversion that you want????????????????????????????
???????????????????????????????????? (3,9): "1.13.1",
?????????????????? }??
