kube 설치 및 노드 업그레이드

cd HashiCorp
code Vagrantfile
vagrant plugin install vagrant-vbguest
vagrant up

 

 

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

*kubeadm, kubelet 및 kubectl 설치(k-control, k-node1...각각 들어가서 설치)
(설치 참고: https://kubernetes.io/ko/docs/setup/production-environment/tools/kubeadm/install-kubeadm/)

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet=1.19.12-00 kubeadm=1.19.12-00 kubectl=1.19.12-00
sudo apt-mark hold kubelet kubeadm kubectl


* k-control
(https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)
sudo kubeadm init --control-plane-endpoint 192.168.200.50 --pod-network-cidr 192.168.0.0/16 --apiserver-advertise-address 192.168.200.50
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
ls -l ~/.kube/config
-rw------- 1 vagrant vagrant 5594 Jul  6 06:44 /home/vagrant/.kube/config



(https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises)

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
kubectl get nodes     -> ready 상태인지 확인
kubeadm token list
TOKEN                     TTL         EXPIRES                USAGES                   DESCRIPTION                                                EXTRA GROUPS
bkoyxl.3aht0jk9ws0d4nfj   23h         2021-07-07T06:42:57Z   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
>    openssl dgst -sha256 -hex | sed 's/^.* //'
015ac96083dbcdeece03ff30344df57cf26c0785570975dbeece3c90f0eff824


*k-node123 가서 명령어 입력
(ubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash>)
sudo kubeadm join 192.168.200.50:6443 --token bkoyxl.3aht0jk9ws0d4nfj --discovery-token-ca-cert-hash sha256:015ac96083dbcdeece03ff30344df57cf26c0785570975dbeece3c90f0eff824 

*다시 k-control 가서 노드 확인
kubectl get nodes

*cka (쿠버네티스 자격증)->설치 업그레이드 할줄알아야함 안그러면 빵점

sudo apt-cache madison kubeadm

sudo apt install kubeadm=1.18.19-00 kubectl=1.18.19-00 kubelet=1.18.19-00

 

*sudo kubeadm reset

*리셋말고 vagrant destroy -f 해서 다시 설치

sudo apt install kubeadm=1.18.19-00 kubectl=1.18.19-00 kubelet=1.18.19-00

 

 

*노드 확인시 계속 not ready 상태일 경우 k-control, k-node123 전부 이대로 pull 해주기

sudo docker login
sudo docker pull calico/node:v3.19.1
sudo docker pull calico/pod2daemon-flexvol:v3.19.1
sudo docker pull calico/cni:v3.19.1
sudo docker pull calico/kube-controllers:v3.19.1
kubectl get nodes             -> k-control 에서 확인

<컨트롤 플레인 업그레이드>
*kubeadm 업그레이드
sudo apt-cache madison kubeadm
sudo apt-get update && \
sudo apt-get install -y --allow-change-held-packages kubeadm=1.18.20-00
kubeadm version
kubeadm upgrade plan
sudo kubeadm upgrade apply v1.18.20

*kubelet과 kubectl 업그레이드
sudo apt-get update && \
sudo apt-get install -y --allow-change-held-packages kubelet=1.18.20-00 kubectl=1.18.20-00
kubelet --version
kubectl version
sudo systemctl daemon-reload
sudo systemctl restart kubelet



<노드 업그레이드>
*kubeadm 업그레이드
sudo apt-get update && \
sudo apt-get install -y --allow-change-held-packages kubeadm=1.18.20-00
sudo kubeadm upgrade node

*kubelet과 kubectl 업그레이드
sudo apt-get update && \
sudo apt-get install -y --allow-change-held-packages kubelet=1.18.20-00 kubectl=1.18.20-00
sudo systemctl daemon-reload
sudo systemctl restart kubelet



*컨트롤 플레인에 가서 버전 제대로 올라왔는지 확인
kubectl get nodes
kubectl get pods -A인        -> 러닝되는지 확인

sudo kubeadm reset
sudo rm -rf /var/lib/kubelet/
sudo rm -rf /etc/kubernetes/
sudo rm -rf /etc/cni/net.d
sudo rm -rf ~/.kube/
sudo apt remove kubeadm kubelet kubectl

* 컨트롤 플레인에서 node3 삭제 후 1.19.12 새로 설치
sudo apt-get install -y --allow-change-held-packages kubeadm=1.19.12-00
sudo kubeadm upgrade apply v1.19.12
sudo apt-get install -y --allow-change-held-packages kubelet=1.19.12-00 kubectl=1.19.12-00
sudo systemctl daemon-reload
sudo systemctl restart kubelet
kubectl get nodes

*node1, 2 가서 1.19.12 다시 설치
sudo apt-get install -y --allow-change-held-packages kubeadm=1.19.12-00
sudo kubeadm upgrade node
sudo apt-get install -y --allow-change-held-packages kubelet=1.19.12-00 kubectl=1.19.12-00
sudo systemctl daemon-reload
sudo systemctl restart kubelet

*컨트롤로 가서 각 노드들 버전이 제대로 바뀌었는지 확인
kubectl get nodes

*컨트롤 노드 다 해주기
vagrant@k-control:~$ vi .vimrc
vagrant@k-control:~$ cat .vimrc
syntax on
autocmd FileType yaml setlocal ts=2 sts=2 sw=2 expandtab autoindent


*컨트롤에서 해주기
vagrant@k-control:~$ vi myapp-pod.yaml
vagrant@k-control:~$ cat myapp-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
spec:
  containers:
  - image: ghcr.io/c1t1d0s7/go-myweb
    name: myapp
    ports:
    - containerPort: 8080
 vagrant@k-control:~$ kubectl create -f myapp-pod.yaml
vagrant@k-control:~$ kubectl get pods
NAME        READY   STATUS    RESTARTS   AGE
myapp-pod   1/1     Running   0          46s

 

 ip 를 10.0.~ 이 아닌 192.~(enp0s8)로 바꿔주기 -> 포트포워딩시 오류가 날수있어서 아이피 바꿔줘야함

* 컨트롤(192.168.200.50), 노드1(x.x.x.51), 노드2(x.x.x.52)

ip a s
echo "KUBELET_EXTRA_ARGS='--node-ip 192.168.200.5x'" | sudo tee /etc/default/kubelet -> 컨트롤이랑 각 노드 전부 이것부터 먼저해주고 포트포워딩 명령어 쳐야함. 안그럼 오류뜸
sudo systemctl daemon-reload
sudo systemctl restart kubelet

kubectl get nodes -o wide

kubectl port-forward myapp-pod 8080:8080 -> 포트포워딩

 

*새 터미널 열어서 확인

vagrant@k-control:~$ curl http://localhost:8080
Hello World!
myapp-pod