[Kubernetes] kubeadm cri based containerd

Signed-off-by: Kang Huaishuai <khs1994@khs1994.com>
pull/494/head
Kang Huaishuai 2021-07-16 12:25:33 +08:00
parent 95aed0099e
commit 5caddf4413
No known key found for this signature in database
GPG Key ID: 5E515022F565DA09
3 changed files with 406 additions and 9 deletions

View File

@ -126,7 +126,7 @@
* [](kubernetes/concepts.md) * [](kubernetes/concepts.md)
* [](kubernetes/design.md) * [](kubernetes/design.md)
* [ Kubernetes](kubernetes/setup/README.md) * [ Kubernetes](kubernetes/setup/README.md)
* [使 kubeadm kubernetes](kubernetes/setup/kubeadm.md) * [使 kubeadm kubernetes(CRI 使 containerd)](kubernetes/setup/kubeadm.md)
* [ Docker Desktop 使](kubernetes/setup/docker-desktop.md) * [ Docker Desktop 使](kubernetes/setup/docker-desktop.md)
* [ kubernetes ](kubernetes/setup/systemd.md) * [ kubernetes ](kubernetes/setup/systemd.md)
* [ Dashboard](kubernetes/setup/dashboard.md) * [ Dashboard](kubernetes/setup/dashboard.md)

View File

@ -0,0 +1,187 @@
# 使 kubeadm kubernetes(使 Docker)
`kubeadm` `kubeadm init` `kubeadm join` `kubernetes`
## Docker
[ Docker](../../install) Docker
## **kubelet** **kubeadm** **kubectl**
### Ubuntu/Debian
```bash
$ apt-get update && apt-get install -y apt-transport-https
$ curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
$ cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
$ apt-get update
$ apt-get install -y kubelet kubeadm kubectl
```
### CentOS/Fedora
```bash
$ cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
$ sudo yum install -y kubelet kubeadm kubectl
```
##
```bash
$ cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
#
$ sysctl --system
```
## kubelet
### `kubelet.service`
`/etc/systemd/system/kubelet.service.d/10-proxy-ipvs.conf`
```bash
# ipvs
[Service]
ExecStartPre=-/sbin/modprobe ip_vs
ExecStartPre=-/sbin/modprobe ip_vs_rr
ExecStartPre=-/sbin/modprobe ip_vs_wrr
ExecStartPre=-/sbin/modprobe ip_vs_sh
```
```bash
$ sudo systemctl daemon-reload
```
##
### master
```bash
$ sudo kubeadm init --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \
--pod-network-cidr 10.244.0.0/16 \
--v 5 \
--ignore-preflight-errors=all
```
* `--pod-network-cidr 10.244.0.0/16` CNI `flannel`
>
```bash
...
[addons] Applied essential addon: CoreDNS
I1116 12:35:13.270407 86677 request.go:538] Throttling request took 181.409184ms, request: POST:https://192.168.199.100:6443/api/v1/namespaces/kube-system/serviceaccounts
I1116 12:35:13.470292 86677 request.go:538] Throttling request took 186.088112ms, request: POST:https://192.168.199.100:6443/api/v1/namespaces/kube-system/configmaps
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.199.100:6443 --token cz81zt.orsy9gm9v649e5lf \
--discovery-token-ca-cert-hash sha256:5edb316fd0d8ea2792cba15cdf1c899a366f147aa03cba52d4e5c5884ad836fe
```
### node
**** **** kubelet
```bash
$ kubeadm join 192.168.199.100:6443 --token cz81zt.orsy9gm9v649e5lf \
--discovery-token-ca-cert-hash sha256:5edb316fd0d8ea2792cba15cdf1c899a366f147aa03cba52d4e5c5884ad836fe
```
##
Docker
###
* `apiserver` RESTful
* `scheduler` pod
* `controller-manager` endpoint-controller pod replication-controller pod
###
* `proxy` pod 访
###
* Etcd
## 使
`/etc/kubernetes/admin.conf` `~/.kube/config`
`$ kubectl get all -A`
CNI CoreDNS 使 Kubernetes
## CNI
`flannel`
### flannel
podCIDR
```bash
$ kubectl get node -o yaml | grep CIDR
#
podCIDR: 10.244.0.0/16
podCIDRs:
```
```bash
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.11.0/Documentation/kube-flannel.yml
```
## master pod
`kubeadm` 使
```bash
$ kubectl taint nodes --all node-role.kubernetes.io/master-
#
# $ kubectl taint nodes NODE_NAME node-role.kubernetes.io/master=true:NoSchedule
```
##
* [](https://kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/)

View File

@ -1,12 +1,205 @@
# 使 kubeadm kubernetes # 使 kubeadm kubernetes(CRI 使 containerd)
`kubeadm` `kubeadm init` `kubeadm join` `kubernetes` `kubeadm` `kubeadm init` `kubeadm join` `kubernetes`
## Docker ## containerd
[ Docker](../../install) Docker [ Docker](../../install) apt/yum
## **kubelet** **kubeadm** **kubectl** ```bash
# debian
$ sudo apt install containerd.io
# rhel
$ sudo yum install containerd.io
```
## containerd
`/etc/systemd/system/cri-containerd.service`
```
[Unit]
Description=containerd container runtime for kubernetes
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd --config //etc/cri-containerd/config.toml
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
```
`/etc/cri-containerd/config.toml` containerd
```toml
version = 2
# persistent data location
root = "/var/lib/cri-containerd"
# runtime state information
state = "/run/cri-containerd"
plugin_dir = ""
disabled_plugins = []
required_plugins = []
# set containerd's OOM score
oom_score = 0
[grpc]
address = "/run/cri-containerd/cri-containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
# socket uid
uid = 0
# socket gid
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[debug]
address = ""
format = "json"
uid = 0
gid = 0
level = ""
[metrics]
address = "127.0.0.1:1338"
grpc_histogram = false
[cgroup]
path = ""
[timeouts]
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
disable_tcp_service = true
stream_server_address = "127.0.0.1"
stream_server_port = "0"
stream_idle_timeout = "4h0m0s"
enable_selinux = false
selinux_category_range = 1024
sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5"
stats_collect_period = 10
# systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
disable_cgroup = false
disable_apparmor = false
restrict_oom_score_adj = false
max_concurrent_downloads = 3
disable_proc_mount = false
unset_seccomp_profile = ""
tolerate_missing_hugetlb_controller = true
disable_hugetlb_controller = true
ignore_image_defined_volumes = false
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "overlayfs"
default_runtime_name = "runc"
no_pivot = false
disable_snapshot_annotations = false
discard_unpacked_layers = false
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
pod_annotations = []
container_annotations = []
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
# SystemdCgroup enables systemd cgroups.
SystemdCgroup = true
# BinaryName is the binary name of the runc binary.
# BinaryName = "runc"
# BinaryName = "crun"
# NoPivotRoot disables pivot root when creating a container.
# NoPivotRoot = false
# NoNewKeyring disables new keyring for the container.
# NoNewKeyring = false
# ShimCgroup places the shim in a cgroup.
# ShimCgroup = ""
# IoUid sets the I/O's pipes uid.
# IoUid = 0
# IoGid sets the I/O's pipes gid.
# IoGid = 0
# Root is the runc root directory.
Root = ""
# CriuPath is the criu binary path.
# CriuPath = ""
# CriuImagePath is the criu image path
# CriuImagePath = ""
# CriuWorkPath is the criu work path.
# CriuWorkPath = ""
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
max_conf_num = 1
conf_template = ""
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/cri-containerd/certs.d"
[plugins."io.containerd.grpc.v1.cri".registry.headers]
# Foo = ["bar"]
[plugins."io.containerd.grpc.v1.cri".image_decryption]
key_model = ""
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/cri-containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = ""
pool_name = ""
base_image_size = ""
async_remove = false
```
## **kubelet** **kubeadm** **kubectl** **cri-tools** **kubernetes-cni**
### Ubuntu/Debian ### Ubuntu/Debian
@ -77,8 +270,14 @@ $ sudo systemctl daemon-reload
### master ### master
```bash ```bash
$ sudo kubeadm init --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \ $ systemctl enable cri-containerd
$ systemctl start cri-containerd
$ sudo kubeadm init \
--image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \
--pod-network-cidr 10.244.0.0/16 \ --pod-network-cidr 10.244.0.0/16 \
--cri-socket /run/cri-containerd/cri-containerd.sock \
--v 5 \ --v 5 \
--ignore-preflight-errors=all --ignore-preflight-errors=all
``` ```
@ -119,13 +318,23 @@ kubeadm join 192.168.199.100:6443 --token cz81zt.orsy9gm9v649e5lf \
**** **** kubelet **** **** kubelet
```bash ```bash
$ kubeadm join 192.168.199.100:6443 --token cz81zt.orsy9gm9v649e5lf \ $ systemctl enable cri-containerd
--discovery-token-ca-cert-hash sha256:5edb316fd0d8ea2792cba15cdf1c899a366f147aa03cba52d4e5c5884ad836fe
$ systemctl start cri-containerd
$ kubeadm join 192.168.199.100:6443 \
--token cz81zt.orsy9gm9v649e5lf \
--discovery-token-ca-cert-hash sha256:5edb316fd0d8ea2792cba15cdf1c899a366f147aa03cba52d4e5c5884ad836fe \
--cri-socket /run/cri-containerd/cri-containerd.sock
``` ```
## ##
Docker `crictl`
```bash
CONTAINER_RUNTIME_ENDPOINT=/run/cri-containerd/cri-containerd.sock crictl ps -a
```
### ###
@ -185,3 +394,4 @@ $ kubectl taint nodes --all node-role.kubernetes.io/master-
## ##
* [](https://kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/) * [](https://kubernetes.io/zh/docs/setup/production-environment/tools/kubeadm/install-kubeadm/)
* [Container runtimes](https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd)