云攻防-k8s手动搭建

参考视频：
https://www.bilibili.com/video/BV1P17fzBES1?spm_id_from=333.788.videopod.episodes&vd_source=89bf25153801ebc942aaf90aa2af1675&p=5
看官网搭建：
https://kubernetes.io/zh-cn/docs/tasks/tools/install-kubectl-linux/

国内网络环境 优化的 CentOS 7 部署 Kubernetes 集群详细步骤,及常见问题

一、环境准备（所有节点执行）

1. 配置主机名与 hosts 解析

1
2
3
4
5
6
7
8
9
10
11
# Master 节点
hostnamectl set-hostname k8s-master
 
# Worker 节点（如 node1）
hostnamectl set-hostname k8s-node1
 
# 所有节点编辑 /etc/hosts
cat <<EOF >> /etc/hosts
<Master节点内网IP> k8s-master
<Worker节点内网IP> k8s-node1
EOF

2. 关闭防火墙、SELinux 和 Swap

1
2
3
4
5
systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab  # 永久关闭 Swap

3. 配置内核参数

1
2
3
4
5
6
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system

4. 加载内核模块

1
2
modprobe br_netfilter
modprobe overlay

二、安装容器运行时（所有节点执行）

CentOS 7 yum 无法使用

CentOS 7 仓库已经被归档，当前的镜像地址无法找到所需的文件。CentOS 7 的官方支持已经结束，部分仓库已被移至归档库。这导致了 yum 命令无法找到所需的元数据文件。CentOS 7 的官方仓库在 2024 年 6 月 30 日之后已经停止维护。因此，使用最新的 CentOS 7 官方仓库可能会遇到问题。

参考文章：

https://blog.csdn.net/weixin_68792404/article/details/147272888

解决方法:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
cd /etc/yum.repos.d
cp  CentOS-Base.repo   CentOS-Base.repo.backup
vi CentOS-Base.repo
修改内容
# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the 
# remarked out baseurl= line instead.
#
#
 
[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
#baseurl=http://vault.centos.org/7.9.2009/x86_64/os/
baseurl=http://vault.centos.org/7.9.2009/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#released updates 
[updates]
name=CentOS-$releasever - Updates
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
#baseurl=http://vault.centos.org/7.9.2009/x86_64/os/
baseurl=http://vault.centos.org/7.9.2009/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
 
#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras&infra=$infra
#$baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
#baseurl=http://vault.centos.org/7.9.2009/x86_64/os/
baseurl=http://vault.centos.org/7.9.2009/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
 
#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
#baseurl=http://vault.centos.org/7.9.2009/x86_64/os/
baseurl=http://vault.centos.org/7.9.2009/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

保存后执行

1
2
sudo yum clean all
sudo yum makecache

然后执行以下命令

1
2
3
4
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
 
sudo yum clean all
sudo yum makecache

1. 安装 Docker（国内镜像加速）

1
2
3
# 使用阿里云 Docker 仓库
yum install -y yum-utils
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
1
2
# 安装指定版本 Docker
yum install -y docker-ce-20.10.23 docker-ce-cli-20.10.23 containerd.io
1
2
3
4
5
6
7
8
9
10
11
# 配置 Docker 镜像加速和 cgroup 驱动
mkdir -p /etc/docker
cat <<EOF > /etc/docker/daemon.json
{
  "registry-mirrors": ["https://<你的阿里云镜像加速地址>.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {"max-size": "100m"},
  "storage-driver": "overlay2"
}
EOF
1
2
# 启动 Docker
systemctl enable docker && systemctl start docker

三、安装 Kubernetes 组件（所有节点执行）

1. 配置阿里云 Kubernetes 仓库

1
2
3
4
5
6
7
8
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
EOF

2. 安装 kubeadm、kubelet、kubectl

1
2
3
4
5
# 指定版本安装（示例版本 1.28.2）
yum install -y kubelet-1.28.2 kubeadm-1.28.2 kubectl-1.28.2 --disableexcludes=kubernetes
 
# 启动 kubelet
systemctl enable kubelet && systemctl start kubelet

四、初始化 Master 节点

安装网络插件:
使用containerd配置k8s

1
2
3
4
5
6
7
8
9
//没有文件可以创建/etc/containerd
# mkdir /etc/containerd
containerd config default > /etc/containerd/config.tomlgrep sandboximage /etc/containerd/config.toml

sed -i "s#k8s.gcr.io/pause#registry.aliyuncs.com/google containers/pause#g" /etc/containerd/config.toml

sed -i "s#registry.k8s.io/pause#registry.aliyuncs.com/google_containers/pause#g" /etc/containerd/config.toml
#配置containerd cgroup驱动程序systemd：
sed -i 's#Systemdcgroup =false#Systemdcgroup = true#g' /etc/containerd/config.toml
1
2
3
4
5
6
#修改配置文件/etc/containerd/config.toml，145行添加config_path
144   [plugins."io.containerd.grpc.v1.cri".registry]
145        config_path="/etc/containerd/certs.d"

#创建对应目录
mkdir -p /etc/containerd/certs.d/docker.io
1
2
3
4
5
6
7
8
9
10
11
#配置加速
#配置阿里云镜像
cat >/etc/containerd/certs.d/docker.io/hosts.toml <<EOF
server ="https://docker.io"
[host."https://ms9glx6x.mirror.aliyuncs.com"]
 capabilities =["pull","resolve"]
[host."https://docker.mirrors.ustc.edu.cn"]
 capabilities =["pull","resolve"]
[host."https://registry-1.docker.io"]
 capabilities =["pull","resolve" ,"push"]
EOF

上面步骤所有节点都要操作

只操作k8s-master，生成管理节点的组件

1
2
3
4
5
6
7
8
9
10
11
12
13
kubeadm config print init-defaults > kubeadm.yaml
kubeadm init --config kubeadm.yaml
vim kueadm.yaml
#kueadm.yaml
advertiseAddress: 192.168.200.148   #更换为master主机ip
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: k8s-master    #更换为master的主机名
imageRepository: registry.aliyuncs.com/google_containers  # 替换为国内镜像源
kubernetsVersion: 1.24.4   #改为本机安装的版本
在networking:下加入
podSubnet: 10.244.0.0/16  #添加pod子网
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#查看需要使用的镜像列表,若无问题，将得到如下列表
$ kubeadm config images list --config kubeadm.yaml
registry.aliyuncs.com/google 
containers/kube-apiserver:v1.24.4registry.aliyuncs.com/google_containers/kube-controller-manager:v1.24.4
registry.aliyuncs.com/google containers/kube-scheduler:v1.24.4
registry.aliyuncs.com/google containers/kube-proxy:v1.24.4
registry.aliyuncs.com/google containers/pause:3.7
registry.aliyuncs.com/google containers/etcd:3.5.3-0
registry.aliyuncs.com/google containers/coredns:v1.8.6

#提前下载镜像到本地
[root@k8s-master ~]#kubeadm config images pull --config kubeadm.yaml

[config/images] Pulled registry.aliyuncs.com/google containers/kube-apiserver:v1.24.4
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.24.4
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.24.4
[config/images] Pulled registry.aliyuncs.com/google containers/kube-proxy:v1.24.4
[config/images] Pulled registry.aliyuncs.com/google containers/pause:3.7
[config/images] Pulled registry.aliyuncs.com/google containers/etcd:3.5.3-0
[config/images] Pulled registry.aliyuncs.com/google containers/coredns:v1.8.6
**配置 kubelet 使用 Docker**

初始化 master 节点

1
kubeadm init --config kubeadm.yaml

根据提示执行以下命令

1
2
3
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

会出现以下提示，将node节点加入到master中

kubeadm token create –print-join-command 可以查看加入集群的命令

1
2
3
4
kubeadm join 192.168.200.148:6443 --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:9639e444338138f5324bb769ff4898f67e906d43471adeadf2459529f681ef2d 
# 如果出现一种情况，检测到多个 CRI（容器运行时接口）端点，无法自动选择使用哪一个，需要指定CRI
--cri-socket unix:///var/run/containerd/containerd.sock #指定containerd

这样就成功加入到master集群中了

由于它们是Noready状态，所以需要加入网络

安装网络插件:
flannel安装

1
2
3
4
5
6
7
8
9
10
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

#下载下来后，如果主机有多个网卡，需要指定网卡
#修改配置文件，kube-flannel.yml
#在文件的160行左右
args:
- --ip-masq
- --kube-subnet-mgr
#添加以下代码
- --iface=ens33
1
2
kubectl apply -f kube-flannel.yml   #取下载镜像
kubectl -n kube-flannel get po -owide -w #等待初始化镜像的拉取
# vi kube-flannel.yml
net-conf.json: |
  {
    "Network" : 10.244.0.0/16   #需要与配置pod子网一致
    "Backend" : {}
  }