K8S测试/生产环境下部署metrics-server整理、记录
一、测试环境部署
1.1 获取yaml
wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
1.2 修改yaml
vim components.yaml
#部分内容如下
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: metrics-server
name: metrics-server
namespace: kube-system
spec:
selector:
matchLabels:
k8s-app: metrics-server
strategy:
rollingUpdate:
maxUnavailable: 0
template:
metadata:
labels:
k8s-app: metrics-server
spec:
containers:
- args:
- --cert-dir=/tmp
- --secure-port=4443
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --kubelet-use-node-status-port
- --metric-resolution=15s
#添加参数，禁用 TLS 验证
- --kubelet-insecure-tls
#修改镜像地址为阿里云
image: registry.aliyuncs.com/google_containers/metrics-server:v0.6.4
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /livez
port: https
scheme: HTTPS
periodSeconds: 10
name: metrics-server
ports:
- containerPort: 4443
name: https
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /readyz
port: https
scheme: HTTPS
initialDelaySeconds: 20
periodSeconds: 10
resources:
requests:
cpu: 100m
memory: 200Mi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
volumeMounts:
- mountPath: /tmp
name: tmp-dir
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
serviceAccountName: metrics-server
volumes:
- emptyDir: {}
name: tmp-dir
1.3 部署metrics-server服务
#部署服务 kubectl apply -f components.yaml #验证服务 kubectl kubectl get apiservices |grep metrics #内容如下， AVAILABLE显示True则正常 NAME SERVICE AVAILABLE AGE v1beta1.metrics.k8s.io kube-system/metrics-server True 4h28m #查看节点指标 kubectl top nodes #内容如下 NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% master01 1410m 70% 1216Mi 31% master02 1443m 72% 1188Mi 31% node01 531m 26% 591Mi 15% node02 489m 24% 637Mi 16%
1.4 检查metrics-server的API Service是否正常
kubectl get apiservices |grep metrics
#排错参考
#官网 https://github.com/kubernetes-sigs/metrics-server #CSDN https://blog.csdn.net/qq_45097352/article/details/121509213 #阿里云 https://help.aliyun.com/document_detail/291404.html
二、生产环境部署
1.部署方式：kubeadm
2.k8s版本：v1.28.2
3.使用
--kubelet-insecure-tls 选项来配置Metrics Server会禁用 TLS 验证，这会降低安全性，因此不建议在生产环境中使用。
2.1 证书生成（参考二进制部署）
2.1.1 生成CA证书请求文件
vim ca-csr.json
#文件内容如下
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Hubei",
"L": "Wuhan",
"O": "k8s",
"OU": "system"
}
],
"ca": {
"expiry": "87600h"
}
}
#执行生成
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2.1.2 生成CA证书文件
vim ca-config.json
#文件内容如下
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
}
2.1.3 生成metrics-server证书
vim metrics-server.json
#文件内容如下
{
"CN": "metrics-server",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Hubei",
"L": "Wuhan",
"O": "k8s",
"OU": "system"
}
]
}
#生成证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes metrics-server.json | cfssljson -bare ./metrics-server
2.2 配置K8S服务
2.2.1 在master主机开启apiserver聚合服务
vim /etc/kubernetes/manifests/kube-apiserver.yaml #spec.containers.command 在这一行下面手动添加如下一行 --enable-aggregator-routing=true
2.2.2 开启TLSBootstrap
2.2.2.1 修改config.yaml
vim /var/lib/kubelet/config.yaml #所有节点（master和node）节点 serverTLSBootstrap: true
2.2.2.2 修改configmap
kubectl get configmap -n kube-system |grep kubelet-config #内容如下 kubelet-config 1 22d kubectl edit configmap kubelet-config -n kube-system #添加内容同上
2.2.3 重启kubelet服务
systemctl restart kubelet
2.2.4 证书签发
#查看需要签发的证书 kubectl get certificatesigningrequests 或 kubectl get csr
#内容如下
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
csr-9j9r8 28s kubernetes.io/kubelet-serving system:node:master01 Pending
csr-sp7hc 28s kubernetes.io/kubelet-serving system:node:node01 Pending
csr-sqflv 28s kubernetes.io/kubelet-serving system:node:master02 Pending
csr-wcg7c 28s kubernetes.io/kubelet-serving system:node:node02 Pending
# 签发证书 kubectl certificate approve <CSR-名称>
2.2.5 RBAC配置
确保 Metrics Server 具有适当的 RBAC（Role-Based Access Control）权限，以便能够访问 kubelet API。在生产环境中，最好使用最小特权原则，只授予 Metrics Server 所需的权限。
vim metrics-server-rbac.yaml #内容如下 apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-server rules: - apiGroups: - "" resources: - pods - nodes - nodes/stats - namespaces verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system
2.2.6 部署 Metrics Server
2.2.6.1 获取yaml
wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
2.2.6.2 修改yaml
#仅修改镜像地址，其他无需修改，内容如下 registry.aliyuncs.com/google_containers/metrics-server:v0.6.4
2.2.6.3 部署服务
kubectl apply -f metrics-server-deployment.yaml
2.2.7 服务验证
kubectl kubectl get apiservices |grep metrics #内容如下， AVAILABLE显示True则正常 NAME SERVICE AVAILABLE AGE v1beta1.metrics.k8s.io kube-system/metrics-server True 4h28m #查看节点指标 kubectl top nodes #内容如下 NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% master01 1410m 70% 1216Mi 31% master02 1443m 72% 1188Mi 31% node01 531m 26% 591Mi 15% node02 489m 24% 637Mi 16%
三、参考文献
http://www.javashuo.com/article/p-tfsnltqj-ke.html https://blog.csdn.net/xhredeem/article/details/127314979 https://blog.csdn.net/vic_qxz/article/details/127569544