-
Choerodon平台版本: 0.6.0
-
遇到问题的执行步骤:
ansible-playbook -i inventory.ini 90-init-cluster.yml -
文档地址: https://choerodon.io/zh/docs/installation-configuration/steps/kubernetes/
-
环境信息(如:节点信息):
安装集群的环境是两个内网ip: 172.22.0.3, 172.22.0.4 -
报错日志:
included: /home/harry/kubeadm-ha/roles/kube-certificates/tasks/distribute.yml for 172.22.0.4, 172.22.0.3
TASK [kube-certificates : 获取 kubernetes master 节点相关证书] *************************
ok: [172.22.0.4 -> 172.22.0.4] => (item=admin.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=admin.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=apiserver.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=apiserver.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=apiserver-kubelet-client.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=apiserver-kubelet-client.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=ca.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=ca.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=front-proxy-ca.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=front-proxy-ca.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=front-proxy-client.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=front-proxy-client.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=kube-controller-manager.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=kube-scheduler.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=kube-scheduler.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=sa.key)
ok: [172.22.0.4 -> 172.22.0.4] => (item=sa.pub)
TASK [kube-certificates : 获取 kubelet 服务端证书] ************************************
ok: [172.22.0.4 -> 172.22.0.4] => (item=kubelet.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=kubelet.key)
TASK [kube-certificates : 分发 kubelet 服务端证书到所有节点] *******************************
changed: [172.22.0.3] => (item=None)
changed: [172.22.0.3] => (item=None)
changed: [172.22.0.3]
TASK [kube-certificates : 获取 kubelet 客户端证书列表] **********************************
ok: [172.22.0.4]
ok: [172.22.0.3]
TASK [kube-certificates : 获取 kubelet 客户端证书] ************************************
ok: [172.22.0.4 -> 172.22.0.4] => (item=/etc/kubernetes/pki/ca.crt)
ok: [172.22.0.3 -> 172.22.0.4] => (item=/etc/kubernetes/pki/ca.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=/var/lib/kubelet/pki/kubelet-client-172.22.0.4.crt)
ok: [172.22.0.3 -> 172.22.0.4] => (item=/var/lib/kubelet/pki/kubelet-client-172.22.0.3.crt)
ok: [172.22.0.4 -> 172.22.0.4] => (item=/var/lib/kubelet/pki/kubelet-client-172.22.0.4.key)
ok: [172.22.0.3 -> 172.22.0.4] => (item=/var/lib/kubelet/pki/kubelet-client-172.22.0.3.key)
TASK [kube-certificates : 分发 kubelet 客户端证书] ************************************
ok: [172.22.0.3] => (item=None)
changed: [172.22.0.3] => (item=None)
changed: [172.22.0.3] => (item=None)
changed: [172.22.0.3]
TASK [kube-certificates : 读取 kubelet.conf 文件 stat 信息] **************************
ok: [172.22.0.3]
ok: [172.22.0.4]
PLAY [kube-master,new-master] **************************************************
TASK [kube-master : 读取 docker sock 文件 stat 信息] *********************************
ok: [172.22.0.4]
TASK [kube-master : 设置 container_manager_detected 变量] **************************
ok: [172.22.0.4]
TASK [kube-master : 创建 kubernetes 相关目录] ****************************************
ok: [172.22.0.4] => (item=/var/lib/kubelet)
ok: [172.22.0.4] => (item=/etc/kubernetes/config)
ok: [172.22.0.4] => (item=/etc/kubernetes/pki)
ok: [172.22.0.4] => (item=/etc/kubernetes/config)
ok: [172.22.0.4] => (item=/etc/kubernetes/manifests)
ok: [172.22.0.4] => (item=/var/log/kubernetes/audit)
ok: [172.22.0.4] => (item=/usr/share/bash-completion/completions)
TASK [kube-master : 读取 kubelet.conf 文件 stat 信息] ********************************
ok: [172.22.0.4]
included: /home/harry/kubeadm-ha/roles/kube-master/tasks/kubeadm-config.yml for 172.22.0.4
TASK [kube-master : 确认 kubeadm 版本] *********************************************
changed: [172.22.0.4]
TASK [kube-master : 设置 kubeadm api version 为 v1beta2] **************************
ok: [172.22.0.4]
TASK [kube-master : 获取 Docker Cgroup Driver 值] *********************************
changed: [172.22.0.4]
TASK [kube-master : 设置 docker 为容器运行时 kubelet_cgroup_driver_detected 变量] ********
ok: [172.22.0.4]
TASK [kube-master : 创建 kubeadm 的配置文件] ******************************************
fatal: [172.22.0.4]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: Unable to look up a name or access an attribute in template string (apiVersion: kubeadm.k8s.io/v1beta2\nkind: InitConfiguration\nlocalAPIEndpoint:\n advertiseAddress: {{ CURRENT_HOST_IP }}\n bindPort: 6443\nnodeRegistration:\n kubeletExtraArgs:\n network-plugin: cni\n root-dir: {{ kubelet_root_dir }}\n hostname-override: {{ inventory_hostname }}\n pod-infra-container-image: {{ pod_infra_container_image }}\n criSocket: {{ CRI_SOCKET }}\n name: {{ inventory_hostname }}\n{% if inventory_hostname in (groups['kube-master'] + groups['new-master']) and inventory_hostname not in (groups['kube-worker'] + groups['new-worker']) %}\n taints:\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n{% else %}\n taints: []\n{% endif %}\nbootstrapTokens:\n- groups:\n - system:bootstrappers:kubeadm:default-node-token\n token: {{ kubeadm_token }}\n ttl: 0s\n usages:\n - signing\n - authentication\n---\napiVersion: kubeadm.k8s.io/v1beta2\nkind: ClusterConfiguration\nkubernetesVersion: v{{ kube_version }}\nclusterName: kubernetes\ncontrolPlaneEndpoint: \"{{ KUBE_APISERVER_IP | trim }}:{{ lb_kube_apiserver_port | trim }}\"\ncertificatesDir: /etc/kubernetes/pki\ndns:\n type: CoreDNS\netcd:\n external:\n endpoints:\n{% for host in ((groups['etcd'] | union(groups['new-etcd'])) | difference(groups['del-etcd'])) %}\n{% if hostvars[host]['ansible_host'] is defined %}\n - https://{{ hostvars[host]['ansible_host'] }}:2379\n{% else %}\n - https://{{ host }}:2379\n{% endif %}\n{% endfor %}\n caFile: /etc/kubernetes/pki/etcd/ca.crt\n certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt\n keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key\nimageRepository: \"{{ kube_image_repository }}\"\nnetworking:\n dnsDomain: {{ kube_dns_domain }}\n podSubnet: {{ kube_pod_subnet }}\n serviceSubnet: {{ kube_service_subnet }}\napiServer:\n extraArgs:\n allow-privileged: \"true\"\n apiserver-count: \"{{ groups['kube-master']|length + groups['new-master']|length }}\"\n{% if kubernetes_audit %}\n audit-log-path: /var/log/audit/kube-apiserver-audit.log\n audit-log-maxage: \"{{ audit_log_maxage }}\"\n audit-log-maxbackup: \"{{ audit_log_maxbackups }}\"\n audit-log-maxsize: \"{{ audit_log_maxsize }}\"\n audit-log-truncate-enabled: \"true\"\n audit-policy-file: {{ audit_policy_file }}\n{% endif %}\n{% if kube_apiserver_enable_admission_plugins|length > 0 %}\n enable-admission-plugins: {{ kube_apiserver_enable_admission_plugins | join(',') }}\n{% endif %}\n{% if kube_apiserver_disable_admission_plugins|length > 0 %}\n disable-admission-plugins: {{ kube_apiserver_disable_admission_plugins | join(',') }}\n{% endif %}\n encryption-provider-config: /etc/kubernetes/pki/secrets-encryption.yaml\n kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt\n kubelet-client-certificate: /etc/kubernetes/pki/apiserver-kubelet-client.crt\n kubelet-client-key: /etc/kubernetes/pki/apiserver-kubelet-client.key\n kubelet-https: \"true\"\n profiling: \"false\"\n service-node-port-range: {{ kube_service_node_port_range }}\n{% if kube_kubeadm_apiserver_extra_args|length > 0 %}\n{% for key in kube_kubeadm_apiserver_extra_args %}\n {{ key }}: \"{{ kube_kubeadm_apiserver_extra_args[key] }}\"\n{% endfor %}\n{% endif %}\n extraVolumes:\n - hostPath: /etc/localtime\n mountPath: /etc/localtime\n pathType: File\n readOnly: true\n name: localtime\n{% if kubernetes_audit %}\n - hostPath: \"{{ audit_policy_file | dirname }}\"\n mountPath: \"{{ audit_policy_file | dirname }}\"\n pathType: DirectoryOrCreate\n readOnly: true\n name: audit-policy\n - hostPath: \"{{ audit_log_hostpath }}\"\n mountPath: /var/log/audit/\n pathType: DirectoryOrCreate\n name: audit-logs\n{% endif %}\n{% for volume in apiserver_extra_volumes %}\n - name: {{ volume.name }}\n hostPath: {{ volume.hostPath }}\n mountPath: {{ volume.mountPath }}\n readOnly: {{ volume.readOnly | d(not (volume.writable | d(false))) }}\n{% endfor %}\n certSANs:\n - localhost\n - kubernetes\n - kubernetes.default\n - kubernetes.default.svc\n{% for sub_domain in kube_dns_domain.split('.') %}\n{% set outer_loop = loop %}\n - kubernetes.default.svc.{% for domain in kube_dns_domain.split('.') %}{% if loop.index <= outer_loop.index %}{{ domain }}{% if loop.index < outer_loop.index %}.{% endif %}{% endif %}{% endfor %}\n \n{% endfor %}\n{% if hostvars[inventory_hostname]['ansible_host'] is defined %}\n{% for host in (groups['kube-master'] + groups['new-master']| default([])) | unique %}\n - {{ host }}\n{% endfor %}\n{% endif %}\n{% for domain in kube_master_external_domain %}\n - {{ domain }}\n{% endfor %}\n - 127.0.0.1\n - 0:0:0:0:0:0:0:1\n - {{ KUBERNETES_SERVICE_IP }}\n{% if lb_kube_apiserver_ip is defined %}\n - {{ lb_kube_apiserver_ip | trim }}\n{% endif %}\n{% for host in (groups['kube-master'] + groups['new-master'] | default([])) | unique %}\n - {% if hostvars[host]['ansible_host'] is defined %}{{ hostvars[host]['ansible_host'] }}{% else %}{{ host }}{% endif %}\n \n{% endfor %}\n{% for ip in kube_master_external_ip %}\n - {{ ip }}\n{% endfor %}\ncontrollerManager:\n extraArgs:\n bind-address: 127.0.0.1\n experimental-cluster-signing-duration: \"{{kube_certs_expired|int * 24}}h0m0s\"\n feature-gates: \"RotateKubeletServerCertificate=true\"\n profiling: \"false\"\n node-monitor-grace-period: {{ kube_controller_node_monitor_grace_period }}\n node-monitor-period: {{ kube_controller_node_monitor_period }}\n pod-eviction-timeout: {{ kube_controller_pod_eviction_timeout }}\n terminated-pod-gc-threshold: \"{{ kube_controller_terminated_pod_gc_threshold }}\"\n node-cidr-mask-size: \"{{ kube_network_node_prefix }}\"\n{% if kube_kubeadm_controller_extra_args|length > 0 %}\n{% for key in kube_kubeadm_controller_extra_args %}\n {{ key }}: \"{{ kube_kubeadm_controller_extra_args[key] }}\"\n{% endfor %}\n{% endif %}\n extraVolumes:\n - hostPath: /etc/localtime\n mountPath: /etc/localtime\n pathType: File\n readOnly: true\n name: localtime\n{% for volume in controller_manager_extra_volumes %}\n - name: {{ volume.name }}\n hostPath: {{ volume.hostPath }}\n mountPath: {{ volume.mountPath }}\n readOnly: {{ volume.readOnly | d(not (volume.writable | d(false))) }}\n{% endfor %}\nscheduler:\n extraArgs:\n bind-address: 127.0.0.1\n profiling: \"false\"\n{% if kube_kubeadm_scheduler_extra_args|length > 0 %}\n{% for key in kube_kubeadm_scheduler_extra_args %}\n {{ key }}: \"{{ kube_kubeadm_scheduler_extra_args[key] }}\"\n{% endfor %}\n{% endif %}\n extraVolumes:\n - hostPath: /etc/localtime\n mountPath: /etc/localtime\n pathType: File\n readOnly: true\n name: localtime\n{% for volume in scheduler_extra_volumes %}\n - name: {{ volume.name }}\n hostPath: {{ volume.hostPath }}\n mountPath: {{ volume.mountPath }}\n readOnly: {{ volume.readOnly | d(not (volume.writable | d(false))) }}\n{% endfor %}\n---\napiVersion: kubelet.config.k8s.io/v1beta1\nkind: KubeletConfiguration\naddress: 0.0.0.0\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 2m0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 5m0s\n cacheUnauthorizedTTL: 30s\ncgroupDriver: {{ kubelet_cgroup_driver|default(kubelet_cgroup_driver_detected) }}\ncgroupsPerQOS: true\nclusterDNS:\n- {{ CLUSTER_DNS_SERVICE_IP }}\nclusterDomain: {{ kube_dns_domain }}\nconfigMapAndSecretChangeDetectionStrategy: Watch\ncontainerLogMaxFiles: 5\ncontainerLogMaxSize: 10Mi\ncontentType: application/vnd.kubernetes.protobuf\ncpuCFSQuota: true\ncpuCFSQuotaPeriod: 100ms\ncpuManagerPolicy: none\ncpuManagerReconcilePeriod: 10s\nenableControllerAttachDetach: true\nenableDebuggingHandlers: true\nenforceNodeAllocatable:\n- pods\neventBurst: 10\neventRecordQPS: 5\nevictionHard:\n imagefs.available: {{ eviction_hard_imagefs_available }}\n memory.available: {{ eviction_hard_memory_available }}\n nodefs.available: {{ eviction_hard_nodefs_available }}\n nodefs.inodesFree: {{ eviction_hard_nodefs_inodes_free }}\nkubeReserved:\n cpu: {{ kube_cpu_reserved }}\n memory: {{ kube_memory_reserved|regex_replace('Mi', 'M') }}\n{% if system_reserved_enabled is defined and system_reserved_enabled %}\nsystemReserved:\n cpu: {{ system_cpu_reserved|default('500m') }}\n memory: {{ system_memory_reserved|default('512M')|regex_replace('Mi', 'M') }}\n ephemeral-storage: {{ system_ephemeral_storage_reserved|default('10Gi')|regex_replace('Gi', 'G') }}\n{% endif %}\nevictionPressureTransitionPeriod: 5m0s\nfailSwapOn: true\nfeatureGates: \n RotateKubeletServerCertificate: true\nfileCheckFrequency: 20s\nhairpinMode: promiscuous-bridge\nhealthzBindAddress: 127.0.0.1\nhealthzPort: 10248\nhttpCheckFrequency: 20s\nimageGCHighThresholdPercent: 85\nimageGCLowThresholdPercent: 80\nimageMinimumGCAge: 2m0s\niptablesDropBit: 15\niptablesMasqueradeBit: 14\nkubeAPIBurst: 10\nkubeAPIQPS: 5\nmakeIPTablesUtilChains: true\nmaxOpenFiles: 1000000\nmaxPods: {{ kube_max_pods }}\nnodeLeaseDurationSeconds: 40\nnodeStatusReportFrequency: 1m0s\nnodeStatusUpdateFrequency: 10s\noomScoreAdj: -999\npodPidsLimit: -1\nport: 10250\nprotectKernelDefaults: true\nreadOnlyPort: 0\nregistryBurst: 10\nregistryPullQPS: 5\nresolvConf: /etc/resolv.conf\nrotateCertificates: true\nruntimeRequestTimeout: 2m0s\nserializeImagePulls: true\nstaticPodPath: /etc/kubernetes/manifests\nstreamingConnectionIdleTimeout: 4h0m0s\nsyncFrequency: 1m0s\ntlsCertFile: /var/lib/kubelet/pki/kubelet.crt\ntlsPrivateKeyFile: /var/lib/kubelet/pki/kubelet.key\nvolumeStatsAggPeriod: 1m0s\n---\napiVersion: kubeproxy.config.k8s.io/v1alpha1\nkind: KubeProxyConfiguration\nbindAddress: 0.0.0.0\nclientConnection:\n acceptContentTypes: \"\"\n burst: 10\n contentType: application/vnd.kubernetes.protobuf\n kubeconfig: /var/lib/kube-proxy/kubeconfig.conf\n qps: 5\nclusterCIDR: {{ kube_pod_subnet }}\nconfigSyncPeriod: 15m0s\nconntrack:\n maxPerCore: 32768\n min: 131072\n tcpCloseWaitTimeout: 1h0m0s\n tcpEstablishedTimeout: 24h0m0s\nenableProfiling: false\nhealthzBindAddress: 0.0.0.0:10256\nhostnameOverride: \"\"\niptables:\n masqueradeAll: false\n masqueradeBit: 14\n minSyncPeriod: 0s\n syncPeriod: 30s\nipvs:\n excludeCIDRs: null\n minSyncPeriod: 0s\n scheduler: \"\"\n strictARP: false\n syncPeriod: 30s\nmetricsBindAddress: 0.0.0.0:10249\nmode: {{ kube_proxy_mode }}\nnodePortAddresses: null\noomScoreAdj: -999\nportRange: \"\"\nudpIdleTimeout: 250ms\nwinkernel:\n enableDSR: false\n networkName: \"\"\n sourceVip: \"\").\nMake sure your variable name does not contain invalid characters like '-': argument of type 'AnsibleUndefined' is not iterable"}
NO MORE HOSTS LEFT *************************************************************
PLAY RECAP *********************************************************************
172.22.0.3 : ok=79 changed=20 unreachable=0 failed=0 skipped=41 rescued=0 ignored=0
172.22.0.4 : ok=167 changed=57 unreachable=0 failed=1 skipped=60 rescued=0 ignored=0
AnsibleUndefinedVariable: Unable to look up a name or access an attribute in template string (apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: {{ CURRENT_HOST_IP }}
bindPort: 6443
nodeRegistration:
kubeletExtraArgs:
network-plugin: cni
root-dir: {{ kubelet_root_dir }}
hostname-override: {{ inventory_hostname }}
pod-infra-container-image: {{ pod_infra_container_image }}
criSocket: {{ CRI_SOCKET }}
name: {{ inventory_hostname }}
{% if inventory_hostname in (groups['kube-master'] + groups['new-master']) and inventory_hostname not in (groups['kube-worker'] + groups['new-worker']) %}
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
{% else %}
taints: []
{% endif %}
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: {{ kubeadm_token }}
ttl: 0s
usages:
- signing
- authentication
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v{{ kube_version }}
clusterName: kubernetes
controlPlaneEndpoint: "{{ KUBE_APISERVER_IP | trim }}:{{ lb_kube_apiserver_port | trim }}"
certificatesDir: /etc/kubernetes/pki
dns:
type: CoreDNS
etcd:
external:
endpoints:
{% for host in ((groups['etcd'] | union(groups['new-etcd'])) | difference(groups['del-etcd'])) %}
{% if hostvars[host]['ansible_host'] is defined %}
- https://{{ hostvars[host]['ansible_host'] }}:2379
{% else %}
- https://{{ host }}:2379
{% endif %}
{% endfor %}
caFile: /etc/kubernetes/pki/etcd/ca.crt
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
imageRepository: "{{ kube_image_repository }}"
networking:
dnsDomain: {{ kube_dns_domain }}
podSubnet: {{ kube_pod_subnet }}
serviceSubnet: {{ kube_service_subnet }}
apiServer:
extraArgs:
allow-privileged: "true"
apiserver-count: "{{ groups['kube-master']|length + groups['new-master']|length }}"
{% if kubernetes_audit %}
audit-log-path: /var/log/audit/kube-apiserver-audit.log
audit-log-maxage: "{{ audit_log_maxage }}"
audit-log-maxbackup: "{{ audit_log_maxbackups }}"
audit-log-maxsize: "{{ audit_log_maxsize }}"
audit-log-truncate-enabled: "true"
audit-policy-file: {{ audit_policy_file }}
{% endif %}
{% if kube_apiserver_enable_admission_plugins|length > 0 %}
enable-admission-plugins: {{ kube_apiserver_enable_admission_plugins | join(',') }}
{% endif %}
{% if kube_apiserver_disable_admission_plugins|length > 0 %}
disable-admission-plugins: {{ kube_apiserver_disable_admission_plugins | join(',') }}
{% endif %}
encryption-provider-config: /etc/kubernetes/pki/secrets-encryption.yaml
kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
kubelet-client-certificate: /etc/kubernetes/pki/apiserver-kubelet-client.crt
kubelet-client-key: /etc/kubernetes/pki/apiserver-kubelet-client.key
kubelet-https: "true"
profiling: "false"
service-node-port-range: {{ kube_service_node_port_range }}
{% if kube_kubeadm_apiserver_extra_args|length > 0 %}
{% for key in kube_kubeadm_apiserver_extra_args %}
{{ key }}: "{{ kube_kubeadm_apiserver_extra_args[key] }}"
{% endfor %}
{% endif %}
extraVolumes:
- hostPath: /etc/localtime
mountPath: /etc/localtime
pathType: File
readOnly: true
name: localtime
{% if kubernetes_audit %}
- hostPath: "{{ audit_policy_file | dirname }}"
mountPath: "{{ audit_policy_file | dirname }}"
pathType: DirectoryOrCreate
readOnly: true
name: audit-policy
- hostPath: "{{ audit_log_hostpath }}"
mountPath: /var/log/audit/
pathType: DirectoryOrCreate
name: audit-logs
{% endif %}
{% for volume in apiserver_extra_volumes %}
- name: {{ volume.name }}
hostPath: {{ volume.hostPath }}
mountPath: {{ volume.mountPath }}
readOnly: {{ volume.readOnly | d(not (volume.writable | d(false))) }}
{% endfor %}
certSANs:
- localhost
- kubernetes
- kubernetes.default
- kubernetes.default.svc
{% for sub_domain in kube_dns_domain.split('.') %}
{% set outer_loop = loop %}
- kubernetes.default.svc.{% for domain in kube_dns_domain.split('.') %}{% if loop.index <= outer_loop.index %}{{ domain }}{% if loop.index < outer_loop.index %}.{% endif %}{% endif %}{% endfor %}
{% endfor %}
{% if hostvars[inventory_hostname]['ansible_host'] is defined %}
{% for host in (groups['kube-master'] + groups['new-master']| default([])) | unique %}
- {{ host }}
{% endfor %}
{% endif %}
{% for domain in kube_master_external_domain %}
- {{ domain }}
{% endfor %}
- 127.0.0.1
- 0:0:0:0:0:0:0:1
- {{ KUBERNETES_SERVICE_IP }}
{% if lb_kube_apiserver_ip is defined %}
- {{ lb_kube_apiserver_ip | trim }}
{% endif %}
{% for host in (groups['kube-master'] + groups['new-master'] | default([])) | unique %}
- {% if hostvars[host]['ansible_host'] is defined %}{{ hostvars[host]['ansible_host'] }}{% else %}{{ host }}{% endif %}
{% endfor %}
{% for ip in kube_master_external_ip %}
- {{ ip }}
{% endfor %}
controllerManager:
extraArgs:
bind-address: 127.0.0.1
experimental-cluster-signing-duration: "{{kube_certs_expired|int * 24}}h0m0s"
feature-gates: "RotateKubeletServerCertificate=true"
profiling: "false"
node-monitor-grace-period: {{ kube_controller_node_monitor_grace_period }}
node-monitor-period: {{ kube_controller_node_monitor_period }}
pod-eviction-timeout: {{ kube_controller_pod_eviction_timeout }}
terminated-pod-gc-threshold: "{{ kube_controller_terminated_pod_gc_threshold }}"
node-cidr-mask-size: "{{ kube_network_node_prefix }}"
{% if kube_kubeadm_controller_extra_args|length > 0 %}
{% for key in kube_kubeadm_controller_extra_args %}
{{ key }}: "{{ kube_kubeadm_controller_extra_args[key] }}"
{% endfor %}
{% endif %}
extraVolumes:
- hostPath: /etc/localtime
mountPath: /etc/localtime
pathType: File
readOnly: true
name: localtime
{% for volume in controller_manager_extra_volumes %}
- name: {{ volume.name }}
hostPath: {{ volume.hostPath }}
mountPath: {{ volume.mountPath }}
readOnly: {{ volume.readOnly | d(not (volume.writable | d(false))) }}
{% endfor %}
scheduler:
extraArgs:
bind-address: 127.0.0.1
profiling: "false"
{% if kube_kubeadm_scheduler_extra_args|length > 0 %}
{% for key in kube_kubeadm_scheduler_extra_args %}
{{ key }}: "{{ kube_kubeadm_scheduler_extra_args[key] }}"
{% endfor %}
{% endif %}
extraVolumes:
- hostPath: /etc/localtime
mountPath: /etc/localtime
pathType: File
readOnly: true
name: localtime
{% for volume in scheduler_extra_volumes %}
- name: {{ volume.name }}
hostPath: {{ volume.hostPath }}
mountPath: {{ volume.mountPath }}
readOnly: {{ volume.readOnly | d(not (volume.writable | d(false))) }}
{% endfor %}
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
address: 0.0.0.0
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 2m0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 5m0s
cacheUnauthorizedTTL: 30s
cgroupDriver: {{ kubelet_cgroup_driver|default(kubelet_cgroup_driver_detected) }}
cgroupsPerQOS: true
clusterDNS:
- {{ CLUSTER_DNS_SERVICE_IP }}
clusterDomain: {{ kube_dns_domain }}
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 100ms
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
imagefs.available: {{ eviction_hard_imagefs_available }}
memory.available: {{ eviction_hard_memory_available }}
nodefs.available: {{ eviction_hard_nodefs_available }}
nodefs.inodesFree: {{ eviction_hard_nodefs_inodes_free }}
kubeReserved:
cpu: {{ kube_cpu_reserved }}
memory: {{ kube_memory_reserved|regex_replace('Mi', 'M') }}
{% if system_reserved_enabled is defined and system_reserved_enabled %}
systemReserved:
cpu: {{ system_cpu_reserved|default('500m') }}
memory: {{ system_memory_reserved|default('512M')|regex_replace('Mi', 'M') }}
ephemeral-storage: {{ system_ephemeral_storage_reserved|default('10Gi')|regex_replace('Gi', 'G') }}
{% endif %}
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
featureGates:
RotateKubeletServerCertificate: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: {{ kube_max_pods }}
nodeLeaseDurationSeconds: 40
nodeStatusReportFrequency: 1m0s
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
protectKernelDefaults: true
readOnlyPort: 0
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
tlsCertFile: /var/lib/kubelet/pki/kubelet.crt
tlsPrivateKeyFile: /var/lib/kubelet/pki/kubelet.key
volumeStatsAggPeriod: 1m0s
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
bindAddress: 0.0.0.0
clientConnection:
acceptContentTypes: ""
burst: 10
contentType: application/vnd.kubernetes.protobuf
kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
qps: 5
clusterCIDR: {{ kube_pod_subnet }}
configSyncPeriod: 15m0s
conntrack:
maxPerCore: 32768
min: 131072
tcpCloseWaitTimeout: 1h0m0s
tcpEstablishedTimeout: 24h0m0s
enableProfiling: false
healthzBindAddress: 0.0.0.0:10256
hostnameOverride: ""
iptables:
masqueradeAll: false
masqueradeBit: 14
minSyncPeriod: 0s
syncPeriod: 30s
ipvs:
excludeCIDRs: null
minSyncPeriod: 0s
scheduler: ""
strictARP: false
syncPeriod: 30s
metricsBindAddress: 0.0.0.0:10249
mode: {{ kube_proxy_mode }}
nodePortAddresses: null
oomScoreAdj: -999
portRange: ""
udpIdleTimeout: 250ms
winkernel:
enableDSR: false
networkName: ""
sourceVip: "").
Make sure your variable name does not contain invalid characters like '-': argument of type 'AnsibleUndefined' is not iterable
- 原因分析:
安装过一次 k8s 环境,可能是之前乱配置了一下东西,
想卸载之后重新安装, 我运行了ansible-playbook -i inventory.ini 99-reset-cluster.yml