-
K8S 版本:
v1.8.5 -
kubeadm-ansible 版本： master分支最新脚本
-
遇到问题的执行步骤:
使用 kubeadm-ansible 部署一个 3节点的 K8S 集群，所有的工作节点的状态全部为 NotReady，节点的信息及日志请参见 Github 上的对应的 Issue。 -
Github Issue 引用: https://github.com/choerodon/kubeadm-ansible/issues/14
算是吧，是广州开发中心的三台服务器，彼此可以互相访问。
在master节点
curl https://k8s-node01:6443 -k
看下有返回结果吗
我发现 /opt/cni/bin/portmap 仅在主节点有，其他工作节点还是没有这个文件
[root@k8s-master bin] ○ curl https://k8s-node01:6443 -k
curl: (7) Failed connect to k8s-node01:6443; 拒绝连接
[root@k8s-master bin] ○
我们再检查下 这个应该每个节点都有的
您的master节点同时也是node吗
是的。
[root@k8s-master bin] ○ cat /u01/choerodon/tools/kubeadm-ansible/inventory/hosts
[all]
k8s-master
k8s-node01
k8s-node02
[kube-master]
k8s-master
[etcd]
k8s-master
[kube-node]
k8s-master
k8s-node01
k8s-node02
好的 我们先测试一下
请在每个节点执行下面命令 看看是否正常
执行过了，还是不正常的。
@vinkdong 我发现 cluster.yml 里面仅在 kube-master 下才会执行 addons/flannel
而 portmap 的安装是在 addons/flannel/tasks/main.yml 下的。。
这个问题已修正。
请在所有节点都执行一下netstat -luntp，查看一下端口号开启情况
k8s-master:
[root@k8s-master kubeadm-ansible] ○ netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 8816/rpcbind
tcp 0 0 0.0.0.0:20048 0.0.0.0:* LISTEN 8889/rpc.mountd
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 2914/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 7679/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 7621/cupsd
tcp 0 0 0.0.0.0:35896 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1927/master
tcp 0 0 0.0.0.0:52637 0.0.0.0:* LISTEN 8840/rpc.statd
tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN -
tcp6 0 0 :::111 :::* LISTEN 8816/rpcbind
tcp6 0 0 :::20048 :::* LISTEN 8889/rpc.mountd
tcp6 0 0 :::51922 :::* LISTEN 8840/rpc.statd
tcp6 0 0 :::22 :::* LISTEN 7679/sshd
tcp6 0 0 ::1:631 :::* LISTEN 7621/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 1927/master
tcp6 0 0 :::2049 :::* LISTEN -
tcp6 0 0 :::32932 :::* LISTEN -
udp 0 0 0.0.0.0:36387 0.0.0.0:* -
udp 0 0 0.0.0.0:20048 0.0.0.0:* 8889/rpc.mountd
udp 0 0 0.0.0.0:45702 0.0.0.0:* 7780/avahi-daemon:
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:5353 0.0.0.0:* 7780/avahi-daemon:
udp 0 0 0.0.0.0:56501 0.0.0.0:* 8840/rpc.statd
udp 0 0 192.168.122.1:53 0.0.0.0:* 9028/dnsmasq
udp 0 0 192.168.122.1:53 0.0.0.0:* 2914/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 9028/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 2914/dnsmasq
udp 0 0 0.0.0.0:111 0.0.0.0:* 8816/rpcbind
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 127.0.0.1:323 0.0.0.0:* 18712/chronyd
udp 0 0 0.0.0.0:935 0.0.0.0:* 8816/rpcbind
udp 0 0 127.0.0.1:960 0.0.0.0:* 8840/rpc.statd
udp 0 0 0.0.0.0:2049 0.0.0.0:* -
udp6 0 0 :::20048 :::* 8889/rpc.mountd
udp6 0 0 :::37281 :::* 8840/rpc.statd
udp6 0 0 :::4789 :::* -
udp6 0 0 :::53998 :::* -
udp6 0 0 :::111 :::* 8816/rpcbind
udp6 0 0 ::1:323 :::* 18712/chronyd
udp6 0 0 :::935 :::* 8816/rpcbind
udp6 0 0 :::2049 :::* -
k8s-node01:
[root@k8s-node01 ~] ○ netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 2445/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 7604/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 7546/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1684/master
tcp6 0 0 :::22 :::* LISTEN 7604/sshd
tcp6 0 0 ::1:631 :::* LISTEN 7546/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 1684/master
udp 0 0 192.168.122.1:53 0.0.0.0:* 8786/dnsmasq
udp 0 0 192.168.122.1:53 0.0.0.0:* 2445/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 8786/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 2445/dnsmasq
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:33533 0.0.0.0:* 7705/avahi-daemon:
udp 0 0 0.0.0.0:5353 0.0.0.0:* 7705/avahi-daemon:
udp6 0 0 :::4789 :::* -
k8s-node02
[root@k8s-node02 ~] ○ netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 2756/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 7616/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 7558/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2073/master
tcp6 0 0 :::22 :::* LISTEN 7616/sshd
tcp6 0 0 ::1:631 :::* LISTEN 7558/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 2073/master
udp 0 0 0.0.0.0:40704 0.0.0.0:* 7717/avahi-daemon:
udp 0 0 192.168.122.1:53 0.0.0.0:* 8829/dnsmasq
udp 0 0 192.168.122.1:53 0.0.0.0:* 2756/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 8829/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 2756/dnsmasq
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 127.0.0.1:323 0.0.0.0:* 18505/chronyd
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:5353 0.0.0.0:* 7717/avahi-daemon:
udp6 0 0 ::1:323 :::* 18505/chronyd
udp6 0 0 :::4789 :::* -
我重新又部署了一次，我在ansible的执行日志中始终只能看见 portmap 在 kube-master 这个hosts 上执行过。 但我不确定会不会和我的 NotReady 有关。
另外，请问您说的 问题已修复指的是哪个问题呢？ @TimeBye
修正的代码刚刚才合并进去哈，建议你再reset一下集群，然后拉取一下master分支最新代码进行部署试一试。
多谢，刚刚重新部署，portmap 在所有节点都已经有了。
不过问题仍然未解决~
[root@k8s-master kubeadm-ansible] ○ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 3m v1.8.5
k8s-node01 NotReady <none> 3m v1.8.5
k8s-node02 NotReady <none> 3m v1.8.5
[root@k8s-master kubeadm-ansible] ○ kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
default-http-backend-6dd4d5b7c9-sb9rr 1/1 Running 0 3m
heapster-746d67c7b9-sflll 1/1 Running 0 3m
kube-apiserver-k8s-master 1/1 Running 0 3m
kube-controller-manager-k8s-master 1/1 Running 0 3m
kube-dns-79d99555df-m9kbt 3/3 Running 0 4m
kube-flannel-45w2k 1/1 Running 0 3m
kube-flannel-9vzwr 0/1 Pending 0 2m
kube-flannel-bd7nw 0/1 Pending 0 2m
kube-lego-6f45757db7-mtshx 1/1 Running 0 3m
kube-proxy-b6247 0/1 Pending 0 2m
kube-proxy-gb6xp 0/1 Pending 0 2m
kube-proxy-jsml7 1/1 Running 0 4m
kube-scheduler-k8s-master 1/1 Running 0 3m
kubernetes-dashboard-dc8fcdbc5-htw2k 1/1 Running 0 3m
nginx-ingress-controller-5d77d4945d-bzqkb 1/1 Running 0 3m
nginx-proxy-k8s-node01 1/1 Running 0 3m
nginx-proxy-k8s-node02 1/1 Running 0 3m
[root@k8s-master kubeadm-ansible] ○
k8s-master
[root@k8s-master kubeadm-ansible] ○ netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:10251 0.0.0.0:* LISTEN 28338/kube-schedule
tcp 0 0 192.168.123.155:2379 0.0.0.0:* LISTEN 27594/etcd
tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN 27594/etcd
tcp 0 0 127.0.0.1:10252 0.0.0.0:* LISTEN 28358/kube-controll
tcp 0 0 192.168.123.155:2380 0.0.0.0:* LISTEN 27594/etcd
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 8816/rpcbind
tcp 0 0 192.168.123.155:80 0.0.0.0:* LISTEN 28030/kube-proxy
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 28624/kube-apiserve
tcp 0 0 0.0.0.0:20048 0.0.0.0:* LISTEN 8889/rpc.mountd
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 2914/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 7679/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 7621/cupsd
tcp 0 0 0.0.0.0:35896 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1927/master
tcp 0 0 192.168.123.155:443 0.0.0.0:* LISTEN 28030/kube-proxy
tcp 0 0 0.0.0.0:52637 0.0.0.0:* LISTEN 8840/rpc.statd
tcp 0 0 192.168.123.155:4001 0.0.0.0:* LISTEN 27594/etcd
tcp 0 0 127.0.0.1:4001 0.0.0.0:* LISTEN 27594/etcd
tcp 0 0 0.0.0.0:2049 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 28869/kubelet
tcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN 28030/kube-proxy
tcp6 0 0 :::6443 :::* LISTEN 28624/kube-apiserve
tcp6 0 0 :::10255 :::* LISTEN 28869/kubelet
tcp6 0 0 :::111 :::* LISTEN 8816/rpcbind
tcp6 0 0 :::10256 :::* LISTEN 28030/kube-proxy
tcp6 0 0 :::20048 :::* LISTEN 8889/rpc.mountd
tcp6 0 0 :::51922 :::* LISTEN 8840/rpc.statd
tcp6 0 0 :::22 :::* LISTEN 7679/sshd
tcp6 0 0 ::1:631 :::* LISTEN 7621/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 1927/master
tcp6 0 0 :::2049 :::* LISTEN -
tcp6 0 0 :::4194 :::* LISTEN 28869/kubelet
tcp6 0 0 :::32932 :::* LISTEN -
tcp6 0 0 :::10250 :::* LISTEN 28869/kubelet
udp 0 0 0.0.0.0:36387 0.0.0.0:* -
udp 0 0 0.0.0.0:20048 0.0.0.0:* 8889/rpc.mountd
udp 0 0 0.0.0.0:45702 0.0.0.0:* 7780/avahi-daemon:
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:5353 0.0.0.0:* 7780/avahi-daemon:
udp 0 0 0.0.0.0:56501 0.0.0.0:* 8840/rpc.statd
udp 0 0 192.168.122.1:53 0.0.0.0:* 9028/dnsmasq
udp 0 0 192.168.122.1:53 0.0.0.0:* 2914/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 9028/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 2914/dnsmasq
udp 0 0 0.0.0.0:111 0.0.0.0:* 8816/rpcbind
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 127.0.0.1:323 0.0.0.0:* 18712/chronyd
udp 0 0 0.0.0.0:935 0.0.0.0:* 8816/rpcbind
udp 0 0 127.0.0.1:960 0.0.0.0:* 8840/rpc.statd
udp 0 0 0.0.0.0:2049 0.0.0.0:* -
udp6 0 0 :::20048 :::* 8889/rpc.mountd
udp6 0 0 :::37281 :::* 8840/rpc.statd
udp6 0 0 :::4789 :::* -
udp6 0 0 :::53998 :::* -
udp6 0 0 :::111 :::* 8816/rpcbind
udp6 0 0 ::1:323 :::* 18712/chronyd
udp6 0 0 :::935 :::* 8816/rpcbind
udp6 0 0 :::2049 :::* -
k8s-node01
[root@k8s-node01 ~] ○ netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 2445/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 7604/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 7546/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1684/master
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 126424/kubelet
tcp6 0 0 :::10255 :::* LISTEN 126424/kubelet
tcp6 0 0 :::22 :::* LISTEN 7604/sshd
tcp6 0 0 ::1:631 :::* LISTEN 7546/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 1684/master
tcp6 0 0 :::4194 :::* LISTEN 126424/kubelet
tcp6 0 0 :::10250 :::* LISTEN 126424/kubelet
udp 0 0 192.168.122.1:53 0.0.0.0:* 8786/dnsmasq
udp 0 0 192.168.122.1:53 0.0.0.0:* 2445/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 8786/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 2445/dnsmasq
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:33533 0.0.0.0:* 7705/avahi-daemon:
udp 0 0 0.0.0.0:5353 0.0.0.0:* 7705/avahi-daemon:
udp6 0 0 :::4789 :::* -
k8s-node02
[root@k8s-node02 ~] ○ netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 2756/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 7616/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 7558/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2073/master
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 126212/kubelet
tcp6 0 0 :::10255 :::* LISTEN 126212/kubelet
tcp6 0 0 :::22 :::* LISTEN 7616/sshd
tcp6 0 0 ::1:631 :::* LISTEN 7558/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 2073/master
tcp6 0 0 :::4194 :::* LISTEN 126212/kubelet
tcp6 0 0 :::10250 :::* LISTEN 126212/kubelet
udp 0 0 0.0.0.0:40704 0.0.0.0:* 7717/avahi-daemon:
udp 0 0 192.168.122.1:53 0.0.0.0:* 8829/dnsmasq
udp 0 0 192.168.122.1:53 0.0.0.0:* 2756/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 8829/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 2756/dnsmasq
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 127.0.0.1:323 0.0.0.0:* 18505/chronyd
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:5353 0.0.0.0:* 7717/avahi-daemon:
udp6 0 0 ::1:323 :::* 18505/chronyd
udp6 0 0 :::4789 :::* -