Kubeadm: Kubeadm init 阻止“如果必须拉取控制平面图像，这可能需要一分钟或更长时间

创建于 2018-01-31 · 67评论 · 资料来源: kubernetes/kubeadm

版本

kubeadm 版本（使用kubeadm version ）：

环境：

Kubernetes 版本（使用kubectl version ）：v1.9.2
云提供商或硬件配置：Virtual Box
操作系统（例如来自 /etc/os-release）：Ubuntu 16.04.0 LTS (Xeniak Xerus) amd64
内核（例如uname -a ）：linux 4.4.0-62-generic
其他：kubeadm 版本：v1.9.2：amd64，kubelet 版本：v1.9.2 amd64，kubernetes-cni 版本：0.6.0-00 amd64， docker 版本：17.03.2-ce

发生了什么？

当我尝试运行 kubeadm init 时，它挂起
xx@xx :~$ sudo kubeadm init --kubernetes-version=v1.9.2

[init] 使用 Kubernetes 版本：v1.9.2
[init] 使用授权模式：[Node RBAC]
[飞行前] 运行飞行前检查。
[警告 FileExisting-crictl]：在系统路径中找不到 crictl
[预检] 启动 kubelet 服务
[证书] 生成的 ca 证书和密钥。
[证书] 生成的 apiserver 证书和密钥。
[证书] 为 DNS 名称 [kickseed kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] 和 IP [10.96.0.1 172.17.41.15] 签署 apiserver 服务证书
[证书] 生成 apiserver-kubelet-client 证书和密钥。
[证书] 生成 sa 密钥和公钥。
[证书] 生成的 front-proxy-ca 证书和密钥。
[证书] 生成的前端代理客户端证书和密钥。
[证书] 有效的证书和密钥现在存在于“/etc/kubernetes/pki”中
[kubeconfig] 将 KubeConfig 文件写入磁盘：“admin.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“kubelet.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“controller-manager.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“scheduler.conf”
[controlplane] 将组件 kube-apiserver 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-apiserver.yaml”
[controlplane] 将组件 kube-controller-manager 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-controller-manager.yaml”
[controlplane] 将组件 kube-scheduler 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-scheduler.yaml”
[etcd] 将本地 etcd 实例的静态 Pod 清单写入“/etc/kubernetes/manifests/etcd.yaml”
[init] 等待 kubelet 作为静态 Pod 从目录“/etc/kubernetes/manifests”启动控制平面。
[init] 如果必须拉取控制平面图像，这可能需要一分钟或更长时间。

然后我检查 kubelet 日志：
xx@xx :~$ sudo journalctl -xeu kubelet:
Jan 31 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.280984 28516 remote_runtime.go:92] RunPodSandbox 从运行时服务失败：rpc 错误：代码 = 未知 desc = 提取图像失败“gcr.io/google_containers/ pause-amd64:3.0"：来自守护程序的错误响应：获取https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时
1 月31 日 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.281317 28516 kuberuntime_sandbox.go:54] pod“kube-scheduler-kickseed_kube-system(69c12074e336b0dbbd0a1666ce05226a1666ce05226a 错误代码 = 未知 desc)”的 CreatePodSandbox = 拉取图像“gcr.io/google_containers/pause-amd64:3.0”失败：来自守护进程的错误响应：获取https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时
1 月31 日 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.281580 28516 kuberuntime_manager.go:647] 为 pod“kube-scheduler-kickseed_kube-system(69c12074e336b0dbbd0a1666ce05226 错误代码 = 未知 desc)”创建PodSandbox = 拉取图像“gcr.io/google_containers/pause-amd64:3.0”失败：来自守护进程的错误响应：获取https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时
1月31日https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时”
1 月31 日 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.380290 28516 event.go:209] 无法写入事件：'补丁https://172.17.41.15:6443/api/v1/namespaces/default /events/kickseed.150ecf46afb098b7: dial tcp 172.17.41.15:6443: getsockopt: connection denied' （可以在休眠后重试）
1 月31 日 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.933783 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: 无法列出 *v1。 Pod：获取https://172.17.41.15 :6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.934707 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: 无法列出 *v1.Node:获取https://172.17.41.15 :6443/api/v1/nodes?fieldSelector=metadata.name%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:45:03 kickseed kubelet[28516]: E0131 14:45:03.935921 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: 无法列出 *v1.Service:获取https://172.17.41.15 :6443/api/v1/services?limit=500&resourceVersion=0: 拨打 tcp 172.17.41.15:6443: getsockopt: 连接被拒绝
1 月31 日 14:45:04 kickseed kubelet[28516]: E0131 14:45:04.281024 28516 remote_runtime.go:92] RunPodSandbox 从运行时服务失败：rpc 错误：代码 = 未知 desc = 提取图像失败“gcr.io/google_containers/ pause-amd64:3.0"：来自守护程序的错误响应：获取https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时
1 月31 日 14:45:04 kickseed kubelet[28516]: E0131 14:45:04.281352 28516 kuberuntime_sandbox.go:54] 为 pod “kube-controller-manager-kickseed_kube-system(6546d6faf0b50c9fc6712ce25ee9b = rb) 错误代码创建PodSandbox”未知 desc = 拉取图像失败“gcr.io/google_containers/pause-amd64:3.0”：来自守护进程的错误响应：获取https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时
1 月31 日 14:45:04 kickseed kubelet[28516]: E0131 14:45:04.281634 28516 kuberuntime_manager.go:647] 为 pod “kube-controller-manager-kickseed_kube-system(6546d6faf0b50c9fc6712ce25ee9b = rb) 错误代码创建PodSandbox”未知 desc = 拉取图像失败“gcr.io/google_containers/pause-amd64:3.0”：来自守护进程的错误响应：获取https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443：i/o 超时
1月31日https://gcr.io/v1/_ping ：拨打 tcp 172.217.6.127:443: i/o暂停”
1 月31 日 14:45:04 kickseed kubelet[28516]: E0131 14:45:04.934694 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: 无法列出 *v1。 Pod：获取https://172.17.41.15 :6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:45:04 kickseed kubelet[28516]: E0131 14:45:04.935613 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: 无法列出 *v1.Node:获取https://172.17.41.15 :6443/api/v1/nodes?fieldSelector=metadata.name%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:45:04 kickseed kubelet[28516]: E0131 14:45:04.936669 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: 无法列出 *v1.Service:获取https://172.17.41.15 :6443/api/v1/services?limit=500&resourceVersion=0: 拨打 tcp 172.17.41.15:6443: getsockopt: 连接被拒绝
Jan 31 14:45:05 kickseed kubelet[28516]: W0131 14:45:05.073692 28516 cni.go:171] 无法更新 cni 配置：在 /etc/cni/net.d 中找不到网络
Jan 31 14:45:05 kickseed kubelet[28516]: E0131 14:45:05.074106 28516 kubelet.go:2105] 容器运行时网络未准备好：NetworkReady=false原因：NetworkPluginNotReady 消息：docker ：网络插件未准备好：cni config未初始化
1 月31 日 14:45:05 kickseed kubelet[28516]: E0131 14:45:05.935680 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: 无法列出 *v1。 Pod：获取https://172.17.41.15 :6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:45:05 kickseed kubelet[28516]: E0131 14:45:05.937423 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: 无法列出 *v1.Node:获取https://172.17.41.15 :6443/api/v1/nodes?fieldSelector=metadata.name%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:45:05 kickseed kubelet[28516]: E0131 14:45:05.937963 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: 无法列出 *v1.Service:获取https://172.17.41.15 :6443/api/v1/services?limit=500&resourceVersion=0: 拨打 tcp 172.17.41.15:6443: getsockopt: 连接被拒绝
Jan 31 14:45:05 kickseed kubelet[28516]: I0131 14:45:05.974034 28516 kubelet_node_status.go:273] 设置节点注释以启用卷控制器附加/分离
Jan 31 14:45:06 kickseed kubelet[28516]: I0131 14:45:06.802447 28516 kubelet_node_status.go:273] 设置节点注释以启用卷控制器附加/分离
Jan 31 14:45:06 kickseed kubelet[28516]: I0131 14:45:06.804242 28516 kubelet_node_status.go:82] 尝试注册节点 kickseed
Jan 31 14:45:06 kickseed kubelet[28516]: E0131 14:45:06.804778 28516 kubelet_node_status.go:106] 无法向 API 服务器注册节点“kickseed”：发布https://172.17.41.15:6443/api/ v1/nodes：拨打 tcp 172.17.41.15:6443：getsockopt：con

xx@xx :~$ sudo systemctl status kubelet:

kubelet.service - kubelet：Kubernetes 节点代理
已加载：已加载（/lib/systemd/system/kubelet.service；已启用；供应商预设：已启用）
插入：/etc/systemd/system/kubelet.service.d
└─11-kubeadm.conf、10-kubeadm1.conf、90-local-extras.conf
活跃：自 2018 年 1 月 31 日星期三 13:53:46 CST 以来活跃（运行）； 49 分钟前
文档：http: //kubernetes.io/docs/
主PID：28516（kubelet）
任务：13
内存：37.8M
CPU：22.767s
CGroup：/system.slice/kubelet.service
└─28516 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --pod-manifest-path=/etc/kubernetes/ manifests --allow-privileged=true --cgroup-driver=cgroupfs --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni /bin --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt --cadvisor-port =0 --rotate-certificates=true --cert-dir=/var/lib/kubelet/pki --fail-swap-on=false

1 月31 日 14:43:17 kickseed kubelet[28516]: E0131 14:43:17.862590 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: 无法列出 *v1.Node:获取https://172.17.41.15 :6443/api/v1/nodes?fieldSelector=metadata.name%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:43:17 kickseed kubelet[28516]: E0131 14:43:17.863474 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: 无法列出 *v1.Service:获取https://172.17.41.15 :6443/api/v1/services?limit=500&resourceVersion=0: 拨打 tcp 172.17.41.15:6443: getsockopt: 连接被拒绝
1 月31 日 14:43:18 kickseed kubelet[28516]: E0131 14:43:18.621818 28516 event.go:209] 无法写入事件：'补丁https://172.17.41.15 :6443/api/v1/namespaces/default /events/kickseed.150ecf46afb098b7: dial tcp 172.17.41.15:6443: getsockopt: connection denied' （可以在休眠后重试）
1 月31 日 14:43:18 kickseed kubelet[28516]: E0131 14:43:18.862440 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: 无法列出 *v1。 Pod：获取https://172.17.41.15 :6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:43:18 kickseed kubelet[28516]: E0131 14:43:18.863379 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: 无法列出 *v1.Node:获取https://172.17.41.15 :6443/api/v1/nodes?fieldSelector=metadata.name%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:43:18 kickseed kubelet[28516]: E0131 14:43:18.864424 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: 无法列出 *v1.Service:获取https://172.17.41.15 :6443/api/v1/services?limit=500&resourceVersion=0: 拨打 tcp 172.17.41.15:6443: getsockopt: 连接被拒绝
1 月31 日 14:43:19 kickseed kubelet[28516]：E0131 14:43:19.255460 28516 eviction_manager.go:238] 驱逐管理器：意外错误：无法获取节点信息：找不到节点“kickseed”
1 月31 日 14:43:19 kickseed kubelet[28516]: E0131 14:43:19.863266 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: 无法列出 *v1。 Pod：获取https://172.17.41.15 :6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:43:19 kickseed kubelet[28516]: E0131 14:43:19.864238 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: 无法列出 *v1.Node:获取https://172.17.41.15 :6443/api/v1/nodes?fieldSelector=metadata.name%3Dkickseed&limit=500&resourceVersion=0：拨打 tcp 172.17.41.15:6443：getsockopt：连接被拒绝
1 月31 日 14:43:19 kickseed kubelet[28516]: E0131 14:43:19.865262 28516 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: 无法列出 *v1.Service:获取https://172.17.41.15 :6443/api/v1/services?limit=500&resourceVersion=0: 拨打 tcp 172.17.41.15:6443: getsockopt: 连接被拒绝

一些 docker 镜像如下：
gcr.io/google_containers/kube-apiserver-amd64:v1.9.2
gcr.io/google_containers/kube-controller-manager-amd64:v1.9.2
gcr.io/google_containers/kube-scheduler-amd64:v1.9.2
gcr.io/google_containers/kube-proxy-amd64:v1.9.2
gcr.io/google_containers/etcd-amd64:3.2.14
gcr.io/google_containers/pause-amd64:3.1
gcr.io/google_containers/kube-dnsmasq-amd64:1.4.1
gcr.io/google_containers/kubernetes-dashboard-amd64:v1.8.2
gcr.io/google_containers/kubedns-amd64:1.9
gcr.io/google_containers/kube-discovery-amd64:1.0
gcr.io/google_containers/exechealthz-amd64:v1.2.0
gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.8
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.8
gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.8
gcr.io/google_containers/dnsmasq-metrics-amd64:1.0.1

你预期会发生什么？

kubeadm init 应该完成

如何重现它（尽可能最小和精确）？

带有 Ubuntu 16.04 和 kubeadm 1.9.2 的 virtualbox

还有什么我们需要知道的吗？

areUX lifecyclactive prioritimportant-soon

资料来源

PhoenixAD

👍12

最有用的评论

https://github.com/kubernetes/kubernetes/issues/59680#issuecomment -364646304
禁用 selinux 帮助了我。

avielb 于 2018-03-19

😕7 👍7 👎6 🎉2 ❤1

所有67条评论

上面列出的这些 docker 镜像是在运行 "kubeadm init --kubernetes-version v1.9.2" 之前从我的私有存储库中提取的，由于 GFW，我无法直接访问 gcr.io/google-containers。

PhoenixAD 于 2018-01-31

这里同样的问题！

hedinasr 于 2018-02-06

我在 CentOS 7 上有同样的问题

6l20 于 2018-02-07

fluidjeel 于 2018-02-08

newhomie 于 2018-02-09

tmilleville 于 2018-02-09

+1
vultr上的服务器，也卡在这里。

WindGreen 于 2018-02-09

grantcurell 于 2018-02-09

humorcode 于 2018-02-12

CFAndy 于 2018-02-12

作为解决方法

1/ 在你的 kubernetes master 上创建一个 docker 注册表

2/ 在 /etc/hosts 中声明你的 kubernetes master 为 gcr.io

3/ 在可以上网的机器上，登录 gogle cloud 并下载图片
例子：
gloud docker -- 拉 gcrio/goole_container/pause-amd64:3.0
gloud docker -- save -o /tmp/pause-amd.tar gcrio/goole_container/pause-amd64:3.0

4/ 将图像上传到 docker repo 注册表
docker load -i /tmp/pause-amd64.tar
码头工人标签 gcr.io/Google_containers/pause-amd64:3.0 yourdoke rregistry:pause-amd64 :3.0
docker push yourdoke rregistry:pause-amd64 :3.0

5/ 在你的 kebernetes 主机上作为 gcr.io docker 注册表

从您的 docker 注册表存储库中获取图像
docker pull yourdocke rregistry:pause-amd64 :3.0

拉到本地 gcr.io docker 注册表
码头工人标签 yourdocke rregistry:pause-amd64 :3.0 gcr.io/google_containers/pause-amd64:3.0
docker push gcr.io/google_containers/pause-amd64:3.0

下载 kubeadm init 使用的所有镜像。参见 /etc/kubernetes/manifest/*.yaml

tmilleville 于 2018-02-12

👎31 👍1

它在 1.9.3 中修复了吗？

alexpekurovsky 于 2018-02-16

edwardofclt 于 2018-02-23

+1 - 这仅在我第二次运行 kubeadm init 时出现。第一次踢得很好。我不确定第一次运行时是否有一些状态没有通过 kubeadm reset 正确清理。

patrickstjohn 于 2018-02-25

emisty 于 2018-02-27

centos 7 我在 /etc/env 中设置代理然后显示为👎

emisty 于 2018-02-27

😕1

simis2626 于 2018-02-28

这里同样的问题。 Centos7，最新的 kube 安装（1.9.3），尝试了 hightower 的文档，以及所有的 kubernetes 文档。 etcd 和 flannel 正在工作并活跃起来。使用 NO_PROXY 环境变量将我的外部 IP 放入其中，这样它就不会尝试与其他连接的代理连接，但实际上从未达到这一点，并得到与上述其他所有人相同的错误。

multipathmaster 于 2018-02-28

robertBrem 于 2018-03-02

我有同样的问题，centos 7，kubelet v1.9.3；
但似乎图像已成功下载，
docker images
gcr.io/google_containers/kube-apiserver-amd64 v1.9.3 360d55f91cbf 4 weeks ago 210.5 MB gcr.io/google_containers/kube-controller-manager-amd64 v1.9.3 83dbda6ee810 4 weeks ago 137.8 MB gcr.io/google_containers/kube-scheduler-amd64 v1.9.3 d3534b539b76 4 weeks ago 62.71 MB gcr.io/google_containers/etcd-amd64 3.1.11 59d36f27cceb 3 months ago 193.9 MB gcr.io/google_containers/pause-amd64 3.0 99e59f495ffa 22 months ago 746.9 kB

wangsenyuan 于 2018-03-08

❤1

我这里有一个 CentOS 7 虚拟机，并且已经用我们的代理服务器对其进行了配置。
我收到了相同的超时消息，但是 docker 映像已被拉出并启动并运行。

mvsouza 于 2018-03-08

我也遇到了同样的问题。有关更多信息，请参阅输出和日志。

```[ root@kube01 ~]# kubeadm init
[init] 使用 Kubernetes 版本：v1.9.3
[init] 使用授权模式：[Node RBAC]
[飞行前] 运行飞行前检查。
[警告主机名]：无法访问主机名“kube01”
[警告主机名]：主机名“kube01”在 10.10.0.81:53 上查找 kube01：服务器行为不端
[警告 FileExisting-crictl]：在系统路径中找不到 crictl
[预检] 启动 kubelet 服务
[证书] 生成的 ca 证书和密钥。
[证书] 生成的 apiserver 证书和密钥。
[证书] 为 DNS 名称 [kube01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] 和 IP [10.96.0.1 10.25.123.11] 签署 apiserver 服务证书
[证书] 生成 apiserver-kubelet-client 证书和密钥。
[证书] 生成 sa 密钥和公钥。
[证书] 生成的 front-proxy-ca 证书和密钥。
[证书] 生成的前端代理客户端证书和密钥。
[证书] 有效的证书和密钥现在存在于“/etc/kubernetes/pki”中
[kubeconfig] 将 KubeConfig 文件写入磁盘：“admin.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“kubelet.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“controller-manager.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“scheduler.conf”
[controlplane] 将组件 kube-apiserver 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-apiserver.yaml”
[controlplane] 将组件 kube-controller-manager 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-controller-manager.yaml”
[controlplane] 将组件 kube-scheduler 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-scheduler.yaml”
[etcd] 将本地 etcd 实例的静态 Pod 清单写入“/etc/kubernetes/manifests/etcd.yaml”
[init] 等待 kubelet 作为静态 Pod 从目录“/etc/kubernetes/manifests”启动控制平面。
[init] 如果必须拉取控制平面图像，这可能需要一分钟或更长时间。

In the meantime, while watching `docker ps` this is what I see:
***Note:*** Don't mind the length of time that the containers have been up — this is my third attempt and it's always the same.

```CONTAINER ID        IMAGE
              COMMAND                  CREATED              STATUS              PORTS               NAMES
c422b3fd67f9        gcr.io/google_containers/kube-apiserver-amd64<strong i="5">@sha256</strong>:a5382344aa373a90bc87d3baa4eda5402507e8df5b8bfbbad392c4fff715f0
43            "kube-apiserver --req"   About a minute ago   Up About a minute                       k8s_kube-apiserver_kube-apiserver-k
ube01_kube-system_3ff6faac27328cf290a026c08ae0ce75_1
4b30b98bcc24        gcr.io/google_containers/kube-controller-manager-amd64<strong i="6">@sha256</strong>:3ac295ae3e78af5c9f88164ae95097c2d7af03caddf067cb35599
769d0b7251e   "kube-controller-mana"   2 minutes ago        Up 2 minutes                            k8s_kube-controller-manager_kube-co
ntroller-manager-kube01_kube-system_d556d9b8ccdd523a5208b391ca206031_0
71c6505ed125        gcr.io/google_containers/kube-scheduler-amd64<strong i="7">@sha256</strong>:2c17e637c8e4f9202300bd5fc26bc98a7099f49559ca0a8921cf692ffd4a16
75            "kube-scheduler --add"   2 minutes ago        Up 2 minutes                            k8s_kube-scheduler_kube-scheduler-k
ube01_kube-system_6502dddc08d519eb6bbacb5131ad90d0_0
9d01e2de4686        gcr.io/google_containers/pause-amd64:3.0
              "/pause"                 3 minutes ago        Up 2 minutes                            k8s_POD_kube-controller-manager-kub
e01_kube-system_d556d9b8ccdd523a5208b391ca206031_0
7fdaabc7e2a7        gcr.io/google_containers/pause-amd64:3.0
              "/pause"                 3 minutes ago        Up 2 minutes                            k8s_POD_kube-apiserver-kube01_kube-
system_3ff6faac27328cf290a026c08ae0ce75_0
a5a2736e6cd0        gcr.io/google_containers/pause-amd64:3.0
              "/pause"                 3 minutes ago        Up 2 minutes                            k8s_POD_kube-scheduler-kube01_kube-
system_6502dddc08d519eb6bbacb5131ad90d0_0
ea82cd3a27da        gcr.io/google_containers/pause-amd64:3.0
              "/pause"                 3 minutes ago        Up 2 minutes                            k8s_POD_etcd-kube01_kube-system_727
8f85057e8bf5cb81c9f96d3b25320_0

gcr.io/google_containers/kube-apiserver-amd64@sha256 的日志输出：a5382344aa373a90bc87d3baa4eda5402507e8df5b8bfbbad392c4fff715f043

I0309 19:59:29.570990       1 server.go:121] Version: v1.9.3
I0309 19:59:29.756611       1 feature_gate.go:190] feature gates: map[Initializers:true]
I0309 19:59:29.756680       1 initialization.go:90] enabled Initializers feature as part of admission plugin setup
I0309 19:59:29.760396       1 master.go:225] Using reconciler: master-count
W0309 19:59:29.789648       1 genericapiserver.go:342] Skipping API batch/v2alpha1 because it has no resources.
W0309 19:59:29.796731       1 genericapiserver.go:342] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
W0309 19:59:29.797445       1 genericapiserver.go:342] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
W0309 19:59:29.804841       1 genericapiserver.go:342] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
[restful] 2018/03/09 19:59:29 log.go:33: [restful/swagger] listing is available at https://10.25.123.11:6443/swaggerapi
[restful] 2018/03/09 19:59:29 log.go:33: [restful/swagger] https://10.25.123.11:6443/swaggerui/ is mapped to folder /swagger-ui/
[restful] 2018/03/09 19:59:30 log.go:33: [restful/swagger] listing is available at https://10.25.123.11:6443/swaggerapi
[restful] 2018/03/09 19:59:30 log.go:33: [restful/swagger] https://10.25.123.11:6443/swaggerui/ is mapped to folder /swagger-ui/
I0309 19:59:32.393800       1 serve.go:89] Serving securely on [::]:6443
I0309 19:59:32.393854       1 apiservice_controller.go:112] Starting APIServiceRegistrationController
I0309 19:59:32.393866       1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
I0309 19:59:32.393965       1 controller.go:84] Starting OpenAPI AggregationController
I0309 19:59:32.393998       1 crdregistration_controller.go:110] Starting crd-autoregister controller
I0309 19:59:32.394012       1 controller_utils.go:1019] Waiting for caches to sync for crd-autoregister controller
I0309 19:59:32.394034       1 customresource_discovery_controller.go:152] Starting DiscoveryController
I0309 19:59:32.394057       1 naming_controller.go:274] Starting NamingConditionController
I0309 19:59:32.393855       1 crd_finalizer.go:242] Starting CRDFinalizer
I0309 19:59:32.394786       1 available_controller.go:262] Starting AvailableConditionController
I0309 19:59:32.394815       1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
I0309 20:00:06.434318       1 trace.go:76] Trace[12318713]: "Create /api/v1/nodes" (started: 2018-03-09 19:59:32.431463052 +0000 UTC m=+2.986431803) (total time: 34.002792758s):
Trace[12318713]: [4.00201898s] [4.001725343s] About to store object in database
Trace[12318713]: [34.002792758s] [30.000773778s] END
E0309 20:00:32.406206       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *core.LimitRange: the server was unable to return a response in the time allotted, but may still be processing the request (get limitranges)
E0309 20:00:32.406339       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *core.Secret: the server was unable to return a response in the time allotted, but may still be processing the request (get secrets)
E0309 20:00:32.406342       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/client/informers/internalversion/factory.go:73: Failed to list *apiregistration.APIService: the server was unable to return a response in the time allotted, but may still be processing the request (get apiservices.apiregistration.k8s.io)
E0309 20:00:32.408094       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *core.Pod: the server was unable to return a response in the time allotted, but may still be processing the request (get pods)
E0309 20:00:32.415692       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *core.PersistentVolume: the server was unable to return a response in the time allotted, but may still be processing the request (get persistentvolumes)
E0309 20:00:32.415818       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/apiextensions-apiserver/pkg/client/informers/internalversion/factory.go:73: Failed to list *apiextensions.CustomResourceDefinition: the server was unable to return a response in the time allotted, but may still be processing the request (get customresourcedefinitions.apiextensions.k8s.io)
E0309 20:00:32.415862       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *rbac.ClusterRoleBinding: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io)
E0309 20:00:32.415946       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Namespace: the server was unable to return a response in the time allotted, but may still be processing the request (get namespaces)
E0309 20:00:32.416029       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *core.ResourceQuota: the server was unable to return a response in the time allotted, but may still be processing the request (get resourcequotas)
E0309 20:00:32.416609       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *rbac.ClusterRole: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterroles.rbac.authorization.k8s.io)
E0309 20:00:32.416684       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *rbac.RoleBinding: the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io)
E0309 20:00:32.420305       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Endpoints: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints)
E0309 20:00:32.440196       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *storage.StorageClass: the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io)
E0309 20:00:32.440403       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: the server was unable to return a response in the time allotted, but may still be processing the request (get services)
E0309 20:00:32.448018       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *core.ServiceAccount: the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts)
E0309 20:00:32.448376       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:85: Failed to list *rbac.Role: the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io)
E0309 20:00:33.395988       1 storage_rbac.go:175] unable to initialize clusterroles: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterroles.rbac.authorization.k8s.io)
I0309 20:00:43.455564       1 trace.go:76] Trace[375160879]: "Create /api/v1/nodes" (started: 2018-03-09 20:00:13.454506587 +0000 UTC m=+44.009475397) (total time: 30.001008377s):
Trace[375160879]: [30.001008377s] [30.000778516s] END

================================================== =====================

日志输出gcr.io/google_containers/ KUBE-控制器经理，AMD64 @ SHA256 ：3ac295ae3e78af5c9f88164ae95097c2d7af03caddf067cb35599769d0b7251e

I0309 19:51:35.248083       1 controllermanager.go:108] Version: v1.9.3
I0309 19:51:35.257251       1 leaderelection.go:174] attempting to acquire leader lease...
E0309 19:51:38.310839       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:51:41.766358       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:51:46.025824       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:51:49.622916       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:51:52.675648       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:51:55.697734       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:51:59.348765       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:01.508487       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:03.886473       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:06.120356       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:08.844772       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:12.083789       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:16.038882       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:18.555388       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:21.471034       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:24.236724       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:27.363968       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:30.045776       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:32.751626       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:36.383923       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:38.910958       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:41.400748       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:44.268909       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:47.640891       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:51.713420       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:54.419154       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:52:57.134430       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:00.942903       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:03.440586       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:07.518362       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:53:12.968927       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:16.228760       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:18.299005       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:20.681915       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:24.141874       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:28.484775       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:30.678092       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:34.107654       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:36.251647       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:39.914756       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:42.641017       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:45.058876       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:48.359511       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:51.667554       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:54.338101       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:53:57.357894       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:00.633504       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:03.244353       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:05.923510       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:09.817627       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:12.688349       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:16.803954       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:19.519269       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:23.668226       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:25.903217       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:30.248639       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:32.428029       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:34.962675       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:38.598370       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:41.179039       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:43.927574       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:54:48.190961       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:51.974141       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:55.898687       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:54:59.653210       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:02.094737       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:05.125275       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:09.280324       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:12.920886       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:17.272605       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:21.488182       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:23.708198       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:26.893696       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:31.121014       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:35.414628       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:38.252001       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:41.912479       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:45.621133       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:48.976244       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:52.537317       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:55.863737       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:55:59.682009       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:02.653432       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:04.968939       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:09.336478       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:56:13.488850       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:56:16.262967       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:56:22.685928       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:26.235497       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:28.442915       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:32.051827       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:35.547277       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:38.437120       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:41.007877       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:44.295081       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:46.746424       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:49.321870       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:52.831866       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:55.138333       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:56:57.815491       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:00.802112       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:03.848363       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:07.350593       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:10.672982       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:14.171660       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:17.923995       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:21.919624       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:23.923165       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:27.692006       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:30.654447       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:33.851703       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:57:37.302382       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:40.286552       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:42.358940       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:44.364982       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:46.372569       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:50.571683       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:53.988093       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:57:57.648006       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:01.607961       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:05.717138       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:08.819600       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:12.262314       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:14.327626       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:18.359683       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:20.961212       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:24.503457       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:27.099581       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:29.518623       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:32.943210       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:36.900236       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:40.567479       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:42.642410       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:45.938839       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:50.282483       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:54.086558       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:58:56.794469       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:00.604370       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:02.968978       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:05.825551       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:09.824458       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:12.383249       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:15.891164       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:59:19.088375       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:59:21.305063       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:59:23.366258       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:59:26.308481       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://10.25.123.11:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:59:32.440045       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:36.673744       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:40.049109       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:43.463730       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:46.454431       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:49.782639       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:52.964468       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 19:59:57.265527       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:01.181219       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:03.441468       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:07.324053       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:10.269835       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:12.584906       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:15.042928       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:18.820764       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:22.392476       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:24.630702       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:27.881904       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:30.123513       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:32.490088       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:34.675420       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:37.433904       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:39.819475       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"
E0309 20:00:42.152164       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get endpoints in the namespace "kube-system"

================================================== =====================

日志输出gcr.io/google_containers/ KUBE-调度，AMD64 @ SHA256 ：2c17e637c8e4f9202300bd5fc26bc98a7099f49559ca0a8921cf692ffd4a1675

W0309 19:51:34.800737       1 server.go:159] WARNING: all flags than --config are deprecated. Please begin using a config file ASAP.
I0309 19:51:34.812848       1 server.go:551] Version: v1.9.3
I0309 19:51:34.817093       1 server.go:570] starting healthz server on 127.0.0.1:10251
E0309 19:51:34.818028       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: Get https://10.25.123.11:6443/apis/policy/v1beta1/poddisruptionbudgets?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.818279       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: Get https://10.25.123.11:6443/api/v1/pods?fieldSelector=spec.schedulerName%3Ddefault-scheduler%2Cstatus.phase%21%3DFailed%2Cstatus.phase%21%3DSucceeded&limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.818346       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: Get https://10.25.123.11:6443/api/v1/persistentvolumeclaims?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.818408       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: Get https://10.25.123.11:6443/apis/apps/v1beta1/statefulsets?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.819028       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: Get https://10.25.123.11:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.819386       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: Get https://10.25.123.11:6443/api/v1/nodes?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.820217       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: Get https://10.25.123.11:6443/apis/extensions/v1beta1/replicasets?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.820659       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: Get https://10.25.123.11:6443/api/v1/persistentvolumes?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:34.821783       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: Get https://10.25.123.11:6443/api/v1/replicationcontrollers?limit=500&resourceVersion=0: dial tcp 10.25.123.11:6443: getsockopt: connection refused
E0309 19:51:38.320455       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:38.329101       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:38.329733       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:38.332670       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:38.332707       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:38.332734       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:38.334248       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:38.334568       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:38.334594       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:39.322884       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:39.331726       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:39.333093       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:39.335939       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:39.335988       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:39.336229       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:39.336514       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:39.337881       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:39.338784       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:40.323869       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:40.332910       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:40.334120       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:40.337188       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:40.338218       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:40.339267       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:40.340635       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:40.342035       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:40.343070       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:41.325987       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:41.334782       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:41.336320       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:41.338996       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:41.339923       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:41.340904       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:41.342304       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:41.343675       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:41.344622       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:42.328038       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:42.336744       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:42.338239       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:42.340719       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:42.341878       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:42.342835       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:42.344100       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:42.345231       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:42.346405       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:43.330230       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:43.338706       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:43.339941       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:43.342476       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:43.343584       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:43.344615       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:43.345792       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:43.346976       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:43.348050       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:44.332307       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:44.340659       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:44.341607       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:44.344223       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:44.345380       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:44.346247       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0309 19:51:44.347536       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0309 19:51:44.348664       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0309 19:51:44.349648       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0309 19:51:45.334228       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0309 19:51:45.342638       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0309 19:51:45.343460       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0309 19:51:45.345969       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0309 19:51:45.347140       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0309 19:51:45.348176       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope

================================================== =====================

gcr.io/google_containers/pause-amd64:3.0 的日志输出

================================================== =====================

gcr.io/google_containers/pause-amd64:3.0 的日志输出

================================================== =====================

gcr.io/google_containers/pause-amd64:3.0 的日志输出

================================================== =====================

gcr.io/google_containers/pause-amd64:3.0 的日志输出

================================================== =====================

edwardofclt 于 2018-03-09

+1
更新：
在尽我所能查看之后（我对 k8s 有点陌生），我终于发现kubectl describe pod -n kube-system kube-dns-<sha>表明我正在安装的虚拟服务器只有 1 个 CPU，并且 kube-dns 没有启动到期由于缺乏CPU。奇怪的是kubectl logs pod -n kube-system kube-dns-<sha>没有显示此信息。

它在重新安装操作系统后工作（因为 kubeadm 安装后重新启动会使 k8s 主服务器无法正常启动）。
（抱歉忘记捕获输出）

bbruun 于 2018-03-12

QQ-ing 于 2018-03-14

我遇到了同样的问题，取消并运行reset然后init与以前相同，但使用--apiserver-advertise-address=<my_host_public_ip_address> - 它有效。

VaverDariush 于 2018-03-15

👍2

https://github.com/kubernetes/kubernetes/issues/59680#issuecomment -364646304
禁用 selinux 帮助了我。

avielb 于 2018-03-19

😕7 👍7 👎6 🎉2 ❤1

降级到 1.8.10 为我解决了这个问题。

cbluth 于 2018-03-24

👍5

gabrielfsousa 于 2018-03-27

nevotheless 于 2018-04-01

Ubuntu 16.04（无 selinux）上的 v1.9.3 也存在同样的问题

jpmondet 于 2018-04-01

+1 同样的问题

duongmngo 于 2018-04-02

与 arm64 上的 Ubuntu 16.04 上的 v1.10 相同的问题。

farfeduc 于 2018-04-04

👍8

与 arm64 上的 ubuntu 16.04 上的 v1.10 相同的问题（无 selinux）

therealqxing 于 2018-04-05

检查您安装的硬件上的 CPU 数量 - 正如我在 3 周前在上面所写的那样，主设备上需要 2 个才能安装。

bbruun 于 2018-04-05

😕1 👍1

@bbruun使用的硬件是： https ://www.pine64.org/?page_id=1491 所以有 4 个内核，因此可以正确检测到它们。那么硬件应该不是问题。但无论如何，谢谢你的提示。也许@qxing3虽然不使用相同的硬件......

farfeduc 于 2018-04-05

@farfeduc这是我遇到的障碍，连续几次尝试重新安装我的虚拟机以测试安装并了解 k8s，但是从系统中获取可用日志是一件麻烦事，我尝试从任何地方获取它我可以直到我收到一条关于没有足够 cpu 可用的消息。现在我已经购买了 3 个 Udoo x86 Ultra 来运行一个小型集群，以便在我们使用稍微大一点的实例的工作中在家玩 :-)

bbruun 于 2018-04-05

@bbruun我为我的虚拟机配置了 2 个 cpu，无论如何感谢您的提示。

therealqxing 于 2018-04-05

/分配@liztio

timothysc 于 2018-04-07

wu0407 于 2018-04-08

+1 v1.10.0

manojakondi 于 2018-04-13

+1 v1.10.0 和 1.10.1

TechnoRoss 于 2018-04-13

cleversonledur 于 2018-04-13

有趣的是，我发现增量取决于我部署的位置。我希望有时间进一步探索，但到目前为止，我知道这一点。如果我使用我的 Mac/VMware Fusion 并运行 CentOS 7 虚拟机，我可以完全成功地使用 kubeadm 1.8。我从来没有在本地工作过 v1.9 或 v1.10。但是，在 Digital Ocean 使用 CentOS 7 镜像，我可以成功运行 v1.8.x、v1.10.0 和 v1.10.1；出于某种原因，v1.9 似乎“只是依赖”。因此，现在需要深入研究两种环境之间的细微差异，找出触发开关的原因。我知道内核/补丁级别匹配，以及 Docker 引擎等。DO 安装 cloud-init 的东西，我的本地 VM 没有，等等。找出不同之处并非易事。我什至尝试匹配磁盘大小（认为较小的磁盘可能会在某处掩盖一些错误，但也消除了它）。

在所有情况下，拉取图像总是有效的，它只是让 API 服务做出响应，并且当它确实失败时不会每隔几分钟就继续回收。

TechnoRoss 于 2018-04-14

问候，

您可以参考https://docs.docker.com/config/daemon/systemd/#httphttps -proxy 为 docker daemon 设置代理。

guhuajun 于 2018-04-16

+1 在树莓派 3 上的 HypriotOS 上运行

hansamann 于 2018-04-18

我能够通过安装 v1.9.6 而不是最新版本来使其工作。
因此它在 v1.9.6 上正常工作，但在 sopine 板上 arm64 上的 ubuntu 16.04 上的 v1.10.0 和 v1.10.1 上失败。

farfeduc 于 2018-04-18

👍1

我在 Raspberry Pi 3 HypriotOS 上遇到了同样的问题。降级到 1.9.7-00 也对我有用。

JoeJasinski 于 2018-04-22

👍1

+1，kubeadm v1.10.1，树莓派 3b，hypriotOS

kocoten1992 于 2018-04-22

就我而言，我发现 etcd 容器正在启动，然后退出并出现错误，这导致kubeadm init挂起并最终超时。

要检查这是否对您不利，请运行docker ps -a并检查 etcd 容器的状态。如果它没有运行，请检查 etcd 容器（ docker logs <container-id> ）的日志，看看它是否抱怨无法绑定到地址。请参阅此问题报告： https ://github.com/kubernetes/kubernetes/issues/57709

我刚才提到的问题有一个解决方法，但请确保这是您首先遇到的问题。

loudsong 于 2018-04-27

👍1

确保您的防火墙允许6443上的入站流量

例如，如果您使用的是 Ubuntu，请运行ufw status以查看它是否已启用

然后ufw allow 6443打开端口

aaronkjones 于 2018-05-01

是否可以列出下面的图像，我们通过代理手动提取它们，然后再次初始化 kubeadm。
它会起作用吗？
因为，我们在中国，你知道，GFW。
而且我是k8s的新手，在centos7上设置时卡在这里。

HanYL-coder 于 2018-05-17

❤1

对于那些支持伟大防火墙的中国人

thanch2n 于 2018-05-17

👍1

@thanch2n非常感谢。我要试试。

HanYL-coder 于 2018-05-18

我向 docker 添加了代理，使用它，图像似乎都已经下载，但仍然卡住“[init] 如果必须拉取控制平面图像，这可能需要一分钟或更长时间。”。

列出下面自动拉出的图像。

k8s.gcr.io/kube-apiserver-amd64 v1.10.2 e774f647e259 2 周前 225 MB
k8s.gcr.io/kube-scheduler-amd64 v1.10.2 0dcb3dea0db1 2 周前 50.4 MB
k8s.gcr.io/kube-controller-manager-amd64 v1.10.2 f3fcd0775c4e 2 周前 148 MB
k8s.gcr.io/etcd-amd64 3.1.12 52920ad46f5b 2 个月前 193 MB
k8s.gcr.io/pause-amd64 3.1 da86e6ba6ca1 4 个月前 742 kB

HanYL-coder 于 2018-05-18

我花了很多时间试图弄清楚这一点。我禁用了 ufw，关闭了 selinux，确保 ip forwading 已打开，并且 /proc/sys/net/bridge/bridge-nf-call-iptables 设置为 1。似乎没有什么能解决问题。

最后我决定降级然后升级。

sudo apt-get -y --allow-downgrades install kubectl=1.5.3-00 kubelet=1.5.3-00 kubernetes-cni=0.3.0.1-07a8a2-00和

curl -Lo /tmp/old-kubeadm.deb https://apt.k8s.io/pool/kubeadm_1.6.0-alpha.0.2074-a092d8e0f95f52-00_amd64_0206dba536f698b5777c7d210444a8ace18f48e045ab78687327631c6c694f42.deb

从 1.10 降级，然后只是

sudo apt-get -y install kubectl kubelet kubernetes-cni kubeadm

Etcd 正在重新启动，并且 api-server 正在超时。一段时间后，api-server 重新启动并抱怨无法连接。有没有办法可以打开 DEBUG 级别的日志记录？现在确定是什么原因造成的。但它现在工作。我肯定想重现这个并解决它。

mendoncangelo 于 2018-05-19

👍2

我知道我卡住的原因了。
我在 vmware 中运行它并位于 1G RAM，k8s 至少需要 2G RAM。
有没有可能为此添加通知？

HanYL-coder 于 2018-05-22

在 CentOS 7 上 +1 kubeadm 1.10.2
4GB RAM 2CPU

FNaughton 于 2018-05-25

在具有 6GB RAM 和 1 个 VCPU 的 HyperV VM 上的 Debian Stretch (go1.9.3) 上 +1 kubeadm 1.10.1...

过去它运行良好，因为我多次重新生成集群......

simmessa 于 2018-05-28

尝试在 HyperV 中切换到 2 个 VCPU，没有任何变化。

simmessa 于 2018-05-28

+1！

imedaouidene 于 2018-05-29

+1。 kubeadm 1.10.1，Debian 伸展。以前工作过

egism 于 2018-06-01

发现在 Centos 7 上使用 docker 1.13.1 时，我们看到了存储驱动程序的问题。 Docker 日志显示“readlink /var/lib/docker/overlay2/l：无效参数”。迁移到 docker 18.03.1-ce 似乎解决了这个问题，并且 kubeadm init 不再挂起。

FNaughton 于 2018-06-01

我有同样的问题。结果 etcd 拉取了 linux 机器主机名（somedomain.example.com），在 DNS 服务器上搜索它，找到通配符域（*.example.com）的答案并尝试绑定到返回的 IP 地址而不是apiserver 广告地址。

Evengard 于 2018-06-06

已经对预拉以及枢轴超时检测进行了许多修复，因此关闭了这个问题。

timothysc 于 2018-06-12

tejp1002 于 2018-10-30

尝试了让 k8sadmin 拉下图像的标准方法，尝试了很多次然后拉下图像，重置，试图忽略错误，总是失败。

pi@master-node-001 :~ $ sudo kubeadm reset
[reset] 警告：“kubeadm init”或“kubeadm join”对此主机所做的更改将被还原。
[preflight] 运行飞行前检查
[reset] 停止 kubelet 服务
[reset] 卸载“/var/lib/kubelet”中的挂载目录
[reset] 删除有状态目录的内容：[/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/etcd]
[reset] 删除配置目录的内容：[/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] 删除文件：[/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.配置]
pi@master-node-001 :~ $ kubeadm 配置图像拉
[config/images] 拉取k8s.gcr.io/kube-apiserver:v1.12.2
[config/images] 拉取k8s.gcr.io/kube-controller-manager:v1.12.2
[config/images] 拉取 k8s.gcr.io/kube- scheduler:v1.12.2
[config/images] 拉取k8s.gcr.io/kube-proxy:v1.12.2
[config/images] 拉取k8s.gcr.io/pause:3.1
[config/images] 拉取k8s.gcr.io/etcd:3.2.24
[config/images] 拉取k8s.gcr.io/coredns:1.2.2
pi@master-node-001 :~ $ sudo kubeadm init --token-ttl=0 --ignore-preflight-errors=all
[init] 使用 Kubernetes 版本：v1.12.2
[preflight] 运行飞行前检查
[preflight/images] 拉取建立 Kubernetes 集群所需的镜像
[预检/图像] 这可能需要一两分钟，具体取决于您的互联网连接速度
[preflight/images] 您也可以使用“kubeadm config images pull”预先执行此操作
[kubelet] 将带有标志的 kubelet 环境文件写入文件“/var/lib/kubelet/kubeadm-flags.env”
[kubelet] 将 kubelet 配置写入文件“/var/lib/kubelet/config.yaml”
[预检] 激活 kubelet 服务
[证书] 生成 etcd/ca 证书和密钥。
[证书] 生成 apiserver-etcd-client 证书和密钥。
[证书] 生成的 etcd/server 证书和密钥。
[证书] etcd/server 服务证书已为 DNS 名称 [master-node-001 localhost] 和 IP [127.0.0.1 ::1] 签名
[证书] 生成的 etcd/peer 证书和密钥。
[证书] etcd/peer 服务证书已为 DNS 名称 [master-node-001 localhost] 和 IP [192.168.0.100 127.0.0.1 ::1] 签名
[证书] 生成 etcd/healthcheck-client 证书和密钥。
[证书] 生成的 ca 证书和密钥。
[证书] 生成的 apiserver 证书和密钥。
[证书] 为 DNS 名称 [master-node-001 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] 和 IP [10.96.0.1 192.168.0.100] 签署 apiserver 服务证书
[证书] 生成 apiserver-kubelet-client 证书和密钥。
[证书] 生成的 front-proxy-ca 证书和密钥。
[证书] 生成的前端代理客户端证书和密钥。
[证书] 有效证书和密钥现在存在于“/etc/kubernetes/pki”中
[证书] 生成 sa 密钥和公钥。
[kubeconfig] 将 KubeConfig 文件写入磁盘：“/etc/kubernetes/admin.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“/etc/kubernetes/kubelet.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“/etc/kubernetes/controller-manager.conf”
[kubeconfig] 将 KubeConfig 文件写入磁盘：“/etc/kubernetes/scheduler.conf”
[controlplane] 将组件 kube-apiserver 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-apiserver.yaml”
[controlplane] 将组件 kube-controller-manager 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-controller-manager.yaml”
[controlplane] 将组件 kube-scheduler 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-scheduler.yaml”
[etcd] 将本地 etcd 实例的静态 Pod 清单写入“/etc/kubernetes/manifests/etcd.yaml”
[init] 等待 kubelet 作为静态 Pod 从目录“/etc/kubernetes/manifests”启动控制平面
[init] 如果必须拉取控制平面图像，这可能需要一分钟或更长时间

不幸的是，发生了错误：
等待条件超时

此错误可能是由以下原因引起的：
- kubelet 没有运行
- kubelet 不健康，因为节点配置错误（必需的 cgroups 已禁用）

如果您使用的是 systemd 驱动的系统，则可以尝试使用以下命令解决错误：
- 'systemctl 状态 kubelet'
- 'journalctl -xeu kubelet'

此外，控制平面组件在由容器运行时启动时可能已经崩溃或退出。
要进行故障排除，请使用您首选的容器运行时 CLI（例如 docker）列出所有容器。
这是一个示例，您可以列出在 docker 中运行的所有 Kubernetes 容器：
- '码头工人ps -a | grep 库 | grep -v 暂停'
找到失败的容器后，您可以使用以下命令检查其日志：
- '码头工人日志 CONTAINERID'
无法初始化 Kubernetes 集群
pi@master-node-001 :~ $ docker images
存储库标签图像 ID 创建大小
k8s.gcr.io/kube-controller-manager v1.12.2 4bc6cae738d8 7 天前 146MB
k8s.gcr.io/kube-apiserver v1.12.2 8bfe044a05e1 7 天前 177MB
k8s.gcr.io/kube-scheduler v1.12.2 3abf5566fec1 7 天前 52MB
k8s.gcr.io/kube-proxy v1.12.2 328ef67ca54f 7 天前 84.5MB
k8s.gcr.io/kube-proxy v1.12.1 8c06fbe56458 3 周前 84.7MB
k8s.gcr.io/kube-controller-manager v1.12.1 5de943380295 3 周前 146MB
k8s.gcr.io/kube-scheduler v1.12.1 1fbc2e4cd378 3 周前 52MB
k8s.gcr.io/kube-apiserver v1.12.1 ab216fe6acf6 3 周前 177MB
k8s.gcr.io/etcd 3.2.24 e7a8884c8443 5 周前 222MB
k8s.gcr.io/coredns 1.2.2 ab0805b0de94 2 个月前 33.4MB
k8s.gcr.io/kube-scheduler v1.11.0 0e4a34a3b0e6 4 个月前 56.8MB
k8s.gcr.io/kube-controller-manager v1.11.0 55b70b420785 4 个月前 155MB
k8s.gcr.io/etcd 3.2.18 b8df3b177be2 6 个月前 219MB
k8s.gcr.io/pause 3.1 e11a8cbeda86 10 个月前 374kB
pi@master-node-001 :~ $ h | grep kubectl
-bash：h：找不到命令
pi@master-node-001 :~ $ 历史 | grep kubectl
9 kubectl pod 列表
10 个 kubectl 列表 pod
11 kubectl --帮助
12 kubectl 获取 pods -o 宽
14 kubectl 获取 pod -o 宽
32 小时 | grep kubectl
33 历史 | grep kubectl
pi@master-node-001 :~ $ !12
kubectl 获取 pods -o 宽
无法连接到服务器：net/http: TLS 握手超时
pi@master-node-001 :~ $ 历史 | grep 暂停
17 码头工人ps -a | grep 库 | grep -v 暂停
35 历史 | grep 暂停
pi@master-node-001 :~ $ !17
码头工人ps -a | grep 库 | grep -v 暂停
41623613679e 8bfe044a05e1 "kube-apiserver --au…" 29 秒前上升 14 秒 k8s_kube-apiserver_kube-apiserver-master-node-001_kube-system_1ec53f8ef96c76af95c78c809252f05c_3
0870760b9ea0 8bfe044a05e1 "kube-apiserver --au..." 2 分钟前退出 (0) 33 秒前 k8s_kube-apiserver_kube-apiserver-master-node-001_kube-system_1ec53f8ef96c76af95c78c809252f05c_2
c60d65fab8a7 3abf5566fec1 "kube-scheduler --ad..." 6 分钟前向上 5 分钟 k8s_kube-scheduler_kube-scheduler-master-node-001_kube-system_ee7b1077c61516320f4273309e9b4690_0
26c58f6c68e9 e7a8884c8443 "etcd --advertise-cl…" 6 分钟前向上 5 分钟 k8s_etcd_etcd-master-node-001_kube-system_d01dcc7fc79b875a52f01e26432e6745_0
65546081ca77 4bc6cae738d8 "kube-controller-man..." 6 分钟前向上 5 分钟 k8s_kube-controller-manager_kube-controller-manager-master-node-001_kube-system_07e19bc9fea1626927c12244604bdb2f_0
pi@master-node-001 :~ $ kubectl get pods -o wide
^C
pi@master-node-001 :~ $ sudo reboot
远程主机关闭与 192.168.0.100 的连接。
与 192.168.0.100 的连接已关闭。
karl@karl-PL62-7RC :~$ ping 192.168.0.100
PING 192.168.0.100 (192.168.0.100) 56(84) 字节数据。
^C
--- 192.168.0.100 ping 统计 ---
2个包发送，0个接收，100%丢包，时间1015ms

karl@karl-PL62-7RC :~$ ssh [email protected]
ssh_exchange_identification：读取：对等方重置连接
karl@karl-PL62-7RC :~$ ssh [email protected]
[email protected]的密码：
Linux master-node-001 4.14.71-v7+ #1145 SMP Fri Sep 21 15:38:35 BST 2018 armv7l

Debian GNU/Linux 系统中包含的程序是免费软件；
每个程序的确切分配条款在
/usr/share/doc/*/copyright 中的单个文件。

Debian GNU/Linux 绝对没有担保，在某种程度上
适用法律允许。
最后登录时间：2018 年 10 月 31 日星期三 21:36:13
pi@master-node-001 :~ $ kubectl get pods -o wide
与服务器 192.168.0.100:6443 的连接被拒绝 - 您是否指定了正确的主机或端口？
pi@master-node-001 :~ $ sudo kubeadm init --token-ttl=0 --ignore-preflight-errors=all
[init] 使用 Kubernetes 版本：v1.12.2
[preflight] 运行飞行前检查
[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml 已经存在
[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]：/etc/kubernetes/manifests/kube-controller-manager.yaml 已经存在
[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml 已经存在
[WARNING FileAvailable--etc-kubernetes-manifests-etcd.yaml]: /etc/kubernetes/manifests/etcd.yaml 已经存在
[警告端口 10250]：端口 10250 正在使用中
[警告 DirAvailable--var-lib-etcd]: /var/lib/etcd 不为空
[preflight/images] 拉取建立 Kubernetes 集群所需的镜像
[预检/图像] 这可能需要一两分钟，具体取决于您的互联网连接速度
[preflight/images] 您也可以使用“kubeadm config images pull”预先执行此操作
[kubelet] 将带有标志的 kubelet 环境文件写入文件“/var/lib/kubelet/kubeadm-flags.env”
[kubelet] 将 kubelet 配置写入文件“/var/lib/kubelet/config.yaml”
[预检] 激活 kubelet 服务
[证书] 使用现有的 etcd/peer 证书和密钥。
[证书] 使用现有的 apiserver-etcd-client 证书和密钥。
[证书] 使用现有的 etcd/server 证书和密钥。
[证书] 使用现有的 etcd/healthcheck-client 证书和密钥。
[证书] 使用现有的 apiserver 证书和密钥。
[证书] 使用现有的 apiserver-kubelet-client 证书和密钥。
[证书] 使用现有的前端代理客户端证书和密钥。
[证书] 有效证书和密钥现在存在于“/etc/kubernetes/pki”中
[证书] 使用现有的 sa 密钥。
[kubeconfig] 使用现有的最新 KubeConfig 文件：“/etc/kubernetes/admin.conf”
[kubeconfig] 使用现有的最新 KubeConfig 文件：“/etc/kubernetes/kubelet.conf”
[kubeconfig] 使用现有的最新 KubeConfig 文件：“/etc/kubernetes/controller-manager.conf”
[kubeconfig] 使用现有的最新 KubeConfig 文件：“/etc/kubernetes/scheduler.conf”
[controlplane] 将组件 kube-apiserver 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-apiserver.yaml”
[controlplane] 将组件 kube-controller-manager 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-controller-manager.yaml”
[controlplane] 将组件 kube-scheduler 的静态 Pod 清单写入“/etc/kubernetes/manifests/kube-scheduler.yaml”
[etcd] 将本地 etcd 实例的静态 Pod 清单写入“/etc/kubernetes/manifests/etcd.yaml”
[init] 等待 kubelet 作为静态 Pod 从目录“/etc/kubernetes/manifests”启动控制平面
[init] 如果必须拉取控制平面图像，这可能需要一分钟或更长时间

不幸的是，发生了错误：
等待条件超时

此错误可能是由以下原因引起的：
- kubelet 没有运行
- kubelet 不健康，因为节点配置错误（必需的 cgroups 已禁用）

如果您使用的是 systemd 驱动的系统，则可以尝试使用以下命令解决错误：
- 'systemctl 状态 kubelet'
- 'journalctl -xeu kubelet'

fenwk0 于 2018-10-31

13 须藤 kubeadm 初始化 --token-ttl=0
14 kubectl 获取 pod -o 宽
15 sudo kubeadm 重置
16 须藤 kubeadm 初始化 --token-ttl=0
17 码头工人ps -a | grep 库 | grep -v 暂停
18个kubeadm配置镜像拉--kubernetes-version=v1.11.0
19 须藤 kubeadm 重置
20 历史 > notes.txt
还有21个notes.txt
22 须藤重启
23 kubeadm 配置镜像列表
24个kubeadm配置镜像拉--kubernetes-version=v1.11.0
25 sudo kubeadm init --token-ttl=0 --ignore-preflight-errors=all
26 个 kubeadm 配置镜像拉取
27 sudo kubeadm init --token-ttl=0 --ignore-preflight-errors=all
28 个 kubeadm 配置镜像拉取
29 须藤 kubeadm 重置
30 个 kubeadm 配置镜像拉取
31 sudo kubeadm 初始化 --token-ttl=0 --ignore-preflight-errors=all
32 个码头图像
33 小时 | grep kubectl
34 历史 | grep kubectl
35 kubectl 获取 pods -o 宽
36 历史 | grep 暂停
37 码头工人ps -a | grep 库 | grep -v 暂停
38 kubectl 获取 pods -o 宽
39 须藤重启
40 kubectl 获取 pods -o 宽
41 sudo kubeadm init --token-ttl=0 --ignore-preflight-errors=all

fenwk0 于 2018-10-31

此页面是否有帮助？

0 / 5 - 0 等级

Kubeadm: Kubeadm init 阻止“如果必须拉取控制平面图像，这可能需要一分钟或更长时间

版本

发生了什么？

你预期会发生什么？

如何重现它（尽可能最小和精确）？

还有什么我们需要知道的吗？

最有用的评论

所有67条评论

gcr.io/google_containers/kube-apiserver-amd64@sha256 的日志输出：a5382344aa373a90bc87d3baa4eda5402507e8df5b8bfbbad392c4fff715f043

日志输出gcr.io/google_containers/ KUBE-控制器经理，AMD64 @ SHA256 ：3ac295ae3e78af5c9f88164ae95097c2d7af03caddf067cb35599769d0b7251e

日志输出gcr.io/google_containers/ KUBE-调度，AMD64 @ SHA256 ：2c17e637c8e4f9202300bd5fc26bc98a7099f49559ca0a8921cf692ffd4a1675

gcr.io/google_containers/pause-amd64:3.0 的日志输出

gcr.io/google_containers/pause-amd64:3.0 的日志输出

gcr.io/google_containers/pause-amd64:3.0 的日志输出

gcr.io/google_containers/pause-amd64:3.0 的日志输出

相关问题