<p>kubeadm init 卡在“第一个节点已注册，但尚未准备好”</p>

在提交这个问题之前，您在 Kubernetes 问题中搜索了哪些关键字？ （如果您发现任何重复项，您应该在那里回复。）：kubeadm

这是错误报告还是功能请求？ （选择一项）：错误报告

Kubernetes 版本（使用kubectl version ）：1.6.0

环境:

• 云提供商或硬件配置：Raspberry Pi 3 Model B
• 操作系统（例如来自 /etc/os-release）：Hypriot 1.4.0（Docker 手动降级到 1.12.6，Hypriot 1.4.0 附带 Docker 17.03.0-ce）
• 内核（例如uname -a ）：4.4.50-hypriotos-v7+
• 安装工具：kubeadm
• 其他：

发生了什么：

完全遵循kubeadm 入门指南：

# kubeadm init --apiserver-cert-extra-sans redacted --pod-network-cidr 10.244.0.0/16
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.6.0
[init] Using Authorization mode: RBAC
[preflight] Running pre-flight checks
[certificates] Generated CA certificate and key.
[certificates] Generated API server certificate and key.
[certificates] API Server serving cert is signed for DNS names [kube-01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local redacted] and IPs [10.96.0.1 10.0.1.101]
[certificates] Generated API server kubelet client certificate and key.
[certificates] Generated service account token signing key and public key.
[certificates] Generated front-proxy CA certificate and key.
[certificates] Generated front-proxy client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 206.956919 seconds
[apiclient] Waiting for at least one node to register and become ready
[apiclient] First node has registered, but is not ready yet
[apiclient] First node has registered, but is not ready yet
[apiclient] First node has registered, but is not ready yet

最后一条消息“第一个节点已注册，但尚未准备好”无限重复，kubeadm 永远不会结束。 我在另一个会话中连接到主服务器，以查看所有 Docker 容器是否按预期运行，它们是：

$ docker ps
CONTAINER ID        IMAGE                                                                                                                          COMMAND                  CREATED             STATUS              PORTS               NAMES
54733aa1aae3        gcr.io/google_containers/kube-controller-manager-arm<strong i="6">@sha256</strong>:22f30303212b276b6868b89c8e92c5fb2cb93641e59c312b254c6cb0fa111b2a   "kube-controller-mana"   10 minutes ago      Up 10 minutes                           k8s_kube-controller-manager_kube-controller-manager-kube-01_kube-system_d44abf63e3ab24853ab86643e0b96d81_0
55b6bf2cc09e        gcr.io/google_containers/etcd-arm<strong i="7">@sha256</strong>:0ce1dcd85968a3242995dfc168abba2c3bc03d0e3955f52a0b1e79f90039dcf2                      "etcd --listen-client"   11 minutes ago      Up 11 minutes                           k8s_etcd_etcd-kube-01_kube-system_90ab26991bf9ad676a430c7592d08bee_0
bd0dc34d5e77        gcr.io/google_containers/kube-apiserver-arm<strong i="8">@sha256</strong>:c54b8c609a6633b5397173c763aba0656c6cb2601926cce5a5b4870d58ba67bd            "kube-apiserver --ins"   12 minutes ago      Up 12 minutes                           k8s_kube-apiserver_kube-apiserver-kube-01_kube-system_4d99c225ec157dc715c26b59313aeac8_1
1c4c7b69a3eb        gcr.io/google_containers/kube-scheduler-arm<strong i="9">@sha256</strong>:827449ef1f3d8c0a54d842af9d6528217ccd2d36cc2b49815d746d41c7302050            "kube-scheduler --kub"   13 minutes ago      Up 13 minutes                           k8s_kube-scheduler_kube-scheduler-kube-01_kube-system_3ef1979df7569495bb727d12ac1a7a6f_0
4fd0635f9439        gcr.io/google_containers/pause-arm:3.0                                                                                         "/pause"                 14 minutes ago      Up 14 minutes                           k8s_POD_kube-controller-manager-kube-01_kube-system_d44abf63e3ab24853ab86643e0b96d81_0
cfb4a758ad96        gcr.io/google_containers/pause-arm:3.0                                                                                         "/pause"                 14 minutes ago      Up 14 minutes                           k8s_POD_etcd-kube-01_kube-system_90ab26991bf9ad676a430c7592d08bee_0
a631d8b6c11c        gcr.io/google_containers/pause-arm:3.0                                                                                         "/pause"                 14 minutes ago      Up 14 minutes                           k8s_POD_kube-scheduler-kube-01_kube-system_3ef1979df7569495bb727d12ac1a7a6f_0
309b62fff122        gcr.io/google_containers/pause-arm:3.0                                                                                         "/pause"                 14 minutes ago      Up 14 minutes                           k8s_POD_kube-apiserver-kube-01_kube-system_4d99c225ec157dc715c26b59313aeac8_0

我将 admin kubeconfig 复制到我的本地机器并使用 kubectl (1.6.0) 查看 kubeadm 声称已注册的节点发生了什么：

$ kubectl describe node kube-01
Name:           kube-01
Role:
Labels:         beta.kubernetes.io/arch=arm
            beta.kubernetes.io/os=linux
            kubernetes.io/hostname=kube-01
Annotations:        node.alpha.kubernetes.io/ttl=0
            volumes.kubernetes.io/controller-managed-attach-detach=true
Taints:         <none>
CreationTimestamp:  Tue, 28 Mar 2017 22:06:40 -0700
Phase:
Conditions:
  Type          Status  LastHeartbeatTime           LastTransitionTime          Reason              Message
  ----          ------  -----------------           ------------------          ------              -------
  OutOfDisk         False   Tue, 28 Mar 2017 22:17:24 -0700     Tue, 28 Mar 2017 22:06:40 -0700     KubeletHasSufficientDisk    kubelet has sufficient disk space available
  MemoryPressure    False   Tue, 28 Mar 2017 22:17:24 -0700     Tue, 28 Mar 2017 22:06:40 -0700     KubeletHasSufficientMemory  kubelet has sufficient memory available
  DiskPressure      False   Tue, 28 Mar 2017 22:17:24 -0700     Tue, 28 Mar 2017 22:06:40 -0700     KubeletHasNoDiskPressure    kubelet has no disk pressure
  Ready         False   Tue, 28 Mar 2017 22:17:24 -0700     Tue, 28 Mar 2017 22:06:40 -0700     KubeletNotReady         runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Addresses:      10.0.1.101,10.0.1.101,kube-01
Capacity:
 cpu:       4
 memory:    882632Ki
 pods:      110
Allocatable:
 cpu:       4
 memory:    780232Ki
 pods:      110
System Info:
 Machine ID:            9989a26f06984d6dbadc01770f018e3b
 System UUID:           9989a26f06984d6dbadc01770f018e3b
 Boot ID:           7a77e2e8-dd62-4989-b9e7-0fb52747162a
 Kernel Version:        4.4.50-hypriotos-v7+
 OS Image:          Raspbian GNU/Linux 8 (jessie)
 Operating System:      linux
 Architecture:          arm
 Container Runtime Version: docker://1.12.6
 Kubelet Version:       v1.6.0
 Kube-Proxy Version:        v1.6.0
PodCIDR:            10.244.0.0/24
ExternalID:         kube-01
Non-terminated Pods:        (4 in total)
  Namespace         Name                        CPU Requests    CPU Limits  Memory Requests Memory Limits
  ---------         ----                        ------------    ----------  --------------- -------------
  kube-system           etcd-kube-01                0 (0%)      0 (0%)      0 (0%)      0 (0%)
  kube-system           kube-apiserver-kube-01          250m (6%)   0 (0%)      0 (0%)      0 (0%)
  kube-system           kube-controller-manager-kube-01     200m (5%)   0 (0%)      0 (0%)      0 (0%)
  kube-system           kube-scheduler-kube-01          100m (2%)   0 (0%)      0 (0%)      0 (0%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits  Memory Requests Memory Limits
  ------------  ----------  --------------- -------------
  550m (13%)    0 (0%)      0 (0%)      0 (0%)
Events:
  FirstSeen LastSeen    Count   From            SubObjectPath   Type        Reason          Message
  --------- --------    -----   ----            -------------   --------    ------          -------
  14m       14m     1   kubelet, kube-01            Normal      Starting        Starting kubelet.
  14m       10m     55  kubelet, kube-01            Normal      NodeHasSufficientDisk   Node kube-01 status is now: NodeHasSufficientDisk
  14m       10m     55  kubelet, kube-01            Normal      NodeHasSufficientMemory Node kube-01 status is now: NodeHasSufficientMemory
  14m       10m     55  kubelet, kube-01            Normal      NodeHasNoDiskPressure   Node kube-01 status is now: NodeHasNoDiskPressure

这揭示了 kubelet 未准备就绪的原因：

“运行时网络未准备好：NetworkReady=false原因：NetworkPluginNotReady消息：docker ：网络插件未准备好：cni 配置”

在我使用 kubeadm 1.5 的实验中，不需要 CNI 来启动主节点，所以这很令人惊讶。 甚至入门指南也建议kubeadm init应该在您继续部署 CNI 插件之前成功完成。

无论如何，我在本地机器上使用 kubectl 部署了 flannel：

$ kubectl apply -f kube-flannel.yml

文件内容在哪里：

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "type": "flannel",
      "delegate": {
        "isDefaultGateway": true
      }
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      hostNetwork: true
      nodeSelector:
        beta.kubernetes.io/arch: amd64
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      serviceAccountName: flannel
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.7.0-amd64
        command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr" ]
        securityContext:
          privileged: true
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      - name: install-cni
        image: quay.io/coreos/flannel:v0.7.0-amd64
        command: [ "/bin/sh", "-c", "set -e -x; cp -f /etc/kube-flannel/cni-conf.json /etc/cni/net.d/10-flannel.conf; while true; do sleep 3600; done" ]
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
        - name: run
          hostPath:
            path: /run
        - name: cni
          hostPath:
            path: /etc/cni/net.d
        - name: flannel-cfg
          configMap:
            name: kube-flannel-cfg

但它从未安排：

$ kubectl describe ds kube-flannel-ds -n kube-system
Name:       kube-flannel-ds
Selector:   app=flannel,tier=node
Node-Selector:  beta.kubernetes.io/arch=amd64
Labels:     app=flannel
        tier=node
Annotations:    kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"extensions/v1beta1","kind":"DaemonSet","metadata":{"annotations":{},"labels":{"app":"flannel","tier":"node"},"name":"kube-flannel-ds","n...
Desired Number of Nodes Scheduled: 0
Current Number of Nodes Scheduled: 0
Number of Nodes Scheduled with Up-to-date Pods: 0
Number of Nodes Scheduled with Available Pods: 0
Number of Nodes Misscheduled: 0
Pods Status:    0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:       app=flannel
            tier=node
  Service Account:  flannel
  Containers:
   kube-flannel:
    Image:  quay.io/coreos/flannel:v0.7.0-amd64
    Port:
    Command:
      /opt/bin/flanneld
      --ip-masq
      --kube-subnet-mgr
    Environment:
      POD_NAME:      (v1:metadata.name)
      POD_NAMESPACE:     (v1:metadata.namespace)
    Mounts:
      /etc/kube-flannel/ from flannel-cfg (rw)
      /run from run (rw)
   install-cni:
    Image:  quay.io/coreos/flannel:v0.7.0-amd64
    Port:
    Command:
      /bin/sh
      -c
      set -e -x; cp -f /etc/kube-flannel/cni-conf.json /etc/cni/net.d/10-flannel.conf; while true; do sleep 3600; done
    Environment:    <none>
    Mounts:
      /etc/cni/net.d from cni (rw)
      /etc/kube-flannel/ from flannel-cfg (rw)
  Volumes:
   run:
    Type:   HostPath (bare host directory volume)
    Path:   /run
   cni:
    Type:   HostPath (bare host directory volume)
    Path:   /etc/cni/net.d
   flannel-cfg:
    Type:   ConfigMap (a volume populated by a ConfigMap)
    Name:   kube-flannel-cfg
    Optional:   false
Events:     <none>

无论如何，我试图加入其他服务器之一，只是为了看看会发生什么。 我使用kubeadm token create手动创建了一个可以从另一台机器使用的令牌。 在另一台机器上：

kubeadm join --token $TOKEN 10.0.1.101:6443
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "10.0.1.101:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.0.1.101:6443"
[discovery] Failed to request cluster info, will try again: [User "system:anonymous" cannot get configmaps in the namespace "kube-public". (get configmaps cluster-info)]
[discovery] Failed to request cluster info, will try again: [User "system:anonymous" cannot get configmaps in the namespace "kube-public". (get configmaps cluster-info)]
[discovery] Failed to request cluster info, will try again: [User "system:anonymous" cannot get configmaps in the namespace "kube-public". (get configmaps cluster-info)]

最后的信息永远重复了。

你期望发生的事情：

kubeadm init应该完成并生成引导令牌。