Kubeadm: CoreDNS is not starting on Ubuntu 18.04.Bionic Beaver

Created on 9 Jul 2018  ·  18Comments  ·  Source: kubernetes/kubeadm

What keywords did you search in kubeadm issues before filing this one?

dns, resolv.conf, coredns

BUG REPORT

Versions

kubeadm version (use kubeadm version):
kubeadm version: &version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:14:41Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}

Environment:

  • Kubernetes version (use kubectl version):
  Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
  • Cloud provider or hardware configuration:
    baremetal (Intel Xeon, 2x2TB HDD, 32GB RAM)
  • OS (e.g. from /etc/os-release):
    Ubuntu 18.04 LTS (Bionic Beaver)
  • Kernel (e.g. uname -a):
    4.15.0-24-generic

What happened?

After installation of kubernetes via kubeadm the coredns Pods won't come up. kubectl get pods --all-namespaces prints this:

NAMESPACE     NAME                                  READY     STATUS    RESTARTS   AGE
kube-system   coredns-78fcdf6894-kgg8d              0/1       Pending   0          2h
kube-system   coredns-78fcdf6894-vl9jf              0/1       Pending   0          2h
kube-system   etcd-beetlejuice                      1/1       Running   0          2h
kube-system   kube-apiserver-beetlejuice            1/1       Running   0          2h
kube-system   kube-controller-manager-beetlejuice   1/1       Running   0          2h
kube-system   kube-proxy-bjdqd                      1/1       Running   0          2h
kube-system   kube-scheduler-beetlejuice            1/1       Running   0          2h

What you expected to happen?

coredns changes into status 'Running' and kubernetes is running without problems

How to reproduce it (as minimally and precisely as possible)?

This is the script I used for installation:

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF

apt-get update
apt-get install -y docker.io
apt-get install -y kubeadm

kubeadm init --pod-network-cidr=10.27.0.0/16

Anything else we need to know?

I think yes, but I don't know what... If you need some kind of logs, just tell me.

kinbug prioritawaiting-more-evidence prioritimportant-soon
Source
picture sergej2705

Most helpful comment

too bad that there are no logs from the scheduler.

The contents of /var/lib/kubelet/kubeadm-flags.env are

the --resolv-conf flag seems to be added, so that's OK.

By now it will be a single node kube.

call kubeadm reset and then kubeadm init ... again.
then copy the config to your user dir and try installing a pod network plugin (weave):

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

and see if the dns pods ready up. this might not make sense, but please give it a try.

picture neolit123 on 9 Jul 2018
👍25 ❤9

All 18 comments

@sergej2705
the output from:
kubectl describe pod <coredns-pod-ids>

and relevant errors from:

'systemctl status kubelet'
'journalctl -xeu kubelet'

would help more.
thanks.

neolit123 picture neolit123 on 9 Jul 2018
👍1

From kubectl describe pod coredns-78fcdf6894-kgg8d -n kube-system:

Name:           coredns-78fcdf6894-kgg8d
Namespace:      kube-system
Node:           <none>
Labels:         k8s-app=kube-dns
                pod-template-hash=3497892450
Annotations:    <none>
Status:         Pending
IP:             
Controlled By:  ReplicaSet/coredns-78fcdf6894
Containers:
coredns:
    Image:       k8s.gcr.io/coredns:1.1.3
    Ports:       53/UDP, 53/TCP, 9153/TCP
    Host Ports:  0/UDP, 0/TCP, 0/TCP
    Args:
    -conf
    /etc/coredns/Corefile
    Limits:
    memory:  170Mi
    Requests:
    cpu:        100m
    memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Environment:  <none>
    Mounts:
    /etc/coredns from config-volume (ro)
    /var/run/secrets/kubernetes.io/serviceaccount from coredns-token-4fqm7 (ro)
Conditions:
Type           Status
PodScheduled   False 
Volumes:
config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      coredns
    Optional:  false
coredns-token-4fqm7:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  coredns-token-4fqm7
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     CriticalAddonsOnly
                node-role.kubernetes.io/master:NoSchedule
                node.kubernetes.io/not-ready:NoExecute for 300s
                node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type     Reason            Age                 From               Message
----     ------            ----                ----               -------
Warning  FailedScheduling  4m (x1436 over 4h)  default-scheduler  0/1 nodes are available: 1 node(s) were not ready.

and from kubectl describe pod coredns-78fcdf6894-vl9jf -n kube-system:

Name:           coredns-78fcdf6894-vl9jf
Namespace:      kube-system
Node:           <none>
Labels:         k8s-app=kube-dns
                pod-template-hash=3497892450
Annotations:    <none>
Status:         Pending
IP:             
Controlled By:  ReplicaSet/coredns-78fcdf6894
Containers:
coredns:
    Image:       k8s.gcr.io/coredns:1.1.3
    Ports:       53/UDP, 53/TCP, 9153/TCP
    Host Ports:  0/UDP, 0/TCP, 0/TCP
    Args:
    -conf
    /etc/coredns/Corefile
    Limits:
    memory:  170Mi
    Requests:
    cpu:        100m
    memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Environment:  <none>
    Mounts:
    /etc/coredns from config-volume (ro)
    /var/run/secrets/kubernetes.io/serviceaccount from coredns-token-4fqm7 (ro)
Conditions:
Type           Status
PodScheduled   False 
Volumes:
config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      coredns
    Optional:  false
coredns-token-4fqm7:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  coredns-token-4fqm7
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     CriticalAddonsOnly
                node-role.kubernetes.io/master:NoSchedule
                node.kubernetes.io/not-ready:NoExecute for 300s
                node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type     Reason            Age                 From               Message
----     ------            ----                ----               -------
Warning  FailedScheduling  1m (x1467 over 4h)  default-scheduler  0/1 nodes are available: 1 node(s) were not ready.

I don't see any errors in systemctl status kubelet, here is the complete output:

● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
        └─10-kubeadm.conf, override.conf
Active: active (running) since Mon 2018-07-09 17:43:53 CEST; 4h 7min ago
    Docs: http://kubernetes.io/docs/
Main PID: 26710 (kubelet)
    Tasks: 32 (limit: 4915)
CGroup: /system.slice/kubelet.service
        └─26710 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-co

in the journalctl -xeu kubelet I see follwing lines multiple times:

Jul 09 21:54:48 beetlejuice kubelet[26710]: E0709 21:54:48.883071   26710 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninit
Jul 09 21:54:49 beetlejuice kubelet[26710]: E0709 21:54:49.566069   26710 dns.go:131] Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 213.133.99.99 213.133.98.98 213.133.100.100
Jul 09 21:54:53 beetlejuice kubelet[26710]: W0709 21:54:53.884846   26710 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d

fyi, the /etc/resolv.conf:

### Hetzner Online GmbH installimage
# nameserver config
nameserver 213.133.99.99
nameserver 213.133.98.98
nameserver 213.133.100.100
nameserver 2a01:4f8:0:1::add:1010
nameserver 2a01:4f8:0:1::add:9999
nameserver 2a01:4f8:0:1::add:9898

where Hetzner is the name of the data center operator.

sergej2705 picture sergej2705 on 9 Jul 2018

@sergej2705
the scheduler fails but it's unclear to me why.

you can find the ID of the kube-scheduler pod with
kubectl get pods --all-namespace

then you could try this to start a terminal in that pod:
kubectl exec -ti <POD-ID-HERE> bash -n kube-system

and from there you could try looking at the log:
cat /var/log

no guaranties that it has something relevant.

neolit123 picture neolit123 on 9 Jul 2018

a couple of other things:

  • are you installing a pod-network after init (asking just in case)?
  • after init what contents do you get in /var/lib/kubelet/kubeadm-flags.env?
neolit123 picture neolit123 on 9 Jul 2018

kubectl exec -ti kube-scheduler-beetlejuice bash -n kube-system says

OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"bash\": executable file not found in $PATH": unknown command terminated with exit code 126

so i tried kubectl exec -ti kube-scheduler-beetlejuice sh -n kube-system so sh instead of bash... but there is no file /var/log

ls -lAh /var:

drwxr-xr-x    3 root     root        4.0K May 22 17:00 spool
drwxr-xr-x    2 root     root        4.0K May 22 17:00 www

By now it will be a single node kube.

The contents of /var/lib/kubelet/kubeadm-flags.env are

KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni --resolv-conf=/run/systemd/resolve/resolv.conf
sergej2705 picture sergej2705 on 9 Jul 2018

too bad that there are no logs from the scheduler.

The contents of /var/lib/kubelet/kubeadm-flags.env are

the --resolv-conf flag seems to be added, so that's OK.

By now it will be a single node kube.

call kubeadm reset and then kubeadm init ... again.
then copy the config to your user dir and try installing a pod network plugin (weave):

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

and see if the dns pods ready up. this might not make sense, but please give it a try.

neolit123 picture neolit123 on 9 Jul 2018
👍25 ❤9

after fresh initialization pods look like this (again):

NAMESPACE     NAME                                  READY     STATUS    RESTARTS   AGE
kube-system   coredns-78fcdf6894-lcmg6              0/1       Pending   0          1m
kube-system   coredns-78fcdf6894-wd9nt              0/1       Pending   0          1m
kube-system   etcd-beetlejuice                      1/1       Running   0          18s
kube-system   kube-apiserver-beetlejuice            1/1       Running   0          36s
kube-system   kube-controller-manager-beetlejuice   1/1       Running   0          12s
kube-system   kube-proxy-zrhgj                      1/1       Running   0          1m
kube-system   kube-scheduler-beetlejuice            1/1       Running   0          24s

after installing the weave plugin it looks like this:

NAMESPACE     NAME                                  READY     STATUS    RESTARTS   AGE
kube-system   coredns-78fcdf6894-lcmg6              1/1       Running   0          2m
kube-system   coredns-78fcdf6894-wd9nt              1/1       Running   0          2m
kube-system   etcd-beetlejuice                      1/1       Running   0          1m
kube-system   kube-apiserver-beetlejuice            1/1       Running   0          1m
kube-system   kube-controller-manager-beetlejuice   1/1       Running   0          58s
kube-system   kube-proxy-zrhgj                      1/1       Running   0          2m
kube-system   kube-scheduler-beetlejuice            1/1       Running   0          1m
kube-system   weave-net-ldxg5                       2/2       Running   0          24s

now it works 👍

I installed kubernetes-dashboard now without problems, this wasn't working before. But is this the solution for this problem?

sergej2705 picture sergej2705 on 10 Jul 2018

thank you for testing.

But is this the solution for this problem?

i would say - yes, for now.

both the CLI and the documentation instruct the users to install a pod-network plugin right after init.
we haven't documented exactly what happens is this step is skipped, but it is expected that the cluster will not work properly...

if someone feels like this issue should not be closed, please re-open.

neolit123 picture neolit123 on 10 Jul 2018

I have the exact problem, although the weave plugin didn't do the trick. I still see coredns pods at ContainerCreating status. It's been almost an hour now, so ...

drpaneas picture drpaneas on 14 Aug 2018 
linux-uwkw:~ # kubectl cluster-info
Kubernetes master is running at https://192.168.178.163:6443
KubeDNS is running at https://192.168.178.163:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

linux-uwkw:~ # cat /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni
  • the --resolv-conf flag is missing. So I guess this is a problem. How I can configure kubeqdm to add it?
  • the --cni-bin-dir=/opt/cni/bin is wrong. In my system I don't have this.
linux-uwkw:~ # rpm -ql cni
/etc/cni
/etc/cni/net.d
/etc/cni/net.d/99-loopback.conf.sample
/usr/lib/cni
/usr/lib/cni/noop
/usr/sbin/cnitool
/usr/share/doc/packages/cni
/usr/share/doc/packages/cni/CONTRIBUTING.md
/usr/share/doc/packages/cni/DCO
/usr/share/doc/packages/cni/README.md
/usr/share/licenses/cni
/usr/share/licenses/cni/LICENSE

I guess I have to put there /usr/sbin, right?

Also the logs from the scheduler are:

linux-uwkw:~ # docker logs k8s_kube-scheduler_kube-scheduler-linux-uwkw_kube-system_a00c35e56ebd0bdfcd77d53674a5d2a1_0
I0813 21:18:19.816990       1 server.go:126] Version: v1.11.2
W0813 21:18:19.821719       1 authorization.go:47] Authorization is disabled
W0813 21:18:19.821744       1 authentication.go:55] Authentication is disabled
I0813 21:18:19.821787       1 insecure_serving.go:47] Serving healthz insecurely on 127.0.0.1:10251
E0813 21:18:25.603025       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0813 21:18:25.603122       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0813 21:18:25.603161       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0813 21:18:25.603253       1 reflector.go:205] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:176: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0813 21:18:25.603286       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list storageclasses.storage.k8s.io at the cluster scope
E0813 21:18:25.603335       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0813 21:18:25.603364       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0813 21:18:25.603437       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0813 21:18:25.603491       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0813 21:18:25.605642       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
E0813 21:18:26.603723       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list persistentvolumes at the cluster scope
E0813 21:18:26.606225       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list services at the cluster scope
E0813 21:18:26.606295       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list replicationcontrollers at the cluster scope
E0813 21:18:26.607860       1 reflector.go:205] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:176: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list pods at the cluster scope
E0813 21:18:26.611457       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list storageclasses.storage.k8s.io at the cluster scope
E0813 21:18:26.612777       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list persistentvolumeclaims at the cluster scope
E0813 21:18:26.616076       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1beta1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list statefulsets.apps at the cluster scope
E0813 21:18:26.616779       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list poddisruptionbudgets.policy at the cluster scope
E0813 21:18:26.619308       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1beta1.ReplicaSet: replicasets.extensions is forbidden: User "system:kube-scheduler" cannot list replicasets.extensions at the cluster scope
E0813 21:18:26.620048       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:130: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list nodes at the cluster scope
I0813 21:18:28.429769       1 controller_utils.go:1025] Waiting for caches to sync for scheduler controller
I0813 21:18:28.533687       1 controller_utils.go:1032] Caches are synced for scheduler controller
I0813 21:18:28.533868       1 leaderelection.go:185] attempting to acquire leader lease  kube-system/kube-scheduler...
I0813 21:18:28.539621       1 leaderelection.go:194] successfully acquired lease kube-system/kube-scheduler
drpaneas picture drpaneas on 14 Aug 2018

I have the exact problem

i don't think it is the same problem.

KubeDNS is running at

CoreDNS is the default dns server in 1.11.x. did you enable that intentionally?

cat /var/lib/kubelet/kubeadm-flags.env

/var/lib/kubelet/kubeadm-flags.env is auto-generated on kubeadm runtime and should not be edited.
you can add flags in /etc/default/kubelet

see the info here:
https://github.com/kubernetes/kubernetes/blob/master/build/debs/10-kubeadm.conf

the --resolv-conf flag is missing.

it would be only added if your distro uses this:
https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html

--cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d

these are the default values, but AFAIK this should not be auto-added at runtime.

hard to say what the problem is in your case.
better open up a new issue and follow the issue report template.

neolit123 picture neolit123 on 14 Aug 2018

I found my problem after all. I was missing the loopback binary of CNI at /opt/cni/bin.

cd /opt/cni/bin
curl -L -O https://github.com/containernetworking/cni/releases/download/v0.4.0/cni-amd64-v0.4.0.tgz
tar -xf cni-amd64-v0.4.0.tgz
systemctl restart kubelet
drpaneas picture drpaneas on 14 Aug 2018

I met the same problem and solved it by installing flannel. After installing flannel, coredns and other pods can be created correctly.

Why installing flannel can solve the problem?

I also init my K8s cluster using kubeadm and add the --pod-network-cidrarg. So I think if flannel or other pod network add-ons are not installed, kubelet can not know how to assign ip to the pod and throw error.

You said that you see the following message using journal -xeu kubelet:

26710 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninit
Jul 09 21:54:49 beetlejuice kubelet[26710]: E0709 21:54:49.566069   26710 dns.go:131] Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 213.133.99.99 213.133.98.98 213.133.100.100
Jul 09 21:54:53 beetlejuice kubelet[26710]: W0709 21:54:53.884846   26710 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d

I also saw similar logs on my machine and so I think this is the cause of the problem.

Yusen1997 picture Yusen1997 on 15 Aug 2018

Why installing flannel can solve the problem?

a CNI plugin must be installed:
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm
Installing a pod network add-on
You must install a pod network add-on so that your pods can communicate with each other.

neolit123 picture neolit123 on 15 Aug 2018

a CNI plugin must be installed:
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm

My coredns can not start because a pod network add-on is not installed after initing the kubernetes and now I know it, thank you.

Yusen1997 picture Yusen1997 on 16 Aug 2018

too bad that there are no logs from the scheduler.

The contents of /var/lib/kubelet/kubeadm-flags.env are

the --resolv-conf flag seems to be added, so that's OK.

By now it will be a single node kube.

call kubeadm reset and then kubeadm init ... again.
then copy the config to your user dir and try installing a pod network plugin (weave):

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

and see if the dns pods ready up. this might not make sense, but please give it a try.

it works for me! thanks a lot

yixblog picture yixblog on 13 May 2019

too bad that there are no logs from the scheduler.

The contents of /var/lib/kubelet/kubeadm-flags.env are

the --resolv-conf flag seems to be added, so that's OK.

By now it will be a single node kube.

call kubeadm reset and then kubeadm init ... again.
then copy the config to your user dir and try installing a pod network plugin (weave):

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

and see if the dns pods ready up. this might not make sense, but please give it a try.

serviceaccount/weave-net configured
clusterrole.rbac.authorization.k8s.io/weave-net configured
clusterrolebinding.rbac.authorization.k8s.io/weave-net configured
role.rbac.authorization.k8s.io/weave-net configured
rolebinding.rbac.authorization.k8s.io/weave-net configured
unable to recognize no matches for kind "DaemonSet" in version "extensions/v1beta1"

Unable to configure

sampleref picture sampleref on 23 Sep 2019

unable to recognize no matches for kind "DaemonSet" in version "extensions/v1beta1"

that's a bug on the CNI plugin side.
try the Callico CNI plugin instead.

neolit123 picture neolit123 on 24 Sep 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mattmoyer picture mattmoyer  ·  4Comments
chuckha picture chuckha  ·  3Comments
luxas picture luxas  ·  3Comments
ep4eg picture ep4eg  ·  3Comments
shenshouer picture shenshouer  ·  3Comments