RELATÓRIO DE ERRO
versão do kubeadm (use kubeadm version
): "v1.12.0-alpha.0.957 + 1235adac3802fd-dirty"
Criei um nó de plano de controle com kubeadm init
. Eu executei kubeadm join
em um nó separado e recebi esta mensagem de erro:
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
configmaps "kubelet-config-1.12" is forbidden: User "system:bootstrap:4ipkdk" cannot get configmaps in the namespace "kube-system"
Eu esperava que kubeadm join
terminasse com sucesso
Pelo que eu posso dizer, execute kubeadm init
e kubeadm join
em outro nó. Eu tenho muito código / yaml extra que não deve estar influenciando o mapa de configuração (necessário para a implantação do happy aws). Mas se isso não puder ser reproduzido, fornecerei instruções mais detalhadas.
Também acho que kubeadm join
e kubeadm init
estão nomeando os mapas de configuração de forma inconsistente. O comando init
usa o kubernetesVersion
especificado no arquivo de configuração e o comando join
usa a versão kubelet para o nome do mapa de configuração (por exemplo, kubelet-config-1.1). Isso é bom, a menos que você tenha versões incompatíveis, então não é bom.
O comando init
cria regras RBAC para acesso anônimo a mapas de configuração no namespace kube-public
mas não parece colocar a configuração do kubelet no namespace público e, portanto, a junção do nó não tem acesso a isto.
Isso é bom, a menos que você tenha versões incompatíveis, então não é bom.
@chuckha Se bem me lembro kubelet-*
mapas de configuração devem ser kube-system
e kubeadm cria regras para permitir o acesso ao token de bootstrap e nós. Mas vou verificar isso novamente após a última rodada de mudanças
@chuckha Concluí a junção com sucesso em um cluster com todos os componentes construídos a partir do mestre + número da versão forçados para v1.11.0
kubelet-config-1.11
foi criado em kube-system
kubeadm:kubelet-config-1.11
foi criada em kube-system
com obter permissão no mapa de configuraçãokubeadm:kubelet-config-1.11
foi criado em kube-system
para system:nodes
e system:bootstrappers:kubeadm:default-node-token
Então, IMO:
A parte que ainda falta investigar é
O comando init usa o kubernetesVersion especificado no arquivo de configuração e o comando join usa a versão do kubelet para o nome do mapa de configuração (por exemplo, kubelet-config-1.1)
Sim, deve ser um problema de versão.
Não configurei a versão quando criei, então todos os binários pensam que são 1.12.0+, mas instalei e forcei o kubeadm a usar v1.11.
Isso resultou em
root@ip-10-0-0-7:~# k get cm -n kube-system
NAME DATA AGE
...
kubelet-config-1.11 1 32m
e então ao entrar:
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
configmaps "kubelet-config-1.12" is forbidden: User "system:bootstrap:u3ns6m" cannot get configmaps in the namespace "kube-system"
Vou reconstruir os binários e forçá-los para a versão correta e tentar novamente.
Tudo está bem quando as versões do kubelet e do kubeadm coincidem. Fechando em favor de uma correção menos urgente em relação à inconsistência (talvez intencional) entre escrever o configmap e buscar o configmap.
Não estou usando o kubeadm init, mas chamando as fases individualmente. Não há configmap no kube-system nem permissões para configurá-lo adequadamente. Que fase é essa?
@drewwells Eu encontrei o mesmo problema que você. Estou executando as fases individualmente e não há mpas de configuração.
sudo kubectl get cm -n kube-system --kubeconfig=/etc/kubernetes/admin.conf
NAME DATA AGE
calico-config 2 11m
coredns 1 15m
extension-apiserver-authentication 6 15m
kube-proxy 2 15m
Você encontrou a solução?
Devo também mencionar, todos os componentes são 1.11.4
Isso vai além. Eu inicializo um cluster com kubeadm init e tenho o mapa de configuração correto no lugar agora:
ubuntu@master-1-test2:~$ sudo kubectl get cm -n kube-system --kubeconfig=/etc/kubernetes/admin.conf
NAME DATA AGE
coredns 1 41m
extension-apiserver-authentication 6 41m
kube-proxy 2 41m
kubeadm-config 1 41m
kubelet-config-1.11 1 41m
ubuntu@master-1-test2:~$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.4", GitCommit:"bf9a868e8ea3d3a8fa53cbb22f566771b3f8068b", GitTreeState:"clean", BuildDate:"2018-10-25T19:13:39Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
No nó:
$ sudo kubeadm -v=10 join k8s.oz.noris.de:6443 --token 36etul.nv5lz6hjfifdd4c9 --discovery-token-unsafe-skip-ca-verification I1107 12:57:01.340638 3631 join.go:226] [join] found NodeName empty
I1107 12:57:01.340816 3631 join.go:227] [join] considered OS hostname as NodeName
[preflight] running pre-flight checks
I1107 12:57:01.341152 3631 join.go:238] [preflight] running various checks on all nodes
I1107 12:57:01.341265 3631 checks.go:253] validating the existence and emptiness of directory /etc/kubernetes/manifests
I1107 12:57:01.341677 3631 checks.go:291] validating the existence of file /etc/kubernetes/pki/ca.crt
I1107 12:57:01.341774 3631 checks.go:291] validating the existence of file /etc/kubernetes/kubelet.conf
I1107 12:57:01.341857 3631 checks.go:291] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I1107 12:57:01.341947 3631 kernelcheck_linux.go:45] validating the kernel module IPVS required exists in machine or not
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs_sh ip_vs ip_vs_rr ip_vs_wrr] or no builtin kernel ipvs support: map[ip_vs:{} ip_
vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support
I1107 12:57:01.349448 3631 checks.go:138] validating if the service is enabled and active
I1107 12:57:01.361957 3631 checks.go:340] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I1107 12:57:01.362034 3631 checks.go:340] validating the contents of file /proc/sys/net/ipv4/ip_forward
I1107 12:57:01.362076 3631 checks.go:653] validating whether swap is enabled or not
I1107 12:57:01.362134 3631 checks.go:381] validating the presence of executable crictl
I1107 12:57:01.362204 3631 checks.go:381] validating the presence of executable ip
I1107 12:57:01.362244 3631 checks.go:381] validating the presence of executable iptables
I1107 12:57:01.362281 3631 checks.go:381] validating the presence of executable mount
I1107 12:57:01.362320 3631 checks.go:381] validating the presence of executable nsenter
...
[discovery] Trying to connect to API Server "mycluster.example.com:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://mycluster.example.com:6443"
I1107 12:57:01.487256 3631 round_trippers.go:386] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.11.4 (linux/amd64) kubernetes/bf9a868" 'https://mycluster.example.com:6443/api/v1/namespaces/kube-public/config
maps/cluster-info'
I1107 12:57:01.504539 3631 round_trippers.go:405] GET https://mycluster.example.com:6443/api/v1/namespaces/kube-public/configmaps/cluster-info 200 OK in 17 milliseconds
I1107 12:57:01.504720 3631 round_trippers.go:411] Response Headers:
I1107 12:57:01.504818 3631 round_trippers.go:414] Content-Type: application/json
I1107 12:57:01.504914 3631 round_trippers.go:414] Content-Length: 2217
I1107 12:57:01.505003 3631 round_trippers.go:414] Date: Wed, 07 Nov 2018 12:57:01 GMT
I1107 12:57:01.505174 3631 request.go:942] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","selfLink":"/api/v1/namespaces/kube-public/configmaps/cluster-info","uid":"97e9
a3d1-e286-11e8-9272-fa163ef9e3af","resourceVersion":"425","creationTimestamp":"2018-11-07T12:13:53Z"},"data":{"jws-kubeconfig-36etul":"eyJhbGciOiJIUzI1NiIsImtpZCI6IjM2ZXR1bCJ9..zRgexonkjOpLJS0q3IignURwTcpBuQy7gv35Qhhsl_k","jws-kubeconfig-
eth6o8":"eyJhbGciOiJIUzI1NiIsImtpZCI6ImV0aDZvOCJ9..kWj4cI2j1WgKfNG07IGiIij4CSb9kWUbaM2mixlYThY","jws-kubeconfig-rbxd02":"eyJhbGciOiJIUzI1NiIsImtpZCI6InJieGQwMiJ9..HwIWDwfIbAjNM1EGbWdXYOhC8z1MxgwuzhjlJRaZ_pc","kubeconfig":"apiVersion: v1\n
clusters:\n- cluster:\n certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNE1URXdNekEwTlRrME4xb1hEVEk0TVRBek1UQTB
OVGswTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTU9BCkNHQU5jUjVRQWV3MlljY2V0eWVyYktiODd4RWRPVlp2aUdneElrbkpKTTZwZFVBbzMwSWVxckRqSnlFaTFVeDcKU0c5NS9sRlBqU1htdHhhNHMvc1g1KzNTVW4zZ
EtFRWw5TFhXa0lzeTRJYzRFUTMwWE9WcnNuYTYwN1UzNmQyaAp3NHdTK1dveE5QR3dqZDM2bXQzMFR4bUluYk54ZVl5d2NnVU1tMlZFZXM4dGhVaVhZMXB1N1Y2SUNCY243cE9NCkdoT2xlRXg4SmlEVnhuSGlpSm9oYytCbGNIdHdLU1pzK2cvZUhwdGdlSDdaQlZNRC8zZVFvZXVsUGVvTEkwamEKc09jTENMTkpEVVB
LUWJqRnRNbkFZSXVvOENHSXpFTzBDaDZNeW5vb1pTL0E0bEs1MXJmTVdkTkZ4N0dVdnQxYQo5KzZzMHo2NEpHeVFBdmtBcWhVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEbWs1NEYzZ1BqOS91NzlRbTg2V1Mzc
k5YaFoKZG16Wmt3TXRDajRuTXdsSndGQy9iZUU4ZUdsWnFxWDcrdEpYUDVaY0xLNE1pSnM1U2JTMjd5NDF3WTRRTFFWaQpVWmRocEFHUTBOSlpHSGhWMTVDczlVQTA1ZTFNajNCaHZ6SG5VV2t1ZUhYbW84VmI4SkI5RGloeGdiUW5GY2FQCjRWcVhWY0pBemxVQ0V5aXhreVRGendZTklJbzJHdGtCdlI1YkxCM0doT2R
sQURmQzEwdzgvTmQveFFmRnRWdmYKL3lHaktpbW8rT2xERkV5YittcHVKMVdiN3Y3bnJJSzlSSy9WbVhUWENiOWZLQ3BmQ0hMU0hpa0lEWklZK0wxTQpwbWpXYXZFcjFLSlE5UEJIYmdZSHkxK1F0bkpXRDNjNnJrOUtoNU1zMFhTVmpBc2Z1RWdXaG9CYnlVdz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\n
server: https://k8s.oz.noris.de:6443\n name: \"\"\ncontexts: []\ncurrent-context: \"\"\nkind: Config\npreferences: {}\nusers: []\n"}}
[discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "k8s.oz.noris.de:6443"
[discovery] Successfully established connection with API Server "mycluster.example.com:6443"
I1107 12:57:01.509945 3631 join.go:260] [join] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I1107 12:57:01.617006 3631 loader.go:359] Config loaded from file /etc/kubernetes/bootstrap-kubelet.conf
I1107 12:57:01.617871 3631 join.go:283] Stopping the kubelet
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
I1107 12:57:01.627838 3631 round_trippers.go:386] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.11.4 (linux/amd64) kubernetes/bf9a868" -H "Authorization: Bearer 36etul.nv5lz6hjfifdd4c9" 'https://mycluster.example.com:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config-1.12'
I1107 12:57:01.639396 3631 round_trippers.go:405] GET https://mycluster.example.com:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config-1.12 403 Forbidden in 11 milliseconds
I1107 12:57:01.639427 3631 round_trippers.go:411] Response Headers:
I1107 12:57:01.639443 3631 round_trippers.go:414] Content-Length: 311
I1107 12:57:01.639464 3631 round_trippers.go:414] Date: Wed, 07 Nov 2018 12:57:01 GMT
I1107 12:57:01.639477 3631 round_trippers.go:414] Content-Type: application/json
I1107 12:57:01.639492 3631 round_trippers.go:414] X-Content-Type-Options: nosniff
I1107 12:57:01.639525 3631 request.go:942] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"kubelet-config-1.12\" is forbidden: User \"system:bootstrap:36etul\" cannot get confi
gmaps in the namespace \"kube-system\"","reason":"Forbidden","details":{"name":"kubelet-config-1.12","kind":"configmaps"},"code":403}
configmaps "kubelet-config-1.12" is forbidden: User "system:bootstrap:36etul" cannot get configmaps in the namespace "kube-system"
notou o material estranho?
Aparentemente, isso tudo faz desde então, se você verificar a versão do kubelet! Tive a versão 1.12.2 do kubelet.
Eu peguei a dica do código
Estou tendo o mesmo problema, exceto uma versão mais recente.
No mestre:
$ lsb_release -d
Description: Ubuntu 16.04.5 LTS
$ dpkg -l | grep kub
ii kubeadm 1.12.1-00 amd64 Kubernetes Cluster Bootstrapping Tool
ii kubectl 1.12.1-00 amd64 Kubernetes Command Line Tool
ii kubelet 1.12.1-00 amd64 Kubernetes Node Agent
ii kubernetes-cni 0.6.0-00 amd64 Kubernetes CNI
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T20:56:12Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
De onde veio a versão 1.13.0 do servidor? Eu não instalei isso.
De qualquer forma, antes de executar o kubeadm init nesta VM, eu o clonei, então tenho outro pronto para ser outro nó no cluster. Por ser um clone, também possui 1.12.1. Quando tento entrar:
$ kubeadm join --token blahblah 10.138.0.3:6443 --discovery-token-ca-cert-hash sha256:deadbeefdeadbeefetc
[preflight] running pre-flight checks
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh] or no builtin kernel ipvs support: map[ip_vs_sh:{} nf_conntrack_ipv4:{} ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support
[discovery] Trying to connect to API Server "10.138.0.3:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.138.0.3:6443"
[discovery] Requesting info from "https://10.138.0.3:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will
use API Server "10.138.0.3:6443"
[discovery] Successfully established connection with API Server "10.138.0.3:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace configmaps "kubelet-config-1.12" is forbidden: User "system:bootstrap:3ai26q" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
E por que não pode obter o kubelet-config-1.12 configmap? Porque não existe um. De volta ao mestre:
$ sudo kubectl get cm -n kube-system --kubeconfig=/etc/kubernetes/admin.conf
NAME DATA AGE
calico-config 4 26m
coredns 1 29m
extension-apiserver-authentication 6 29m
kube-proxy 2 29m
kubeadm-config 2 29m
kubelet-config-1.13 1 29m
@brianriceca : enfrentando exatamente o mesmo problema que você ... qualquer solução para isso ..
Mestre: ram @ k8master1 : ~ $ kubeadm versão
kubeadm version: & version.Info {Major: "1", Minor: "12", GitVersion: "v1.12.1", GitCommit: "4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState: "clean", BuildDate: "2018-10T16: 43-05T16 08Z ", GoVersion:" go1.10.4 ", Compilador:" gc ", Plataforma:" linux / amd64 "}
ram @ k8master1 : ~ $ dpkg -l | grep kub
ii kubeadm 1.12.1-00 amd64 Ferramenta de inicialização de cluster do Kubernetes
ii kubectl 1.12.1-00 amd64 Ferramenta de linha de comando do Kubernetes
ii kubelet 1.12.1-00 amd64 Agente de nó do Kubernetes
ii kubernetes-cni 0.6.0-00 amd64 Kubernetes CNI
ram @ k8master1 : ~ $ kubectl version
Versão do cliente: version.Info {Principal: "1", Secundária: "12", GitVersion: "v1.12.1", GitCommit: "4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState: "clean", BuildDate: "2018-10-05T16: 46: 06Z ", GoVersion:" go1.10.4 ", Compilador:" gc ", Plataforma:" linux / amd64 "}
Versão do servidor: version.Info {Major: "1", Minor: "13", GitVersion: "v1.13.0", GitCommit: "ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState: "clean", BuildDate: "2018-12-03T20: 56: 12Z ", GoVersion:" go1.11.2 ", Compilador:" gc ", Plataforma:" linux / amd64 "}
ram @ k8master1 : ~ $ kubectl get node
NOME STATUS ROLES IDADE VERSÃO
k8master1.example.com Ready master 101m v1.12.1
ram @ k8master1 : ~ $ kubectl get pods --all-namespaces
NAMESPACE NOME PRONTO STATUS RESTARTS IDADE
kube-system calico-node-r248v 2/2 Running 0 99m
kube-system coredns-869f847d58-72lqd 1/1 Executando 0 101m
kube-system coredns-869f847d58-p2zzs 1/1 Executando 0 101m
kube-system etcd-k8master1.example.com 1/1 Executando 0 100m
kube-system kube-apiserver-k8master1.example.com 1/1 Executando 0 100m
kube-system kube-controller-manager-k8master1.example.com 1/1 Executando 0 100m
kube-system kube-proxy-77qbx 1/1 Executando 0 101m
kube-system kube-scheduler-k8master1.example.com 1/1 Executando 0 100m
Nós de trabalho:
root @ k8worker1 : ~ # dpkg -l | grep -i kub
ii kubeadm 1.12.1-00 amd64 Ferramenta de inicialização de cluster do Kubernetes
ii kubectl 1.12.1-00 amd64 Ferramenta de linha de comando do Kubernetes
ii kubelet 1.12.1-00 amd64 Agente de nó do Kubernetes
ii kubernetes-cni 0.6.0-00 amd64 Kubernetes CNI
root @ k8worker1 : ~ # kubectl version
Versão do cliente: version.Info {Principal: "1", Secundária: "12", GitVersion: "v1.12.1", GitCommit: "4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState: "clean", BuildDate: "2018-10-05T16: 46: 06Z ", GoVersion:" go1.10.4 ", Compilador:" gc ", Plataforma:" linux / amd64 "}
A conexão com o servidor localhost: 8080 foi recusada - você especificou o host ou a porta correta?
root @ k8worker1 : ~ # kubeadm join 10.0.0.61:6443 --token xjxgqa.h2vnld3x9ztgf3pr --discovery-token-ca-cert-hash sha256: 7c18b654baf623ee84164bb0dfa7940f3pr --discovery-token-ca-cert-hash sha256: 7c18b654baf623ee84164bb0dfa79409c82139846e1a968843434
[preflight] executando verificações pré-voo
[AVISO RequiredIPVSKernelModulesAvailable]: o proxy IPVS não será usado, porque os seguintes módulos do kernel necessários não são carregados: [ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh] ou nenhum kernel integrado com suporte ipvs: map [nf_conntrack_ipv4: {} ip_vs} ip_vs_wrr: {} ip_vs_sh: {}]
você pode resolver esse problema com os seguintes métodos:
[descoberta] Tentando conectar ao servidor API "10.0.0.61:6443"
[descoberta] Cliente de descoberta de informações de cluster criado, solicitando informações de " https://10.0.0.61 : 6443"
[descoberta] Solicitando informações de " https://10.0.0.61 : 6443" novamente para validar o TLS em relação à chave pública fixada
[descoberta] A assinatura e o conteúdo das informações do cluster são válidos e o certificado TLS é validado em relação às raízes fixadas; usará o servidor API "10.0.0.61:6443"
[descoberta] Conexão estabelecida com sucesso com o servidor API "10.0.0.61:6443"
[kubelet] Baixando a configuração do kubelet do ConfigMap "kubelet-config-1.12" no namespace do sistema kube
configmaps "kubelet-config-1.12" é proibido: O usuário " system: bootstrap : xjxgqa" não pode obter o recurso "configmaps" no grupo de API "" no namespace "kube-system"
root @ k8worker1 : ~ # kubeadm join 10.0.0.61:6443 --token xjxgqa.h2vnld3x9ztgf3pr --discovery-token-ca-cert-hash sha256: 7c18b654baf623ee84164bb0dfa7940f3pr --discovery-token-ca-cert-hash sha256: 7c18b654baf623ee84164bb0dfa7940f572139846e0ad3434
[preflight] executando verificações pré-voo
[AVISO RequiredIPVSKernelModulesAvailable]: o proxy IPVS não será usado, porque os seguintes módulos do kernel necessários não são carregados: [ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh] ou nenhum kernel integrado suporte ipvs: map [ip_vs_rr: {} ip_vs_wrr: {} ip_vs_wrr: {} nf_conntrack_ipv4: {} ip_vs: {}]
você pode resolver esse problema com os seguintes métodos:
[preflight] Ocorreram alguns erros fatais:
[ERROR FileAvailable - etc-kubernetes-bootstrap-kubelet.conf]: /etc/kubernetes/bootstrap-kubelet.conf já existe
[ERROR FileAvailable - etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt já existe
[preflight] Se você sabe o que está fazendo, pode tornar um cheque não fatal com --ignore-preflight-errors=...
igual a @brianriceca e @ramanjk. As versões do kubadm e do kubelet são 1.12.1 em ambos os nós, o kubelet-config-1.13
configmap está lá e também está vendo configmaps "kubelet-config-1.12" is forbidden
D'oh! O que eu ainda não sabia é que o kubeadm sempre baixa a versão mais recente do plano de controle do Kubernetes do gcr.io, a menos que você diga o contrário. Portanto, se eu quiser instalar o 1.12.1 mesmo quando o 1.13 estiver disponível, preciso fazer
kubeadm init --kubernetes-version 1.12.1 --pod-network-cidr
_qualquer / qualquer_qualquer coisa_
Excluí ambos os nós e tentei novamente usando a versão 1.12.2
de tudo, e o problema não ocorreu desta vez. Houve alguma saída sobre a existência de uma versão mais recente, mas que será revertida para 1.12 (eu perdi agora)
Estou sofrendo do mesmo bug, embora (aparentemente) usando a mesma versão no semeador e no nó.
No semeador (após a inicialização):
$ kubeadm version -o json
{
"clientVersion": {
"major": "1",
"minor": "13",
"gitVersion": "v1.13.0",
"gitCommit": "ddf47ac13c1a9483ea035a79cd7c10005ff21a6d",
"gitTreeState": "clean",
"buildDate": "2018-12-11T17:03:40Z",
"goVersion": "go1.11.2",
"compiler": "gc",
"platform": "linux/amd64"
}
}
$
$ kubectl get cm --all-namespaces
NAMESPACE NAME DATA AGE
kube-public cluster-info 2 174m
kube-system coredns 1 174m
kube-system extension-apiserver-authentication 6 174m
kube-system flannel-plugin-config-map 2 174m
kube-system kube-proxy 2 174m
kube-system kubeadm-config 2 174m
kube-system kubelet-config-1.13 1 174m
kube-system kubic-init-config-seeder 1 174m
$
$
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"archive", BuildDate:"2018-12-07T12:00:00Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T20:56:12Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
No nó:
$ cat config.txt
apiVersion: kubeadm.k8s.io/v1beta1
caCertPath: /etc/kubernetes/pki/ca.crt
discovery:
bootstrapToken:
apiServerEndpoint: 192.168.100.1:6443
token: 94dcda.c271f4ff502789ca
unsafeSkipCAVerification: true
timeout: 5m0s
tlsBootstrapToken: 94dcda.c271f4ff502789ca
kind: JoinConfiguration
nodeRegistration:
criSocket: /var/run/crio/crio.sock
kubeletExtraArgs:
cni-bin-dir: /var/lib/kubelet/cni/bin
cni-conf-dir: /etc/cni/net.d
container-runtime-endpoint: unix:///var/run/crio/crio.sock
network-plugin: cni
$
$ kubeadm join --v=8 --config=config.txt
I1220 11:55:56.879023 7 join.go:299] [join] found NodeName empty; using OS hostname as NodeName
I1220 11:55:56.880357 7 joinconfiguration.go:72] loading configuration from the given file
[preflight] Running pre-flight checks
I1220 11:55:56.890498 7 join.go:328] [preflight] Running general checks
I1220 11:55:56.891937 7 checks.go:245] validating the existence and emptiness of directory /etc/kubernetes/manifests
I1220 11:55:56.893051 7 checks.go:283] validating the existence of file /etc/kubernetes/kubelet.conf
I1220 11:55:56.894239 7 checks.go:283] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I1220 11:55:56.895384 7 checks.go:104] validating the container runtime
I1220 11:55:57.072517 7 checks.go:373] validating the presence of executable crictl
I1220 11:55:57.073553 7 checks.go:332] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I1220 11:55:57.074479 7 checks.go:332] validating the contents of file /proc/sys/net/ipv4/ip_forward
I1220 11:55:57.075518 7 checks.go:644] validating whether swap is enabled or not
I1220 11:55:57.076499 7 checks.go:373] validating the presence of executable ip
I1220 11:55:57.077424 7 checks.go:373] validating the presence of executable iptables
I1220 11:55:57.078594 7 checks.go:373] validating the presence of executable mount
I1220 11:55:57.079564 7 checks.go:373] validating the presence of executable nsenter
I1220 11:55:57.080425 7 checks.go:373] validating the presence of executable ebtables
I1220 11:55:57.081391 7 checks.go:373] validating the presence of executable ethtool
I1220 11:55:57.082170 7 checks.go:373] validating the presence of executable socat
I1220 11:55:57.084207 7 checks.go:373] validating the presence of executable tc
I1220 11:55:57.085250 7 checks.go:373] validating the presence of executable touch
I1220 11:55:57.086132 7 checks.go:515] running all checks
I1220 11:55:57.137681 7 checks.go:403] checking whether the given node name is reachable using net.LookupHost
I1220 11:55:57.150619 7 checks.go:613] validating kubelet version
I1220 11:55:57.450319 7 checks.go:130] validating if the service is enabled and active
I1220 11:55:57.554984 7 checks.go:208] validating availability of port 10250
I1220 11:55:57.556700 7 checks.go:283] validating the existence of file /etc/kubernetes/pki/ca.crt
I1220 11:55:57.557579 7 checks.go:430] validating if the connectivity type is via proxy or direct
[preflight] Some fatal errors occurred:
[ERROR FileAvailable--etc-kubernetes-bootstrap-kubelet.conf]: /etc/kubernetes/bootstrap-kubelet.conf already exists
[ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
sh-4.4# rm -f /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/pki/ca.crt
sh-4.4# kubeadm join --v=8 --config=config.txt
I1220 11:56:10.073469 30 join.go:299] [join] found NodeName empty; using OS hostname as NodeName
I1220 11:56:10.074575 30 joinconfiguration.go:72] loading configuration from the given file
[preflight] Running pre-flight checks
I1220 11:56:10.085937 30 join.go:328] [preflight] Running general checks
I1220 11:56:10.086871 30 checks.go:245] validating the existence and emptiness of directory /etc/kubernetes/manifests
I1220 11:56:10.087809 30 checks.go:283] validating the existence of file /etc/kubernetes/kubelet.conf
I1220 11:56:10.088573 30 checks.go:283] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I1220 11:56:10.089370 30 checks.go:104] validating the container runtime
I1220 11:56:10.126939 30 checks.go:373] validating the presence of executable crictl
I1220 11:56:10.128075 30 checks.go:332] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I1220 11:56:10.129096 30 checks.go:332] validating the contents of file /proc/sys/net/ipv4/ip_forward
I1220 11:56:10.129993 30 checks.go:644] validating whether swap is enabled or not
I1220 11:56:10.131006 30 checks.go:373] validating the presence of executable ip
I1220 11:56:10.132983 30 checks.go:373] validating the presence of executable iptables
I1220 11:56:10.139740 30 checks.go:373] validating the presence of executable mount
I1220 11:56:10.140267 30 checks.go:373] validating the presence of executable nsenter
I1220 11:56:10.140738 30 checks.go:373] validating the presence of executable ebtables
I1220 11:56:10.141092 30 checks.go:373] validating the presence of executable ethtool
I1220 11:56:10.141459 30 checks.go:373] validating the presence of executable socat
I1220 11:56:10.142799 30 checks.go:373] validating the presence of executable tc
I1220 11:56:10.145062 30 checks.go:373] validating the presence of executable touch
I1220 11:56:10.145954 30 checks.go:515] running all checks
I1220 11:56:10.189173 30 checks.go:403] checking whether the given node name is reachable using net.LookupHost
I1220 11:56:10.204103 30 checks.go:613] validating kubelet version
I1220 11:56:10.529594 30 checks.go:130] validating if the service is enabled and active
I1220 11:56:10.556043 30 checks.go:208] validating availability of port 10250
I1220 11:56:10.557915 30 checks.go:283] validating the existence of file /etc/kubernetes/pki/ca.crt
I1220 11:56:10.559380 30 checks.go:430] validating if the connectivity type is via proxy or direct
I1220 11:56:10.560242 30 join.go:334] [preflight] Fetching init configuration
I1220 11:56:10.561013 30 join.go:601] [join] Discovering cluster-info
[discovery] Trying to connect to API Server "192.168.100.1:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.100.1:6443"
I1220 11:56:10.567171 30 round_trippers.go:383] GET https://192.168.100.1:6443/api/v1/namespaces/kube-public/configmaps/cluster-info
I1220 11:56:10.568131 30 round_trippers.go:390] Request Headers:
I1220 11:56:10.568891 30 round_trippers.go:393] Accept: application/json, */*
I1220 11:56:10.569609 30 round_trippers.go:393] User-Agent: kubeadm/v1.13.0 (linux/amd64) kubernetes/ddf47ac
I1220 11:56:10.586461 30 round_trippers.go:408] Response Status: 200 OK in 16 milliseconds
I1220 11:56:10.587241 30 round_trippers.go:411] Response Headers:
I1220 11:56:10.588006 30 round_trippers.go:414] Content-Type: application/json
I1220 11:56:10.588757 30 round_trippers.go:414] Content-Length: 1991
I1220 11:56:10.589497 30 round_trippers.go:414] Date: Thu, 20 Dec 2018 11:56:11 GMT
I1220 11:56:10.590141 30 request.go:942] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","selfLink":"/api/v1/namespaces/kube-public/configmaps/cluster-info","uid":"c8b93b6b-0436-11e9-b4e4-4845202d6379","resourceVersion":"368","creationTimestamp":"2018-12-20T09:08:15Z"},"data":{"jws-kubeconfig-94dcda":"eyJhbGciOiJIUzI1NiIsImtpZCI6Ijk0ZGNkYSJ9..qJePAaUQp5APwTC-dSSzvL3MEVE8PQxgbvipbsC1faA","kubeconfig":"apiVersion: v1\nclusters:\n- cluster:\n certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNE1USXlNREE1TURjMU0xb1hEVEk0TVRJeE56QTVNRGMxTTFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTkF3CmRET2tVSXI4M3BDMkY2bFE2UFZMYzlMbGhtenNnc3NPMWRQZWhVWTZ0azJwNUZIdmRlNEwwdkVNWHpaZU5oSGUKNFNnd1A1cTMxd1F0Wkx3aEFKWDRDR1dzNGVFZG9LNnZqMDVJYzQ1SDZhMDNvN3RqSWhNcUsvTHVrdDAzR2Q1Lwp0OENXUllHYjV5TWtBNldIMkxDZkwvSXlYQWU1NTV0STR1RTRuTTFSUXNSMFNMdy8rR2FK [truncated 967 chars]
[discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "192.168.100.1:6443"
[discovery] Successfully established connection with API Server "192.168.100.1:6443"
I1220 11:56:10.596836 30 join.go:608] [join] Retrieving KubeConfig objects
[join] Reading configuration from the cluster...
[join] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
I1220 11:56:10.600835 30 round_trippers.go:383] GET https://192.168.100.1:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config
I1220 11:56:10.601554 30 round_trippers.go:390] Request Headers:
I1220 11:56:10.602287 30 round_trippers.go:393] User-Agent: kubeadm/v1.13.0 (linux/amd64) kubernetes/ddf47ac
I1220 11:56:10.603124 30 round_trippers.go:393] Accept: application/json, */*
I1220 11:56:10.603831 30 round_trippers.go:393] Authorization: Bearer 94dcda.c271f4ff502789ca
I1220 11:56:10.633321 30 round_trippers.go:408] Response Status: 200 OK in 28 milliseconds
I1220 11:56:10.634283 30 round_trippers.go:411] Response Headers:
I1220 11:56:10.635127 30 round_trippers.go:414] Date: Thu, 20 Dec 2018 11:56:11 GMT
I1220 11:56:10.635912 30 round_trippers.go:414] Content-Type: application/json
I1220 11:56:10.636635 30 round_trippers.go:414] Content-Length: 1316
I1220 11:56:10.637413 30 request.go:942] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/configmaps/kubeadm-config","uid":"c8069fb3-0436-11e9-b4e4-4845202d6379","resourceVersion":"173","creationTimestamp":"2018-12-20T09:08:14Z"},"data":{"ClusterConfiguration":"apiServer:\n certSANs:\n - 192.168.100.1\n extraArgs:\n authorization-mode: Node,RBAC\n oidc-ca-file: /etc/kubernetes/pki/ca.crt\n oidc-client-id: kubernetes\n oidc-groups-claim: group\n oidc-issuer-url: https://192.168.0.154:32000\n oidc-username-claim: email\n timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta1\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrolPlaneEndpoint: \"\"\ncontrollerManager: {}\ndns:\n type: CoreDNS\netcd:\n local:\n dataDir: /var/lib/etcd\n imageRepository: registry.opensuse.org/devel/kubic/containers/container/kubic\n imageTag: \"3.3\"\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVer [truncated 292 chars]
I1220 11:56:10.643565 30 round_trippers.go:383] GET https://192.168.100.1:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy
I1220 11:56:10.644430 30 round_trippers.go:390] Request Headers:
I1220 11:56:10.645126 30 round_trippers.go:393] Accept: application/json, */*
I1220 11:56:10.645791 30 round_trippers.go:393] User-Agent: kubeadm/v1.13.0 (linux/amd64) kubernetes/ddf47ac
I1220 11:56:10.646455 30 round_trippers.go:393] Authorization: Bearer 94dcda.c271f4ff502789ca
I1220 11:56:10.654053 30 round_trippers.go:408] Response Status: 200 OK in 6 milliseconds
I1220 11:56:10.655099 30 round_trippers.go:411] Response Headers:
I1220 11:56:10.655921 30 round_trippers.go:414] Content-Type: application/json
I1220 11:56:10.656796 30 round_trippers.go:414] Content-Length: 1655
I1220 11:56:10.657597 30 round_trippers.go:414] Date: Thu, 20 Dec 2018 11:56:11 GMT
I1220 11:56:10.658883 30 request.go:942] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kube-proxy","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/configmaps/kube-proxy","uid":"c8de0370-0436-11e9-b4e4-4845202d6379","resourceVersion":"229","creationTimestamp":"2018-12-20T09:08:15Z","labels":{"app":"kube-proxy"}},"data":{"config.conf":"apiVersion: kubeproxy.config.k8s.io/v1alpha1\nbindAddress: 0.0.0.0\nclientConnection:\n acceptContentTypes: \"\"\n burst: 10\n contentType: application/vnd.kubernetes.protobuf\n kubeconfig: /var/lib/kube-proxy/kubeconfig.conf\n qps: 5\nclusterCIDR: 172.16.0.0/13\nconfigSyncPeriod: 15m0s\nconntrack:\n max: null\n maxPerCore: 32768\n min: 131072\n tcpCloseWaitTimeout: 1h0m0s\n tcpEstablishedTimeout: 24h0m0s\nenableProfiling: false\nhealthzBindAddress: 0.0.0.0:10256\nhostnameOverride: \"\"\niptables:\n masqueradeAll: false\n masqueradeBit: 14\n minSyncPeriod: 0s\n syncPeriod: 30s\nipvs:\n excludeCIDRs: null\n minSyncPeriod: 0s\n scheduler: \"\"\n syncPeriod: 30s\nkind: Kub [truncated 631 chars]
I1220 11:56:10.664746 30 round_trippers.go:383] GET https://192.168.100.1:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config-1.13
I1220 11:56:10.665883 30 round_trippers.go:390] Request Headers:
I1220 11:56:10.666731 30 round_trippers.go:393] User-Agent: kubeadm/v1.13.0 (linux/amd64) kubernetes/ddf47ac
I1220 11:56:10.667616 30 round_trippers.go:393] Authorization: Bearer 94dcda.c271f4ff502789ca
I1220 11:56:10.668451 30 round_trippers.go:393] Accept: application/json, */*
I1220 11:56:10.676896 30 round_trippers.go:408] Response Status: 200 OK in 7 milliseconds
I1220 11:56:10.677820 30 round_trippers.go:411] Response Headers:
I1220 11:56:10.680010 30 round_trippers.go:414] Content-Type: application/json
I1220 11:56:10.681115 30 round_trippers.go:414] Content-Length: 2134
I1220 11:56:10.682015 30 round_trippers.go:414] Date: Thu, 20 Dec 2018 11:56:11 GMT
I1220 11:56:10.683204 30 request.go:942] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubelet-config-1.13","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/configmaps/kubelet-config-1.13","uid":"c80c1601-0436-11e9-b4e4-4845202d6379","resourceVersion":"176","creationTimestamp":"2018-12-20T09:08:14Z"},"data":{"kubelet":"address: 0.0.0.0\napiVersion: kubelet.config.k8s.io/v1beta1\nauthentication:\n anonymous:\n enabled: false\n webhook:\n cacheTTL: 2m0s\n enabled: true\n x509:\n clientCAFile: /etc/kubernetes/pki/ca.crt\nauthorization:\n mode: Webhook\n webhook:\n cacheAuthorizedTTL: 5m0s\n cacheUnauthorizedTTL: 30s\ncgroupDriver: cgroupfs\ncgroupsPerQOS: true\nclusterDNS:\n- 172.24.0.10\nclusterDomain: cluster.local\nconfigMapAndSecretChangeDetectionStrategy: Watch\ncontainerLogMaxFiles: 5\ncontainerLogMaxSize: 10Mi\ncontentType: application/vnd.kubernetes.protobuf\ncpuCFSQuota: true\ncpuCFSQuotaPeriod: 100ms\ncpuManagerPolicy: none\ncpuManagerReconcilePeriod: 10s\nenableControllerAttachDetach: tr [truncated 1110 chars]
I1220 11:56:10.688139 30 interface.go:384] Looking for default routes with IPv4 addresses
I1220 11:56:10.688797 30 interface.go:389] Default route transits interface "eth0"
I1220 11:56:10.689612 30 interface.go:196] Interface eth0 is up
I1220 11:56:10.690375 30 interface.go:244] Interface "eth0" has 2 addresses :[192.168.100.220/24 fe80::d0a8:62ff:fe54:b6e9/64].
I1220 11:56:10.690995 30 interface.go:211] Checking addr 192.168.100.220/24.
I1220 11:56:10.691796 30 interface.go:218] IP found 192.168.100.220
I1220 11:56:10.692489 30 interface.go:250] Found valid IPv4 address 192.168.100.220 for interface "eth0".
I1220 11:56:10.693168 30 interface.go:395] Found active IP 192.168.100.220
I1220 11:56:10.694393 30 join.go:341] [preflight] Running configuration dependant checks
I1220 11:56:10.695211 30 join.go:478] [join] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I1220 11:56:10.942159 30 loader.go:359] Config loaded from file /etc/kubernetes/bootstrap-kubelet.conf
I1220 11:56:10.943961 30 join.go:503] Stopping the kubelet
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
I1220 11:56:10.977300 30 round_trippers.go:383] GET https://192.168.100.1:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config-1.12
I1220 11:56:10.978035 30 round_trippers.go:390] Request Headers:
I1220 11:56:10.978844 30 round_trippers.go:393] User-Agent: kubeadm/v1.13.0 (linux/amd64) kubernetes/ddf47ac
I1220 11:56:10.979502 30 round_trippers.go:393] Accept: application/json, */*
I1220 11:56:10.980081 30 round_trippers.go:393] Authorization: Bearer 94dcda.c271f4ff502789ca
I1220 11:56:10.983223 30 round_trippers.go:408] Response Status: 403 Forbidden in 2 milliseconds
I1220 11:56:10.984240 30 round_trippers.go:411] Response Headers:
I1220 11:56:10.985065 30 round_trippers.go:414] Content-Type: application/json
I1220 11:56:10.985883 30 round_trippers.go:414] X-Content-Type-Options: nosniff
I1220 11:56:10.987515 30 round_trippers.go:414] Content-Length: 342
I1220 11:56:10.989207 30 round_trippers.go:414] Date: Thu, 20 Dec 2018 11:56:11 GMT
I1220 11:56:10.990506 30 request.go:942] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"kubelet-config-1.12\" is forbidden: User \"system:bootstrap:94dcda\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden","details":{"name":"kubelet-config-1.12","kind":"configmaps"},"code":403}
configmaps "kubelet-config-1.12" is forbidden: User "system:bootstrap:94dcda" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
$
$
$ kubeadm version -o json
{
"clientVersion": {
"major": "1",
"minor": "13",
"gitVersion": "v1.13.0",
"gitCommit": "ddf47ac13c1a9483ea035a79cd7c10005ff21a6d",
"gitTreeState": "archive",
"buildDate": "2018-12-07T12:00:00Z",
"goVersion": "go1.11.2",
"compiler": "gc",
"platform": "linux/amd64"
}
}
Por alguma razão, parece que ele está procurando por kubelet-config-1.12
enquanto o ConfigMap correto deve ser kubelet-config-1.13
.
Depois de verificar kubelet --version
, vejo que:
$ kubelet --version
Kubernetes v1.12.0
O nome do configmap é derivado disso?
@inercia o nome do configmap é derivado da versão do kubelet. Veja meu link acima.
Obrigado pelo esclarecimento @ oz123.
Eu queria saber o que aconteceria com as atualizações. Por exemplo,
1) uma semeadora é instalada com my-distribution-1.13
que inclui kubeadm-1.13
2) a semeadora está init
ializada
3) algum tempo depois, um nó é instalado com a mesma distro, instalando o kubeadm-1.13
e kubelet-1.13
4) mas algumas atualizações são aplicadas no final da instalação, e o novo kubelet-1.14
é instalado
5) o kubeadm join
tenta procurar o configmap por 1.14, mas não está lá ...
Estou tendo este problema ao instalar o k8s 1.13.1 com uma versão correspondente do kubeadm, mas isolado para o kube-proxy:
kubeadm join --config /etc/kubernetes/kubeadm-client.conf --ignore-preflight-errors=all
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "xxx.xxx.xxx.xxx:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://xxx.xxx.xxx.xxx:6443"
[discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "xxx.xxx.xxx.xxx:6443"
[discovery] Successfully established connection with API Server "xxx.xxx.xxx.xxx:6443"
[join] Reading configuration from the cluster...
[join] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
unable to fetch the kubeadm-config ConfigMap: failed to get component configs: configmaps "kube-proxy" is forbidden: User "system:bootstrap:3tw24k" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
Se eu criar uma solução alternativa RoleBinding manualmente, poderei unir nós.
# on controlplane node
kubectl create rolebinding -n kube-system --role kube-proxy --group system:bootstrappers:kubeadm:default-node-token kubeadm:kube-proxy-bootstrap
# on joining node
...
[join] Reading configuration from the cluster...
[join] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.13" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "ci-pdk1-debug4144-k8sne-1" as an annotation
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the master to see this node join the cluster.
Um kube-proxy
RoleBinding para o grupo system:bootstrappers:kubeadm:default-node-token
deve ser preenchido automaticamente? Alguma ideia do que está acontecendo aqui?
EDIT: Também descobri que executar kubeadm init phase addon kube-proxy
explicitamente após meu primeiro kubeadm init
cria um Kube-proxy RoleBinding para system:bootstrappers:kubeadm:default-node-token
. Ainda não sei por que meu kubeadm init
pula a criação deste RoleBinding.
Isso funcionou para mim:
Definitivamente, verifique sua versão do kubeadm e kubelet, certifique-se de que a mesma versão desses pacotes seja usada em todos os seus nós. Antes de instalar, você deve "marcar e manter" suas versões em seus hosts:
verifique a versão atual de cada um:
kubelet --version
verificar kubeadm
versão kubeadm
se forem diferentes, você terá problemas. Você deve reinstalar a mesma versão entre todos os seus nós e permitir downgrades. Minhas versões no comando abaixo são provavelmente mais antigas do que o que está disponível atualmente, você pode substituir o número da versão por um mais atualizado, mas isso funcionará:
sudo apt-get install -y docker-ce = 18.06.1 ~ ce ~ 3-0 ~ ubuntu kubelet = 1.12.2-00 kubeadm = 1.12.2-00 kubectl = 1.12.2-00 --permitir downgrades
então, uma vez que estiverem instalados, marque-os e segure-os para que não possam ser atualizados automaticamente e interrompa seu sistema
sudo apt-mark hold docker-ce kubelet kubeadm kubectl
Comentários muito úteis