Kubeadm: Kubeadm ๊ตฌ์„ฑ ๋ฌธ์„œ/์˜ˆ์‹œ?

์— ๋งŒ๋“  2017๋…„ 01์›” 24์ผ  ยท  12์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: kubernetes/kubeadm

kubeadm init๋ฅผ ์‹คํ–‰ํ•  ๋•Œ kubeadm์— ๊ตฌ์„ฑ์„ ์ œ๊ณตํ•˜๋ ค๊ณ  ํ•ฉ๋‹ˆ๋‹ค.
ํ† ํฐ์„ ๋ณ€๊ฒฝํ•˜๋Š” ๋“ฑ ์ •ํ™•ํ•œ ํ˜•์‹์˜ ๊ตฌ์„ฑ์ด ์žˆ์Œ์„ ํ™•์ธํ–ˆ์Šต๋‹ˆ๋‹ค.

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
kubernetesVersion: v1.4.4
secrets:
  givenToken: 123456.3123fe2e098799f3

๊ทธ๋ƒฅ ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋ฆฌ๊ณ  kubeadm ์ถœ๋ ฅ์—์„œ

์ œ๊ณต๋œ ํ† ํฐ ๊ฒ€์ฆ
ํ—ˆ์šฉ๋œ ์ œ๊ณต๋œ ํ† ํฐ

๊ทธ๋Ÿฌ๋‚˜ ๋‹ค๋ฅธ yaml์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ. ๊ธฐ๋ณธ์ ์œผ๋กœ ๋‚ด ์ž์‹ ์˜ crt ํŒŒ์ผ์„ ์ง€์ •ํ•˜๊ณ  ์‹ถ์Šต๋‹ˆ๋‹ค.

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
kubernetesVersion: v1.4.4
etcd:
  caFile : /root/easy-rsa-master/easyrsa3/pki/ca.crt
  certFile : /root/easy-rsa-master/easyrsa3/pki/issued/kubernetes-master.crt
  keyFile : /root/easy-rsa-master/easyrsa3/pki/private/kubernetes-master.key

์ถœ๋ ฅ์กฐ์ฐจ ๋˜์ง€ ์•Š๋Š”๋‹ค.์ œ๊ณต๋œ ํ† ํฐ ๊ฒ€์ฆ
kubeadm์ด config(?)

์ด๊ฒƒ์„ ๋””๋ฒ„๊น…ํ•˜๋Š” ๋ฐฉ๋ฒ•์ด ์žˆ์Šต๋‹ˆ๊นŒ?
kubeadm config ๊ตฌ๋ฌธ์— ๋Œ€ํ•œ ๋ฌธ์„œ๋Š” ์–ด๋””์— ์žˆ์Šต๋‹ˆ๊นŒ?
https://kubernetes.io/docs/admin/kubeadm/ ๋์— ์ƒ˜ํ”Œ ๋งˆ์Šคํ„ฐ ๊ตฌ์„ฑ๋งŒ ์žˆ์Šต๋‹ˆ๋‹ค.
https://kubernetes.io/docs/admin/kube-apiserver/ ์—์„œ ์ง€์›๋˜๋Š” api-servers์— ๋Œ€ํ•œ ๋‹ค๋ฅธ ๋ช…๋ น์ค„ ๊ฐ’์„ ์ง€์ •ํ•˜๋ ค๋Š” ๊ฒฝ์šฐ ํ•ด๋‹น ์ด๋ฆ„์€ ๋ฌด์—‡์ž…๋‹ˆ๊นŒ?

์˜ˆ๋ฅผ ๋“ค์–ด kubeadm init config.yaml์—์„œ api ์„œ๋ฒ„์šฉ --tls-private-key-file์„ ์ง€์ •ํ•˜๋ ค๋ฉด ํ•ด๋‹น ์ด๋ฆ„์€ ๋ฌด์—‡์ž…๋‹ˆ๊นŒ? tlsPrivateKeyFile์ž…๋‹ˆ๊นŒ?

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

๊ตฌ์„ฑ ์˜ˆ์ œ๊ฐ€ ๋‹ค์Œ ๋ฌธ์„œ์— ์ถ”๊ฐ€๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

https://kubernetes.io/docs/admin/kubeadm/#config -ํŒŒ์ผ

๋ชจ๋“  12 ๋Œ“๊ธ€

@Lokicity ๋‚ด๊ฐ€ ์ดํ•ดํ•˜๋Š” ๋ฐ”์— ๋”ฐ๋ฅด๋ฉด ์‹คํ–‰ ์ค‘์—๋Š” ์ œ์–ด ๊ณ„ํš์˜ ๊ตฌ์„ฑ์„ ๋™์ ์œผ๋กœ ๋ณ€๊ฒฝํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค. ํ•ด๋‹น ๊ตฌ์„ฑ ์š”์†Œ๋ฅผ ๋‹ค์‹œ ์‹œ์ž‘ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

ํ† ํฐ ๋ฌธ์ œ์˜ ๊ฒฝ์šฐ ํ† ํฐ์„ ์ „๋‹ฌํ•˜์ง€ ์•Š์œผ๋ฉด ์ž๋™์œผ๋กœ ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.

๋„ค. ๊ทธ๋Ÿฌ๋‚˜ kubeadm init์—๋Š” ์‚ฌ์šฉ์ž๊ฐ€ ๊ธฐ๋ณธ ๋ฐ์ดํ„ฐ๋ฅผ ์ „๋‹ฌํ•  ์ˆ˜ ์žˆ๋Š” --config๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค. API ์„œ๋ฒ„์— ๋Œ€ํ•ด ์›ํ•˜๋Š” ํŠน์ • ๋ฐ์ดํ„ฐ๋ฅผ ์ „๋‹ฌํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ? ์˜ˆ๋ฅผ ๋“ค์–ด, kubeadm์—์„œ ์ƒ์„ฑ๋œ ๊ธฐ๋ณธ ์„œ๋ฒ„๊ฐ€ ์•„๋‹Œ api ์„œ๋ฒ„์— ์ž์ฒด --tls-private-key-file์„ ์‚ฌ์šฉํ•˜๊ณ  ์‹ถ์Šต๋‹ˆ๋‹ค.
kubeadm ์ดˆ๊ธฐํ™” ์‹œ๊ฐ„์— ๊ทธ๋ ‡๊ฒŒ ํ•  ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ์›ํ•ฉ๋‹ˆ๋‹ค.

๊ตฌ์„ฑ ์˜ˆ์ œ๊ฐ€ ๋‹ค์Œ ๋ฌธ์„œ์— ์ถ”๊ฐ€๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

https://kubernetes.io/docs/admin/kubeadm/#config -ํŒŒ์ผ

@jistr ๊ท€ํ•˜์˜ ์ƒ์œ„ ๋งํฌ๋ฅผ ๋”ฐ๋ฅด์‹ญ์‹œ์˜ค. ์•„๋ž˜ yaml ํŒŒ์ผ์„ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
etcd:
  endpoints:
  - http://127.0.0.1:2379

config ํŒŒ์ผ๋กœ ์ €์žฅํ•˜์ง€๋งŒ etcd ์—”๋“œํฌ์ธํŠธ๋Š” ๋ฐ˜์˜ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค(์—ฌ์ „ํžˆ nil).

kubeadm.MasterConfiguration{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, API:kubeadm.API{AdvertiseAddress:"192.168.180.53", BindPort:6443}, Etcd:kubeadm.Etcd{Endpoints:[]string(nil), CAFil
e:"", CertFile:"", KeyFile:""}, Networking:kubeadm.Networking{ServiceSubnet:"10.96.0.0/12", PodSubnet:"", DNSDomain:"cluster.local"}, KubernetesVersion:"v1.6.0", CloudProvider:"", AuthorizationM
ode:"RBAC", Token:"8f6cc5.9fdf609145758fbd", TokenTTL:0, SelfHosted:false, APIServerExtraArgs:map[string]string(nil), ControllerManagerExtraArgs:map[string]string(nil), SchedulerExtraArgs:map[st
ring]string(nil), APIServerCertSANs:[]string(nil), CertificatesDir:"/etc/kubernetes/pki"}

๋‚ด๊ฐ€ ์–ด๋””์—์„œ ์‹ค์ˆ˜๋ฅผ ํ–ˆ๋Š”์ง€ ์•„๋‹ˆ๋ฉด ์ฝ”๋“œ ๋ฒ„๊ทธ์ธ์ง€ ๊ถ๊ธˆํ•ฉ๋‹ˆ๋‹ค.

์•ˆ๋…•ํ•˜์„ธ์š”,

@WIZARD-CXY์™€ ๋™์ผํ•œ ๋ฌธ์ œ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.
์ด์— ๋Œ€ํ•œ ์—…๋ฐ์ดํŠธ๊ฐ€ ์žˆ์Šต๋‹ˆ๊นŒ?

๊ฐ์‚ฌ ํ•ด์š”!

cc @jamiehannaford
etcd ์Šฌ๋ผ์ด์Šค๊ฐ€ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ๊ตฌ๋ฌธ ๋ถ„์„๋˜์ง€ ์•Š๋Š”์ง€ ์‚ดํŽด๋ณด๊ณ  ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ?

@WIZARD-CXY @akarasik kubeadm ๋ฐ etcd ์˜ ๋ฒ„์ „์€ ๋ฌด์—‡์ž…๋‹ˆ๊นŒ? ๋‚˜๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์‹œ๋„ํ–ˆ๊ณ  ๊ทธ๊ฒƒ์€ ๋‚˜๋ฅผ ์œ„ํ•ด ์ผํ–ˆ์Šต๋‹ˆ๋‹ค.

root<strong i="9">@c1</strong>: kubeadm version
kubeadm version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.3", GitCommit:"0480917b552be33e2dba47386e51decb1a211df6", GitTreeState:"clean", BuildDate:"2017-05-10T15:38:08Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}

root<strong i="10">@c1</strong>: curl 127.0.0.1:2379/version
{"etcdserver":"3.1.7","etcdcluster":"3.1.0"}

root<strong i="11">@c1</strong>: cat config.yaml
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
etcd:
  endpoints:
  - http://127.0.0.1:2379

root<strong i="12">@c1</strong>: kubectl get pods --all-namespaces
NAMESPACE     NAME                         READY     STATUS    RESTARTS   AGE
kube-system   kube-controller-manager-c1   1/1       Running   0          49s
kube-system   kube-dns-3913472980-vxldm    0/3       Pending   0          45s
kube-system   kube-proxy-2d3q4             1/1       Running   0          45s
kube-system   kube-scheduler-c1            1/1       Running   0          56s

๋˜ํ•œ kubeadm.MasterConfiguration ๊ตฌ์กฐ์ฒด์— ์•ก์„ธ์Šคํ•œ ์œ„์น˜๋Š” ์–ด๋””์ž…๋‹ˆ๊นŒ? ์ž๋™์œผ๋กœ ๋คํ”„๋˜์—ˆ์Šต๋‹ˆ๊นŒ ์•„๋‹ˆ๋ฉด ํŒŒ์ผ์— ๋””๋ฒ„๊ทธ ๋…ผ๋ฆฌ๋ฅผ ์ถ”๊ฐ€ํ–ˆ์Šต๋‹ˆ๊นŒ?

@jamiehannaford kubeadm 1.6.1 etcd 3.0.17
๊ฐ„๋‹จํ•œ ๋””๋ฒ„๊ทธ ์ธ์‡„๋ฅผ ํ†ตํ•ด kubeadm.MasterConfiguration ๊ตฌ์กฐ์ฒด๋ฅผ ์–ป์—ˆ์Šต๋‹ˆ๋‹ค. ์•„๋งˆ๋„ kubeadm์„ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๊ณ  ์˜ค๋Š˜ ๋‹ค์‹œ ์‹œ๋„ํ•˜๊ณ  ์—ฌ๊ธฐ์—์„œ ๊ฒฐ๊ณผ๋ฅผ ์—…๋ฐ์ดํŠธํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ๋‹น์‹ ์˜ ๋„์›€์„ ์ฃผ์…”์„œ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค

@jamiehannaford kubeadm 1.6.3์€ ๋งค๋ ฅ์ฒ˜๋Ÿผ ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค.

@Lokicity ์ตœ์‹  ๋ฒ„์ „์˜ kubeadm์—์„œ ์—ฌ์ „ํžˆ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๊นŒ?

๊ณ ์ • ์ข…๋ฃŒ... :+1:

https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1 ์—์„œ v1.14์˜ ์˜ˆ๋ฅผ ์ฐธ์กฐํ•˜์‹ญ์‹œ์˜ค.

apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
bootstrapTokens:
- token: "9a08jv.c0izixklcxtmnze7"
  description: "kubeadm bootstrap token"
  ttl: "24h"
- token: "783bde.3f89s0fje9f38fhf"
  description: "another bootstrap token"
  usages:
  - authentication
  - signing
  groups:
  - system:bootstrappers:kubeadm:default-node-token
nodeRegistration:
  name: "ec2-10-100-0-1"
  criSocket: "/var/run/dockershim.sock"
  taints:
  - key: "kubeadmNode"
    value: "master"
    effect: "NoSchedule"
  kubeletExtraArgs:
    cgroup-driver: "cgroupfs"
localAPIEndpoint:
  advertiseAddress: "10.100.0.1"
  bindPort: 6443
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
etcd:
  # one of local or external
  local:
    imageRepository: "k8s.gcr.io"
    imageTag: "3.2.24"
    dataDir: "/var/lib/etcd"
    extraArgs:
      listen-client-urls: "http://10.100.0.1:2379"
    serverCertSANs:
    -  "ec2-10-100-0-1.compute-1.amazonaws.com"
    peerCertSANs:
    - "10.100.0.1"
  # external:
    # endpoints:
    # - "10.100.0.1:2379"
    # - "10.100.0.2:2379"
    # caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt"
    # certFile: "/etcd/kubernetes/pki/etcd/etcd.crt"
    # keyFile: "/etcd/kubernetes/pki/etcd/etcd.key"
networking:
  serviceSubnet: "10.96.0.0/12"
  podSubnet: "10.100.0.1/24"
  dnsDomain: "cluster.local"
kubernetesVersion: "v1.12.0"
controlPlaneEndpoint: "10.100.0.1:6443"
apiServer:
  extraArgs:
    authorization-mode: "Node,RBAC"
  extraVolumes:
  - name: "some-volume"
    hostPath: "/etc/some-path"
    mountPath: "/etc/some-pod-path"
    readOnly: false
    pathType: File
  certSANs:
  - "10.100.1.1"
  - "ec2-10-100-0-1.compute-1.amazonaws.com"
  timeoutForControlPlane: 4m0s
controllerManager:
  extraArgs:
    "node-cidr-mask-size": "20"
  extraVolumes:
  - name: "some-volume"
    hostPath: "/etc/some-path"
    mountPath: "/etc/some-pod-path"
    readOnly: false
    pathType: File
scheduler:
  extraArgs:
    address: "10.100.0.1"
extraVolumes:
- name: "some-volume"
  hostPath: "/etc/some-path"
  mountPath: "/etc/some-pod-path"
  readOnly: false
  pathType: File
certificatesDir: "/etc/kubernetes/pki"
imageRepository: "k8s.gcr.io"
useHyperKubeImage: false
clusterName: "example-cluster"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
# kubelet specific options here
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
# kube-proxy specific options here
์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰