Kubernetes: kubeadm์€ ABAC๋ฅผ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ์ง€์›ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

์— ๋งŒ๋“  2018๋…„ 03์›” 06์ผ  ยท  2์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: kubernetes/kubernetes

๋ฒ„๊ทธ ๋ณด๊ณ ์„œ ๋˜๋Š” ๊ธฐ๋Šฅ ์š”์ฒญ์ž…๋‹ˆ๊นŒ? :
/ ์ข…๋ฅ˜์˜ ๋ฒ„๊ทธ

๋ฌด์Šจ ์ผ์ด ์ผ์–ด ๋‚ฌ์Šต๋‹ˆ๊นŒ?
์•„๋ž˜ ๊ตฌ์„ฑ ํŒŒ์ผ์„ ์‚ฌ์šฉํ•˜์—ฌ ABAC ์ธ์ฆ์ด ํ™œ์„ฑํ™” ๋œ kubeadm ๋„๊ตฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๋งŒ๋“ค๋ ค๊ณ ํ–ˆ์Šต๋‹ˆ๋‹ค.

~ / config.yaml

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
networking:
  podSubnet: 10.244.0.0/16
authorizationModes:
- Node
- RBAC
- ABAC
apiServerExtraArgs:
  basic-auth-file: /etc/kubernetes/basic_auth.csv
  authorization-policy-file: /etc/kubernetes/abac_policy.json

~ / basic_auth.csv

admin,admin,1,"system:masters"

~ / abac_policy.json

{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"*", "nonResourcePath": "*", "readonly": true}}
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"admin", "namespace": "*", "resource": "*", "apiGroup": "*"}}
{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"group":"system:serviceaccounts", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*"}}

kubeadm ๋ช…๋ น์ด ๋‹ค์Œ ์ถœ๋ ฅ๊ณผ ํ•จ๊ป˜ ์ค‘๋‹จ๋ฉ๋‹ˆ๋‹ค ...

[init] Using Kubernetes version: v1.9.3
[init] Using Authorization modes: [Node RBAC ABAC]
[preflight] Running pre-flight checks.
        [WARNING FileExisting-crictl]: crictl not found in system path
[preflight] Starting the kubelet service
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [####### #######]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".
[init] This might take a minute or longer if the control plane images have to be pulled.

kube-apiserver ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ๊ณ„์† ๋‹ค์‹œ ์‹œ์ž‘๋˜๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

kube-apiserver ์ปจํ…Œ์ด๋„ˆ์˜ ๋กœ๊ทธ ์ถœ๋ ฅ์€

I0306 12:39:33.179719       1 server.go:121] Version: v1.9.3
invalid authentication config: open /etc/kubernetes/basic_auth.csv: no such file or directory

์˜ˆ์ƒ ํ•œ ์ผ :
kube-apiserver ๊ฐ€ ์‹œ์ž‘๋  ๊ฒƒ์œผ๋กœ ์˜ˆ์ƒํ–ˆ์Šต๋‹ˆ๋‹ค.

์žฌํ˜„ ๋ฐฉ๋ฒ• (๊ฐ€๋Šฅํ•œ ํ•œ ์ตœ์†Œํ•œ์œผ๋กœ ์ •ํ™•ํ•˜๊ฒŒ) :

mkdir -p /etc/kubernetes
cp ~/basic_auth.csv /etc/kubernetes
cp ~/abac_policy.json /etc/kubernetes
kubeadm init --config ~/config.yaml

์šฐ๋ฆฌ๊ฐ€ ์•Œ์•„์•ผ ํ•  ๋‹ค๋ฅธ ๊ฒƒ์ด ์žˆ์Šต๋‹ˆ๊นŒ? :

ํ™˜๊ฒฝ :

  • Kubernetes ๋ฒ„์ „ ( kubectl version ) : v1.9.3
  • ํด๋ผ์šฐ๋“œ ๊ณต๊ธ‰์ž ๋˜๋Š” ํ•˜๋“œ์›จ์–ด ๊ตฌ์„ฑ : ๋ฒ ์–ด ๋ฉ”ํƒˆ
  • OS (์˜ˆ : / etc / os-release) : 16.04.4 LTS (Xenial Xerus)
  • ์ปค๋„ (์˜ˆ : uname -a ) : Linux k8s-master 4.4.0-116-generic # 140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU / Linux
  • ๋„๊ตฌ ์„ค์น˜ :
  • ๊ธฐํƒ€ :
arekubeadm kinbug sicli sicluster-lifecycle

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

/etc/kubernetes/basic_auth.csv ๋ฐ /etc/kubernetes/abac_policy.json ํŒŒ์ผ์ด kube-apiserver ์ปจํ…Œ์ด๋„ˆ์— ๋งˆ์šดํŠธ๋˜์ง€ ์•Š์€ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.
๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ถ”๊ฐ€ ๋ณผ๋ฅจ ์Šคํƒ ์ž๋ฅผ ์‚ฌ์šฉํ•ด ๋ณด์•˜์ง€๋งŒ ํŒŒ์ผ๊ณผ ํ•จ๊ป˜ ์ž‘๋™ํ•˜๋„๋ก ํ•  ์ˆ˜ ์—†์œผ๋ฉฐ ๋””๋ ‰ํ† ๋ฆฌ์—์„œ๋งŒ ์ž‘๋™ํ•˜๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

apiServerExtraVolumes:
- name: <value|string>
  hostPath: <value|string>
  mountPath: <value|string>

๋‘ ํŒŒ์ผ์„ ํฌํ•จํ•˜๋Š” ๋””๋ ‰ํ† ๋ฆฌ๋ฅผ ๋งˆ์šดํŠธํ•˜๊ณ  ๊ตฌ์„ฑ์—์„œ ์ƒˆ ์œ„์น˜๋ฅผ ์ง€์ •ํ•˜๋ฉด ์—ฌ์ „ํžˆ ๊ธฐ๋ณธ ์œ„์น˜์—์„œ ํŒŒ์ผ์„ ์ฐพ์Šต๋‹ˆ๋‹ค! ์•„๋ž˜์—์„œ ์ƒ์„ฑ ๋œ /etc/kubernetes/manifests/kube-apiserver.yaml ์ฐธ์กฐํ•˜์‹ญ์‹œ์˜ค. --authorization-policy-file ์˜ต์…˜์— ๋Œ€ํ•œ ํ•ญ๋ชฉ์ด ๋‘ ๊ฐœ ์žˆ๊ณ  ๋‘ ๋ฒˆ์งธ ํ•ญ๋ชฉ์ด ์ฒซ ๋ฒˆ์งธ ํ•ญ๋ชฉ๋ณด๋‹ค ์šฐ์„ ํ•ฉ๋‹ˆ๋‹ค.

apiVersion: v1
kind: Pod
metadata:
  annotations:
    scheduler.alpha.kubernetes.io/critical-pod: ""
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --authorization-policy-file=/etc/kubernetes/users/abac_policy.json
    - --basic-auth-file=/etc/kubernetes/users/basic_auth.csv
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota
    - --enable-bootstrap-token-auth=true
    - --allow-privileged=true
    - --requestheader-group-headers=X-Remote-Group
    - --advertise-address=#######
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --insecure-port=0
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --secure-port=6443
    - --requestheader-username-headers=X-Remote-User
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --requestheader-allowed-names=front-proxy-client
    - --service-cluster-ip-range=10.96.0.0/12
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --authorization-policy-file=/etc/kubernetes/abac_policy.json
    - --authorization-mode=Node,RBAC,ABAC
    - --etcd-servers=http://127.0.0.1:2379
    image: gcr.io/google_containers/kube-apiserver-amd64:v1.9.3
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: #######
        path: /healthz
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 15
      timeoutSeconds: 15
    name: kube-apiserver
    resources:
      requests:
        cpu: 250m
    volumeMounts:
    - mountPath: /etc/kubernetes/pki
      name: k8s-certs
      readOnly: true
    - mountPath: /etc/ssl/certs
      name: ca-certs
      readOnly: true
    - mountPath: /etc/kubernetes/users
      name: users
      readOnly: true
  hostNetwork: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki
      type: DirectoryOrCreate
    name: k8s-certs
  - hostPath:
      path: /etc/ssl/certs
      type: DirectoryOrCreate
    name: ca-certs
  - hostPath:
      path: /etc/kubernetes/users
      type: DirectoryOrCreate
    name: users
status: {}

๋ชจ๋“  2 ๋Œ“๊ธ€

/etc/kubernetes/basic_auth.csv ๋ฐ /etc/kubernetes/abac_policy.json ํŒŒ์ผ์ด kube-apiserver ์ปจํ…Œ์ด๋„ˆ์— ๋งˆ์šดํŠธ๋˜์ง€ ์•Š์€ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.
๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ถ”๊ฐ€ ๋ณผ๋ฅจ ์Šคํƒ ์ž๋ฅผ ์‚ฌ์šฉํ•ด ๋ณด์•˜์ง€๋งŒ ํŒŒ์ผ๊ณผ ํ•จ๊ป˜ ์ž‘๋™ํ•˜๋„๋ก ํ•  ์ˆ˜ ์—†์œผ๋ฉฐ ๋””๋ ‰ํ† ๋ฆฌ์—์„œ๋งŒ ์ž‘๋™ํ•˜๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

apiServerExtraVolumes:
- name: <value|string>
  hostPath: <value|string>
  mountPath: <value|string>

๋‘ ํŒŒ์ผ์„ ํฌํ•จํ•˜๋Š” ๋””๋ ‰ํ† ๋ฆฌ๋ฅผ ๋งˆ์šดํŠธํ•˜๊ณ  ๊ตฌ์„ฑ์—์„œ ์ƒˆ ์œ„์น˜๋ฅผ ์ง€์ •ํ•˜๋ฉด ์—ฌ์ „ํžˆ ๊ธฐ๋ณธ ์œ„์น˜์—์„œ ํŒŒ์ผ์„ ์ฐพ์Šต๋‹ˆ๋‹ค! ์•„๋ž˜์—์„œ ์ƒ์„ฑ ๋œ /etc/kubernetes/manifests/kube-apiserver.yaml ์ฐธ์กฐํ•˜์‹ญ์‹œ์˜ค. --authorization-policy-file ์˜ต์…˜์— ๋Œ€ํ•œ ํ•ญ๋ชฉ์ด ๋‘ ๊ฐœ ์žˆ๊ณ  ๋‘ ๋ฒˆ์งธ ํ•ญ๋ชฉ์ด ์ฒซ ๋ฒˆ์งธ ํ•ญ๋ชฉ๋ณด๋‹ค ์šฐ์„ ํ•ฉ๋‹ˆ๋‹ค.

apiVersion: v1
kind: Pod
metadata:
  annotations:
    scheduler.alpha.kubernetes.io/critical-pod: ""
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --authorization-policy-file=/etc/kubernetes/users/abac_policy.json
    - --basic-auth-file=/etc/kubernetes/users/basic_auth.csv
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota
    - --enable-bootstrap-token-auth=true
    - --allow-privileged=true
    - --requestheader-group-headers=X-Remote-Group
    - --advertise-address=#######
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --insecure-port=0
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --secure-port=6443
    - --requestheader-username-headers=X-Remote-User
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --requestheader-allowed-names=front-proxy-client
    - --service-cluster-ip-range=10.96.0.0/12
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --authorization-policy-file=/etc/kubernetes/abac_policy.json
    - --authorization-mode=Node,RBAC,ABAC
    - --etcd-servers=http://127.0.0.1:2379
    image: gcr.io/google_containers/kube-apiserver-amd64:v1.9.3
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: #######
        path: /healthz
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 15
      timeoutSeconds: 15
    name: kube-apiserver
    resources:
      requests:
        cpu: 250m
    volumeMounts:
    - mountPath: /etc/kubernetes/pki
      name: k8s-certs
      readOnly: true
    - mountPath: /etc/ssl/certs
      name: ca-certs
      readOnly: true
    - mountPath: /etc/kubernetes/users
      name: users
      readOnly: true
  hostNetwork: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki
      type: DirectoryOrCreate
    name: k8s-certs
  - hostPath:
      path: /etc/ssl/certs
      type: DirectoryOrCreate
    name: ca-certs
  - hostPath:
      path: /etc/kubernetes/users
      type: DirectoryOrCreate
    name: users
status: {}

@msnelling- ์ผ๋ฐ˜์ ์œผ๋กœ ABAC ์‚ฌ์šฉ์„ ๊ถŒ์žฅํ•˜์ง€

๊ฐ€๋Šฅํ•œ ๊ฒฝ์šฐ https://github.com/kubernetes/kubeadm ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์— ๋Œ€ํ•œ ๋ฌธ์ œ๋ฅผ์—ฌ์‹ญ์‹œ์˜ค.

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰