helm version
μΆλ ₯ : v2.14.3
kubectl version
μΆλ ₯ : ν΄λΌμ΄μΈνΈ : v1.15.3, μλ² : v1.16.0-rc.1
ν΄λΌμ°λ μ 곡μ / νλ«νΌ (AKS, GKE, Minikube λ±) : IBM Cloud Kubernetes Service
$ helm init --service-account tiller
$HELM_HOME has been configured at /Users/xxxx/.helm.
Error: error installing: the server could not find the requested resource
$ helm init --debug --service-account tiller
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: helm
name: tiller
name: tiller-deploy
namespace: kube-system
spec:
.
.
.
helmμ΄ tiller
λ°°ν¬λ₯Ό μμ±νλ €κ³ μλνλ κ² κ°μ΅λλ€. apiVersion: extensions/v1beta1
μ λ°λ₯΄λ©΄ : https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16
λ μ΄μ μ§μλμ§ μμ΅λλ€.
κ³Όκ±°μλ helm init --upgrade
extensions/v1beta1
λ° apps/v1
tiller λ°°ν¬λ₯Ό μ‘°μ νλ 볡μ‘μ±μΌλ‘ μΈν΄ tillerλ₯Ό apps / v1λ‘ μ
λ°μ΄νΈνμ§ μμμ΅λλ€. Kubernetes 1.16.0 μ§μμ μμνλ©΄ μμΌλ‘μ΄ μΌμ΄μ€λ₯Ό μ²λ¦¬νκ³ μ΅μ apiVersionμΌλ‘ λ§μ΄κ·Έλ μ΄μ
ν΄μΌ ν κ² κ°μ΅λλ€.
λ€μμ λ¨κΈ°μ μΈ ν΄κ²° λ°©λ²μ λλ€.
helm init --output yaml | sed 's<strong i="6">@apiVersion</strong>: extensions/v1beta1<strong i="7">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
μ¬μ€ μΆ©λΆνμ§ μμ΅λλ€. μ¬μ ν μ€λ₯κ° λ°μν©λλ€.
error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec
λ€μκ³Ό κ°μ΄ ν¨μΉ ν μ μμ΅λλ€.
/usr/local/bin/kubectl patch --local -oyaml -f - -p '{"spec":{"selector": {"app":"helm","name":"tiller"}}}'
μ’μ! --override
νλκ·Έλ‘ crazy sed hacksλ³΄λ€ λμΌν ν¨κ³Όλ₯Ό μ»μ μ μμ΅λλ€. :)
μ’μ!
--override
νλκ·Έλ‘ crazy sed hacksλ³΄λ€ λμΌν ν¨κ³Όλ₯Ό μ»μ μ μμ΅λλ€. :)
μ, κ·Έλ¬λ κ·Έμ λ―ΈμΉ sed ν΄νΉμ 볡μ¬νμ¬ λΆμ¬ λ£μ μ μμ§λ§ helm init --override "apiVersion"="apps/v1"
λ μλνμ§ μμ΅λλ€. μ’μ, sed ν΄νΉλ μλνμ§ μμ΅λλ€.
νμ¬ ν΄κ²° λ°©λ²μ λ€μκ³Ό κ°μ΅λλ€.
helm init --output yaml> tiller.yaml
tiller.yamlμ μ
λ°μ΄νΈνμμμ€.
---
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: helm
name: tiller
name: tiller-deploy
namespace: kube-system
spec:
replicas: 1
strategy: {}
selector:
matchLabels:
app: helm
name: tiller
....
λ€μ sedλ λλ₯Ό μν΄ μλν©λλ€.
helm init --service-account tiller --output yaml | sed 's<strong i="6">@apiVersion</strong>: extensions/v1beta1<strong i="7">@apiVersion</strong>: apps/v1@' | sed 's@ replicas: 1@ replicas: 1\n selector: {"matchLabels": {"app": "helm", "name": "tiller"}}@' | kubectl apply -f -
@mattymo μ루μ (kubectl patch --local μ¬μ©)μ λ¬Έμ λ μ λ ₯μ μ¬λ¬ 리μμ€ (μ¬κΈ°μλ λ°°ν¬ λ° μλΉμ€)κ° ν¬ν¨ λ κ²½μ° μλνμ§ μλ κ² κ°μ΅λλ€.
Kubernetes 1.16.0μ 2018 λ
9 μ 18 μΌ μ΄μ μΆμλμμ΅λλ€.
μμ ν΄κ²° λ°©λ²μ μ¬μ©νμ§ μμΌλ©΄μ΄ μ΅μ Kubernetes 릴리μ€μμ Helmμ΄ μμλ©λλ€.
μ΄ λ¬Έμ λ μΈμ ν΄κ²°λκ³ Helm 2.15.0
μΈμ μΆμλ©λκΉ?
λ sedλ₯Ό μ¬μ©νλ €λ©΄ :)
helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="6">@apiVersion</strong>: extensions/v1beta1<strong i="7">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
κ°μ¬!
μ€λ λλ κ°μ λ¬Έμ λ₯Ό λ§λ¬κ³ λ μ΄λΈμ μ§μ λ³κ²½νμ΅λλ€. λ μ΄λΈμ apps/v1
νκ³ selector
λΆλΆμ ββμΆκ°ν©λλ€. μ§κΈμ νλ₯νκ² μνλ©λλ€. μλλ λ΄ yaml νμΌμ
λλ€.
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: helm
name: tiller
name: tiller-deploy
namespace: kube-system
spec:
replicas: 1
strategy: {}
selector:
matchLabels:
app: helm
name: tiller
template:
metadata:
creationTimestamp: null
labels:
app: helm
name: tiller
spec:
automountServiceAccountToken: true
containers:
- env:
- name: TILLER_NAMESPACE
value: kube-system
- name: TILLER_HISTORY_MAX
value: "0"
image: gcr.io/kubernetes-helm/tiller:v2.14.3
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
path: /liveness
port: 44135
initialDelaySeconds: 1
timeoutSeconds: 1
name: tiller
ports:
- containerPort: 44134
name: tiller
- containerPort: 44135
name: http
readinessProbe:
httpGet:
path: /readiness
port: 44135
initialDelaySeconds: 1
timeoutSeconds: 1
resources: {}
serviceAccountName: tiller
status: {}
@jbrette λΉμ μ λμ μμ
μ
λλ€! λλ selector
μ€ν μμ μ΄λ €μμ κ²ͺκ³ μμμ΅λλ€.
μ€λ λλ κ°μ λ¬Έμ λ₯Ό λ§λ¬κ³ λ μ΄λΈμ μ§μ λ³κ²½νμ΅λλ€. λ μ΄λΈμ
apps/v1
νκ³selector
λΆλΆμ ββμΆκ°ν©λλ€. μ§κΈμ νλ₯νκ² μνλ©λλ€. μλλ λ΄ yaml νμΌμ λλ€.apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: helm name: tiller name: tiller-deploy namespace: kube-system spec: replicas: 1 strategy: {} selector: matchLabels: app: helm name: tiller template: metadata: creationTimestamp: null labels: app: helm name: tiller spec: automountServiceAccountToken: true containers: - env: - name: TILLER_NAMESPACE value: kube-system - name: TILLER_HISTORY_MAX value: "0" image: gcr.io/kubernetes-helm/tiller:v2.14.3 imagePullPolicy: IfNotPresent livenessProbe: httpGet: path: /liveness port: 44135 initialDelaySeconds: 1 timeoutSeconds: 1 name: tiller ports: - containerPort: 44134 name: tiller - containerPort: 44135 name: http readinessProbe: httpGet: path: /readiness port: 44135 initialDelaySeconds: 1 timeoutSeconds: 1 resources: {} serviceAccountName: tiller status: {}
λ³κ²½νλ λ°©λ²? μμΈν λ΄μ©μ μ€λͺ ν΄ μ£Όμκ² μ΅λκΉ?
μ€λ λλ κ°μ λ¬Έμ λ₯Ό λ§λ¬κ³ λ μ΄λΈμ μ§μ λ³κ²½νμ΅λλ€. λ μ΄λΈμ
apps/v1
νκ³selector
λΆλΆμ ββμΆκ°ν©λλ€. μ§κΈμ νλ₯νκ² μνλ©λλ€. μλλ λ΄ yaml νμΌμ λλ€.apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: helm name: tiller name: tiller-deploy namespace: kube-system spec: replicas: 1 strategy: {} selector: matchLabels: app: helm name: tiller template: metadata: creationTimestamp: null labels: app: helm name: tiller spec: automountServiceAccountToken: true containers: - env: - name: TILLER_NAMESPACE value: kube-system - name: TILLER_HISTORY_MAX value: "0" image: gcr.io/kubernetes-helm/tiller:v2.14.3 imagePullPolicy: IfNotPresent livenessProbe: httpGet: path: /liveness port: 44135 initialDelaySeconds: 1 timeoutSeconds: 1 name: tiller ports: - containerPort: 44134 name: tiller - containerPort: 44135 name: http readinessProbe: httpGet: path: /readiness port: 44135 initialDelaySeconds: 1 timeoutSeconds: 1 resources: {} serviceAccountName: tiller status: {}
@ gm12367 λ³κ²½ λ°©λ²κ³Ό μμΈν λ΄μ©μ μ€λͺ ν μ μμ΅λκΉ?
μ€λ λλ κ°μ λ¬Έμ λ₯Ό λ§λ¬κ³ λ μ΄λΈμ μ§μ λ³κ²½νμ΅λλ€. λ μ΄λΈμ
apps/v1
νκ³selector
λΆλΆμ ββμΆκ°ν©λλ€. μ§κΈμ νλ₯νκ² μνλ©λλ€. μλλ λ΄ yaml νμΌμ λλ€.apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: helm name: tiller name: tiller-deploy namespace: kube-system spec: replicas: 1 strategy: {} selector: matchLabels: app: helm name: tiller template: metadata: creationTimestamp: null labels: app: helm name: tiller spec: automountServiceAccountToken: true containers: - env: - name: TILLER_NAMESPACE value: kube-system - name: TILLER_HISTORY_MAX value: "0" image: gcr.io/kubernetes-helm/tiller:v2.14.3 imagePullPolicy: IfNotPresent livenessProbe: httpGet: path: /liveness port: 44135 initialDelaySeconds: 1 timeoutSeconds: 1 name: tiller ports: - containerPort: 44134 name: tiller - containerPort: 44135 name: http readinessProbe: httpGet: path: /readiness port: 44135 initialDelaySeconds: 1 timeoutSeconds: 1 resources: {} serviceAccountName: tiller status: {}
@ gm12367 λ³κ²½ λ°©λ²κ³Ό μμΈν λ΄μ©μ μ€λͺ ν μ μμ΅λκΉ?
μλ₯Ό λ€μ΄ helm init --service-account tiller --tiller-namespace kube-system --debug
λ₯Ό μ¬μ©νμ¬ YAML νμμ 맀λνμ€νΈλ₯Ό μΈμ ν μ μμ΅λλ€. --debug
μ΅μ
μμ΄ μμ
μ μνν©λλ€.
@ gm12367 μ, μΈμλ¬Όμ λ³Ό μ μμ§λ§ μΆλ ₯ λ§ λ³Ό μ μμ΅λλ€. κ·Έλ λ€λ©΄ μΆλ ₯μ λ³κ²½ν μμλ λͺ λ Ήμ 무μμ λκΉ?
@ gm12367 μ± / v1μ λ³κ²½νκ³ μ νκΈ° λΆλΆμ μΆκ°νκ³ μΆμ΅λλ€.
@ puww1010 λ°©κΈ μΆλ ₯μ νμΌλ‘ 리λλ μ ν λ€μ VIMμ μ¬μ©νμ¬ λ³κ²½νμ΅λλ€. μ°Έμ‘°λ‘ μλ λͺ λ Ή.
# helm init --service-account tiller --tiller-namespace kube-system --debug >> helm-init.yaml
# vim helm-init.yaml
# kubectl apply -f helm-init.yaml
go νκ²½μ΄ μ€μ λμ΄ μκ³ μ΄ λ¬Έμ λ₯Ό ν΄κ²°νλ λ€μ PR [Kubernetes 1.16κ³Ό νΈνλλ Helm init] # 6462κ° λ³ν© λ λκΉμ§ κΈ°λ€λ¦΄ μμλ κ²½μ° μΈμ λ μ§ ν μ μμ΅λλ€.
μ§λ€
mkdir p ${GOPATH}/src/k8s.io
cd ${GOPATH}/src/k8s.io
git clone -b kube16 https://github.com/keleustes/helm.git
cd helm
make bootstrap build
ν μ€νΈ:
kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
/bin/helm init --wait --tiller-image gcr.io/kubernetes-helm/tiller:v2.14.3
Creating /home/xxx/.helm
Creating /home/xxx/.helm/repository
Creating /home/xxx/.helm/repository/cache
Creating /home/xxx/.helm/repository/local
Creating /home/xxx/.helm/plugins
Creating /home/xxx/.helm/starters
Creating /home/xxx/.helm/cache/archive
Creating /home/xxx/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /home/xxx/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
```bash
kubectl get deployment.apps / tiller-deploy -n kube-system -o yaml
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: "2019-09-22T01:01:11Z"
generation: 1
labels:
app: helm
name: tiller
name: tiller-deploy
namespace: kube-system
resourceVersion: "553"
selfLink: /apis/apps/v1/namespaces/kube-system/deployments/tiller-deploy
uid: 124001ca-6f31-417e-950b-2452ce70f522
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: helm
name: tiller
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: helm
name: tiller
spec:
automountServiceAccountToken: true
containers:
- env:
- name: TILLER_NAMESPACE
value: kube-system
- name: TILLER_HISTORY_MAX
value: "0"
image: gcr.io/kubernetes-helm/tiller:v2.14.3
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /liveness
port: 44135
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: tiller
ports:
- containerPort: 44134
name: tiller
protocol: TCP
- containerPort: 44135
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /readiness
port: 44135
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2019-09-22T01:01:23Z"
lastUpdateTime: "2019-09-22T01:01:23Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: "2019-09-22T01:01:11Z"
lastUpdateTime: "2019-09-22T01:01:23Z"
message: ReplicaSet "tiller-deploy-568db6b69f" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
@jbrette μ§μΉ¨μ λ°λ₯Έ νμλ λμΌν λ¬Έμ κ° κ³μ λ°μν©λλ€.
@jbrette μ§μΉ¨μ λ°λ₯Έ νμλ λμΌν λ¬Έμ κ° κ³μ λ°μν©λλ€.
"./bin/helm"λμ "helm"μ μ λ ₯ ν κ² κ°μ΅λλ€ .... λ°λΌμ μ΄μ λ²μ μ λ°μ΄λ리λ₯Ό μ¬μ©νκ³ μμ΅λλ€.
μ±κ³΅μ μΌλ‘ μ΄κΈ°ν ν νμλ extensions / v1beta1μ κ΅μ²΄ ν λκΉμ§ μ μ₯μμμ μ°¨νΈ ν¨ν€μ§λ₯Ό μ€μΉν μ μμ΅λλ€.
λ€μμ k8s v1.16.0 μ© μ μ₯μμ μ°¨νΈλ₯Ό μ‘°μ νλ λ°©λ²μ
λλ€.
μμ λ prometheus μ°¨νΈλ₯Ό κΈ°λ°μΌλ‘ν©λλ€.
git clone https://github.com/helm/charts
cd charts/stable
extensions / v1beta1μ policy / v1beta1 PodSecurityPolicyλ‘ λ°κΏλλ€.
sed -i 's<strong i="11">@apiVersion</strong>: extensions/v1beta1<strong i="12">@apiVersion</strong>: policy/v1beta1@' `find . -iregex ".*yaml\|.*yml" -exec awk '/kind:\s+PodSecurityPolicy/ {print FILENAME}' {} +`
NetworkPolicy apiVersionμ μ¬μ©λλ μ°¨νΈμ λν΄ _helpers.tplμ μν΄ μ μ²λ¦¬λ©λλ€.
Deployment, StatefulSet, ReplicaSet, DaemonSetμμ extensions / v1beta1μ apps / v1λ‘ λ체ν©λλ€.
sed -i 's<strong i="17">@apiVersion</strong>: extensions/v1beta1<strong i="18">@apiVersion</strong>: apps/v1@' `find . -iregex ".*yaml\|.*yml" -exec awk '/kind:\s+(Deployment|StatefulSet|ReplicaSet|DaemonSet)/ {print FILENAME}' {} +`
sed -i 's<strong i="19">@apiVersion</strong>: apps/v1beta2<strong i="20">@apiVersion</strong>: apps/v1@' `find . -iregex ".*yaml\|.*yml" -exec awk '/kind:\s+(Deployment|StatefulSet|ReplicaSet|DaemonSet)/ {print FILENAME}' {} +`
μ ν¨ν€μ§λ₯Ό λ§λλλ€.
helm package ./prometheus
Successfully packaged chart and saved it to: /home/vagrant/charts/stable/prometheus-9.1.1.tgz
κ·Έκ²μ μ€μΉνμμμ€ :
helm install /home/vagrant/charts/stable/prometheus-9.1.1.tgz
https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/ κΈ°λ°
PS μ’
μμ±μ΄μλ μΌλΆ μ°¨νΈμ κ²½μ° helm dependency update
νκ³ ν΄λΉνλ κ²½μ° μ’
μ tgzλ₯Ό ν¨μΉ λ κ²μΌλ‘ λ체ν΄μΌ ν μ μμ΅λλ€.
helm init --history-max 200
μ€νμ λμΌν μ€λ₯κ° λ°μ ν¨
μ°μΆ
$HELM_HOME has been configured at /Users/neil/.helm.
Error: error installing: the server could not find the requested resource
$ helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Error: could not find tiller
μ΄ λΆκΈ°λ https://github.com/keleustes/helm/tree/kube16μμ μλν©λλ€ . https://s3-us-west-2.amazonaws.com/bin.cryptexlabs.com/github.com/keleustes/helm/kube16/darwin/helm ν κ°μ§μ£Όμ ν μ μ λΉλκ° λ¦΄λ¦¬μ€λμ§ μμμΌλ―λ‘ μΉ΄λ리μ μ΄λ―Έμ§ νλκ·Έ helm init --canary-image
λ₯Ό μ¬μ©ν΄μΌνλ€λ κ²μ
λλ€.
μ΄ μμ
μ μννκΈ° μν΄ μΉ΄λ리μ μ΄λ―Έμ§κ° νμνμ§ μμ΅λλ€. # 6462λ₯Ό μλνκ³ μΆλ€λ©΄ μμ μΈκΈ ν @jbrette λ‘ helm init --tiller-image gcr.io/kubernetes-helm/tiller:v2.14.3
λ₯Ό μ¬μ©νλ κ²μ΄ μ’μ΅λλ€.
μ΄μ¨λ PRμ μλνκΈ° μ μ λ¨Όμ μ΄μ μ μ 곡λ ν΄κ²° λ°©λ² μ€ νλ λ₯Ό μλν΄ λ³΄λ κ²μ΄ μ’μ΅λλ€. μ΄λ κ²νλ©΄ μμ§ κ²ν μ€μΈ 컀μ€ν κ°λ° λΈλμΉ λμ Helm 2.14.3μ κ³μ μ¬μ©ν μ μμ΅λλ€.
λ΄κ° λͺ λ Ήμ μννλ©΄ λ°°ν¬νμ§λ§ κ·Έ νμλ podμμ λ³Ό μ μμΌλ©° Error from server (NotFound) : pods "tiller-deploy"not found
helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="7">@apiVersion</strong>: extensions/v1beta1<strong i="8">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
deployment.apps / tiller-deployκ° μμ±λ¨
service / tiller-deploy μμ±λ¨
νμ§λ§ λ΄κ° kubectlμ ν λ pods --all-namespacesλ podλ₯Ό λ³Ό μ μμ΅λλ€.
λ€μ μ€νμ΄μ€ μ΄λ¦ μ€λΉ μν λ€μ μμ λμ΄
kube-system coredns-5644d7b6d9-559hw 1/1 Running 0 23h
kube-system coredns-5644d7b6d9-xcmjd 1/1 Running 0 23h
kube-system etcd-fmp 1/1 Running 0 24h
kube-system kube-apiserver-fmp 1/1 Running 0 24h
kube-system kube-controller-manager-fmp 1/1 Running 1 24h
kube-system kube-flannel-ds-amd64-ffx2g 1/1 μ€ν μ€ 0 23h
kube-system kube-proxy-lfvrz 1/1 Running 0 24h
kube-system kube-scheduler-fmp 1/1 μ€ν μ€ 0 24 μκ°
kubectl get all --all-namespaces | grep νλ³
kube-system service / tiller-deploy ClusterIP xxx
kube-system deployment.apps / tiller-deploy 0/10 0 2m54s
kube-system replicaset.apps / tiller-deploy-77855d9dcf 10 0 2m54s
@DanielIvaylov λλ λΉμ μ΄ κ²½μ΄κΈ° μλΉμ€ κ³μ μ΄ μλ€κ³ μκ°ν©λλ€. κ·Έκ²μ μμ±νλ©΄ λ°°ν¬κ° νΈλ¬ ν¬λλ μμ±ν©λλ€.
κ°μ¬!
@DanielIvaylov λλ λΉμ μ΄ κ²½μ΄κΈ° μλΉμ€ κ³μ μ΄ μλ€κ³ μκ°ν©λλ€. κ·Έκ²μ μμ±νλ©΄ λ°°ν¬κ° νΈλ¬ ν¬λλ μμ±ν©λλ€.
κ°μ¬!
μ£μ‘ν©λλ€, λλ μ΄κ²μ΄ κ·Έκ²μ μμν κ²μ΄λΌλ κ²μ μλ‘μ΅λλ€. κ²½μ΄κΈ° μλΉμ€ κ³μ μ μ΄λ»κ² μμν©λκΉ?
μλ νμΈμ.
kubectl --namespace kube-system create sa tiller
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
μ§μ μ€λ¨ λ APIλ₯Ό μΌμμ μΌλ‘ λ€μ νμ±ν ν API μλ² (/etc/kubernetes/manifests/kube-apiserver.yaml)μ μλ νλκ·Έκ° μΆκ°λμμ΅λλ€.
--runtime-config = apps / v1beta1 = true, apps / v1beta2 = true, extensions / v1beta1 / daemonsets = true, extensions / v1beta1 / deployments = true, extensions / v1beta1 / replicasets = true, extensions / v1beta1 / networkpolicies = true, extensions / v1beta1 / podsecuritypolicies = true
μ΄κ²μ helm v2λ₯Ό μμ νμ΅λλ€.
Windows μ¬μ©μλ₯Ό μν΄ λ€μκ³Ό κ°μ΄ powershellμ ν΅ν΄ tillerλ₯Ό μ€μΉ / μ κ·Έλ μ΄λ ν μμμμ΅λλ€.
$(helm init --output yaml) -replace "extensions/v1beta1","apps/v1"
λ€μμ λ¨κΈ°μ μΈ ν΄κ²° λ°©λ²μ λλ€.
helm init --output yaml | sed 's<strong i="7">@apiVersion</strong>: extensions/v1beta1<strong i="8">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
μ¬μ€ μΆ©λΆνμ§ μμ΅λλ€. μ¬μ ν μ€λ₯κ° λ°μν©λλ€.
error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec
λ€μκ³Ό κ°μ΄ ν¨μΉ ν μ μμ΅λλ€.
/usr/local/bin/kubectl patch --local -oyaml -f - -p '{"spec":{"selector": {"app":"helm","name":"tiller"}}}'
λλ₯Ό μν΄ kubectl ν¨μΉκ° κ±Έλ € μμ΅λλ€.
/ var / log / syslog νμΌμ λ©μμ§κ° μμ΅λλ€.
μλΉμ€ κ³μ μ΄ μ΄λ―Έ μμ΅λλ€.
kubeflow @ masternode : ~ $ kubectl --namespace kube-system create sa tiller
μλ² μ€λ₯ (AlreadyExists) : μλΉμ€ κ³μ "tiller"κ° μ΄λ―Έ μμ΅λλ€.
kubeflow @ masternode : ~ $ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount = kube- system : tiller
μλ² μ€λ₯ (AlreadyExists) : clusterrolebindings.rbac.authorization.k8s.io "tiller"κ° μ΄λ―Έ μμ΅λλ€.
μ‘°μΈ ν΄ μ£Ό μκ² μ΄μ
μλ μ€ν
λ΄λ³΄λ΄κΈ° PATH = $ PATH : / usr / local / bin
μ΄λ μ‘°ν
μ΄λ€ κ²½μ΄κΈ°
ν€ μ€μΉ
-μ΄λ¦ nfs-client-provisioner
--set nfs.server = 10.0.0.4
--set nfs.path = / nfsroot
--set storageClass.name = nfs
--set storageClass.defaultClass = true
stable / nfs-client-provisioner
λμμ¨λ€
/ usr / local / bin / helm
/ usr / local / bin / tiller
μ€λ₯ : κ²½μ΄κΈ°λ₯Ό μ°Ύμ μ μμ΅λλ€.
μ΄κ²μ μ΄μ μΌ μ€ν νΌμ΄λ―λ‘ λμμ μ£Όμ μ κ°μ¬ν©λλ€.
@cyrilthank νΈλ¬ λ°°ν¬κ° μ€νλκ³ μμ§ μκΈ° λλ¬Έμ νΈλ¬ μ€λ₯κ° λ°μν κ² κ°μ΅λλ€. λ€μ λͺ
λ Ήμ μ€ννμ¬ tillerλ₯Ό μ€μΉν΄λ³΄μμμ€.
helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="7">@apiVersion</strong>: extensions/v1beta1<strong i="8">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
helm version -s
κ° μ λλ‘ μλνκ³ μ€νμ€μΈ κ²½μ° μλ² (κ²½μ΄κΈ°) λ²μ μ λ°νν΄μΌν©λλ€.
κ°μ¬ν©λλ€.
kubeflowλ₯Ό λ€μ λ¨κ³λ‘ μ§ννλ λ° λμμ΄λμμ΅λλ€.
μ’μ μ΄μ μ΄ λ¬Έμ κ°μλ κ² κ°μ
https://github.com/kubeflow/kubeflow/issues/4184
λ€μ μ°¨λ¨μ λ‘
https://github.com/kubeflow/kubeflow/issues/4184 μμ λ΄κ° μ΄λ»κ² λμμλ°μ μ μλμ§μ λν μ‘°μΈμ μ£Όμλ©΄ κ°μ¬νκ² μ΅λλ€.
@cyrilthank μμ μ 곡λ λ¨κ³λ₯Ό μλν΄λ³΄μμμ€ : https://github.com/helm/helm/issues/6374#issuecomment -533853888
λ μ΄μ μ¬μ©λμ§ μλ API λ²μ μ μ λ²μ μΌλ‘ κ΅μ²΄ν΄μΌν©λλ€.
kubeflow_workaround_and_error_traces.txt
νΉνμ΄ λ¬Έμ λ₯Ό 곡κ°νλ λ°μμ΄ νμμ λ΅λ³μ κ°μ¬λ립λλ€.
μ£μ‘νμ§λ§ ν΄κ²° λ¨κ³μμ λ¬Έμ κ°μλ κ² κ°μ΅λλ€.
μ²¨λΆ λ μΆμ μ λ¨κ³λ₯Ό κ²ν νκ³ λ΄κ° λ μλͺ»νκ³ μλμ§ μ‘°μΈ ν΄ μ£Όμλ©΄ κ°μ¬νκ² μ΅λλ€.
@cyril κ°μ¬ν©λλ€. kubeflow yamlsμ λν΄ sed
λͺ
λ Ήμ μ€ννμ¬ μ΄μ api νμ₯μ μ νμ₯μΌλ‘ κ΅μ²΄νλ©΄λ©λλ€ (νλ‘ λ©ν
μ°μ€λ₯Ό μ ν λ°°ν¬ ν νμ μμ π). μ νννμ§ λͺ»νλ€λ©΄ λ―Έμν©λλ€.
μ΄ μμ μ κΈ°λ³Έμ μΌλ‘ kubeflow dpl yamlsμμ extensions/v1beta1
λ₯Ό apps/v1
λ‘ λ체νλ κ²μ
λλ€.
μ κ·Έλμ λ©μ²ν μΉ΄νΌλ₯Ό νμ΄ :(
λ΄ KFAPP = / nfsroot / kf-poc
μ¬μ ν λͺ κ°μ§ λ©μμ§μ μ΅μ’ μ€λ₯κ° νμλλ κ² κ°μ΅λλ€.
kubeflowμ λ€μ λ¨κ³λ‘ μ΄λνκΈ° μν΄ μ§κΈ λΉμ μκ² μμ‘΄νλ―λ‘ λμ μ£Όμκ² μ΅λκΉ?
@cyril κ°μ¬ν©λλ€. kubeflow yamlsμ λν΄
sed
λͺ λ Ήμ μ€ννμ¬ μ΄μ api νμ₯μ μ νμ₯μΌλ‘ κ΅μ²΄νλ©΄λ©λλ€ (νλ‘ λ©ν μ°μ€λ₯Ό λ°°ν¬ ν νμκ° μ ν μμ). μ νννμ§ λͺ»νλ€λ©΄ λ―Έμν©λλ€.
μ΄ μμ μ κΈ°λ³Έμ μΌλ‘ kubeflow dpl yamlsμμextensions/v1beta1
λ₯Όapps/v1
λ‘ λ체νλ κ²μ λλ€.
apps/v1beta2
λ apps/v1
https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/
λμμ μ£Όμ
μ κ°μ¬ν©λλ€ @uniuuu
https://github.com/helm/helm/files/3662328/kubeflow_workaround_sed_and_error_traces.txtμ μ°Έμ‘° λ yaml νμΌμ μ΄λμ / μ΄λ»κ² μ»μ μ μλμ§ μλ € μ£Όμκ² μ΅λκΉ?
https://github.com/helm/helm/issues/6374#issuecomment -533840097
https://github.com/helm/helm/issues/6374#issuecomment -533185074
sedλ₯Ό λ³κ²½ ν νμλ μ¬μ νμμ μ°Έμ‘° λ μ€λ₯κ° λ°μνκΈ° λλ¬Έμ μμ²
λ¨κ³κ° μλ€λ©΄ μ‘°μΈ ν΄ μ£Όμκ² μ΅λκΉ?
kubectl λ³ν -f
λ§€λ² μ€νλμ΄μΌν©λλ€.
helm init
μμ
ν λ μμμ μΈκΈ ν ν΄κ²° λ°©λ²μ μ μ©νμ§λ§ helm version
μ κ°μ μμ
μ μλ ν λ λ€μ μ€λ₯κ° κ³μ λ°μνλ€λ©΄ helm
deployment
μ°Ύμ μ μλ€.
Error: could not find tiller
μ€λΉλμ§ μμ μ΄μ λ₯Ό νμΈνλ €λ©΄ kubectl get events --all-namespaces | grep -i tiller
λ₯Ό μ€νν΄μΌν©λλ€.
μλ₯Ό λ€μ΄, microk8s
serviceaccount "tiller"
μ΄ νμνμ§ μκΈ° λλ¬Έμ λ΄ λ¬Έμ λ μλμ κ°μ΅λλ€.
microk8s.kubectl get events --all-namespaces | grep -i tiller
kube-system 23m Warning FailedCreate replicaset/tiller-deploy-77855d9dcf Error creating: pods "tiller-deploy-77855d9dcf-" is forbidden: error looking up service account kube-system/tiller: serviceaccount "tiller" not found
κ·Έλμ μλΉμ€ κ³μ μμ΄ μμ μνμ΅λλ€.
- helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="20">@apiVersion</strong>: extensions/v1beta1<strong i="21">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
+ helm init spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="22">@apiVersion</strong>: extensions/v1beta1<strong i="23">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
@cyrilthank μ¬κΈ°μ κ΄λ ¨λ ν λ‘ κ³Ό κ΄λ ¨μ΄ μκΈ° λλ¬Έμ κ·νμ μ견μ μ κ±°νμ΅λλ€. kubeflow / kubeflow # 4184μμ νμ μ‘°μΉλ₯Ό κ³μνμμμ€. κ°μ¬!
helm init spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="6">@apiVersion</strong>: extensions/v1beta1<strong i="7">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
μ½κ°μ μμ
helm init --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="11">@apiVersion</strong>: extensions/v1beta1<strong i="12">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
+1
@ puww1010 λ°©κΈ μΆλ ₯μ νμΌλ‘ 리λλ μ ν λ€μ VIMμ μ¬μ©νμ¬ λ³κ²½νμ΅λλ€. μ°Έμ‘°λ‘ μλ λͺ λ Ή.
# helm init --service-account tiller --tiller-namespace kube-system --debug >> helm-init.yaml # vim helm-init.yaml # kubectl apply -f helm-init.yaml
λλ μ΄κ²μ μλνλ€. VIMμμ νμΌμ νΈμ§ ν ν kubectl apply
λͺ
λ Ήμ μ¬μ©νμ§λ§ μ무 μμ
λ μννμ§ μλ κ² κ°μ΅λλ€. helm init --service-account tiller --tiller-namespace kube-system --debug >> helm-init.yaml
λ€μ μ€ννκ±°λ helm init --output yaml
νλ©΄ λ³κ²½ μ¬νμ΄ μ μ©λμ§ μμ΅λλ€. λ€λ₯Έ μ¬λμ΄ μ΄κ²μ κ²½νν©λκΉ?
λ sedλ₯Ό μ¬μ©νλ €λ©΄ :)
helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's<strong i="7">@apiVersion</strong>: extensions/v1beta1<strong i="8">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
κ°μ¬!
λ°©κΈ k8sλ₯Ό μ
κ·Έλ μ΄λνκ³ μ΄ λ¬Έμ μ μ§λ©΄νκ³ μμ μ루μ
μ μ¬μ©νμ΅λλ€. λ°°ν¬λ₯Ό μμ±νμ§λ§ 볡μ μΈνΈκ° μ€ν¨νκ³ μ΄κ²μ΄ kubectl describe -n kube-system replicasets.apps tiller-deploy-77855d9dcf
μμ μ»μ κ²μ
λλ€.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 41s (x14 over 82s) replicaset-controller Error creating: pods "tiller-deploy-77855d9dcf-" is forbidden: error looking up service account kube-system/tiller: serviceaccount "tiller" not found
ν΄λΉ μλΉμ€ κ³μ μ λ§λ€κΈ°μν yaml νμΌμ μ΄λμμ μ°Ύμ μ μμ΅λκΉ?
μλ νμΈμ.
kubectl --namespace kube-system create sa tiller kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
μ΄κ²μ λ΄ λ¬Έμ λ₯Ό ν΄κ²°νμ΅λλ€!
λͺ¨λ κ°μ¬ν©λλ€!
μΉ΄λ리μ μ΄λ―Έμ§λμ΄ λ³ν©μ΄ μμ§μλ ν λμΌν μ€λ₯λ₯Ό μμ±ν©λλ€.
@ puww1010 λ°©κΈ μΆλ ₯μ νμΌλ‘ 리λλ μ ν λ€μ VIMμ μ¬μ©νμ¬ λ³κ²½νμ΅λλ€. μ°Έμ‘°λ‘ μλ λͺ λ Ή.
# helm init --service-account tiller --tiller-namespace kube-system --debug >> helm-init.yaml # vim helm-init.yaml # kubectl apply -f helm-init.yaml
λλ μ΄κ²μ μλνλ€. VIMμμ νμΌμ νΈμ§ ν ν
kubectl apply
λͺ λ Ήμ μ¬μ©νμ§λ§ μ무 μμ λ μννμ§ μλ κ² κ°μ΅λλ€.helm init --service-account tiller --tiller-namespace kube-system --debug >> helm-init.yaml
λ€μ μ€ννκ±°λhelm init --output yaml
νλ©΄ λ³κ²½ μ¬νμ΄ μ μ©λμ§ μμ΅λλ€. λ€λ₯Έ μ¬λμ΄ μ΄κ²μ κ²½νν©λκΉ?
κ·Έλ, λλ λ§μ°¬κ°μ§μΌ.
μ΄λ―Έμ§ μμΉλ₯Ό gcr.azk8s.cn/kubernetes-helm/tiller:v2.14.3
λ‘ λ³κ²½ν΄μΌ ν μ μμ΅λλ€. gcr.io
μμΉκ° μ°¨λ¨ λ κ² κ°μ΅λλ€.
μ΄λ―Έμ§ μμΉλ₯Ό gcr.azk8s.cn/kubernetes-helm/ tiller : v2.14.3 μΌλ‘ λ³κ²½ν΄μΌ ν μ μμ΅λλ€. gcr.io μμΉκ° μ°¨λ¨ λ κ² κ°μ΅λλ€.
μμ ν μ ν¨ν λ¬Έμ μ΄μ§λ§μ΄ λ¬Έμ λμ΄ λ¬Έμ μμλ λ¬Έμ μ μ½κ° μ§κ΅νμ§λ§, μνκΉκ²λ gcr.io
λ μ€κ΅μμ μ°¨λ¨λ©λλ€. μμΈν λ΄μ©μ https://github.com/helm/charts/issues/14607 μ μ°Έμ‘°νμμμ€.
kubernetes λ²μ μ 1.15.4λ‘ λ‘€λ°±νμ¬μ΄ λ¬Έμ λ₯Ό ν΄κ²°ν μμμμ΅λλ€.
μ΄κ²μ 곡μ ν΄ μ£Όμ @UmamaheshMaxwell μκ² κ°μ¬λ립λλ€.
kubernetes λ²μ μ λ‘€λ°±νλ λ° μ¬μ©ν λ¨κ³λ₯Ό 곡μ ν΄ μ£Όμκ² μ΅λκΉ?
@cyrilthank λ―Έλ μΏ λ² λΌλ©΄ minikube config set kubernetes-version v1.15.4
μ΄κ²μ 곡μ ν΄ μ£Όμ @UmamaheshMaxwell μκ² κ°μ¬λ립λλ€.
kubernetes λ²μ μ λ‘€λ°±νλ λ° μ¬μ©ν λ¨κ³λ₯Ό 곡μ ν΄ μ£Όμκ² μ΅λκΉ?
@cyrilthank μ°λ¦¬λ μ체 VM (Ubuntu 18+)μ μ¬μ©νκ³ μμ΅λλ€. μλλ k8s λ²μ μ μ€μΉνκΈ°μν setpsμ
λλ€. 1.15.4
--pod-network-cidr=10.244.10.0/16
-νλλ¬
--apiserver-advertise-address=x.x.x.x
-VMμ λΉκ³΅κ° IP (λ§μ€ν°)
--apiserver-cert-extra-sans=x.x.x.x
-VMμ κ³΅μ© IP (λ§μ€ν°) (λ‘컬 μ»΄ν¨ν°μμ λ§μ€ν°μ μ‘μΈμ€νλ €λ κ²½μ° νμν©λλ€.
μ°Έκ³ : μλ λ§ν¬λ₯Ό λ°λΌ μ체 νΈμ€ν Kubernetes ν΄λ¬μ€ν° (http://docs.shippable.com/deploy/tutorial/create-kubeconfig-for-self-hosted-kubernetes-cluster/)μ λν kubeconfig νμΌμ μ€μ νμμμ€.
μ¬μ ν κΆκΈν μ μ΄ μμΌλ©΄ μλ €μ£ΌμΈμ.
@cyrilthank λ§μ½ κ·Έκ²μ΄
κ°μ¬ν©λλ€ @MrSimonEmms κ΄μ°μ λ―Έλκ° μλλλ€. λλ @UmamaheshMaxwell μ λ¨κ³μ ν¨κ» κ°μΌ ν κ²μ΄λΌκ³ μκ°ν©λλ€.
μ΄κ²μ 곡μ ν΄ μ£Όμ @UmamaheshMaxwell μκ² κ°μ¬λ립λλ€.
kubernetes λ²μ μ λ‘€λ°±νλ λ° μ¬μ©ν λ¨κ³λ₯Ό 곡μ ν΄ μ£Όμκ² μ΅λκΉ?@cyrilthank μ°λ¦¬λ μ체 VM (Ubuntu 18+)μ μ¬μ©ν΄ μμ΅λλ€. μλλ k8s λ²μ 1.15.4λ₯Ό μ€μΉνκΈ°μν setpsμ λλ€.
kubeadm μ¬μ€μ
sudo apt-get install kubelet = 1.15.4-00 kubectl = 1.15.4-00 kubeadm = 1.15.4-00
sudo kubeadm init --pod-network-cidr = 10.244.10.0 / 16 --apiserver-advertise-address = xxxx --apiserver-cert-extra-sans = xxxx --kubernetes-version "1.15.4"--pod-network-cidr = 10.244.10.0 / 16-νλλ¬
--apiserver-advertise-address = xxxx-VMμ κ°μΈ IP (λ§μ€ν°)
--apiserver-cert-extra-sans = xxxx-VMμ κ³΅μ© IP (λ§μ€ν°) (λ‘컬 μ»΄ν¨ν°μμ λ§μ€ν°μ μ‘μΈμ€νλ €λ κ²½μ° νμν©λλ€.
μ°Έκ³ : μλ λ§ν¬λ₯Ό λ°λΌ μ체 νΈμ€ν Kubernetes ν΄λ¬μ€ν° (http://docs.shippable.com/deploy/tutorial/create-kubeconfig-for-self-hosted-kubernetes-cluster/)μ λν kubeconfig νμΌμ μ€μ νμμμ€.
μ¬μ ν κΆκΈν μ μ΄ μμΌλ©΄ μλ €μ£ΌμΈμ.
νμ νμ μ κ°μ¬λ립λλ€ @UmamaheshMaxwell
κΈ°μ‘΄ kubernetes 1.16 μ€μ μ΄ μμ΅λλ€.μ΄ λ¨κ³λ₯Ό μ€νν μ μλμ§ νμΈν΄ μ£Όμκ² μ΅λκΉ?
μ΄κ²μ 곡μ ν΄ μ£Όμ @UmamaheshMaxwell μκ² κ°μ¬λ립λλ€.
kubernetes λ²μ μ λ‘€λ°±νλ λ° μ¬μ©ν λ¨κ³λ₯Ό 곡μ ν΄ μ£Όμκ² μ΅λκΉ?
@cyrilthank μ°λ¦¬λ μ체 VM (Ubuntu 18+)μ μ¬μ©ν΄ μμ΅λλ€. μλλ k8s λ²μ 1.15.4λ₯Ό μ€μΉνκΈ°μν setpsμ λλ€.
kubeadm μ¬μ€μ
sudo apt-get install kubelet = 1.15.4-00 kubectl = 1.15.4-00 kubeadm = 1.15.4-00
sudo kubeadm init --pod-network-cidr = 10.244.10.0 / 16 --apiserver-advertise-address = xxxx --apiserver-cert-extra-sans = xxxx --kubernetes-version "1.15.4"
--pod-network-cidr = 10.244.10.0 / 16-νλλ¬
--apiserver-advertise-address = xxxx-VMμ κ°μΈ IP (λ§μ€ν°)
--apiserver-cert-extra-sans = xxxx-VMμ κ³΅μ© IP (λ§μ€ν°) (λ‘컬 μ»΄ν¨ν°μμ λ§μ€ν°μ μ‘μΈμ€νλ €λ κ²½μ° νμν©λλ€.
μ°Έκ³ : μλ λ§ν¬λ₯Ό λ°λΌ μ체 νΈμ€ν Kubernetes ν΄λ¬μ€ν° (http://docs.shippable.com/deploy/tutorial/create-kubeconfig-for-self-hosted-kubernetes-cluster/)μ λν kubeconfig νμΌμ μ€μ νμμμ€.
μ¬μ ν κΆκΈν μ μ΄ μμΌλ©΄ μλ €μ£ΌμΈμ.νμ νμ μ κ°μ¬λ립λλ€ @UmamaheshMaxwell
κΈ°μ‘΄ kubernetes 1.16 μ€μ μ΄ μμ΅λλ€.μ΄ λ¨κ³λ₯Ό μ€νν μ μλμ§ νμΈν΄ μ£Όμκ² μ΅λκΉ?
κ·Έλ @cyrilthank , μ¬μ§μ΄ μ°λ¦¬λ kubernetes 1.16.1μ κ°μ§κ³ μμμ§λ§ μ°λ¦¬λ κ·Έκ²μ 1.15.4
λ‘ λ‘€λ°±ν΄μΌνμ΅λλ€. μ²μλΆν° μ€μ νλ €λ©΄ μλ λ§ν¬κ° μμ΅λλ€.
VMμ OS λ²μ
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic
kuberenetes μ 리
kubeadm reset
sudo apt-get purge kubeadm kubectl kubelet kubernetes-cni kube*
sudo apt-get autoremove
sudo rm -rf ~/.kube
Kubernetes μ€μ (_Master λ° Node_ λͺ¨λ)
https://www.howtoforge.com/tutorial/how-to-install-kubernetes-on-ubuntu/
(μκ°μ μ μ½νλ €λ©΄ μμ λ§ν¬μμ μ μ λ λ¨κ³λ₯Ό μ΅λν μλννλ κ²μ΄ μ’μ΅λλ€.)
λμμ΄ λ νμνλ©΄ μλ €μ£ΌμΈμ. λ‘€λ°±μΌλ‘ ν볡ν μ¬μ :), λΉμ μ΄ κ·Έκ²μΌλ‘ μμ‘°λ‘κ² μ¬ννκΈ°λ₯Ό λ°λλλ€.
μ΄λ―Έμ§ μμΉλ₯Ό gcr.azk8s.cn/kubernetes-helm/ tiller : v2.14.3 μΌλ‘ λ³κ²½ν΄μΌ ν μ μμ΅λλ€. gcr.io μμΉκ° μ°¨λ¨ λ κ² κ°μ΅λλ€.
μμ ν μ ν¨ν λ¬Έμ μ΄μ§λ§μ΄ λ¬Έμ λμ΄ λ¬Έμ μμλ λ¬Έμ μ μ½κ° μ§κ΅νμ§λ§, μνκΉκ²λ
gcr.io
λ μ€κ΅μμ μ°¨λ¨λ©λλ€. μμΈν λ΄μ©μ helm / charts # 14607 μ μ°Έμ‘°νμμμ€.
λλ μ€κ΅μ΄ μλλΌ λ―Έκ΅μ μμ΅λλ€. νμ§λ§ VPNμ΄ ν΄λΉ μ¬μ΄νΈλ₯Ό μ°¨λ¨νλ€κ³ μκ°ν©λλ€. μ΄μ¨λ , μ λμ΄ μ€λ λμ μ€λͺ λ λͺ¨λ λ¨κ³λ₯Ό λ°λκ³ μ΄λ―Έμ§λ₯Ό μλμΌλ‘ κ°μ Έ μ€λ €κ³ μλνμ§λ§ μλ΅νμ§ μλ κ²μ νμΈν λκΉμ§ μλνμ§ λͺ»νμ΅λλ€. λλ₯Ό.
λν μ€λ₯κ° λ°μν©λλ€.
$ helm init
$HELM_HOME has been configured at C:\Users\user\.helm.
Error: error installing: the server could not find the requested resource
λλμ΄ λ¬Έμ λ₯Ό, νΉν μ μνλ μ루μ μλνκ³ μ΄ νλ . κ·Έλ¬λ κ·Έμ λ°λΌ tiller.yaml νμΌμ μμ ν ν ꡬμ±μ μ λ°μ΄νΈ ν μ μμ΅λλ€. λ³κ²½ μ¬νμ μ μ©νκ³ κ΅¬μ±μ μ λ°μ΄νΈνκΈ° μν΄ λ€μ λͺ λ Ήμ μλνκ³ μμ΅λλ€.
$ kubectl apply -f tiller.yaml
deployment.apps/tiller-deploy configured
service/tiller-deploy configured
νμ§λ§ λ΄κ° μ€ννλ©΄ :
$ helm init --output yaml > tiller2.yaml
tiller2.yaml νμΌμ λ€μμ 보μ¬μ€λλ€.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: helm
name: tiller
name: tiller-deploy
namespace: kube-system
spec:
replicas: 1
strategy: {}
template:
κΈ°λ³Έμ μΌλ‘ λ³κ²½ μ¬νμ λ°μλμ§ μμ΅λλ€. λ°λΌμ ꡬμ±μ μ λλ‘ μ λ°μ΄νΈνμ§ μλλ€κ³ κ°μ ν©λλ€. κ·Έκ²μ μννλ μ¬λ°λ₯Έ λ°©λ²μ 무μμ λκΉ?
νΈμ§ : λλ κ·Έκ²μ μ€νν μ μμλ€. μ λ Minikubeλ₯Ό μ¬μ©νκ³ μμΌλ©°μ΄λ₯Ό μ€ννκΈ° μν΄ λ¨Όμ Kubernetes λ²μ μ 1.15.4λ‘ λ€μ΄ κ·Έλ μ΄λνμ΅λλ€.
minikube delete
minikube start --kubernetes-version=1.15.4
κ·Έλ° λ€μ νλ‘μλ₯Ό μ¬μ©νκΈ° λλ¬Έμ Minikubeμ IPλ₯Ό NO_PROXY λͺ©λ‘μ μΆκ°ν΄μΌνμ΅λλ€. μ κ²½μ°μλ 192.168.99.101
μ
λλ€. μ°Έμ‘° : https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
μ°Έκ³ : λͺ κ°μ§ μΆκ° ν
μ€νΈ ν λ€μ΄ κ·Έλ μ΄λκ° νμνμ§ μμ μ μμΌλ©° λλ½ λ κ²μ NO_PROXY λ¨κ³λΏμ
λλ€. 192.168.99.0/24
, 192.168.39.0/24
λ° 10.96.0.0/12
μ λͺ¨λ NO_PROXY μ€μ μ μΆκ°νλλ° μ΄μ μ λλ‘ μλνλ κ² κ°μ΅λλ€.
helm init --service-account tiller --override spec.selector.matchLabels.'name '='tiller ', spec.selector.matchLabels.'app'= 'helm'--output yaml | sed ' s @ apiVersion : extensions / v1beta1 @ apiVersion : apps / v1 @'| kubectl apply -f-
κ·Έκ²μ λλ₯Ό μν΄ μΌνμ΅λλ€, μ λ§ κ°μ¬ν©λλ€
Kubernetes APIκ° λ°μ ν¨μ λ°λΌ APIλ μ£ΌκΈ°μ μΌλ‘ μ¬κ΅¬μ±λκ±°λ μ κ·Έλ μ΄λλ©λλ€. APIκ° λ°μ νλ©΄ μ΄μ APIλ λ μ΄μ μ¬μ©λμ§ μμΌλ©° κ²°κ΅ μ κ±°λ©λλ€.
v1.16 릴리μ€λ λ μλ‘κ³ μμ μ μΈ API λ²μ μ μν΄ λ€μκ³Ό κ°μ μ§μ μ€λ¨ λ API λ²μ μ 곡μ μ€μ§ν©λλ€.
NetworkPolicy (in the extensions/v1beta1 API group)
Migrate to use the networking.k8s.io/v1 API, available since v1.8. Existing persisted data can be retrieved/updated via the networking.k8s.io/v1 API.
PodSecurityPolicy (in the extensions/v1beta1 API group)
Migrate to use the policy/v1beta1 API, available since v1.10. Existing persisted data can be retrieved/updated via the policy/v1beta1 API.
DaemonSet, Deployment, StatefulSet, and ReplicaSet (in the extensions/v1beta1 and apps/v1beta2 API groups)
Migrate to use the apps/v1 API, available since v1.9. Existing persisted data can be retrieved/updated via the apps/v1 API.
v1.20 릴리μ€λ λ μλ‘κ³ μμ μ μΈ API λ²μ μ μν΄ μ¬μ©λμ§ μλ λ€μ API λ²μ μ μ 곡μ μ€μ§ν©λλ€.
Ingress (in the extensions/v1beta1 API group)
Migrate to use the networking.k8s.io/v1beta1 API, serving Ingress since v1.14. Existing persisted data can be retrieved/updated via the networking.k8s.io/v1beta1 API.
μΈμ©νλ€ :
minikubeλ₯Ό μ¬μ©νλ helm n00bλ‘μ kubernetes λ²μ μ λ€μκ³Ό κ°μ΄ μ€μ νμ¬μ΄ λ¬Έμ λ₯Ό ν΄κ²°ν μμμμ΅λλ€.
$ minikube delete
$ minikube start --kubernetes-version=1.15.4
λμμ΄ λμκΈ°λ₯Ό λ°λλλ€.
@PierreF k8s v1.16.1 λ° helm v2.15.0κ³Ό ν¨κ» μ루μ (https://github.com/helm/helm/issues/6374#issuecomment-533186177)μ μ¬μ©νμΌλ©° κ²½μ΄κΈ°κ° μλνμ§ μμ΅λλ€.
Readiness probe failed: Get http://10.238.128.95:44135/readiness: dial tcp 10.238.128.95:44135: connect: connection refused
@ joshprzybyszewski-wf λ€μ λͺ λ Ήμ μ¬μ©νμ΅λλ€.
minikube start --memory=16384 --cpus=4 --kubernetes-version=1.15.4
kubectl create -f istio-1.3.3/install/kubernetes/helm/helm-service-account.yaml
helm init --service-account tiller
helm install istio-1.3.3/install/kubernetes/helm/istio-init --name istio-init --namespace istio-system
helm install istio-1.3.3/install/kubernetes/helm/istio --name istio --namespace istio-system
κ·Έλ¦¬κ³ μ΄μ
Error: validation failed: [unable to recognize "": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3", unable to recognize "": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3", unable to recognize "": no matches for kind "attributemanifest" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "attributemanifest" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "handler" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "handler" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "instance" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "rule" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "rule" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "rule" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "rule" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "rule" in version "config.istio.io/v1alpha2", unable to recognize "": no matches for kind "rule" in version "config.istio.io/v1alpha2"]
λ€μμ λ¨κΈ°μ μΈ ν΄κ²° λ°©λ²μ λλ€.
helm init --output yaml | sed 's<strong i="7">@apiVersion</strong>: extensions/v1beta1<strong i="8">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
μ¬μ€ μΆ©λΆνμ§ μμ΅λλ€. μ¬μ ν μ€λ₯κ° λ°μν©λλ€.
error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec
λ€μκ³Ό κ°μ΄ ν¨μΉ ν μ μμ΅λλ€.
/usr/local/bin/kubectl patch --local -oyaml -f - -p '{"spec":{"selector": {"app":"helm","name":"tiller"}}}'
macthLabels
κ²μλ¬Ό μ νκΈ°λ₯Ό μΆκ°νμ§ λͺ»νμ΅λλ€.
@jbrette μ μ루μ μΌλ‘ μ λ¬λμμ΅λλ€. μ΄κ²μ΄ λ΄κ° κ·Έκ²μ μ€ννμ λ μ»μ κ²μ λλ€.
error: error parsing STDIN: error converting YAML to JSON: yaml: line 11: mapping values are not allowed in this context
Helm 2.16.0 μμ μμ λμμ΅λλ€.
@jbrette μ μ루μ μΌλ‘ μ λ¬λμμ΅λλ€. μ΄κ²μ΄ λ΄κ° κ·Έκ²μ μ€ννμ λ μ»μ κ²μ λλ€.
error: error parsing STDIN: error converting YAML to JSON: yaml: line 11: mapping values are not allowed in this context
yaml νμΌμ νμΈνμΈμ. λλΆλΆμ κ²½μ° μ°Έμ‘° λ μ€μ {} λλ []κ° μκ³ κ·Έ μλμ μ€λ₯λ₯Ό μΌμΌν€λ λ€λ₯Έ νλͺ©μ΄ μ μλμ΄ μμ΅λλ€. λλΆλΆμ κ²½μ° λ¬Έμ λ values.yaml λ΄μ μμ΅λλ€. κ·Έλ μ§ μμΌλ©΄ μ°¨νΈμ ν νλ¦Ώ μΉμ μ νμΈνμΈμ.
@PierreF λ° @mihivagyok μ루μ μ λν μ°Έκ³ μ¬νμ λλ€. λ΄κ° κ°μΈ ν€ μ μ₯μλ₯Ό μ¬μ©ν λ κ·Έκ²λ€μ λλ₯Ό μν΄ μλνμ§ μμμ΅λλ€.
$ helm repo add companyrepo https://companyrepo
Error: Couldn't load repositories file (/home/username/.helm/repository/repositories.yaml).
helm initμ΄ μ€νλμ§ μμκΈ° λλ¬Έμ λ°μνλ κ²μΌλ‘ μκ°νκ³ yaml νμΌμ μμ±ν©λλ€. helm init -c
λ₯Ό μΆκ°λ‘ μ€ννμ¬ λ¬Έμ λ₯Ό ν΄κ²°νμ΅λλ€.
k8s v1.16.6μμ helm init otputμλ spec.selector fyiκ° νμν©λλ€.
νμ¬ ν΄κ²° λ°©λ²μ λ€μκ³Ό κ°μ΅λλ€.
helm init --output yaml> tiller.yaml
tiller.yamlμ μ λ°μ΄νΈνμμμ€.
- apps / v1λ‘ λ³κ²½
- μ νκΈ° νλ μΆκ°
--- apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: helm name: tiller name: tiller-deploy namespace: kube-system spec: replicas: 1 strategy: {} selector: matchLabels: app: helm name: tiller ....
μλν©λλ€. kubernetesκ° apiVersion apps / v1μ λ°°ν¬ μ©μΌλ‘ λ³κ²½νκΈ° λλ¬Έμ λ³κ²½ν΄μΌ ν ν κ°μ§λ μ¬μμ λν μ νκΈ° matchLabelsλ₯Ό μΆκ°ν΄μΌνλ€λ κ²μ λλ€.
λ λ€λ₯Έ ν΄κ²° λ°©λ²μ tillerλ₯Ό μ¬μ©νμ§ μλ Helm 3μ μ¬μ©νλ κ²μ λλ€.
helm init --output yaml | sed ' s @ apiVersion : extensions / v1beta1 @ apiVersion : apps / v1 @'| kubectl apply -f-
μλ νμΈμ, μ΄κ²μ μλνλ λμ μ΄κ²μ μ»μμ΅λλ€.
jenkins @ jenkin : ~ / .kube $ helm init --output yaml | sed ' s @ apiVersion : extensions / v1beta1 @ apiVersion : apps / v1 @'| kubectl apply -f-
'kubectl'λͺ λ Ήμ μ°Ύμ μ μμ§λ§ λ€μκ³Ό ν¨κ» μ€μΉν μ μμ΅λλ€.
kubectl μ€λ
μ€μΉ
κ΄λ¦¬μμκ² λ¬Έμνμμμ€.
jenkins @ jenkin : ~ / .kube $
helm version
μΆλ ₯ : v2.14.3
kubectl version
μΆλ ₯ : ν΄λΌμ΄μΈνΈ : v1.15.3, μλ² : v1.16.0-rc.1
ν΄λΌμ°λ μ 곡μ / νλ«νΌ (AKS, GKE, Minikube λ±) : IBM Cloud Kubernetes Service$ helm init --service-account tiller $HELM_HOME has been configured at /Users/xxxx/.helm. Error: error installing: the server could not find the requested resource $ helm init --debug --service-account tiller --- apiVersion: extensions/v1beta1 kind: Deployment metadata: creationTimestamp: null labels: app: helm name: tiller name: tiller-deploy namespace: kube-system spec: . . .
helmμ΄
tiller
λ°°ν¬λ₯Ό μμ±νλ €κ³ μλνλ κ² κ°μ΅λλ€.apiVersion: extensions/v1beta1
μ λ°λ₯΄λ©΄ : https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16
λ μ΄μ μ§μλμ§ μμ΅λλ€.
μ΄ μ€λ₯κ° λ°μν©λλ€. μ΄λ»κ² ν΄κ²°ν μ μμ΅λκΉ ???
root @ jenkin : ~ # helm init --service-account tiller
$ HELM_HOMEμ΄ /root/.helmμ ꡬμ±λμμ΅λλ€.
μ€λ₯ : μ€λ₯ μ€μΉ : μ μ μμ (λ°°ν¬ ν νμ₯ νλ‘κ·Έλ¨)
root @ jenkin : ~ #
λ€μμ λ¨κΈ°μ μΈ ν΄κ²° λ°©λ²μ λλ€.
helm init --output yaml | sed 's<strong i="7">@apiVersion</strong>: extensions/v1beta1<strong i="8">@apiVersion</strong>: apps/v1@' | kubectl apply -f -
μ¬μ€ μΆ©λΆνμ§ μμ΅λλ€. μ¬μ ν μ€λ₯κ° λ°μν©λλ€.
error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec
λ€μκ³Ό κ°μ΄ ν¨μΉ ν μ μμ΅λλ€.
/usr/local/bin/kubectl patch --local -oyaml -f - -p '{"spec":{"selector": {"app":"helm","name":"tiller"}}}'
μ΄ μ€λ₯κ° λ°μν©λλ€.
jenkins @ jenkin : ~ / .helm $ helm init --output yaml | sed ' s @ apiVersion : extensions / v1beta1 @ apiVersion : apps / v1 @'| kubectl apply -f-
'kubectl'λͺ λ Ήμ μ°Ύμ μ μμ§λ§ λ€μκ³Ό ν¨κ» μ€μΉν μ μμ΅λλ€.
kubectl μ€λ
μ€μΉ
κ΄λ¦¬μμκ² λ¬Έμνμμμ€.
jenkins @ jenkin : ~ / .helm $
jq
μ¬μ©νμ¬ ν΄κ²° λ°©λ² :
helm init -o json | jq '(select(.apiVersion == "extensions/v1beta1") .apiVersion = "apps/v1")' | jq '(select(.kind == "Deployment") .spec.selector.matchLabels.app = "helm")' | jq '(select(.kind == "Deployment") .spec.selector.matchLabels.name = "tiller")' | kubectl create -f -
jq
μ¬μ©νμ¬ ν΄κ²° λ°©λ² :
helm init -o json | jq '(select(.apiVersion == "extensions/v1beta1") .apiVersion = "apps/v1")' | jq '(select(.kind == "Deployment") .spec.selector.matchLabels.app = "helm")' | jq '(select(.kind == "Deployment") .spec.selector.matchLabels.name = "tiller")' | kubectl create -f -
kubectl create
리μμ€λ₯Ό μ
λ°μ΄νΈ ν μ μμ΅λλ€.
@ikarlashovλ 'λ§λ€κΈ°'λ₯Ό 'μ μ©'μΌλ‘ λ체νκΈ°μ μΆ©λΆν©λλ€. μμ ν μ€μ μμ§ λ¦¬μμ€ μμ±μ μλνμ§ μμλ€κ³ κ°μ ν©λλ€.
κ°μ₯ μ μ©ν λκΈ
λ€μ sedλ λλ₯Ό μν΄ μλν©λλ€.
@mattymo μ루μ (kubectl patch --local μ¬μ©)μ λ¬Έμ λ μ λ ₯μ μ¬λ¬ 리μμ€ (μ¬κΈ°μλ λ°°ν¬ λ° μλΉμ€)κ° ν¬ν¨ λ κ²½μ° μλνμ§ μλ κ² κ°μ΅λλ€.