Kubernetes: configmap file mount path results in command not found error

Created on 23 Apr 2017  ·  10Comments  ·  Source: kubernetes/kubernetes

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): No

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): command not found configmap kubernetes

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG

Kubernetes version (use kubectl version):
Client Version: v1.6.1 GitCommit:"b0b7a323cc5a4a2019b2e9520c21c7830b7f708e"
Server Version: v1.6.0 GitCommit:"fff5156092b56e6bd60fff75aad4dc9de6b6ef37

Environment:

  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release): host is ubuntu 16.04
  • Kernel (e.g. uname -a): host is Linux dev1 4.4.0-72-generic #93-Ubuntu SMP Fri Mar 31 14:07:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Others: Running from within minikube version: v0.18.0

What happened:
When I try to create a deployment with a configmap file that is mounted into the same directory as the entry point , the container fails to start with the following error
"Error response from daemon: Container command '/app/app.sh' not found or does not exist."
The pod spec includes a configmap which is mounted into the same directory as the entrypoint

Seems the entry point script is lost after mounting the volume for the config map in the same directory
If I mount the configmap file into a sub directory all works as expected

What you expected to happen:
I expected the config map file to be created in the directory without effecting the existing directory content which in this case contains an entrypoint script

How to reproduce it (as minimally and precisely as possible):

Docker file - note the entry point

FROM busybox:latest

RUN        mkdir /app
COPY       app.sh /app

ENTRYPOINT ["/app/app.sh"]

Entry point script - infinite loop

#!/bin/sh
seq=1
while [[ true ]]; do
    echo "${seq} $(date) working"
    sleep .5s   
    let seq=$((seq + 1))
done

k8s configmap and deployment file

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    product: k8s-demo
  name: demo
data:
  settings.json: |
    {
      "store": {
        "type": "InMemory",
    }

---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  labels:
    product: k8s-demo
  name: demo
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: demo
        product: k8s-demo
    spec:
      containers:
      - name: demo
        image: pmcgrath/shellloop:1
        imagePullPolicy: Always
        volumeMounts:
          - name: demo-config
            mountPath: /app
      volumes:
        - name: demo-config
          configMap:
            name: demo
            items:
              - key: settings.json
                path: settings.json

When I run the kubectl apply -d k8s.yaml and look at the pod I can see the following error

rpc error: code = 2 desc = failed to start container "f9e0112c80ebba568d4b508f99ffb053bf1ae5a4f095ce7f45bff5f38900b617": Error response from daemon: Container command '/app/app.sh' not found or does not exist.

Anything else we need to know:
If I change the mountPath for the volume to any other directory it works as expected

I did test this directly with docker on my host (17.03.0-ce) and it worked as expected 

touch settings.json
docker container run -ti -v $(pwd)/settings.json:/app/settings.json pmcgrath/shellloop:1
picture pmcgrath
👍1

Most helpful comment

@pmcgrath

Checkout here.

it seems I understand your issue. I have the same question before, but there is an answer actually in your situation.

To brief your case, you have a configmap(settings.json: blahblah), and want to mount into a folder /app. Then below is what you need to know:

  1. Once you mount a volume(no matter it is configmap or others), it overrides the mountPath, so in your case, the /app folder will only contain settings.json.
  2. I know it is not what you expected, so you have to specify the mountPath: /app/settings.json, only that way, the original content in /app folders won't be affected.
  3. well, when you do the second step, you remember that configmap is actually a list of key-value pairs, you only need one of the keys(though you actually have only one as well), so you need to tell the volume mounts to use a subpath from you configmap.

This is something that you will get eventually:

containers:
- volumeMounts:
  - name: demo-config
    mountPath: /app/settings.json
    subPath: settings.json
volumes:
- name: demo-config
  configMap:
    name: demo
picture zhouhaibing089 on 25 Apr 2017
👍284 ❤82 🎉54 😄39 👎3 🚀2

All 10 comments

@pmcgrath

Checkout here.

it seems I understand your issue. I have the same question before, but there is an answer actually in your situation.

To brief your case, you have a configmap(settings.json: blahblah), and want to mount into a folder /app. Then below is what you need to know:

  1. Once you mount a volume(no matter it is configmap or others), it overrides the mountPath, so in your case, the /app folder will only contain settings.json.
  2. I know it is not what you expected, so you have to specify the mountPath: /app/settings.json, only that way, the original content in /app folders won't be affected.
  3. well, when you do the second step, you remember that configmap is actually a list of key-value pairs, you only need one of the keys(though you actually have only one as well), so you need to tell the volume mounts to use a subpath from you configmap.

This is something that you will get eventually:

containers:
- volumeMounts:
  - name: demo-config
    mountPath: /app/settings.json
    subPath: settings.json
volumes:
- name: demo-config
  configMap:
    name: demo
zhouhaibing089 picture zhouhaibing089 on 25 Apr 2017
👍284 ❤82 🎉54 😄39 👎3 🚀2

@zhouhaibing089
Thanks for the reply, works based on your suggestion, I appreciate the explanation

I am happy to close this issue
Pat

pmcgrath picture pmcgrath on 26 Apr 2017

For reference, the original mention of this solution seems to be here: https://github.com/kubernetes/kubernetes/issues/23748#issuecomment-230390309

It looks like the documentation for this is missing, which makes this case fairly confusing/misleading, and the projection docs seem to make it more misleading as well -- not sure if it's missing because auto updates apparently don't work as per that issue

agilgur5 picture agilgur5 on 2 Nov 2017

The requirement for the file name to be specified both under mountPath and subPath is counterintuitive.

emwalker picture emwalker on 28 Dec 2017
👍15

The solution provided by @zhouhaibing089 works but the content of the mounted file at subPath doesn't update if we edit it in the resembled ConfigMap.

vishaltelangre picture vishaltelangre on 29 Dec 2017
👍12

IMO, this isn't _really_ solved. There should be an option to append each key rather than overwrite.

Something like:

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-conf
data:
  example1.conf: |
    server {
      server_name example1.com;
      # config here...
    }
  example2.conf: |
    server {
      server_name example2.com;
      # config here...
    }

So that...

- name: nginx-conf
  mountPath: /etc/nginx/conf.d
  append: true

... keeps the existing default.conf and any other artefacts of the Docker image, but augments with example*.conf

Repeating the same info in subPath is just _icky_.

leebenson picture leebenson on 31 Jul 2018
👍19

+1 on @leebenson 's append: true option.

mehemken picture mehemken on 13 Sep 2018

According to @leebenson answer: can anyone explain where from is coming append: true option ??

it does not work for me, i got: 

error: error validating "hdfs/21-namenode-statefulset.yaml": error validating data: ValidationError(StatefulSet.spec.template.spec.containers[1].volumeMounts[1]): unknown field "append" in io.k8s.api.core.v1.VolumeMount; if you choose to ignore these errors, turn validation off with --validate=false

Also, it is not present in api docs for volume mount :
https://k8smeetup.github.io/docs/api-reference/v1.9/#volumemount-v1-core

val1715 picture val1715 on 12 Mar 2019
👍1

According to @leebenson answer: can anyone explain where from is coming append: true option ??

it does not work for me, i got:

error: error validating "hdfs/21-namenode-statefulset.yaml": error validating data: ValidationError(StatefulSet.spec.template.spec.containers[1].volumeMounts[1]): unknown field "append" in io.k8s.api.core.v1.VolumeMount; if you choose to ignore these errors, turn validation off with --validate=false

Also, it is not present in api docs for volume mount :
https://k8smeetup.github.io/docs/api-reference/v1.9/#volumemount-v1-core

it doesn't exist, it's a suggestion. a good one for the use case.

Zvikan picture Zvikan on 18 Mar 2019

I tried to use this method, but I got a Read-only file system error, when I applied the statefulset. Does anyone know how to fix that?
(I'm overwriting an existing file, that contains settings for elasticsearch, actual error message --> /usr/share/elasticsearch/bin/run.sh: line 28: ./config/elasticsearch.yml: Read-only file system)

fbede4 picture fbede4 on 24 Apr 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

montanaflynn picture montanaflynn  ·  3Comments
jadhavnitind picture jadhavnitind  ·  3Comments
theothermike picture theothermike  ·  3Comments
zetaab picture zetaab  ·  3Comments
mml picture mml  ·  3Comments