Portainer: Cannot create/modify/save ENV variables on Synology environment.

Bug only on Synology.

@chreggy have you been able to test this on another environment to confirm that this is only problematic on Synology? Synology Docker GUI works ok?

It's working on my Debian VM and Synology GUI.

@chreggy what error are you seeing? Something in the UI? Can you inspect the container creation HTTP request via your browser dev tools?

Hello,

there is no error on deployment. Where can I see the real HTTP request ?

@chreggy in your browser you should be able to open the "developer tools" (F12 on Chrome for example), then simply refresh the view and inspect the requests in the network panel of the developer tools.

Hello,

2 screenshots :

With Env variable, not saving it :

With a Label and saving OK :

Is it what you want ?

I can confirm having the same issue on my Synology NAS. No error message at all but adding ENV variables just get ignored when deploying/editing a container.

Same problem here with Portainer 1.20.1

1.20.2 218+ same problem

Same issue as above, 1.20.2, Synology 218+ , unable to make changes to ENV variables

Everything else works (creating containers etc)

It might be related to the way docker is installed there. I am experiencing an issue that Synology's docker ignores environment from swarm services commands:
This command works directly on NAS

sudo docker run --rm --env SERVER=google.com alpine:latest sh -c 'ping ${SERVER}'

However while running this one, no environment is set on Synology NAS node, but works on other nodes.

docker service create --mode global --name ping-alpine --env SERVER=google.com alpine:latest sh -c 'ping ${SERVER}'

Docker on Synology NAS seems to be pretty outdated

docker --version
Docker version 17.05.0-ce, build d49b918-synology

and IMO can be the root cause of the issue

Same issue. Changes gets reverted when editing a container.

Portainer 1.21.0
Synology 918+
Docker version 17.05.0-ce, build 371caef-synology

Any of you guys found a solution to this issue ?

Docker for Synology package has just been updated to 18.09.6 version, I have yet to see if that improves the ENV variables handling in Portainer.

Update after testing : no improvement spotted

For me it didn’t

On Fri, 30 Aug 2019 at 0:49 KamistixX notifications@github.com wrote:

Docker for Synology package has just been updated to 18.09.6 version, I
have yet to see if that improves the ENV variables handling in Portainer.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/portainer/portainer/issues/2423?email_source=notifications&email_token=AHJP7PVY72PN477ZG4B2IITQHA75FA5CNFSM4GAPBQ72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5P5UOI#issuecomment-526375481,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHJP7PU4ESTLBQ4PD2KTVE3QHA75FANCNFSM4GAPBQ7Q
.

>

https://lightricks.com/
https://www.facebook.com/Lightricks/ https://twitter.com/Lightricks
https://www.instagram.com/lightricks/
https://www.youtube.com/channel/UCKWhLS9QMr1oNthZL1fZB4A
https://www.linkedin.com/company/lightricks/
Ohad Benita
Senior Backend Developer ▪︎ Lightricks
+972-50-8246047 [email protected]
https://lightricks.com

No changes for me at least. Same problem.
Docker version 18.09.6, build 8cdf373

For me also no change. And watchtower is not able to re-set ENV anymore with this new version.

Kind of known issue for this version: https://community.synology.com/enu/forum/15/post/128146

This has been an issue for me for some time now, I'm glad there's a ticket open for it.

When I use Portainer to recreate the container, and I update an environment variable, the resulting container has the previous environment variable set, not the new value I had just set.

new here, is the advised interim solution(in the synology forum post linked above) 'to use the synology docker UI to set the env var' practical? It would seem not given how a new container is made for each operation?

There’s a new docker release for Synology that solves the problem

On Thu, 10 Oct 2019 at 0:05 scyto notifications@github.com wrote:

new here, is the advised interim solution 'to use the synology docker UI
to set the env var' practical? It would seem not given how a new container
is made for each operation?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/portainer/portainer/issues/2423?email_source=notifications&email_token=AHJP7PSSXLDH5HQEB7ONU6LQNZISBA5CNFSM4GAPBQ72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAZSLGQ#issuecomment-540222874,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHJP7PSULBYOG6OJU4JBFVLQNZISBANCNFSM4GAPBQ7Q
.

>

https://lightricks.com/
https://www.facebook.com/Lightricks/ https://twitter.com/Lightricks
https://www.instagram.com/lightricks/
https://www.youtube.com/channel/UCKWhLS9QMr1oNthZL1fZB4A
https://www.linkedin.com/company/lightricks/
Ohad Benita
Senior Backend Developer ▪︎ Lightricks
+972-50-8246047 [email protected]
https://lightricks.com

There’s a new docker release for Synology that solves the problem

That is not true. With the latest stable Docker release by Synology (18.09.0-0506) the issue still exists. Do you refer another version? On 29th Aug you commented by yourself that it does't work with this version...

Dear Customer,

Thank you for contacting Synology support.

I checked for some global server, there are some issue with Package Center
release.

Please find the following pkg download link and manual update in Package
Center if there isn't update information.

https://global.download.synology.com/download/Package/spk/Docker
/18.09.0-0506/Docker
-x64-18.09.0-0506.spk?model=DS918%2B&bays=4&dsm_version=6.2.2&build_number=24922

Sorry for any inconvenience caused.

On Thu, 10 Oct 2019 at 15:15 r2ixuz notifications@github.com wrote:

There’s a new docker release for Synology that solves the problem

That is not true. With the latest stable Docker release by Synology
(18.09.0-0506) the issue still exists. Do you refer another version? On
29th Aug you commented by yourself that it does't work with this version...

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/portainer/portainer/issues/2423?email_source=notifications&email_token=AHJP7PVC66RUJOMH6WN3R2LQN4THFA5CNFSM4GAPBQ72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA4IBEA#issuecomment-540573840,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHJP7PUYPGPKXO3UIQS4FTLQN4THFANCNFSM4GAPBQ7Q
.

>

https://lightricks.com/
https://www.facebook.com/Lightricks/ https://twitter.com/Lightricks
https://www.instagram.com/lightricks/
https://www.youtube.com/channel/UCKWhLS9QMr1oNthZL1fZB4A
https://www.linkedin.com/company/lightricks/
Ohad Benita
Senior Backend Developer ▪︎ Lightricks
+972-50-8246047 [email protected]
https://lightricks.com

Thanks for the link @ohadbenita , i will give it a try - what terrible engineering bad practice to rev the package but not the package version number?!

I don't know what you two guys talking about. This .spk package is the same one which comes via Package Center. And even if you install this SPK manually the issue where you cannot modify/add ENV via Portainer is still persistent.

@r2ixuz me neither lol, interestingly this package is not the same 18.09.0-0506 one as i previously had , it at least has a later docker version 18.09.8 vs 18.09.6. Interestingly i have two synologies, both had the same 'latest' package from package center installed on different dates. They seem to have different contents. This means folks with the 18.09.0-0506 package don't all have the same package depending on when they downloaded it!?

Also it has not fixed the issue for me with portainer - the env vars are still not present.

For example i just added
1 environment:
CAP_HOST_MANAGEMENT: 1

to my stack definition and pushed new service definition out to the two synology nodes and the env var still doesn't show up even with the latest version.... if you found a way to make it work, i am all ears!

this seems to be caused by the docker compose 1.24.0 version, supposedly updating to 1.24.1 fixes the issue... https://github.com/docker/compose/issues/6889

On my DS1511+ only the Docker client is 18.09.8 with the latest Synology Docker package:

root@syn:~# docker version
Client:
 Version:           18.09.8
 API version:       1.39
 Go version:        go1.11
 Git commit:        2c0a67b
 Built:             Fri Sep  6 02:50:44 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.11
  Git commit:       1d8275b
  Built:            Fri Sep  6 02:51:05 2019
  OS/Arch:          linux/amd64
  Experimental:     false

yes that matches mine, i have the issue what about your docker-compose version?
The issue is with docker-compose not docker.
docker-compose version 1.24.0, build 0aa59064 docker-py version: 3.7.2 CPython version: 3.6.8 OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018

root@syn:~# docker-compose version
docker-compose version 1.24.0, build 0aa59064
docker-py version: 3.7.2
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

dunno what to say, it is not broken for you, awesome, take a look at the docker compose bug link -others are having the issue.

It does not work for me on my Synology too. As I stated multiple times.

But I have another Docker server running on a Gentoo machine and there is no docker-compose installed. I think Portainer does not use docker-compose, because container deploying via Portainer Stack works perfectly fine there. And recreating container with modified/new ENV is also no problem.

your gentoo box has different files in different locations and just proves that this is a synology specific issue, you aske me what i was talking about - i was just saying i would test the package the other one refered to. It a)was different to what i had with same version number and b) doesn't work for me. I am in middle of installing docker-compose 1.24.1

updating to 1.24.1 of docker-compose has not fixed it for me.

More testing.

It seems that when i use a docker-compose.yaml file like this and run just docker-compose up the env var gets passed correctly (note this doesn't result in a working agent, that wasn't the point of the test). Does portainer construct the yaml file on the fly or is it passing the env vars in using -e option?

services:
  agent-test:
    container_name: agent-test
    environment:
      CAP_HOST_MANAGEMENT: '1'
    image: portainerci/agent:develop
    logging:
      driver: json-file
      options:
        max-size: 10m
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/docker/volumes:/var/lib/docker/volumes

@deviantony with help from the compose folks about https://github.com/docker/compose/issues/6889 i can confirm that this issue doesn't seem present on my system when using command line (unless the issue is way more complex.... But that portainer still fails to deploy env vars with stack deployment (it works with container deployments)s, so i think it maybe something different. Can you detail what portainer is doing and how i might replicate via command line?

More info:
I created service by hand in the UI (to remove stacks being a variable here) with just the following set:
name=test
image=nginx:latest
disable access control
add environment variable MYVAR=morty
all other settings default

when this service deployed my env var was not set, how can i replicate this on the command line on the node so i can eliminate portainter completely or not?

ok i think i have found the issue... when i run this no env var seems to get set... did i get synatx right?

 docker service create \
    --name test4 \
    --env MYVAR=rick \
    --mode global \
    nginx:latest

i tired --env and -e and got same result.

@paulkling seems to have confirmed this command is right. As such the issue appears to be that docker service create is not processing the env parameter correctly. I am at a loss what to do next?

official synology response to env vars not working with stacks / swarms

It looks like this is most likely being caused by the swarm function of the Docker, which is an issue that our developers are aware of. They are working on a fix, and it should be available in a future release, but there is no workaround in the meantime.

@deviantony i guess you might want to close this ticket as issue is external and known?

@scyto indeed. Thanks for the investigation and the update.

Closing as this is related to the platform and not Portainer.

Anyone have a fix? Have a synology DS1515+ with Docker 18.09.0-0506 Portainer 1.23.2

When I create a container with Portainer the ENV variables are set fine. But if I click Edit/Duplicate and add/edit a ENV variable it doesn't change on the new container (even selecting delete old container). I must manually enter all settings into a new container for it to take which sucks when there are tons of ENVs and Volume mappings etc.

did you try reverting to old release of docker spk from synology, i know that fixed some elements of this issue for them.

This was the official synology statement i gto from support, we haven't see an update since they said this in October'19. My suspicion is we will have to wait until DMS7 later in H2.

It looks like this is most likely being caused by the swarm function of the Docker, which is an issue that our developers are aware of. They are working on a fix, and it should be available in a future release, but there is no workaround in the meantime.

did you try reverting to old release of docker spk from synology, i know that fixed some elements of this issue for them.

This was the official synology statement i gto from support, we haven't see an update since they said this in October'19. My suspicion is we will have to wait until DMS7 later in H2.

It looks like this is most likely being caused by the swarm function of the Docker, which is an issue that our developers are aware of. They are working on a fix, and it should be available in a future release, but there is no workaround in the meantime.

* off topic * Man I can't wait til DS7. Synology's new Photo app looks amazing.

I'll try an older version. Thanks

i just discovered this workaround, but it is not without issue as it seems some had issues trying to create new networks, which for portainer may be a big issue

https://community.synology.com/enu/forum/1/post/131600

Am I correct in saying this is still an issue with the latest Docker Synology package?

Am I correct in saying this is still an issue with the latest Docker Synology package?

I found my way to this page after Googling for this issue because I have Portainer running on my Synology NAS, and editing a previously created container with new ENV changes do not stick, so I would say yes, it is still an issue.

still in issue here also !

Still issue on on my DS210+. Has anybody been able to find a solution?

solution here https://github.com/markdumay/synology-docker

@siddjellali Thanks for the pointer!

Sad this issue is not addressed in the (un)official Synology docker app.

Anybody has a decent proposal as to where it's best practice to install this (https://github.com/markdumay/synology-docker) folder-wise on my NAS?

The context of my docker usage is that I am trying to get a Conbee 2 dongle up and running on my Nas (although I reckon that does not warrant a different install practice)

solution here https://github.com/markdumay/synology-docker

Sorry, just to clarify. The solution to get Portainer to work properly is to install another script that ensures Docker on Synology is up-to-date?

I'm not sure how this "fixes" the issue with Portainer - changing the ENV files in Portainer previously used to work, so what caused it to stop all of a sudden?

Just seems risky to use a third-party script to update Docker on Synology.

Hi,

Actually this script solves a lot of limitation or issue with the official Synology release.
You can try it !! Script allows backup and restore.
It works for me :P

@Sid-Ahmed

Great you got it working. Honestly like Wesley I'm having a few concerns too. Specifically in terms of stability as the idea was to have a docker instance running the hub in my IoT chain... meaning it has to be pretty stable.

Seems there are limitations in terms of virtualisation vs USB devices (in terms of effectuating firmware upgrades on external devices) so I may have to kill my darlings here.

Anyway, out of curiosity what folder on your Synology Nas did you install this script into?

Guess /usr/local/bin would the correct install path (?)

However, as per the issue tracker of the docker script kindly made available by markdumay seems this docker version does not really integrate well into the Synology ecosystem + GUI in its present version.

I dont know enough bout what is advisable besides I want Synology off their butts to fix this.

Worthwhile comment - potential solution:

Just realised that I am actually able to change container environment variables via the container "Edit" function.

What I had to do was highly advanced - so bear with me:

Press enter after each inserted value. Then hit apply.

Also, related issue - had to do a port binding via the "Port settings" tab to map out the desired port of the app. Adding alternative port under Environment variables did nothing.

Hope that info may be of help to others here.

Press enter after each inserted value. Then hit apply.

That doesn't work. There is no "apply" button in Portainer from what I can see. To be clear, this is about changing environmental variables from within Portainer using the Duplicate / Edit button, then changing the values in the Environment tab. Based on what you've said above @NoSpamMaam, are you talking about the Docker UI, maybe within Synology's DSM?

Also, related issue - had to do a port binding via the "Port settings" tab to map out the desired port of the app. Adding alternative port under Environment variables did nothing.

I don't think that's how you're meant to alter port settings anyway. They aren't environmental variables - there is a specific setting for managing ports (i.e. -p) and it is not under the ENV section, typically speaking.

Hi @NoSpamMaam (funny nickname ^^)

I run the script /root;
The script creates a backup of your configuration and upgrade Docker and Docker Compose versions.
I noticed a new version of docker was available and i did the upgrade :P

root@DSyno:~/synology-docker# sh syno_docker_update.sh update
Update Docker Engine and Docker Compose on Synology to target version

Current DSM version: 6.2.3
Current Docker version: 19.03.13
Current Docker Compose version: 1.27.4
Target Docker version: 19.03.14
Target Docker Compose version: 1.27.4

WARNING! This will replace:
  - Docker Engine
  - Docker Compose
  - Docker daemon log driver

Are you sure you want to continue? [y/N] y
Step 1 from 8: Downloading target Docker binary (https://download.docker.com/linux/static/stable/x86_64/docker-19.03.14.tgz)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 59.5M  100 59.5M    0     0  8689k      0  0:00:07  0:00:07 --:--:-- 7658k
Step 2 from 8: Stopping Docker service
pkgctl-Docker stoped.
Step 3 from 8: Backing up current Docker binaries (/root/synology-docker/docker_backup_20201208_223255.tgz)
bin/
bin/runc
bin/auplink
bin/docker-compose_bak
bin/docker-init
bin/containerd
bin/dockerd
bin/docker
bin/containerd-shim
bin/ctr
bin/docker-compose
bin/docker-proxy
dockerd.json
Step 4 from 8: Extracting target Docker binary (/tmp/docker_update/docker-19.03.14.tgz)
docker/
docker/containerd
docker/docker-init
docker/ctr
docker/containerd-shim
docker/runc
docker/docker-proxy
docker/dockerd
docker/docker
Step 5 from 8: Installing binaries
Step 6 from 8: Configuring log driver
Step 7 from 8: Starting Docker service
pkgctl-Docker started.
Step 8 from 8: Cleaning the temp folder
Done.

For more detail, please continue the discussion here @markdumay is may be more available to me to answer to all your question.

this is still an issue. To try to fix it, i updated Docker with the "syno_docker_update.sh". Now, all the containers lost all env-variables. where are the they saved anyways?
(I have /data mapped to /volume1/docker/portainer)

Hi @pseudorandomnumber, joining the discussion in follow up to the mention of @siddjellali. I created the syno_docker_update.sh script as I was getting quite frustrated with Synology's limited support for Docker myself. Would you mind sharing the steps and logs to reproduce the issues you encounter? At least on my NAS, manually updating the Docker engine to the latest version solves the issues with ENV variables.

I was having the same issue in portainer. I downloaded the syno_docker_update.sh script apparently written by @markdumay, but after updating my Docker on my NAS it broke completely with a message saying the Docker API is not responding.

I restored a backup using the same script and it's working again, but all my environment configuration on every container i have has been completely cleared, which is definitely a huge pain.

Also still having trouble with getting some containers back to work. For me it seems like the only thing the script did was damage to my Docker install. Im not blaming anyone, i understand using the script is at my own risk, but just be aware.

Sorry to hear about your issues @YourNightmar3. I can imagine it gave you quite a headache to get it working again. I'm curious though, do you remember the exact error message? The message you mentioned might indicate an issue with the daemon configuration, found in/var/packages/Docker/etc/dockerd.json on Synology. I'm working on a fix to respect the current settings of this file, and to only update the log driver (instead of overwriting everything else with a default configuration).

As a suggestion, you mention you had to manually recreate the environment settings for the various containers running on your NAS. You might want to consider using Docker Compose instead. It allows you to define the environment variables in a docker-compose.yml file (and optional .env file too). It has saved me a lot of time in recreating containers on the fly. I hardly ever use Synology's Docker UI to create containers anymore, for the same reason you mentioned. Synology's UI still helpful for quickly inspecting the resource usage and container status though.

The exact error message was "Docker API Has Failed", that was all @markdumay . No container would run.

You're right doing it through a docker compose file would have saved me lots of trouble, i will consider changing my docker setup to include such files. Today i got the last broken container back to work because i reentered an environment value wrongly.

I want to clarify a few points that people may or may not know already:

-Synology's built in Docker application is able to change and edit environment variables (for me)
-Portainer hosted on the same synology, is able to do everything EXCEPT change a containers ENV variables. Even duplicating as a new container with changed variables keeps original deployments env variables.

System Version Information:
Docker version 18.09.8, build bfed4f5
DSM 6.2.3-25426 Update 2

The Synology Docker UI just keeps loading on mine.

System:
DSM 6.2.3-25426 Update 3
Docker daemon 18.09.8

Glad I found this, it was driving me crazy! Can't believe this issue has been around for years with no solution. I guess the only work-around is delete the container and recreate from command line with the ENV changes?

@elisimpson you can simply go to the Docker app in DSM, stop the container, do what you need to do with the variables in the container configuration modal, and restart the container. This works fine as far as I'm concerned

@elisimpson you can simply go to the Docker app in DSM, stop the container, do what you need to do with the variables in the container configuration modal, and restart the container. This works fine as far as I'm concerned

If that's the case, then there's no need to even use Portainer on a Synology NAS as you can also edit the environment variables within the Docker app UI. However if @elisimpson is using Portainer for the same reason as me (that it is much better and convenient than the native Docker app within Synology, and less buggy), then a fix to this issue would be much appreciated.

My point is that for the time being there is a work around that does not involve deleting a container AND recreate from the command line, which sounds like more hassle than simply using the UI. No?

My point is that for the time being there is a work around that does not involve deleting a container AND recreate from the command line, which sounds like more hassle than simply using the UI. No?

Oh don't get me wrong, I understand you're helping here. It's more that the fix suggested doesn't actually involve using Portainer, which means that whilst I'm sure it will work, it is not related to the issue. It's like saying you will no longer be scared of flying if you walk everywhere...

FWIW, I've started using Docker Compose a whole lot more now - then I just tweak one file then run docker-compose up -d to recreate any effected containers. Again, this is a "fix" but it's not really related to Portainer as it means defaulting back to command line scripting. It means my use of Portainer is more for a visual oversight of my containers, rather than what it was designed for - to be a graphical way of managing your containers - mostly because the Docker app within Synology is poor (slow, often fails to load unless I reboot my NAS and just general clunky).

There are many settings you cannot modify in the Synology GUI without breaking the container. Most of my containers use at least one of these settings, so the Synology Docker GUI is mostly useless for modifying those containers (it can start and stop them with no problems, but any changes will break those settings that Synology doesn't support)

I just wrote an article on how to run the latest Docker daemon as a Docker container (DinD) on Synology DSM 6.x , and having that Docker daemon configured as the default Docker daemon on the system.
Please read, and try it out:
https://kristoferlundgren.medium.com/synology-run-the-latest-docker-daemon-without-patching-dsm6-x-7bb4834d87bc

For me, it looks like that just the changes made in the formular aren't sent/updated to portainer. (lack of SAVE button to be sure!)
When chaning something in ENV, then reloading the page, the old value comes back.
Maybe just a browser problem?
(me on macOS and Synology too)