Sorry for the late answer. I believe that the way you describe the feature is a bit too technical here, actually I did not even understand the use case behind it. Could you give me more functional details about what you'd like to achieve?

The goal of this request is to be able to automatically update running services managed by Portainer when their corresponding image is updated (built or pushed) in the registry (such as DockerHub).

My first post describes one potential approach to implement it in a secure way. It utilizes webhooks the same way automated builds in DockerHub are notified by associated GitHub/Bitbucket to trigger a new build. DockerHub also has the option to setup webhooks that will ping any URLs specified in it when an image is updated. We can leverage that to implement this feature.

Please let me know if you need more explanation on specific parts.

Oh so an integrated “watchtower” experience...

Rgds,

Neil Cresswell

On 26/02/2018, at 3:25 PM, Sawood Alam <[email protected]notifications@github.com> wrote:

The goal of this request is to be able to automatically update running services managed by Portainer when their corresponding image is updated (built or pushed) in the registry (such as DockerHub).

My first post describes one potential approach to implement it in secure way. It utilizes webhooks the same way automated builds in DockerHub are notified by associated GitHub/Bitbucket to trigger a new build. DockerHub also has the option to setup webhooks that will ping any URLs specified in it when an image is updated. We can leverage that to implement this feature.

Please let me know if you need more explanation on specific parts.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_1663-23issuecomment-2D368372154&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=1nvAiOGea4FI3fcdhFf-8_qK4mfCCBx6tCL6UNPoFQA&s=Ys81d7Tm_31bM7JtEmThReyWJautlF-iTVe1x233E8Y&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlSK983MIrpcawUH3aKts9Myil3INks5tYhXwgaJpZM4SMts1&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=1nvAiOGea4FI3fcdhFf-8_qK4mfCCBx6tCL6UNPoFQA&s=9VEw6zb4ZqgbyM9_OyBEe-lrh0LA1pcWim-2El90RAE&e=.

Kinda related to https://github.com/portainer/portainer/issues/1304

Yes, it is related to #1304, but outlines an implementation mechanism that utilizes the webhook (or in other words notification or pingback) feature of registries or other CI/CD systems rather than using periodic polling that can be wasteful or delayed.

It's like Docker Cloud auto redeploy functionnality, and I can only 👍 . As Docker Cloud is closing, many people will seek for an alternative... and they are now. See last paragraph here https://docs.docker.com/docker-cloud/migration/#what-stays-the-same

I need this feature!

We also like to see this feature.

Our current setup with docker cloud:

1. We push our project on github
2. Docker hub automatically builds a new image
3. Docker cloud redeployed our images if this was configured that way

We used this setup for our development.
We have a dev branch that automatically builds and redeploys on git push
When we merge to a staging this also automatically builds and redeploys
When we merge the staging to production (master) it automatically gets build but we

With the current portainer we need to pull the latest image on the image page and after that we need to go to the service and update the service.

I came back to check for any progress on this request. I was surprised to see the number of thumbs ups to the first post, but no one seems to be interested in working on this. While I have an architectural understanding of how this feature can possibly be implemented (as outlined in the first post), I am not familiar with Portainer code base and for the next few months some other priorities are keeping my hands tied. Though, I would really like if someone could take care of it soon. I will be more than happy to discuss implementation details if necessary.

I have actually assigned this to one of our developers to create a functional spec and give me an estimate for development effort.

Watch this space..

From: Sawood Alam notifications@github.com
Sent: Wednesday, 27 June 2018 12:39 p.m.
To: portainer/portainer portainer@noreply.github.com
Cc: Neil Cresswell neil@cresswell.net.nz; Comment comment@noreply.github.com
Subject: Re: [portainer/portainer] [Feature Request] Automate deployment using registry webhooks (#1663)

I came back to check for any progress on this request. I was surprised to see the number of thumbs ups to the first post, but no one seems to be interested in working on this. While I have an architectural understanding of how this feature can possibly be implemented (as outlined in the first post), I am not familiar with Portainer code base and for the next few months some other priorities are keeping my hands tied. Though, I would really like if someone could take care of it soon. I will be more than happy to discuss implementation details if necessary.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_1663-23issuecomment-2D400506423&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=s5UOJA0LMQhYo4cJoPJMCabVMZ7GXSrYR8W6DLyAWvk&s=uKaL_Difg8k_Uy1LtTzNyAXVR3tVsxJ_oYWf7EN4kHA&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlXbQhPQW3bKQTmCXKNq6unPoUY5bks5uAtQGgaJpZM4SMts1&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=s5UOJA0LMQhYo4cJoPJMCabVMZ7GXSrYR8W6DLyAWvk&s=O6uss9ua_UOTYRVDl6pNSsJgeeCx_4f3_4cWV4YCJo0&e=.

FYI here is the first sketch of the specs:

Service redeploy feature

Portainer can expose a new webhook API that can be used to trigger an update of specific services.

Notes:

• this spec only describe how to associate a webhook to a Swarm service (for Swarm stacks, each service of the stack could be updated independently via the service-details view). We could potentially add the same behavior to containers but we should start with services first.
• it can be interesting to play with https://github.com/imjosh/docker-deploy-webhook to get some extra insights.

Portainer UI (frontend)

In the service-creation/service-details views, a new switch "Create re-deploy webhook" is added.

When enabled, upon successful service creation a HTTP POST request is sent to /api/webhooks (authenticated user policy) with the following payload:

serviceId: docker service identifier of the service,
endpointId: endpoint identifier where the service is located

Portainer API (backend)

The previous request will create a new webhook object:

id: webhook identifier,
serviceId: docker service identifier,
endpointId: endpoint identifier where the service is located

The webhook can then be used by sending a HTTP POST request to api/webhook/:id (public access policy in order to be reached from external services such as DockerHub)

It requires no payload.

Note: as the /api/webhook/:id endpoint is publicly exposed, the webhook identifier format should be in the form of a randomly generated string. We might want to add an extra query parameter that can be matched for access restriction.

When receiving a POST request at /api/webhook/:id, the API will send a request to the specified endpoint to retrieve details about the service associated to the webhook (using the service identifier). It will extract information about the image used in the service details, retrieve any registry authentication details if required (check in database) and send a service update request.

Note: In order to execute Docker requests from the backend, we will probably need to embed the Docker SDK inside the API to allow the backend to act as a Docker client.

In Docker hub you have the ability to create a Webhook
https://docs.docker.com/docker-hub/webhooks/ that triggers a post call whenever an image is build or a tag is added.
So we could trigger an image pull and redeploy in portainer.

@Cinezaster that's the goal, yes

@deviantony we never use the service-creation view to create our service but instead us a stackfile to create our service, would it be possible to add the webhook later via the service view?

Yes, you'll be able to go the service-details view to create a re-deploy webhook as well, I'll update the specs.

Any news on this request?

Yup, it's still in our priority list, I'll assign someone on this soon.

FYI. Not fully same thing but can be useful for some users. I just published my Portainer extension for VSTS: https://marketplace.visualstudio.com/items?itemName=OlliJanatuinen.portainer-deploy

Anybody keen to try the implementation for this? It's available via portainer/portainer:feat1663-add-webhook (Linux amd64 build only).

Any feedback welcome !

@deviantony we will try it and test it @appsaloon

Is this only for swarm services only or for independent containers as well?

@Strum355 only for Swarm services right now.

Ah no worries, is there a rough eta on when we could see this for containers?

No ETA yet

Thanks for pushing this one forward. I just deployed it and now exploring around. I will provide more feedback in the PR #2161.

Should we also plan to generalize this webhook functionality for other entities (as I briefly described in an earlier comment) and make it a top-class entity to show up in the sidebar menu? An immediate extension that I can think of is webhooks for images that would allow us to connect directly from a source repo (such as one on GitHub or BitBucket) and build an image directly (which is made possible by #1667) when the code changes. Optionally, we can also update services/containers when an associated image is updated locally.

If we do decide to make webhooks for more than one entities, we might want to namespace URLs as https://portainer.example.com/api/webhooks/services/12345... and https://portainer.example.com/api/webhooks/images/12345... instead of https://portainer.example.com/api/webhooks/12345....

Though it is not necessary as the hook IDs can be unique across the system and their associated type can be stored in a table in the form of polymorphic association.

@Cinezaster and I tried it out with the webhooks hub.docker provides. And it works, our services are automatically being updated after each push to a repository.
But the tags of the images that were updated are all <none>, both in the image list in portainer as well as when I look at docker images. This doesn't interfere with functionality however. We have multiple services using the same image with different tags, and they're all still using the correct image after the update.

Once development is done, we need someone to document /blog how to link portainer with dockerhub webhooks as you have done, feel like being a volunteer for that?

Rgds,

Neil Cresswell

On 27/08/2018, at 4:37 PM, Jo Jordens <[email protected]notifications@github.com> wrote:

@Cinezasterhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Cinezaster&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=Ppw5k5PG1o8Dc0s32b3QQRfKWrl25HEvMc5Jx0-8vrY&s=TqOWOtwxUrb9DhaLtaZGfZ_4zCYlwaTMfGzIvtX0QtE&e= and I tried it out with the webhooks hub.docker provides. And it works, our services are automatically being updated after each push to a repository.
But the tags of the images that were updated are all , both in the image list in portainer as well as when I look at docker images. This doesn't interfere with functionality however. We have multiple services using the same image with different tags, and they're all still using the correct image after the update.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_1663-23issuecomment-2D416172189&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=Ppw5k5PG1o8Dc0s32b3QQRfKWrl25HEvMc5Jx0-8vrY&s=0UKZwz1EI0L8JYdLqrswGx6dpZ0DT9QlxD_aXH-N6vE&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlVGGU2TyMZz2ErAk-5FseDNYgftzE5ks5uU73VgaJpZM4SMts1&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=Ppw5k5PG1o8Dc0s32b3QQRfKWrl25HEvMc5Jx0-8vrY&s=ofElX8u3XOItT169JEYAQ0UO8f6moEEEoBWaOqBJCMU&e=.

Any feedback from the users that were able to test this yet?

@deviantony, @JoJordens tested the feature, his only comment : tags of the images are <none> after an update.
@ncresswell We're going to write a blog post this or next week about how to use this feature.

I have also been testing this and provided feedback on both the functionality and code.

• Should we namespace webhook URIs (by including services in the URI to extend the functionality to other aspects such as images in future)?
• I think the <none> tag issue is present in the stack functionality. When a stack is deployed/updated without first updating the image explicitly, the image that is pulled does not get tagged properly. I have been seeing this issue for a long time.
• Since we are not utilizing any payload from the webhook pingback, should we also make it work on a GET request and not just on a POST request? Ideally, webhooks should be a PUT request, but existing systems send it via POST, so having it work on POST is important. However, supporting GET will be more convenient to trigger an update from the command line.

Thanks @Cinezaster

Also thanks @ibnesayeed but I was expecting UI/UX feedback :-)

Regarding your technical points:

Should we namespace webhook URIs (by including services in the URI to extend the functionality to other aspects such as images in future)?

I don't think so, but we'll extend the Webhook object to support a new Type field.

I think the tag issue is present in the stack functionality. When a stack is deployed/updated without first updating the image explicitly, the image that is pulled does not get tagged properly. I have been seeing this issue for a long time.

Yup, this is a known issue.

Since we are not utilizing any payload from the webhook pingback, should we also make it work on a GET request and not just on a POST request? Ideally, webhooks should be a PUT request, but existing systems send it via POST, so having it work on POST is important. However, supporting GET will be more convenient to trigger an update from thee command line.

As most of the systems are sending POST requests, I believe that we should keep it this way. While a GET request would be more convenient via the CLI, it's not that much of a hassle to actually send a POST request so I don't think we'll want to support GET requests as well.

On the UI side, I initially thought we might want to add the toggle switch in the tabular listing of services, be it on the main services page or under a specific stack. However, I think adding just the switch is insufficient. So if we do so, we will have to have more UI elements to expose the webhook URI (possibly in the drawer that shows up when a service row is clicked).

I don't think so, but we'll extend the Webhook object to support a new Type field.

As long as future extensions won't break, I am OK. I just pointed it out so that we can make it future-proof.

Yup, this is a known issue.

Do we have a ticket to track this issue?

As most of the systems are sending POST requests, I believe that we should keep it this way. While a GET request would be more convenient via the CLI, it's not that much of a hassle to actually send a POST request so I don't think we'll want to support GET requests as well.

I'm cool with this. I am not a big fan of having too many ways to do the same thing, but I thought it might be useful for some.

@ibnesayeed see https://github.com/portainer/portainer/issues/1486

@kendrickm, I pulled the latest portainer:feat1663-add-webhook image a few minutes ago after your recent commits and redeployed the stack on my server. Now, my existing webhooks are broken. All the services where I have enabled webhooks are now appearing to have disabled associated webhooks. If I try to send a POST request to a webhook that was created earlier, I get a 500 response code and if I send a request to a non-existing random one, I get a 502. I feel like we can improve status codes here, depending on the situation.

I think the reason has to do with the newly added webhook type field which is not populated for existing tokens. If so, is there a way to patch the database on my running instance as I would not like to recreate all those tokens and update records in DockerHub for all associated repos.

$ curl -X POST -i https://portainer.example.com/api/webhooks/<created-earlier-but-now-hidden-token>
HTTP/1.1 500 Internal Server Error
Content-Length: 102
Content-Type: application/json
Date: Thu, 30 Aug 2018 21:23:15 GMT
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block

{"err":"Unsupported webhook type","details":"Webhooks for this resource are not currently supported"}

$ curl -X POST -i https://portainer.example.com/api/webhooks/<non-existing-random-token>
HTTP/1.1 502 Bad Gateway
Date: Thu, 30 Aug 2018 21:23:21 GMT
Content-Length: 11
Content-Type: text/plain; charset=utf-8

Bad Gateway

I was able to fix my deployment by manually editing the Bolt database file using https://github.com/br0xen/boltbrowser.

@ibnesayeed Yeah there was a bug around error handling causing it to segfault vs return an actual error. Are you using a webserver proxy in front of it?

Are you using a webserver proxy in front of it?

Yes, I am using Traefik as a reverse proxy in front of my Portainer instance and everything else.

While I am very happy with this feature being in the pipe of getting merged, I was thinking will it be useful to have the ability to optionally customize the token? I can think of some places where one might have some naming scheme for these tokens to automate batch processing without essentially querying all services and their associated tokens. On the UI side, we can add a button next to the Copy link button that says, "Customize token" or "Vanity token". We can perhaps have both a randomly generated token that has an optional custom alias.

@ibnesayeed we can open a new enhancement request later on for this.

@ibnesayeed we can open a new enhancement request later on for this.

Yes, I agree. I didn't mean to stall the corresponding PR, but to share my thoughts.

@ibnesayeed all good, feel free to open a new feature request for this evolution.

Im assuming a new enhancement request should be made to have automated deployment for individual containers?

Yes @Strum355

Opened #2236 to track the custom token feature.

Does this support Windows?

@goalbased I cannot see any reason why it would not work on Windows. Can you test and let use know?

I can't find the Service tab any idea? Do I need Swarm or something first?

@goalbased It only exists when Swarm mode is enabled and that is because of how Docker it selves works.

Plz join to Portainer Slack and ask these offtopic questions on there.