Portainer: Add the ability to update and redeploy a stack created from a git repository

Do you mean the ability to trigger a git pull and then redeploy the stack?

Yes

@deviantony I am thinking of

• Slider "Authentication"
  - Username
  - Password
  
  
  
  • Rename existing button label from "Update the stack" ==> "Update"
  
  
  • New button with label "Redeploy"
    
    what do you think ?
  
  

In my opinion, the credentials should be persisted in the stack data.

If the stack was created from a repository, a new button should be added "Pull and deploy".

Update..
@deviantony I will go ahead with approach mentioned in your comment.

@deviantony I didn’t go with this approach because credentials could change Between saving with data and redeploy attempt(rare cases but possible). Also in one of thread you mentioned that portainer does not save these github credentials.

I would also like this feature. Until then I wrote a small script to update stacks and hooked it into git - https://github.com/docker-how-to/portainer-bash-scripts

@rahulruikar if you're adding basic auth it'd be great to also be able to use a (read only) SSH key like how Github has deploy keys.

Following the discussion in https://github.com/portainer/portainer/pull/1793#issuecomment-384302019

As I'm not keen to store unencrypted credentials for git repositories inside Portainer I think that the way to go is to implement https://github.com/portainer/portainer/issues/1752 first and then work on this one.

Ok..I will start working on #1752 this week.

On Thu, 26 Apr 2018 at 12:18 am, Anthony Lapenna notifications@github.com
wrote:

Following the discussion in #1793 (comment)
https://github.com/portainer/portainer/pull/1793#issuecomment-384302019

As I'm not keen to store unencrypted credentials for git repositories
inside Portainer I think that the way to go is to implement #1752
https://github.com/portainer/portainer/issues/1752 first and then work
on this one.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/portainer/portainer/issues/1753#issuecomment-384303314,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHXNPSgOjCLCHiH5wMz2reZV7Jg52g8hks5tsIWmgaJpZM4S3Cmi
.

>

Sent from Gmail Mobile

Hi,
I am looking forward to have this option as it would optimize my deployment process! 👍
Two things I have in mind for this:
1) Will sidecar files work? There are some config files I need to mount into the container for e.g. nginx.
2) I'd love to have the feature to trigger these redeployments thru webhooks from dockerhub or CI/CD.

Cheers

@shyd

Not sure about the sidecar files, but if they're located inside the repository and part of the stack deployment should work.

And yes, webhooks for this feature is in our plans.

Hi @deviantony
Any news on this?
Is it on your road map yet? If yes, do you eventually have any eta?

@somq I am currently working on this feature - will let you know once it's ready for testing :)

@ssbkang great news thanks!
Will be happy to test it, if you need any more help let me know. I can test it as soon as API endpoints are reachable just to let you know.
Just to understand, do you have any idea if it's a matter of days, weeks, more?

@ssbkang Happy to hear this is worked on... we are looking at having a ci/cd with gitlab to deploy stacks but if this is handled by portainer... all the better.

I understand that the functionalities thaht are worked on are:

• update stack from compose file in github
• webhook to trigger and update of the stack

As i am curious:

• how are webhooks foreseen?
  
  
  
  • like what is done with services
  
  
  • are caller specificities included?
    
    github and gitlab may be able to send webhooks but maybe some degree of filtering may be needed (commit, release, branch, tags, ...). They may send a webhooks for a broader scope of change than what is desired in portainer.
  
  
• is there a way to follow up the code changes? (https://github.com/ssbkang/portainer?)

@deviantony is there news from this... from what i saw @ssbkang did some work already so i don't want to overstep anything there. I think the first step would be to look at pull request #2972.

--i just want to help on this as this would help us very much keeping a simple workflow. currently the alternative is to go ci/cd which would bypass portainer stack management--

@cblomart yes, I am aware of that and I need to review the backend implementation in #2972

However, I am a busy on other topics at the moment but this is in our backlog.

If you want to help, feel free to give a go to the implementation in #2972, it is available through the API only at the moment and can be tested via portainerci/portainer:pr2972. Ask @ssbkang for more details.

@deviantony thanks. i am now running on the pr of @ssbkang. I will need to check how to use the api.

@deviantony Any update on this ?

No update yet, wasn't able to do the review of #2972.

Feel free to give a try to portainerci/portainer:pr2972, see the comment above.

Gave a try but not easy to test as you need to go trough api.

We do use portainer but have more and more pression to store our stack definition elswhere (than in portainer). I think storing them in git and using reposistories is a good way to gitops but we need flexibility from this (update/redeploy).

I don't think the PR addresses what @cblomart would like. It creates private/public keys and stores them in boltdb to be then used/configured with your SCM?

Haven't spend too much time in the code, but isn't that somewhat of a security issue? I don't think Portainer should enter the business of creating security keys.

We usually setup deploy accounts and provide them with read-only access to required repositories and then this user can git clone (or git pull).

In order to secure this further, you can use tokens (on Github: Personal Access Tokens) which limit the scope, etc. further. I am sure Gitlab has something similar. These tokens can be revoked. Where to store these (e.g. in vault?) is another question.

I would also like the ability to re-deploy a stack from a git repository as well. But currently the credentials are not saved by portainer.

So a simple "re-deploy" is a bit of a challenge this way. Can be achieved though with various API calls to populate "the fields".

I would like to have the ability though to prevent changes on the UI when this happens so it's always from git. Maybe an explicit override ("You are about to make changes to a git-based configuration. These changes are not permanent and likely overwritten on a re-deploy.")

IMO this feature is important and doesn't need to be blocked on landing git-over-SSH.

There are valid uses for it _unauthenticated_ (public compose config, config repo is public behind the firewall), and even with the user/pass config, the credentials could simply be prompted for each time the redeploy button is hit.

I second the value and importance of having stack config updated from GIT even starting with just unauthenticated git repos. Our teams use local and public github repos that don't require any auth since they are open source projects.

Hi I just tried to find out about the current status of this issue and all its side-issues.
Can I help out somewhere to get this feature.
I would love to have the ability to redeploy a stack based on a compose-file in a git repo.
Maybe even make it possible to poll and auto redeploy in a configurable intervall.

Just saw this - would be nice! It's quite easy to simply copy paste a compose file from github into portainer, so I was surprised that the currently functionality basically only does that.

Hey guys, this seems like a great idea. I'd like to get the ball rolling on this as there seems to be a lot of people interested 👍

@deviantony I don't think #2972 needs to be blocking this feat. IIRC it pertains to ssh cloning, but to begin with we could simply prompt for credentials/PAT for a private repo

If its easier for everyone, we have a slack server where we are quite active. There is a dev channel there that would be a great place to engage with you all and get your input on this

@itsconquest sorry, I simply don't get slack. How is the status on this. It sounds like a critical issue to deal with, since after creating a stack from a git repo you essentially have to remove it.

Or is there a workaround I missed?

Yeah. Slack is a challenge, since most of you are down under. ;)

We still can not auto-update docker-compose from deploy by git repository for now.
It's easy to use in Kubernetes, called GitOps.

How is going on on this improvement?

Would this fix the problem which prevents to use the start/stop stack button from the stack page on Portainer 2 ?
in my case, when restarting a stack on a docker swarm cluster, the services using images from private repos do not start (No such image)

We're planning to add support for this feature later this year, this is currently earmarked around Q2 2021.

will it be one-way (re-deploy) or two-way? I mean, if you change something in the UI, will it commit/push to the repo as well, or it will only be able to pull and re-deploy but not push?

@upcFrost our vision is that Portainer should keep in sync with a git repo and not the other way around, so we'll think about our UX around stacks that are "synced" but at the moment it was stated to be one-way I think.

deviantony: will be this feature available in 2.2 ? maybe another question, will be in portainer support for notifications/webhooks ? for example, if stack is succesfully synced, out of sync, failed, etc ? thanks

@lukasmrtvy I can't ensure that this will be present in the next release of Portainer at this stage. But with the lot of activity around this topic, you can expect us coming back with some ideas to be discussed in here and we'll consider the existing discussion/ideas in that topic for sure.

For me, it would also be a very useful feature.
Also nice to have a webhook for updates of the stack created from git.

I would certainly use this feature.

will it be one-way (re-deploy) or two-way? I mean, if you change something in the UI, will it commit/push to the repo as well, or it will only be able to pull and re-deploy but not push?

I agree, pushing a commit should save a lot of work as it simplify the project management.

We built a CLI tool to do this. It's not 100% compatible with stock portainer right now (because authentication is a bit messed up when you use SSO), but we are considering opensourcing it later.

will it be one-way (re-deploy) or two-way? I mean, if you change something in the UI, will it commit/push to the repo as well, or it will only be able to pull and re-deploy but not push?

I agree, pushing a commit should save a lot of work as it simplify the project management.

To be fair, with one-way sync I don't really see the point of linking portainer to git. It will make migration to portainer easier, that's true, but it basically means losing the changes history with this migration. Or just using portainer as a read-only UI, which would be a bit strange

This Issue was opened 3 years ago and is very popular. But until now this feature was not implemented.
I read the discussions and @deviantony said it will be there in Q2 of 2021.

Is there a beta version of it at the moment or where can I find that part of code to test it?

@Mr-Markus there is no beta available at the moment, the development on this piece is planned to start next week! We're targeting CE 2.5 at the end of May but as we have many things in our work pipe and a few delays have happened already it might be part of CE 2.6 at the latest.

We're considering it as one of the most important feature to be added next so it will definitely happen soon!

I was just about to open a new ticket when I deployed a stack using git Repository, and then it didn't offer a re-deploy after I'd made changes option. And I'd made deploy tokens and everything.