This would be awesome and make Portainer a real competitor with Rancher if you can integrate into the template profiles.

Any thoughts on creating a portainer container to package swarm as an easily deployed "agent"?

If there is interest I can contribute...

@jslatten can you describe what do you mean by 'integrate into the templates profiles' ?

Also, I don't think there's a need to 'package' Swarm at the moment, especially with the arrival of Docker swarm mode, it has never been easier to deploy a Swarm cluster.

I meant that the templates could reference docker-compose templates instead of basic references to standard docker images

Yes, the way I see it it would be to add another section in the app 'stacks', 'bundle' or something where you would be able to upload your compose file to start the associated stack.

great idea

Use compose files and deploy multiple images linked its the most useful in dockers enviroments

@deviantony thanks for setting this on a milestone.

I think it would be interesting for Portainer to do a GET from a URI for the compose file, file upload, or straight up typing one in.

Then to have a vew options like: run, up, down, rm and etc..

2016-12-14 22:17 GMT+01:00 Paul Kling notifications@github.com:

I think it would be interesting for Portainer to do a GET from a URI for
the compose file, file upload, or straight up typing one in.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/portainer/portainer/issues/257#issuecomment-267159246,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJG6XY0arR1jyBwB_ojQrgWqwwogluhKks5rIFztgaJpZM4KOcDQ
.

--

Yeah. get_from_url option should please be included
On Dec 14, 2016 5:46 PM, "scher200" notifications@github.com wrote:

Then to have a vew options like: run, up, down, rm and etc..

2016-12-14 22:17 GMT+01:00 Paul Kling notifications@github.com:

I think it would be interesting for Portainer to do a GET from a URI for
the compose file, file upload, or straight up typing one in.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
issuecomment-267159246>,
or mute the thread
ojQrgWqwwogluhKks5rIFztgaJpZM4KOcDQ>
.

--

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/portainer/portainer/issues/257#issuecomment-267181058,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGsC46DElksfsM6dbScPmxlJ9-gHWzC0ks5rIHGzgaJpZM4KOcDQ
.

as an active user of compose I'd really like such a feature :+1:

I wouldn't mind getting rid of the current template system and just move to compose 😄. It's in essence exactly the same, but with more options.

yeah it would work wonderfull with: composerregistry

Search API
full text search: https://www.composeregistry.com/api/v1/search?query={QUERY}

implementation for search would be equal to:

e.g. curl -H 'x-key: default' https://www.composeregistry.com/api/v1/search?query=elk

{
"items": [
  {
    "name": "docker-elk",
    "owner": "deviantony",
    "stars": 345,
    "id": "AVBCF9PggLKoASuOwp_8"
  },
  {
    "name": "st2contrib",
    "owner": "StackStorm",
    "stars": 45,
    "id": "AVBR45mWwBuN0C9fxHc5"
  },
  ...
  ]
}

Compose support would be truly awesome.

I wanted to give some examples how we use compose on a swarm in production, see screenshot below:

(env = swarm environment)
(rectangle = docker-compose-stack)
(arrow = merging, default config, external files)

Actually we're only using native features, such as compose .env-files, container-env-files or merging multiple files. It would be nice if we could organize files in different folders.

Btw: we've created the swarm with roj, which is fully compatible to portainer, since both use native Docker tools.

How would you guys like to have compose support in Portainer? The way I see it at the moment :

We should give the users the ability to define 'stacks'. When creating a stack, you provide a name and a docker-compose YML file. Once the stack is created Portainer should give you the ability to manage it (up down etc) and also a way to manage each service in that stack (start stop a service etc). From the stacks view you should be able to manage your stacks (add, edit, delete) and access to a detailed view for each stack (which allows you to manage it as well as giving a visual representation of the containers of the stack).

Stack, or Apps ? I thought that "stack" as a term, is being used in Docker 1.13.

Rgds,

Neil Cresswell

On 15/01/2017, at 10:04 AM, Anthony Lapenna <[email protected]notifications@github.com> wrote:

How would you guys like to have compose support in Portainer? The way I see it at the moment :

We should give the users the ability to define 'stacks'. When creating a stack, you provide a name and a docker-compose YML file. Once the stack is created Portainer should give you the ability to manage it (up down etc) and also a way to manage each service in that stack (start stop a service etc). From the stacks view you should be able to manage your stacks (add, edit, delete) and access to a detailed view for each stack (which allows you to manage it as well as giving a visual representation of the containers of the stack).

-
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_257-23issuecomment-2D272659965&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=Fw9ndRCBfdaqM_QJPyMAN8dnAJGj7OhptUL9ydxCALg&s=p25fvB-hAknn38kSn_OEJqEqB3TuGZ8-UqFZwealFtI&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlTwoVUv-2D5p-5FIce0kU9jEY68YRJF0ks5rSVR1gaJpZM4KOcDQ&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=Fw9ndRCBfdaqM_QJPyMAN8dnAJGj7OhptUL9ydxCALg&s=V2ivYy3sHro5tBndkdegDkzw5Q8cGUU0pQXiAJjAFWc&e=.

I used the term 'stack' with some quotes as an example. We might need to figure out another term.

@deviantony I am using https://github.com/francescou/docker-compose-ui with portainer.

I like the way it behaves, just like you've described.
Don't got any spare time now, but would have tried to embed features of compose-ui to portainer.

How would you guys like to have compose support in Portainer? The way I see it at the moment :
We should give the users the ability to define 'stacks'. When creating a stack, you provide a name and a docker-compose YML file.

From my point of view I'd suggest rather to point to a directory containing the corresponding files. A docker-compose configuration can consist of several files, not only one docker-compose.yml, but i.e. a .env defining the .yml files to include and a project name.
Moreover it would be very important to connect containers to their docker-compose "stacks" - that can be done by parsing the com.compose labels of the container.

I am using https://github.com/francescou/docker-compose-ui with portainer.

I know that project and it is nice, definitely worth a look.

Stack, or Apps ? I thought that "stack" as a term, is being used in Docker 1.13.

The problem with Docker sometimes is that they mix their own terms, while a stack is now much more related to Docker-cloud (see also stackfiles) the platform started out as place for sharing docker-compose.yml files.

Or this one called neam/docker-stack - stared on Mar 1, 2015.

But yes, we should differentiate it clearly :)

Moreover it would be very important to connect containers to their docker-compose "stacks" - that can be done by parsing the com.compose labels of the container.

Could you give us more details about that one? Would you like to be able to see the associated 'stack' in the container view if a container is part of a compose 'stack' ?

Even both ways, so from stack -> container and from container -> stack

For example, if you inspect a container, you'll get

"Labels": {
    "com.docker.compose.config-hash": "0ac7cfd1b1be53c964dfcc90c728dbc543493e5d38cc34822676b274822914b5",
[...]
    "com.docker.compose.project": "myproject",
[...]
}

I would be very helpful, if containers are grouped by their project-names (myproject), even if you have no information about the "stack" (project) itself.
If you would create/upload a stack-folder containing a docker-compose.yml and .env file, you could kind of attach the config to the running containers.

Moreover, Portainer could warn you, if you're using a COMPOSE_PROJECT_NAME twice, or if there are illegal characters set, i.e. there are no - or _ used for the container name.

yes, this would ensure a good view on your 'stack', no matter how you ran it.

Really looking forward to have compose integrated into portainer as well!

An interesting feature in addition to compose would be the notion of questions as Rancher does.

See: https://docs.rancher.com/rancher/v1.2/zh/catalog/private-catalog/#questions-in-the-rancher-composeyml

It is really handy to populate the compose file when it has to contain secret data or specific configuration.

I would prefere the way followed by docker-compose-ui that would be to set the secrets data in a .env file read at same time of docker-compose.yml. As in compose-ui, it would be great to specify a git url which could contains directories for each stacks instead of loading one by one and then the git still be the real source of the compose file

The .env file is not specific to docker-compose-ui but it is native from compose (https://docs.docker.com/compose/environment-variables/#/the-env-file). So maybe the idea would be to implement an UI to fill in this .env file. That would look like to this "question" feature integrated in rancher-compose.

This is not mandatory but that would be a nice feature ;)

And just a few questions that come into my mind when thinking about a implementation of compose in portainer:

• Compose would need to be installed into portainer container?
• What would be the library used to interact with compose (as it does not provide any standard REST api...)? compose-ui uses directly python calls wheareas Rancher uses libcompose (that may be more appropriate as written in go).

@loric-

About your questions:

Compose would need to be installed into portainer container?

No, Portainer will take care of interfacing Docker Compose

What would be the library used to interact with compose (as it does not provide any standard REST api...)? compose-ui uses directly python calls wheareas Rancher uses libcompose (that may be more appropriate as written in go).

Yes, we will probably use libcompose. I already had a look at it and it seems easy to integrate inside Portainer.

Compose would need to be installed into portainer container?

I think that would be one way to go, I experimented with linking of static binaries from another container, which would be possible in this case, but it is not best practice.

What would be the library used to interact with compose (as it does not provide any standard REST api...)? compose-ui uses directly python calls wheareas Rancher uses libcompose (that may be more appropriate as written in go).

Because you already have code written in go, libcompose is the thing to choose, IMO.

Just for reference, we tried to create a similar thing with PHP and this lib https://github.com/omauger/docker-compose-php - but portainer is light years ahead :) Having a REST API for compose is definitely something to think about.

With swarm mode and 1.13 the docker team added ability V3 of docker-compose file.

docker stack deploy --compose-file=docker-compose.yml my_stack

Yet docker-compose doesn't support V3 yet.

@deviantony @paulkling could there be a way to sort/group containers launched by the following stack?:

docker stack deploy --compose-file=docker-compose.yml my_stack

Hi !

We use docker-compose v3 files with "docker stack deploy" command
This creates services (swarm mode)
Some labels are present in services too when created by a compose file :

                    "Labels": {
                        "com.docker.stack.namespace": "stack_name"
                    },

If we can see those stacks in portainer, and manage our stack templates too, it would be awesome

I think only the docker lib is necessary for that, not libcompose

Tell me if you have questions about that

Yup, from what I've read, compose v3 is an entirely new implementation of compose inside the docker library. @WTFKr0 do you have more info about that?

So I doubt libcompose would be useful here. And I really want the ability to reproduce the behavior of docker stack deploy into Portainer.

Thanx for reply

I think we can add a "Stacks" panel on the left
This panel have to list stack name (docker swarm)
Docker api doesn't have "list stack", so we have to list services, and get the Service.Spec.Labels array to find com.docker.stack.namespace value

When we select a stack, a summary of the stack can be composed of :

• List of services - "docker stack services stack_name" command line
• List of tasks (by services) - "docker stack ps stack_name" command line
• Container associated with tasks (with link to container view)

Docker API services and tasks can be filtered by stacks like that :

curl -g --unix-socket /var/run/docker.sock 'http:/v1.26/services?filters={"label":{"com.docker.stack.namespace=stack_name":true}}' | python -m json.tool

See : https://github.com/docker/docker/blob/master/cli/command/stack/services.go#L50

@WTFKr0 @deviantony Yes this would be a great solution!

Concerning the parameterization of the template, I like the way rancher do it with questions too, and would like to go further, like get the possible values for a field from external url
See my comment on docker-compose ui for the idea :
https://github.com/francescou/docker-compose-ui/issues/31#issuecomment-224575619

Today, no API is present for listing/updating stacks in docker 1.13
Docker cli is doing that
No ETA for implementing stack API, so I think we can't wait for that

So we have to code that in portainer
What do you prefer :

• Implement that on API (go), based on https://github.com/docker/docker/tree/master/cli/command/stack and have a new portainer api endpoint like /api/stacks
• Implement that client side, calling multiple existing API endpoint

?

My preference is client side

Rgds,

Neil Cresswell

On 1/03/2017, at 3:44 AM, WTFKr0 <[email protected]notifications@github.com> wrote:

Today, no API is present for listing/updating stacks in docker 1.13
Docker cli is doing that
No ETA for implementing stack API, so I think we can't wait for that

So we have to code that in portainer
What do you prefer :

• Implement that on API (go), based on https://github.com/docker/docker/tree/master/cli/command/stackhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_docker_docker_tree_master_cli_command_stack&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=woJBAGvM3Rz9Goy1FAu7X9SpAOmzFLk6kWnUJbu5kT8&s=ydq_ckUHyfVCtAFRLosfwSln63160TJ61XtGtWb6Whw&e= and have a new portainer api endpoint like /api/stacks
• Implement that client side, calling multiple existing API endpoint

?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_257-23issuecomment-2D283057380&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=woJBAGvM3Rz9Goy1FAu7X9SpAOmzFLk6kWnUJbu5kT8&s=yAkCaVoPo5DT07Z-50aM05R2FmNwj_8HhCijAehH9y8&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlSNYboDHJYtv5WI2sINu4xGMhl36ks5rhDKmgaJpZM4KOcDQ&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=woJBAGvM3Rz9Goy1FAu7X9SpAOmzFLk6kWnUJbu5kT8&s=hVCGqyH5ePC-22L-yD7LyJ93EtepgUC4TpmGuyCdWEA&e=.

I ain't a fan of docker stack yet, since it limits itself to a swarm deployment. Swarm isn't really something I need for my projects, mostly I get 1 server to deploy stuff on.

@thomas15v one server should not hold you back. You can swarm with only 1 server total. Anyway you can upgrade your growing apps later simply

@WTFKr0 my preference would be to re-use existing code (compose management library, dunno if libcompose is still the good way to go) and add that inside the API.

IMO, we should not try to simulate the behavior of compose via Portainer client side, this will not be easy to maintain in the long term.

If you want to re-use existing code, I think the simple way is to use this code in the portainer go api :
https://github.com/docker/docker/tree/master/cli/command/stack
Once docker cli migrate to use docker API, we just have to adapt the code
I can work on a branch to add a /api/stacks endpoint on portainer api to do actions like "docker stack" do

Yup, we should expose a new endpoint in the API. If you're keen to start working on it, please do.

Note that at the moment the API is under heavy work to integrate multi-users / UAC, all of this should be committed soon so you might want to wait for that to be integrated into develop before working on it.

Would you prefer I work on the feat550-user-access-control branch instead for the moment ?

You can base your work on this branch, yes. But is not finished yet and I'm still working on it at the moment ! So unless you really want to start working on it right now you but I advise you to wait for 2 days at least (time for me to finish the work on that branch).

Hi everybody,

Thanks for that work and excuse me for the duplicate request.

Talking about how we handle deployments in production. We layout the compose projects in the file system using an yum/rpm based tooling. Each rpm comprises of a compose project along with the yaml file, some meta and a SysVinit service laid out in the file system. We use the linux sysvinit service commands to start and stop the projects.

Portainer is fascinating since it gives us a handle to the entire equation of start|stop|status of the services in a host, from a neat gui.

Now, what we really want is to have portainer recognize these projects that are laid out by these RPMS and provide means to manage them as individual docker-compose projects. Ie. A way to point my projects in the file system to the portainer apis followed by a way to modulate my individual docker-compose projects. I guess you may need the handle of docker-compose binary path as well. Also you have to CD into the project, before you do any docker-compose commands since the .env files in the project has to be honored.

Kindly consider my request.

@deviantony - not sure how you are trying to go with this feature, but as a user we wold still like to maintain the compose files outside of portainer and use the portainer to recognise these projects, which are already laid out on the file system, and give management interface. I have explained this in the above comment, which I would like to bring to your attention. The following is the layout of our compose projects on a physical host in production.

/path/to/my/projects/
|-- catalog
|   |-- compose-common.yml
|   |-- docker-compose
|   |-- docker-compose.yml
|   |-- launch.sh
|   `-- .env
|-- search
|   |-- compose-common.yml
|   |-- docker-compose
|   |-- docker-compose.yml
|   |-- launch.sh
|   `-- .env
`-- traefik
    |-- compose-common.yml
    |-- docker-compose
    |-- docker-compose.yml
    |-- launch.sh
    `-- .env

@alwaysastudent thanks for your detailed use case ! We'll take it into account !

Never mind I re-read your first post and discovered it is binary. Cool I thought I missing something on how compose worked.

@deviantony - Thank you for considering my request.

@paulkling - Right, it is the binary. We pack the binary as well along with the project, so to be independent and self-contained. In fact there is one another file in the layout that I didn't mention above earlier (made edits now) which is a plain bash launch script that will be symlink'ed as a project specific SysVinit service during the RPM post install. We use these project specific SysVinit services to interact with their own binary using the start|stop|status|reload commands. All this fun is to simplify automation and keep operations under control in a production environment, since we have not graduated to the cluster managers.

@alwaysastudent
From my point of view and from experience, we have to split templates information (store in the docker-compsose file) and parameters informations (generally provide by env file, or often in compose file too)
If we can split these too informations, it will be easy to duplicate envrionnements for dev/tests/integration
Here we have a git repo with compose v3 templates (with some variables in), and a form view to param them and "deploy stack" them

@alwaysastudent
You hit the right idea there. If you can choose a git repo including all these files and hit [deploy stack], and with that serve the compose (v3) file on your stack.

Has anyone considered support for a sub-environment as a precursor to docker-compose support?

For myself (And I feel, most of you others), the primary advantage of docker-compose is that I can specify multiple inter-connected services in a single file and deploy them all as a single entity and manage it as a single "unit".

In our deployment system, we have multiple projects ($prjA, $prjB) running in multiple environments (testing, production, etc). Portainer forces us to manage them all in a single list rather than building them all as part of isolated environments. When viewing my running containers, I may have $prjA-testing running right next to $prjB-production. If I could split that up into multiple environments, that would be amazing, and this would probably be something that compose support could evolve naturally from.

The way I see it right now is, even if compose support was added in, there will still need to be a new interface implemented since you're not going to list a single compose file (Made up of multiple containers) in the global container listing, and the current interface is not designed to support such a thing.

Check out how Rancher handles their environment management. While I don't think Portainer needs to be as in-depth (Rancher has different hardware nodes per environment, which I feel is outside the scope of Portainer's goals), there's definitely a need for container grouping in terms of effective management.

@mwthink yes I agree this is the first step

I think a stack view is the key for that. I've open a PR for stack view in docker swarm mode (based on services labels). This can be updated to show compose stack too (based on container labels)

I think this issue can be splitted in multiple PR :

1) A stack view
The view should display informations on stacks (created by docker-compose up or by docker stack deploy
This PR should only be a read only view of stacks, with links to services/containers
I start a work on that on #668

2) Api to manage docker stack deploy - compose v3
This PR should add api calls to deploy/destroy/update stacks (swarm mode)
There is no docker api for that, so we have to use antoher method
I started a work on #695

3) Api to manage docker-compose - compose v1/v2
This PR should add api calls to up/down/scale a docker-compose project
I think it should use https://github.com/docker/libcompose
No PR started yet

4) Add view to manage those stacks
Once the 2 api are here, we can update the view with buttons like delete/scale in stack view
No PR started yet

5) Add templates
Rework the template view to add stack template
A stack template should be a docker-compose file and a config file (json) to manage user parameters
The template should be deployed with docker-compose or docker stack deploy if compatible
No PR started yet

WDYT ?

I would also like to propose showing which stack a resource (service, container, network, secret, ...) belongs to in it's own view by adding a "Stack" column to the existing views. This would help to distinguish (and sort/filter) faster.

Agree, this will definitely be part of the implementation :-)

great!

To anyone following this issue, I have a WIP for compose-support available. Details are in #959

The WIP image is available under portainer/portainer:pr257 (build for Linux AMD64 only).

Who'd be up to test it and give some feedback? I've created a room in our Slack channel to discuss it, please join us in the room #feat257. If you don't want to join us on Slack, you can also post comments in #959

I'd especially love to get feedback on the following topics:

• Overall UX/UI
• How would you like the ability to register a stack that has been created from outside Portainer?
• Missing features

Waauw, did a awesome job there man,

Little Bug:

• at the end of my test I saw many "fake" nginx stacks compared to "docker stack ls"

UI:

• I would like to remove my stacks directly from stack screen
• A bar like this included, https://www.composeregistry.com/, where you can search, launch and go with one click

Outside registered stacks:

• I would probably use a git repo and point to that
• Else just select or maybe just type path of the project folder on the mounted host path

Features:

• looking forward to the other features especially docker stack deploy

Keep going it's awesome!

Could you give me more details about your bug in #959 directly?

Little Bug:
at the end of my test I saw many "fake" nginx stacks compared to "docker stack ls"

@deviantony the bug was me, it was all cool, because of deploying many tries, portainer launched many nginx stacks with errors. I assumed that they never came existance or would be removed automatically because I got an error in my portainer screen. Actually they did start and sort of stopped then and stayed like that.

@scher200 can you try to reproduce and give me the error that is displayed in your Portainer screen? Also, could you comment in #959 ?

In the list stacks status set to "Up". If you go inside the stack, the status is "down".

Yes, I need to review how the status is determined. This is on my todo ;)

Additionally. In release pr275 there is an unidentified network "0_8376449903853701618_default"

@MaliciousGenius can you give me steps to reproduce ? And comment in #959 ?

@deviantony already

I've been testing the pr257 build - and noticed that any of my non registered stacks (I am assuming I can't register them yet) which contain many '-' are being removed.
I tried to add a stack from the UI where I could add a '-' i.e. postfix-relay however when starting the stack it created a whole new stack that I couldn't manage again. Looking great so far though and like where it is going!

The docker-compose.yml in question is:
version: '2'

services:
postfix-relay:
image: xxx.xxx
container_name: postfix-relay
restart: always
ports:
- 127.0.0.1:25:25
volumes:
- /etc/localtime:/etc/localtime:ro
environment:
- SMTP_HOST=xxx.xxx
- SMTP_PORT=xxx.xxx
- SMTP_USER=xxx.xxx
- SMTP_PASS=xxx.xxx
- SERVER_NAME=xxx.xxx
- ACCEPTED_NETWORKS=xxx.xx
- USE_TLS=yes
- TLS_VERIFY=may
- ZABBIX_HOSTNAME=postfix-relay
networks:
xxx-tier:
aliases:
- postfix-relay
restart: always

networks:
xxx-tier:
external:
name: xxx-proxy`

@tiredofit thanks for the report ! I'll work on this.

Any news?

No ETA on this yet, there is still a lot to work on.

I love this feature since I worked exclusively with compose before portainer.

If it's not too late I'd like to add an idea that builds on this and the "Duplicate / Edit" container function:

• select one or more containers from the list
• hit export button
• select networks and volumes that are related to the selected containers in a popup
• download the generated compose file

So a simple way to save the changed compose file after you've changed some container run config on-the-fly via the ui.

1280 has just been merged ! Support for docker stack deploy is now available in Portainer. docker-compose is not supported though.

@deviantony What does it mean when you say docker-compose isn't supported? Don't you need a compose file to deploy a stack? Not following.

Hey @mister2d, we probably need to clarify this. Stack deploy requires a compose file (version 3+) to deploy services inside a cluster, indeed you're right about this. This this the equivalent of docker stack deploy ... when using the Docker CLI.

What I mean by docker-compose is not supported is that you cannot deploy containers using the new stack feature (that would be the equivalent of docker-compose up).

So, to summarize:

• You can deploy a stack inside a Swarm cluster using a Compose file
• You cannot deploy containers using a Compose file

Is that clear enough?

You cannot deploy containers using a Compose file

Does it mean: You cannot deploy containers using a Compose file by docker-compose binaries?

You can use portainer to deploy a stack, defined in a compose-file, into a docker swarm.
You cannot use it to deploy it onto a stand-alone docker node.

Does it mean: You cannot deploy containers using a Compose file by docker-compose binaries?

Yup, you can't.

You can use portainer to deploy a stack, defined in a compose-file, into a docker swarm.
You cannot use it to deploy it onto a stand-alone docker node.

Exactly.

The main difference (CLI wise) being when you use a Compose file and docker-compose in a stand-alone Docker node, you deploy containers. When you use a Compose file and docker stack deploy in a Swarm cluster, you deploy services.

May I ask why docker-compose wasn't implemented?

@samprog1 docker-compose and docker stack deploy are two different features and both cannot be managed via the Docker API. We still need to check if we can integrate docker-compose the same way we did for docker stack deploy.

@deviantony stacking works great for my swarm cluster. Thanks for explaining.

There is one thing I'm still struggling to understand with the compose support. I use a private registry (with auth) that is defined within Portainer. How do I have my deployed stack pull from that registry? I get errors every time I deploy with private registry entries for my images.

Thanks.

@mister2d this is not available in 1.15.0, we implemented it recently (see https://github.com/portainer/portainer/issues/1313) and will be available in 1.15.1 (to be released soon).

In the meantime, you can try it in portainer/portainer:develop (linux amd64 build).

Does it support building image ?

Well, that should be something we should have a look at, you mean the equivalent of docker-compose build ?

Yes, supporting such docker-compose.yml files

version: "2"
services:
  web:
    build: ./docker/web
    ...

This should build image located in ./docker/web relative to the docker client working directory.

Problem is that Docker API doesn't support sending the build context over API. It only supports downloading it from a tarball URI, which can make things quite complicated.

See https://docs.docker.com/engine/api/v1.33/#operation/ImageBuild

remote: _string_
A Git repository URI or HTTP/HTTPS context URI. If the URI points to a single text file, the file’s contents are placed into a file called Dockerfile and the image is built from that file. If the URI points to a tarball, the file is downloaded by the daemon and the contents therein used as the context for the build. If the URI points to a tarball and the dockerfile parameter is also specified, there must be a file with the corresponding path inside the tarball.

Any news on this?

Docker Stack support was implemented. Your Docker host needs to be in swarm mode for this feature though.

Docker compose was not yet implemented

I think what we have here is already awesome. I have a similar use case to some above using a set of folders per stack (e.g. @alwaysastudent, @schmunk42 ). I'd suggest to get something like that to work, would be great to build a custom "stack manager" container that interfaces to portainer using the API (API PR and potentially separate project for the manager), rather than adding this complexity to the core project. I'd imagine you could build something very basic which links into app templates and just uses a list of folders/compose files which you could volume mount / build into the "stack manager". I can also see for my use case I'd probably want something more custom as my current stacks use powershell deployment scripts. I'll have to have a play around with the API and see what is available.

Any news on docker-compose support? It would be nice to have the ability to start / stop / restart the whole compose stack ..

I hope it will be available with load balancing option

Any news on docker-compose support?

We will not be adding compose support, instead we will focus on extending stack support.

Rgds,

Neil Cresswell

On 11/02/2018, at 12:51 PM, rtmaster <[email protected]notifications@github.com> wrote:

Any news on docker-compose support?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_257-23issuecomment-2D364724146&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=fjctoKmoiDjEKvN0vAjtGyk-e7veH02nkv_oWuJyo9g&s=oUMEJlbUVvsU329BdxgNIXwx4HkXALFwBPLrPVmuIKU&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlQV-2DO-2DCftdiSig2ma4E4-5FaU4l3mxks5tTn-5FsgaJpZM4KOcDQ&d=DwMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=fjctoKmoiDjEKvN0vAjtGyk-e7veH02nkv_oWuJyo9g&s=EWWxopT1zWm5twM4X9Jcz4L724wqzgH2N2mJuj1XJxI&e=.

This decision makes sense. A regular compose file started on a standalone Docker Engine has no corresponding entity that is listable or manageable via Docker CLI/API. If there were, it would be a functional subset of a Swarm Mode stack (lacking the deploy options).

It would be very confusing if Portainer's UI tried to implement Swarm Mode stacks (docker stack ...) AND a faux "compose stack".

Of course, this means everyone who runs on a single host (vs a cluster) will probably want to run a single node Swarm Mode cluster just so they can manage stacks. Doesn't seem like a big deal.

This issue should be closed, then?

What about multiple compose files, like @schmunk42 showed? Is that somehow supported?
We are using portainer and started using multiple compose files using docker stack deploy -c docker-compose.yml -c docker.compose.prod.yml stack_name etc.

@ronlut this is not supported yet but you can open a feature request for this.

Any update on this?

Wanted to post a quick workaround to this:

1. docker-compose up
2. Open Portainer web-ui and go to the container
3. Select Duplicate/Edit
4. Remove the stack from the LABELS section where is says com.docker.compose.project
5. Deploy edited container

-- done