Cloud & DevOps
291 verified issues
Configuring one provider with a dynamic attribute from another (was: depends_on for providers)
This issue was inspired by this question on Google Groups. > I've got some Terraform code that doesn't work because the EC2 instance running the Docker daemon doesn't exist yet so I get "\* Error pinging Docker server: Get http://${aws_instance.docker.public_ip}:2375/_ping: dial tcp: lookup ${aws_instance.docker.public_ip}: no such host" if I run plan or apply. There are providers (docker and consul - theoretically also openstack but that's a stretch) that can be implemented with Terraform itself using other providers like AWS; if there are other resources in a Terraform deployment that use the (docker or consul) provider they cannot be provisioned or managed in any way until and unless the other resources that implement the docker server or consul cluster have been successfully provisioned. If there were a `depends_on` clause for providers like docker and consul, this kind of dependency could be managed automatically. In the absence of this, it may be possible to add `depends_on` clauses for all the resources using the docker or consul provider, but that does not fully address the problem as Terraform will attempt (and fail, if they are not already provisioned) to discover the state of the docker/consul resources during the planning stage, long before it has completed the computation of dependencies. Multiple plan/apply runs may be able to resolve that specific problem, but having a `depends_on` clause for providers would allow everything to be managed in a single pass
terraform get: can't use variable in module source parameter?
I'm trying to avoid hard-coding module sources; the simplest approach would be: [code block] The result I get while attempting to run `terraform get -update` is [code block]
Support use cases with conditional logic
It's been important from the beginning that Terraform's configuration language is declarative, which has meant that the core team has intentionally avoided adding flow-control statements like conditionals and loops to the language. But in the real world, there are still plenty of perfectly reasonable scenarios that are difficult to express in the current version of Terraform without copious amounts of duplication because of the lack of conditionals. We'd like Terraform to support these use cases one way or another. I'm opening this issue to collect some real-world example where, as a config author, it seems like an `if` statement would really make things easier. Using these examples, we'll play around with different ideas to improve the tools Terraform provides to the config author in these scenarios. So please feel free to chime in with some specific examples - ideally with with blocks of Terraform configuration included. If you've got ideas for syntax or config language features that could form a solution, those are welcome here too. (No need to respond with just "+1" / :+1: on this thread, since it's an issue we're already aware is important.)
Create Ansible provisioner
I'd like to use Terraform to build infrastructure then configure it with Ansible, similar to the Chef provisioner.
Depends_on for module
Possible workarounds For module to module dependencies, this workaround by @phinze may help. Original problem This issue was promoted by this question on Google Groups. Terraform version: `Terraform v0.3.7` I have two terraform modules for creating a digital ocean VM and DNS records that are kept purposely modular so they can be reused by others in my organisation. I want to add a series of provisioners using local_exec after a VM has been created and DNS records made. Attempted solution I tried adding a provisioner directly to my terraform file (i.e. not in a resource) which gave an error. I then tried using the `null_resource` which worked but was executed at the wrong time as it didn't know to wait for the other modules to execute first. I then tried adding a `depends_on` attribute to the null resource using a reference to a module but this doesn't seem to be supported using this syntax: [code block] Expected result Either a way for a resource to depend on a module as a dependency or a way to "inject" (for lack of a better word) some provisioners for a resource into a module without having to make a custom version of that module (I realise that might be a separate issue but it would solve my original problem). Terraform config used [code block]
Define host volumes in the `volumes` section of the Compose file
It seems that there is not the possibility to mount a host directory using the `volumes` section using the versrion 2 I would propose an configuration example [code block] My use case: docker-compose.yml [code block] Override docker-compose.prod.yml [code block]
entrypoint defined in docker-compose.yml wipes out CMD defined in Dockerfile
Hi, I am following the example on https://docs.docker.com/compose/startup-order/ to make sure the database is running before I start the application. My `Dockerfile` contains the command [code block] and in my `docker-compose.yml` I have [code block] but after the postgres database starts the service container just exits straight away. [code block] Looking inside the running container I can see the entrypoint does not have the command appended [code block] If both command and entrypoint are in the same place (either in the `Dockerfile` or in the `docker-compose.yml`) the application starts up properly. [code block] Any idea is this is a bug or I am doing something wrong? Thanks
docker-compose dry run
Would like insight to what docker-compose generates/executes
[1.7] CTRL+C fails to stop containers when running compose in the foreground
I can't reproduce it but I would say about 10% of the time when I hit CTRL+C, the containers that were started with `docker-compose up` fail to stop. I'll get an ABORT message in the terminal but they are still up. When it works, it says "gracefully stopping..." instead of ABORT. At this point I need to `docker-compose stop` them to "really" stop them. It was much worse in earlier versions of Compose but it's still happening often enough that I feel like it needs some attention. If you need more info let me know. Environment System [code block] Docker [code block] Docker Compose [code block] Potentially relevant information When I encounter the issue I'm always running multiple containers, such as postgres, redis, and multiple web services. I never run 1 container so I can't say for sure if it would happen with only 1. It happens with and without using `links` and both the legacy and v2 `docker-compose.yml` files.
Hooks/ Plugins support to run arbitrary scripts
It would be nice if `docker-compose up|down|..` would look for custom scripts in the current directory, like: [code block] I guess this is mostly helpful when docker-compose is used to run the development environment. These scripts should not be meant to configure containers. My current use-cases include: - set up hostnames in `/etc/hosts` after container startup - run docker-rsync after container started Eventually docker-compose should allow these scripts to run in the background and stream stdout/stderr to `docker-compose logs`. Related https://github.com/docker/compose/pull/74 https://github.com/docker/compose/issues/57
Docker Compose mounts named volumes as 'root' exclusively
It's about named volumes (so no "data volume container", no "volumes-from") and docker-compose.yml. The goal here is to use docker-compose to manage two services 'appserver' and 'server-postgresql' in two separate containers and use the "volumes:" docker-compose.yml feature to make data from service 'server-postgresql' persistent. The Dockerfile for 'server-postgresql' looks like this: [code block] Adn the docker-compose.yml looks like this: [code block] Then I start everything with `docker-compose up -d`, I enter my server-postgresql container with `docker-compose exec server-postgresql bash`, a quick `ls` does reveal `/volume_data`, I then `cd` into it and try `touch testFile` and got "permission denied. Which is normal because a quick `ls -l` show that `volume_data` is owned by `root:root`. Now what I think is happening is that since I have `USER postgres` in the Dockerfile, when I run `docker-compose exec` I am logged in as user 'postgres' (and the postgresql daemon runs as user 'postgres' as well, so it won't be able to write to `/volume_data`). This is confirmed because when I run this instead: `docker-compose exec --user root server-postgresql bash` and retry to `cd /volume_data` and `touch testFile`, it does work (it's not a permission error between the host and the container, as it is somtimes the case when the container mounts a host folder, this is a typical unix permission error because `/volume_data` is mounted as 'root:root' while user 'postgres' is tryin
docker-compose hangs
Description of the issue In a new server installation (on a VPS provided by OVH) running Ubuntu 18.x the Docker system works flawlessy and fast but docker-compose sometimes hangs indefinitely for no reason. And for every command, from version to ps, up, logs and so on. I tried everything regarding DNS and name resolution, even disabling systemd-resolved. When it works it is fast. When it hangs I can see the `/tmp/_Mxyz123123` directory created. Running with verbose hangs before any output. And then completes flawlessy. Context information (for bug reports) Output of `docker-compose version` [code block] Output of `docker version` [code block] Output of `docker-compose config` (Make sure to add the relevant `-f` and other flags) [code block] Steps to reproduce the issue 1. docker-compose <command> Observed result Sometimes hangs Expected result Run fast. Stacktrace / full error message No stacktrace but i managed to strace the behavior. [code block] Additional information Linux vps[omit] 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Ubuntu 18.04.2 LTS docker-compose installed with curl I have lot of other docker-compose environments working and this is the first time I see this behavior. Any idea on what to check or where to look?
The default naming scheme for containers created by Compose
> The default naming scheme for containers created by Compose in this version > has changed from `<project>_<service>_<index>` to > `<project>_<service>_<index>_<slug>` Is there any way to disable this behavior apart from using `container_name` in yaml file? We have many scripts that rely on container names and are not using swarm, just a single container stack. This change is very inconvenient for us.
docker-compose build --push <registry>
This would be a nice feature for creating and deploying of containers on a different (development) machine than the productive ones (which may run via image: option)
Support for NVIDIA GPUs under Docker Compose
Under Docker 19.03.0 Beta 2, support for NVIDIA GPU has been introduced in the form of new CLI API --gpus. https://github.com/docker/cli/pull/1714 talk about this enablement. Now one can simply pass --gpus option for GPU-accelerated Docker based application. [code block] As of today, Compose doesn't support this. This is a feature request for enabling Compose to support for NVIDIA GPU.
Allow for existing containers (allow sharing containers across services)
I think it would be useful if compose could allow you to specify containers that may already exist, eg: [code block] If the container 'shareddata' did not exist, it would be created as usual. If, however, that container already existed, the allow_existing: true setting would not complain about a duplicate container, instead just skipping creation (and perhaps it would try to bring the container up if it were stopped?). I haven't python-ed in a long time, but I might be able to create a PR for this feature if someone wanted to give me a little guidance on where the best place to start looking into the code would be.
docker-compose copy file or directory to container
we miss a possibility to copy a file or directory using docker-compose. I find this really useful. Please check many +1 in premature closed https://github.com/docker/compose/issues/2105
Using shell command in docker-compose.yml
Hello, is there a way to use shell commands in `docker-compose.yml` file? Here is my use case: [code block] Currently it's giving me this error: [code block]
Feature: Ability to clear log history
A feature I've thought would be useful since originally using Fig, and now Compose would be the ability to clear the log history for Composed-managed containers. Long-running or "chatty" containers can end up with a lot of log noise that may not be wanted. I'd expect a command like the following would solve the problem: `$ docker-compose logs --clear [service]`
Docker compose pull doesn't respect local images
During my searches I've seen this issue raised and closed a few times, but even using the latest build of docker-compose, this still doesn't behave the way I understand that it is expected to. Scenario: We're building a composed environment for our CI tests to run. We grab a database image, an api image, and a _local_ copy of the image containing application we are trying to test (it has been built in a previous CI step, and tagged with the build hash provided by our CI environment (CIRCLE_SHA1) docker-compose.yml [code block] The commands we run then are as follows: [code block] Actual Result: No matter what I do, docker compose always tries to pull my CIRCLE_SHA1 tagged version from docker hub. it doesn't exist there, I never want to push it (until it passes tests and is re-tagged as :latest and/or :release I have a unique tag CIRCLE_SHA1 which only exists inside the build environment, meaning no confusion for docker-compose when it tries to pull, and yet, it seems to try to fetch it anyway, and fail even though that exact tag exists locally. Expected Result: I'd expect the fact that there is no remote build tagged with CIRCLE_SHA1 to cause docker-compose to use the local copy it finds. I need to do pull, because I want everything else to be the latest. I'd suggest that if there is confusion where `image:` refers to a remote repository, then perhaps we could use `local:` instead, to reference a local image?
Execute a command after run
Hi, It will be very helpful to have something like "onrun" in the YAML to be able to run commands after the run. Similar to https://github.com/docker/docker/issues/8860 [code block] After the mongodb start, It will dump db2dump.domain.lan and restore it. When I will stop and then start the container, onrun part will no be executed to preserve idempotency. EDIT 15 June 2020 5 years later, Compose wan't to "standardize" specifications, please check https://github.com/compose-spec/compose-spec/issues/84
docker-compose up doesn't pull down latest image if the image exists locally
It would be nice if there were an option to check for new versions of images when running `docker-compose up`. We already have this functionality with `docker build --pull` as was discussed here https://github.com/docker/docker/issues/4238 and there is an open issue to bring `--pull` to `docker run` here https://github.com/docker/docker/issues/13331. I propose adding `--pull` to `up` to always attempt to pull a newer version of the images in the compose file.
Is there a way to delay container startup to support dependant services with a longer startup time
I have a MySQL container that takes a little time to start up as it needs to import data. I have an Alfresco container that depends upon the MySQL container. At the moment, when I use fig, the Alfresco service inside the Alfresco container fails when it attempts to connect to the MySQL container... ostensibly because the MySQL service is not yet listening. Is there a way to handle this kind of issue in Fig?
Docker-compose up failing because "port is already allocated"
My docker container is able to successfully build but when I enter the command docker-compose build, the following error is returned: Starting docker_etl_1 ... Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 ... Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 ... error ERROR: for 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 Cannot start service postgis: driver failed programming external connectivity on endpoint 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 (91464afbee8bf7212061797ec0f4c017a56cc3c30c9bdaf513127a6e6a4a5a52): Error starting userland prStarting docker_etl_1 ... done ERROR: for postgis Cannot start service postgis: driver failed programming external connectivity on endpoint 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 (91464afbee8bf7212061797ec0f4c017a56cc3c30c9bdaf513127a6e6a4a5a52): Error starting userland proxy: Bind for 0.0.0.0:5432 failed: port is already allocated Here is my docker-compose.yaml version: '2' services: postgis: build: ./postgis volumes: - ../src/main/sql:/sql ports: - "5432:5432" etl: build: ./etl volumes: - ..:/national-voter-file entrypoint: - python3 - /national-voter-file/load/loader.py and here is the Dockerfile: FROM m
docker-compose slow on docker for mac os beta
docker-compose is slow with docker for mac os beta on my home network. Here is my workaround for now: - docker-compose up (take ages) - shut down wifi - docker-compose up (really fast) - re-enable wifi I do not reproduce the issue on another network than mine, my work network for instance do not make it slow. I already had a potentially related issue with the docker client itself which couldn't pull any image (going to bizarre local ips instead of the docker hub registry) but it has been fixed since one of the latest docker for mac os beta update. The issue is not reproduced against the docker-toolbox, only the "native" docker for mac. My version of docker-compose is : `docker-compose version 1.7.0, build 0d7bf73` My version of docker for mac is: `Version 1.11.1-beta10 (build: 6662)` The docker-compose file I'm trying to run is: [code block]
Provide RPM and DEB packages for compose
Not everyone likes to install binaries (#1288). It might be good to provide a system package for compose. We should use https://github.com/spotify/dh-virtualenv for the debian package and look into a similar approach for the rpm.
allow removing something in docker-compose.override.yml
My usecase is that the base `docker-compose.yml` maps a certain port in the `ports` section. In my `docker-compose.override.yml` I'd like to change that port since I already have a different service running there on my host. As far as I understand the current implementation it is only possible to add stuff, override extends the other file. As far as I can tell there is no way to remove an entry. And I can't think of an obvious syntax for it.
Define services which are not started by default
Users quite often define maintenance scripts, test suites, debugging systems in their Compose files which they don't want to run when they do `docker-compose up`. There should be some way of defining what services are started by default, but can be still be run manually by doing `docker-compose up servicename` or `docker-compose run servicename ...`. Possible solutions 1) Recommend users to use a separate Compose file 2) Add an option to services to make them not start by default 3) Add a top-level configuration option to define the default services 4) Add a concept of a thing like a service, but is just for one-off commands ("scripts", "tasks", etc...) (Please suggest others if you have ideas.) Data points: - https://github.com/docker/compose/issues/697 - https://github.com/docker/compose/issues/912 - https://github.com/docker/compose/issues/942 - https://github.com/docker/compose/issues/1439 - https://github.com/docker/compose/issues/1547 - #542 - "test" service in https://github.com/heroku/logplex/blob/master/docker-compose.yml - ~~@cpuguy83: "I've got a little helper service in my compose yaml that injects a bunch of stuff into redis for hipache, which of course exits when it is done. Can't use `compose up` w/o the `-d`"~~ this is fixed now that we don't exit from `compose up` until all services have exited.
key sequence to detach from `docker-compose up`
this has been topic in #291, however i want to make aware of this feature request a little more . i vote for a key-sequence that you can use to detach from `docker-compose up` without stopping all containers (i.e. when you forgot to pass `-d`). There seems to be a key-sequence for attached docker-containers (`Ctrl-p` `Ctrl-c`). It seems plausible to reuse this in docker-compose.
Problem with dependant module resolution if the path is relative
Terraform Version 0.12.13 Terraform Configuration Files Here you can see two examples: https://github.com/xocasdashdash/terraform-test-case One works perfectly with 0.11, same one fails on 0.12.13 (and on dev too). Debug Output [code block] Expected Behavior It should resolve to the correct module path for the "d-module". Actual Behavior It does not. But if I change the route to use a local symlink and add a double "//" on the last folder before the module folder "//a-module" and set up a symlink from the module to the parent folder it does work correctly. Steps to Reproduce Run terraform init in each of the three folders with the last working version (0.11.14 and 0.12.13). Additional Context I've tried to fix it myself and I think the fix should go to this function: https://github.com/hashicorp/terraform/blob/6f66aad03262441521829ca3a678da2bb6bf51d9/internal/initwd/module_install.go#L226 I'm gonna try some more to make it work but I believe a bigger change will be needed to get this to work in all cases