Environment variable contents cause YAML error in `docker-compose pull` after upgrading to Docker Desktop 3.6.0 with Docker Compose v2.0.0-rc.1

[vitaly-krugl]

Description

After upgrading to Docker Desktop 3.6.0 with Docker Compose version v2.0.0-rc.1 automatically enabled, I started getting the error "yaml: line 5: mapping values are not allowed in this context" when running

docker-compose -f compose.yml pull

Steps to reproduce the issue:
1.I created a minimal set of files to reproduce this issue. Create the following files in the same directory:
Dockerfile:

FROM python:3.8.3-slim-buster

compose.yml:

version: '3.8'


services:
  dev:
    working_dir: /a/myapp

    build:
      args:
        DEV_ENV_BUILD_ARG: ${DEV_ENV_BUILD_ARG}
      context: ${PWD}
      dockerfile: ${PWD}/Dockerfile

arg1.txt:

:

value

2. Now, run these commands from bash shell in the same directory that contains the above files

export DEV_ENV_BUILD_ARG="$(cat ./arg1.txt)"

docker-compose -f compose.yml pull

yaml: line 5: mapping values are not allowed in this context

echo $?
15

Describe the results you received:
docker-compose failed with the error message: "yaml: line 5: mapping values are not allowed in this context"

Describe the results you expected:
Success

Additional information you deem important (e.g. issue happens only occasionally):
This worked successfully prior to docker-compose v2. In fact, if I disable docker-compose v2, my test passes:

docker-compose disable-v2
docker-compose -f compose.yml pull
Pulling dev ... done
echo $?
0

Output of docker-compose --version:

Docker Compose version v2.0.0-rc.1

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:10 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker context show:
You can also run docker context inspect context-name to give us more details but don't forget to remove sensitive content.

default

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.1)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 355
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 1.939GiB
 Name: docker-desktop
 ID: EXXX:HMR7:557I:VBT6:IYHC:UXSL:IAWB:S5OS:Z5XD:2O4U:YI6H:PEI7
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 43
  Goroutines: 45
  System Time: 2021-08-13T23:53:08.178874Z
  EventsListeners: 3
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS ECS, Azure ACI, local, etc.):
MacBookPro with Catalina mac os 10.15.7

[vitaly-krugl]

Also, the message like "yaml: line 5: mapping values are not allowed in this context" is incredibly unhelpful for root causing the problem and is terrible for productivity -

1. It doesn't give the name of the file, so when there are multiple YAML files involved, you lose a lot of productivity trying to figure out which one had the problem.
2. The line number (line 5 in this case) doesn't match the line number in the actual YAML file. Perhaps it's not counting the comment lines or something like that, but in any case I lost a lot of productivity trying to chase it down.
3. The message doesn't provide any other actionable context, such as attribute name and/or some of the text where the problem was detected that could help improve productivity.

[skinner2401]

We're seeing the same thing. We reproduced this in a minimal environment by creating the following files:

docker-compose.yml

version: "3.9"
services:
  db:
    image: mysql
    environment:
      SOME_ENV_VAR: "${SOME_ENV_VAR}"

.env

SOME_ENV_VAR="foo: bar"

Run:
$ docker compose build db

Sample output:

yaml: line 4: mapping values are not allowed in this context

Workaround:
Uncheck Use Docker Compose V2 release candidate within the docker preferences:

and then use docker-compose instead of docker compose. When running docker compose after unchecking the V2 rc checkbox, we still ran into this issue.

[skinner2401]

Would also note that this only occurs when environment variable expansion occurs within the docker-compose.yml file. When we replaced the env var within the docker-compose.yml with the string literal contained within the .env variable (removing any possibility of env variable expansion) the issue disappeared, and we could still use foo: bar without issue.

Macbook pro 2017

OS details:
macOS Big Sur
11.5.2 (20G95)

docker-compose version

Docker Compose version v2.0.0-rc.1

zsh --version

zsh 5.8 (x86_64-apple-darwin20.0)