Skip to content

[BUG] using multiple yaml alias not working. mapping key "<<" already defined  #10411

Closed
Closed
[BUG] using multiple yaml alias not working. mapping key "<<" already defined #10411
Assignees
ndeloof
Labels
kind/bug

Description

@arulrajnet
arulrajnet
opened on Mar 29, 2023

Description

docker-compose.yml

version: '3.9'

# yml aliases and anchors
x-mysql: &mysqlenv
  MYSQL_ROOT_PASSWORD: root
  MYSQL_DATABASE: test
  MYSQL_USER: test
  MYSQL_PASSWORD: test

x-mysql-extraenv: &mysqlextraenv
  MY_CUSTOM_VAR: mycustomvalue

services:
  mysql:
    image: mysql:5.7
    environment:
      <<: *mysqlenv
      <<: *mysqlextraenv
    volumes:
      - mysql-data:/var/lib/mysql

volumes:
  mysql-data:

Try docker-compose config. Then getting the following error

yaml: unmarshal errors:
  line 18: mapping key "<<" already defined at line 17

The version of my docker-compose is 2.17.2

The same is working in 2.15.1 , 2.16.0 and even release candidate version 2.17.0-rc.1.

This is broken from version 2.17.0

2023-03-29_09-13

Steps To Reproduce

  1. Windows 11
  2. Docker compose v2.17.0
  3. Copy the above docker-compose.yml
  4. Then docker-compose config
  5. You will see the error

Compose Version

Docker Compose version v2.17.0

Docker Environment

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 9
  Running: 1
  Paused: 0
  Stopped: 8
 Images: 53
 Server Version: 23.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-67-generic
 Operating System: Ubuntu 22.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 8.736GiB
 Name: ubuntuserver2204
 ID: c8839b3e-9c74-4cb6-9444-e5c979394be0
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: arulrajnet
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/

Anything else?

Not Working in 2.17.0

arul@windows ~/Downloads/dc-yaml-alias-bug$~/Downloads/docker-compose-windows-x86_64-2.17.0.exe config
yaml: unmarshal errors:
  line 18: mapping key "<<" already defined at line 17

Working in 2.17.0-rc.1

arul@windows ~/Downloads/dc-yaml-alias-bug$~/Downloads/docker-compose-windows-x86_64-2.17.0-rc.1.exe config
name: dc-yaml-alias-bug
services:
  mysql:
    environment:
      MY_CUSTOM_VAR: mycustomvalue
      MYSQL_DATABASE: test
      MYSQL_PASSWORD: test
      MYSQL_ROOT_PASSWORD: root
      MYSQL_USER: test
    image: mysql:5.7
    networks:
      default: null
    volumes:
    - type: volume
      source: mysql-data
      target: /var/lib/mysql
      volume: {}
networks:
  default:
    name: dc-yaml-alias-bug_default
volumes:
  mysql-data:
    name: dc-yaml-alias-bug_mysql-data
x-mysql:
  MYSQL_DATABASE: test
  MYSQL_PASSWORD: test
  MYSQL_ROOT_PASSWORD: root
  MYSQL_USER: test
x-mysql-extraenv:
  MY_CUSTOM_VAR: mycustomvalue

Activity

  1. arulrajnet
    added
    kind/bug
    status/0-triage
    on Mar 29, 2023
  2. ndeloof
    removed
    status/0-triage
    on Mar 29, 2023
  3. ndeloof
    self-assigned this
    on Mar 29, 2023

  4. ndeloof commented on Mar 29, 2023

    @ndeloof
    ndeloof
    on Mar 29, 2023
    Contributor

    Thanks for reporting this issue, I'll look into this today

  5. ndeloof commented on Mar 29, 2023

    @ndeloof
    ndeloof
    on Mar 29, 2023
    Contributor

    This is an unexpected side-effect upgrading to go-yaml/v3

  6. ndeloof commented on Mar 29, 2023

    @ndeloof
    ndeloof
    on Mar 29, 2023
    Contributor

    goyaml/v3 does not support repeated anchors (due to internal storage using << as key during parsing) but allows use of multi-values in anchors

        environment:
      <<:  [*mysqlenv, *mysqlextraenv]

    According to https://yaml.org/type/merge.html this is the expected syntax

  7. arulrajnet commented on Mar 29, 2023

    @arulrajnet
    arulrajnet
    on Mar 29, 2023
    Author

    Thanks, @ndeloof. This fix works. Will close this.

  8. arulrajnet
    closed this as completedon Mar 29, 2023

  9. gowrizrh commented on Mar 30, 2023

    @gowrizrh
    gowrizrh
    on Mar 30, 2023

    One thing to note that it's not quite the same behaviour.

    Value overrides don't work as before. If you previously had

    x-mysql: &mysqlenv
  MYSQL_USER: test

x-mysql-extraenv: &mysqlextraenv
  MYSQL_USER: overridetest

    environment:
      <<: *mysqlenv
      <<: *mysqlextraenv

    the value of the variable MYSQL_USER would have been overriden to the overridetest.

    However with the new syntax, the value for MYSQL_USER is test.

        environment:
      <<: [*mysqlenv, *mysqlextraenv]

  10. gowrizrh commented on Mar 30, 2023

    @gowrizrh
    gowrizrh
    on Mar 30, 2023

    Sorry for the noise, but it seems to merge in reverse?

    Rewriting it as

        environment:
      <<: [*mysqlextraenv, *mysqlenv]

    Does override values of mysqlenv with values from mysqlextraenv

  11. gowrizrh commented on Mar 30, 2023

    @gowrizrh
    gowrizrh
    on Mar 30, 2023

    Ok, after swifting through https://yaml.org/type/merge.html it does mention this.

    If the value associated with the merge key is a sequence, then this sequence is expected to contain mapping nodes and each of these nodes is merged in turn according to its order in the sequence. Keys in mapping nodes earlier in the sequence override keys specified in later mapping nodes.

    I hope it's helpful to someone else.

  12. mmguero
    added 2 commits that reference this issue on Apr 5, 2023

    Merge pull request #179 from mmguero-dev/v23.04.0_merge_idaholab

    93c34a3

    Merge pull request #247 from mmguero-dev/v23.04.0_merge_cisagov

    37bb7f7
  13. spaceo
    added a commit that references this issue on Apr 13, 2023

    Fix docker compose validation error

    8c43d61

  14. 7 remaining items

  15. spaceo
    added a commit that references this issue on May 9, 2023

    Fix docker compose validation error

    26d0ff0
  16. abates
    added 6 commits that reference this issue on May 11, 2023

    fix: docker-compose fails with multiple merge keys

    89955d5

    fix: docker-compose fails with multiple merge keys

    cd51281

    fix: docker-compose fails with multiple merge keys (#32)

    2093126

    fix: Docker Compose no longer allows duplicate `<<` entries.

    582932a

    fix: Docker Compose no longer allows duplicate `<<` entries.

    a870afa

    fix: Docker Compose no longer allows duplicate `<<` entries.

    6d17265
  17. chrnie
    added a commit that references this issue on May 25, 2023

    join repeated "<<"

    01b5f01
  18. physikerwelt
    added a commit that references this issue on Jul 24, 2023

    Fix yml errors

    8a6cfd9
  19. TanjinAlam
    added a commit that references this issue on Oct 19, 2023

    Update docker-compose.yml

    b638302
  20. Skitionek
    added 4 commits that reference this issue on Jan 8, 2024

    use supported yaml syntax: docker/compose#10411

    ec0f75f

    use supported yaml syntax: docker/compose#10411

    426f951

    use supported yaml syntax: docker/compose#10411

    5b20999

    use supported yaml syntax: docker/compose#10411

    4d7d764
  21. NoNamePaul94
    added a commit that references this issue on Apr 19, 2024

    Fix multiple yaml alias not working

    b099503
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

kind/bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions