Terms defined using list containers only preserve the latest-declared list value?

[trwnh]

Background

Context

In Activity Streams 2.0's normative context document, we define "items" and "orderedItems" separately, but with the same id. The difference is that "orderedItems" uses a list container, and "items" does not:

"items": {
  "@id": "as:items",
  "@type": "@id"
},

"orderedItems": {
  "@id": "as:items",
  "@type": "@id",
  "@container": "@list"
},

Simple example

Consider the input graph:

{
  "@id": "c",
  "https://www.w3.org/ns/activitystreams#items": [
    {"@list": [
      {"@id": "o1"}
    ]},
    {"@list": [
      {"@id": "o2"}
    ]}
  ]
}

You would expect both o1 and o2 to show up in the compacted result in some fashion, perhaps something like this: https://json-ld.org/playground/#startTab=tab-compacted&json-ld=%7B%22%40id%22%3A%22c%22%2C%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%3A%5B%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o1%22%7D%5D%7D%2C%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o2%22%7D%5D%7D%5D%7D&context=%7B%22%40context%22%3A%7B%22items%22%3A%7B%22%40id%22%3A%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%2C%22%40type%22%3A%22%40id%22%7D%7D%7D

{
  "@context": {
    "items": {
      "@id": "https://www.w3.org/ns/activitystreams#items",
      "@type": "@id"
    }
  },
  "@id": "c",
  "items": [
    {
      "@list": [
        "o1"
      ]
    },
    {
      "@list": [
        "o2"
      ]
    }
  ]
}

But in actuality, when both "items" and "orderedItems" are defined, only o2's list is preserved, via "orderedItems": https://json-ld.org/playground/#startTab=tab-compacted&json-ld=%7B%22%40id%22%3A%22c%22%2C%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%3A%5B%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o1%22%7D%5D%7D%2C%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o2%22%7D%5D%7D%5D%7D&context=%7B%22%40context%22%3A%7B%22items%22%3A%7B%22%40id%22%3A%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%2C%22%40type%22%3A%22%40id%22%7D%2C%22orderedItems%22%3A%7B%22%40id%22%3A%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%2C%22%40type%22%3A%22%40id%22%2C%22%40container%22%3A%22%40list%22%7D%7D%7D

{
  "@context": {
    "items": {
      "@id": "https://www.w3.org/ns/activitystreams#items",
      "@type": "@id"
    },
    "orderedItems": {
      "@id": "https://www.w3.org/ns/activitystreams#items",
      "@type": "@id",
      "@container": "@list"
    }
  },
  "@id": "c",
  "orderedItems": [
    "o2"
  ]
}

You can see this happen with the normative activitystreams context document, too: https://json-ld.org/playground/#startTab=tab-compacted&json-ld=%7B%22%40id%22%3A%22c%22%2C%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%3A%5B%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o1%22%7D%5D%7D%2C%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o2%22%7D%5D%7D%5D%7D&context=%7B%22%40context%22%3A%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%22%7D

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "c",
  "orderedItems": ["o2"]
}

Slightly more complex example

Consider the input graph:

{
  "@id": "c",
  "https://www.w3.org/ns/activitystreams#items": [
    {"@list": [
      {"@list": [{"@id": "o1"}]},
      {"@id": "o2"}
    ]},
    {"@id": "o3"},
    {"@list": [
      {"@list": [{"@id": "o4"}]},
      {"@id": "o5"}
    ]},
    {"@id": "o6"}
  ]
}

The JSON-LD playground produces this: https://json-ld.org/playground/#startTab=tab-compacted&json-ld=%7B%22%40id%22%3A%22c%22%2C%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%23items%22%3A%5B%7B%22%40list%22%3A%5B%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o1%22%7D%5D%7D%2C%7B%22%40id%22%3A%22o2%22%7D%5D%7D%2C%7B%22%40id%22%3A%22o3%22%7D%2C%7B%22%40list%22%3A%5B%7B%22%40list%22%3A%5B%7B%22%40id%22%3A%22o4%22%7D%5D%7D%2C%7B%22%40id%22%3A%22o5%22%7D%5D%7D%2C%7B%22%40id%22%3A%22o6%22%7D%5D%7D&context=%7B%22%40context%22%3A%22https%3A%2F%2Fwww.w3.org%2Fns%2Factivitystreams%22%7D

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "c",
  "orderedItems": [
    [
      "o4"
    ],
    "o5"
  ],
  "items": [
    "o3",
    "o6"
  ]
}

Here, we note that o3 and o6 were split out into the "items" property, which defaults to an unordered set container. This is expected.

But we also note that the "orderedItems" property, which is defined as an ordered list container, should refer to both lists in the input graph. Curiously, the list containing the list-of-o1 and then o2 has disappeared from the compacted output entirely.

Questions

What is the correct behavior here?

I can't imagine the disappearance of o1 and o2 to be correct, but there is a difference between [{"@list": []}, {"@list": []}] and "@list": [{"@list": []}, {"@list": []}]. With there being 2 list values in an unordered set, I would guess that the "orderedItems" term is no longer fully appropriate? So maybe the correct answer should be something like applying only the "items" term definition:

{
  "@context": {
    "items": {
      "@id": "https://www.w3.org/ns/activitystreams#items",
      "@type": "@id"
    }
  },
  "@id": "c",
  "items": [
    {
      "@list": [
        "o1"
      ]
    },
    {
      "@list": [
        "o2"
      ]
    }
  ]
}

which applies the @type: @id of the "items" term definition and preserves the lists as @list nodes.

For the more complex example:

{
  "@context": {
    "items": {
      "@id": "https://www.w3.org/ns/activitystreams#items",
      "@type": "@id"
    }
  },
  "@id": "c",
  "items": [
    {
      "@list": [
        {
          "@list": [
            {
              "@id": "o1"
            }
          ]
        },
        "o2"
      ]
    },
    "o3",
    {
      "@list": [
        {
          "@list": [
            {
              "@id": "o4"
            }
          ]
        },
        "o5"
      ]
    },
    "o6"
  ]
}

More precisely, if both "items" and "orderedItems" are defined, then the definition of "orderedItems" will cause some information to be lost. More generally: A term definition using @container: @list will result in information loss with more than 1 list value, regardless if a term definition without @container: @list is present.

Is the bug in the compaction algorithm or in the implementation?

This was observed on the JSON-LD playground, which uses jsonld.js iirc. But it's possible the issue is actually an oversight of the compaction algorithm itself, so I'm filing this here for now. If the compaction algorithm actually accounts for this, then I can refile the issue against jsonld.js and/or the playground.

Etc

I found this possibly related issue:

• List of lists? #428

Although this current issue seems to occur even with "simpler" lists that don't contain other lists.