Inconsistent Behavior of [] between version 1.7.0 and 1.8.3

[jhorbulyk]

Consider the following input:

{
  "singleArray": [{"foo":"bar"}]
}

and JSONata expression:
singleArray[]

In version 1.7.0, the evaluated result is

[
  {
    "foo": "bar"
  }
]

but in 1.8.3 it is

[
  [
    {
      "foo": "bar"
    }
  ]
]

Which evaluation is correct and why?

[andrew-coleman]

This is a result of this PR: #405
to fix this bug: #399

This idea is that the [] syntax turns the sequence into an array even if that sequence contains a single value which itself is an array.

[jhorbulyk]

The problem is that I don't think that change fully works. In JSONata 1.8.3, if we change the JSONata expression to
(singleArray[] ~> $map(function($i){$i}))[]
then the result is still

[
  {
    "foo": "bar"
  }
]

not

[
  [
    {
      "foo": "bar"
    }
  ]
]

and I can't find an JSONata expression that uses $map() to get me to the second output.

[goulderb]

The only solution I've found that will force nested arrays when working with array functions (e.g. $map, $filter) is the following:

[
    [
        $map(singleArray, function($i) {$i})
    ]
]

Using [] at the end of the expression does not work the same as it does without functions involved. In function compositions, this means there isn't a way to force nested arrays as part of the output without wrapping with more functions to force arrays using the wrapping square brackets notation. At least, not that I'm aware of, anyway.

[RedShift1]

This is stupid. Why would a function like $map() ever return anything other than an array? No other language does this.

[andrew-coleman]

This is stupid.

Thanks for the feedback.

In common with XQuery, JSONata treats singleton sequences as equivalent to the value.

But there is potentially an issue highlighted above (slightly more eloquently) that needs investigating.

[knolleary]

Just found this issue as I was expecting $map to always return an array. The docs for $map says it returns an Array and doesn't mention the singleton sequence case.

I found the workaround was to add [] after the $map function - this appears to ensure the result is always an array without causing the double nested array mentioned above:

$map($.payload, function($v) {  {"className":$v.className, "score":$v.score*100, "bbox":$v.bbox}})[]

[dreusel]

Shouldn't changes like this trigger a major version increment, as it breaks applications expecting the previous behavior?

[caralmar]

I ran into this problem while developing a function that adds a namespace to properties in a JSON. The function is recursive and traverses objects and arrays. However, when dealing with arrays that happen to contain another array, and that array was the only element, the function breaks. The issue can be seen by testing a simple JSON:
{
"test" : [[1,2,3]]
}

If we apply this JSONata:
$map(test, function($v){
$v
})
The function returns:
[
1,
2,
3
]

It is not returning an array of arrays, but rather the first element of the outer array. This behavior is different than the "map" function in javascript, which returns [[1,2,3]]. The description for the "map" function:
"Returns an array containing the results of applying the function parameter to each value in the array parameter."

That description is incorrect for the test case I showed above. The map is returning the first ELEMENT of the array that was passed to it, not an array of arrays.

I solved this problem using a "hack" but it works:
(
$singleton := $count(test) = 1;
$result := $map(test, function($v){
$v
});
$singleton and $type(test[0]) = "array" ? [[$result]] : [$result]
)

Using the above logic, the function returns an array of arrays whether the outer array contains a singleton or not.

Function and tester here:
https://try.jsonata.org/ukkhATTKN

[timkindberg]

Cleary there is some hijinx introduced by the recent version? Can we get a confirmation that these changed behaviors are officially considered bugs?

[andrew-coleman]

I have a PR that reverts the behaviour to how it was in v1.7 and added unit tests to ensure it stays that way. However, in order to not break the fix to #399 (which triggered this issue), I've reimplemented the logic such that if the array is explicitly constructed in the expression and marked to promote singleton sequences to arrays (using the [] syntax), then the extra array is added allowing an array of an array to be created.