[BUG] Nested query on wildcard type field will return no result.

[kkewwei]

Describe the bug

In wildcard type fields, the field value is retrieved from DocValues if doc_values is enabled; otherwise, it falls back to the stored field. As of the current implementation (refer to WildcardFieldMapper.java#L325), the default value for doc_values in wildcard fields is false (see 46a0045), which appears to be an unintended error.

For nested fields, sub-documents do not build stored indexes and lack doc_values entirely. Consequently, wildcard queries within nested fields will fail to retrieve values from sub-documents.

Related component

Search:Relevance

To Reproduce

PUT wildcard_index1
{
   "mappings": {
      "properties": {
         "outter_filed": {
            "type": "nested",
            "properties": {
               "wildcard1": {
                   "type": "wildcard"
               }
            }
         }
      }
   }
}
PUT wildcard_index1/_doc/23
{
   "outter_filed": {
      "wildcard1": "abcd"
   }
}

POST wildcard_index1/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "nested": {
            "path": "outter_filed",
            "query": {
              "wildcard": {
                "outter_filed.wildcard1": "*bc*"
              }
            }
          }
        }
      ]
    }
  }
}

And we will return no result.

Expected behavior

• Temporary Mitigation:

Explicitly set doc_values: true when defining wildcard fields to ensure value retrieval from DocValues. This bypasses the default false configuration and enables nested queries to access sub-document values.

• Permanent Fix:

When doc_values is disabled (false), the nested sub-document field will automatically build a stored index to retain retrievable values.

Additional Details

brcnch: main

[getsaurabh02]

Thanks for opening this issue and for the detailed explanation @kkewwei . This is a valid request, especially for dashboards and integrations that expect flattened fields. One approach could be introducing an optional flag in the search response (or a new API param) to return flattened paths for nested objects. Would love to hear more about your use case.

Will you be open to contributing a design proposal or PR on one?

[sandeshkr419]

[Search Triage] Going forward we should make the docValue:true for wildcard type field.

We'd have to take care of backward compatibility maybe - reindexing should fix it though.

[kkewwei]

We have no way of verifying whether a field is a sub-document, nor can we guarantee that the stored field must exist.

I'm wondering if we should disable the docValue setting to ensure it remains consistently true. If the docValue parameter is exposed, users might inadvertently set it to false, which would also result in no result.