False error oas3-valid-schema-example reference resolves to more than one schema

[DavidBiesack]

For support questions, please use the Stoplight Discord Community. This repository's issues are reserved for feature requests and bug reports. If you are unsure if you are experiencing a bug, our Discord is a great place to start.

Please delete this section, any any sections below that you don't use, before creating the issue.

Describe the bug
Spectral is generating a strange error when a schema has a property with the name id.

To Reproduce

1. Given this OpenAPI/AsyncAPI document

openapi: 3.0.3
info:
  title: Spectral Issue
  description: >-
    error oas3-valid-schema-example reference "bf23bc970b78d27691e8" resolves to more than one schema
  version: 0.1.0
  contact:
    name: Apiture

servers:
  - url: /spectral/issues

tags:
  - name: Spectral Issues
    description: Spectral Issues

paths:

  /path:
    get:
      operationId: getResource
      description: Get a resource
      tags:
        - Spectral Issues
      responses:
        '200':
          description: OK.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/collection"


components:

  schemas:

    item:
      type: object
      properties:
        id:
          description: >-
            The unique identifier for this account resource.
            This is an immutable opaque string.
          readOnly: true
          type: string
          pattern: '^[-_:.~$a-zA-Z0-9]+$'
          minLength: 6
          maxLength: 48
          example: bf23bc970b78d27691e8
        url:
          description: >-
            The URL of this account instance. See the [`getAccount`](#op-getAccount) operation.
          readOnly: true
          maxLength: 256
          type: string
          format: uri
          example: https://api.apiture.com/banking/accounts/bf23bc970b78d27691e8
      example:
        id: bf23bc970b78d27691e8
        url: https://api.apiture.com/banking/accounts/bf23bc970b78d27691e8

    collection:
      required:
        - items
      type: object
      allOf:
        - type: object
          properties:
            items:
              description: The array of items in this page of accounts.
              readOnly: true
              type: array
              items:
                $ref: '#/components/schemas/item'
      example:
        items:
          - id: bf23bc970b78d27691e8
            url: https://api.example.com/spectral/issues/bf23bc970b78d27691e8
          - id: 8d27691e8bf23bc970b7
            url: https://api.example.com/spectral/issues/8d27691e8bf23bc970b7

and this .spectral.yaml
3. Run this CLI command: spectral lint -r ./.spectral.yaml openapi.yaml
4. See error

spectral lint -r ./.spectral.yaml openapi.yaml

./spectral-issues/openapi.yaml
 76:15  error  oas3-valid-schema-example  reference "bf23bc970b78d27691e8" resolves to more than one schema  components.schemas.collection.example

✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints)

Expected behavior

There should be no error here.

Environment (remove any that are not applicable):

• Library version: spectral-cli 6.0.0 (newer versions are not working for me; see running spectral CLI yields ERR_PACKAGE_PATH_NOT_EXPORTED error in @stoplight/spectral-ruleset-bundler/package.json #2080 )
• OS: Mac OS
• Browser: N/A

[DavidBiesack]

The error goes away if I globally change id: to resourceId:, or if I remove either of the the example objects in the collection or item schemas or just the first array item in the example in collection.

[DavidBiesack]

The error seems to be coming from ajv which is validating the examples against the schema; ajv is compiling the schemas from #/components/schemas/* and does not know about the example object that OpenAPI defines, so it is interpreting the id as as schema id. I think spectral needs to remove the example object.

addRef (~/dev/tools/spectral/node_modules/ajv/lib/compile/resolve.ts:112)
<anonymous> (~/dev/tools/spectral/node_modules/ajv/lib/compile/resolve.ts:105)
_traverse (~/dev/tools/spectral/node_modules/json-schema-traverse/index.js:69)
_traverse (~/dev/tools/spectral/node_modules/json-schema-traverse/index.js:75)
_traverse (~/dev/tools/spectral/node_modules/json-schema-traverse/index.js:83)
module.exports (~/dev/tools/spectral/node_modules/json-schema-traverse/index.js:14)
getSchemaRefs (~/dev/tools/spectral/node_modules/ajv/lib/compile/resolve.ts:101)
_addSchema (~/dev/tools/spectral/node_modules/ajv/lib/core.ts:706)
compile (~/dev/tools/spectral/node_modules/ajv/lib/core.ts:377)
<anonymous> (~/dev/tools/spectral/packages/functions/src/schema/ajv.ts:93)
schema (~/dev/tools/spectral/packages/functions/src/schema/index.ts:49)
schema (~/dev/tools/spectral/packages/core/src/ruleset/rulesetFunction.ts:136)
oasSchema (~/dev/tools/spectral/packages/rulesets/src/oas/functions/oasSchema.ts:61)
oasSchema (~/dev/tools/spectral/packages/core/src/ruleset/rulesetFunction.ts:136)
oasExample (~/dev/tools/spectral/packages/rulesets/src/oas/functions/oasExample.ts:153)
oasExample (~/dev/tools/spectral/packages/core/src/ruleset/rulesetFunction.ts:136)
lintNode (~/dev/tools/spectral/packages/core/src/runner/lintNode.ts:30)
cb (~/dev/tools/spectral/packages/core/src/runner/runner.ts:52)
mapped.<computed> (~/dev/tools/spectral/packages/core/src/runner/runner.ts:109)
_callbacks.<computed> (~/dev/tools/spectral/node_modules/nimma/dist/legacy/cjs/runtime/proxy-callbacks.js:31)

Note that example is deprecated in OAS 3.1 in favor of examples in JSON Schema, but it is not yet
deprecated in OAS 3.0.x which the example is using.
I can't switch to OAS 3.1, as other tools we use do not yet support it.

[hudlow]

@DavidBiesack Thanks for the work tracking this down. I ran into it independently in case another test case is useful: #2140

[sudoku-lord]

Is there any update on this issue? It's causing problems for me.

[bwl21]

@DavidBiesack

I dropped in the same issue while using stoplight studio. I found ajv-validator/ajv#1426 which seems to deal with the same thing.

The error seems to be coming from ajv which is validating the examples against the schema; ajv is compiling the schemas from #/components/schemas/* and does not know about the example object that OpenAPI defines, so it is interpreting the id as as schema id. I think spectral needs to remove the example object.

If this is true, then it appears to be a bug i stoplight studio. When validating the examples, only the schema relevant for the example should be provided to AJV but not the openapi schema, and also not all schemas within the openapi instance.

the openapi schema says in https://swagger.io/specification/#example-object that "examples" can be any. So stoplight should not mess up the schema for openapi and the schemas mentioned within an openapi specification.

[venilillkall]

Would be great to get this fixed - we have an API spec that has loads of these, it's super annoying

[DavidBiesack]

a workaround is to use unique id values in all the examples, although this is not always possible.

[venilillkall]

yes, we also managed to work around this as suggested, but just having to do that is plain silly

[padamstx]

Any chance we could get some attention on this issue? My team supports a spectral-based validator along with a ruleset that inherits the spectral:oas ruleset, and we've received reports of this problem with the oas3-valid-schema-example rule, along with reports of a similar issue in the oas3-valid-media-example rule as well.

[padamstx]

There might be some hope for this one yet... for the last couple of days, I've been studying the code related to the oas3-valid-schema-example & oas3-valid-media-example rules and trying to understand what's going on. I have a hunch that the resolves to more than one schema error is occurring during the schema compilation with ajv, not during the actual validation of the example value. Also, @DavidBiesack mentioned above:

I think spectral needs to remove the example object.

So with this in mind, and the fact that I really didn't have any other leads 😂, I modified the oasExample() rule function a bit so that it removes any "example" fields in the schema that it's going to pass into oasSchema() to do the actual schema validation of the example value. Surprisingly, that was enough to avoid the false errors. I figured the oasExample() function was the best place to do this, so that other invocations of oasSchema() are not subject to this example filtering.

I still have more testing to do, but I'll try to get a PR opened up for this soon.

[padamstx]

@DavidBiesack I've opened a PR that fixes this issue in case you'd like to give it a try.