✨ Proposal: Reference W3C string-meta for language and direction metadata on strings

[xfq]

Describe the inspiration for your proposal

JSON Schema provides annotation keywords (title, description, etc.), but offers no guidance on how to:

1. Determine or declare the language of a natural-language string value in a JSON instance.
2. Determine or declare the text direction (LTR, RTL, or auto) of a natural-language string value.
3. Establish resource-wide defaults for language or direction that apply to multiple string fields.

Schema authors who need to represent natural language content must invent their own conventions with no standard reference to follow, leading to inconsistent interoperability.

Describe the proposal

This proposal suggests adding the W3C Note Strings on the Web: Language and Direction Metadata ("string-meta") as an informative reference to the JSON Schema spec. string-meta provides best practices for identifying the language and text direction of natural-language strings.

string-meta defines a concrete, widely-applicable serialization pattern for natural language text fields:

"field-name": {
    "value": "The actual string",
    "lang": "en-US",
    "dir": "ltr"
}

This pattern uses existing JSON object structure. No new keywords or validation semantics are required. And it works alongside JSON Schema's existing annotation mechanism.

string-meta is a published W3C Group Note from W3C (and is transitioning to a normative document), with editors who have deep expertise in this domain.

The reference could be placed in the annotation keywords section (title, description) noting that for schemas with title or description values, string-meta provides guidance on language/direction metadata.

(If the JSON Schema develops an i18n-focused vocabulary in the future, string-meta could become a normative reference at that point.)

Describe alternatives you've considered

No response

Additional context

• string-meta Editor's Draft
• string-meta W3C Group Note
• W3C Internationalization Working Group

[jdesrosiers]

This is interesting, but I'm not sure what the proposal is exactly. I think what you're proposing is to allow keywords like title and description to take a string-meta object instead of just a string. I don't think we want to make backward incompatible change, but allowing a string-meta object as well as a plain string could be an option.

"description": {
  "value": "...",
  "lang": "en-US",
  "dir": "ltr"
}

Having a keyword with two different forms isn't ideal.

I feel like were talking about an annotation on an annotation. For example, a "lang" annotation on the "description" keyword. We don't have anything like that in JSON Schema right now. For better compatibility, we could introduce something like,

"description": "...",
"description-lang": "en-US",
"description-dir": "ltr"
"title": "...",
"title-lang": "en-US",
"title-dir": "ltr"

This would fit the JSON Schema paradigm best and have good compatibility and flexibility, but it would require duplicating keywords (description-lang and title-lang instead of just lang).

We could just have one annotation, but it would apply to all values.

"description": "...",
"title": "...",
"lang": "en-US",
"dir": "ltr"

Of course, it's possible to break it up, but it would be cumbersome.

"allOf": [
  {
    "description": "...",
    "lang": "en-US",
    "dir": "ltr"
  },
  {
    "title": "...",
    "lang": "fr-FR",
    "dir": "ltr"
  }
]

And then, what if we want an annotation that applies to the value instead of the annotation? Then we would need two similar annotation keywords.

"type": "string",
"title": "...",
"description": "...",
"value-lang": "en-US",
"annotation-lang": "fr-FR"

I think this is complicated and will need a lot of thought. As you alluded to, I don't think we can do much to solve this problem outside the introduction of a well thought out i18n vocabulary.

[xfq]

Thanks, Jason. I agree that changing the shape of existing JSON Schema annotation keywords such as title and description would be a much bigger problem, and I didn't intend to do that as part of this proposal. My intent for this proposal is narrower:

• Do not change the schema language.
• Do not allow title/description to be either strings or string-meta objects.
• Do not introduce lang, dir, description-lang, value-lang, etc. as JSON Schema keywords.
• Do not try to solve “annotations on annotations” here.

What I’m currently proposing is an informative reference/guidance point for schema authors who need to describe natural-language instance data.

For example, if an API has a localized display name, today people often invent something like:

"name": "مرحبا"

"name": "مرحبا",
"nameLang": "ar",
"nameDirection": "rtl"

"name": {
  "text": "مرحبا",
  "locale": "ar"
}

JSON Schema can validate all of those shapes, but the spec gives schema authors no guidance about which shape is preferable for language and direction metadata. string-meta gives us a well-reviewed reference for that problem.

For example, something like this (my personal thoughts, not yet discussed with the W3C i18n working group):

{
  "type": "object",
  "properties": {
    "displayName": {
      "type": "object",
      "properties": {
        "value": { "type": "string" },
        "lang": { "type": "string" },
        "dir": { "enum": ["ltr", "rtl", "auto"] }
      },
      "required": ["value"]
    }
  }
}

That is just ordinary JSON Schema describing ordinary JSON data. The value of referencing string-meta is that authors and tooling can converge on a common representation instead of every schema inventing its own text/locale, value/language, nameLang, etc. convention.

Another advantage is that many people are unaware of the fact that natural language strings needs lang/dir metadata. Several use cases illustrating this are presented here. Furthermore, while some people believe that language detection alone can resolve all such issues, this is not the case. The string-meta document addressed this point as well. I hope that these best practices will be adopted by a wider audience.

Metadata for JSON Schema’s own annotations is a separate problem. If we want language and direction metadata for title, description, error messages, etc., that probably does require a carefully designed i18n vocabulary. I would consider that as future work and keep it out of scope for this issue.

One concrete next step would be adding a short informative note along these lines:

JSON Schema does not define a built-in mechanism for associating language or base direction metadata with natural-language strings in instance data. Schema authors who need to model such strings are encouraged to consider the patterns described by W3C string-meta.

I hope that would let us improve interoperability while leaving room for a future JSON Schema i18n vocabulary if we want to address schema annotations themselves.

[jdesrosiers]

Ah, I understand now. This doesn't have to do with the title/description meta-data keywords. This would be a schema author best practice recommendation for defining strings that represent natural language. Generally, we've been working to remove things like that from the spec as out of scope and leaving that for documentation on the website or blog posts. Generally, the audience of the spec is implementers of JSON Schema validators and other tools. Then separately we have documentation where the audience is users of the language writing schemas.

We can certainly add something to the documentation (in the website repo) and that has the bonus of not having to wait until a spec release goes out.

If you're interested in contributing a blog post (in the website repo), we could publicize that and get the word out to people about using the string-meta pattern in their schemas.

[xfq]

Okay, thank you. If I want to write a blog post, should I open an issue at https://github.com/json-schema-org/website/issues ?

(BTW, a downside of writing articles in Markdown is that it is not easy to add language and direction metadata ;-)

[jdesrosiers]

If I want to write a blog post, should I open an issue at https://github.com/json-schema-org/website/issues ?

Yes, that's the right place.

[mwadams]

Okay, thank you. If I want to write a blog post, should I open an issue at https://github.com/json-schema-org/website/issues ?

(BTW, a downside of writing articles in Markdown is that it is not easy to add language and direction metadata ;-)

That's why we were given (schematized) JSON frontmatter! 😉