Regular expression pattern specification and testing clarifications

[zx80]

The JSON Schema specifications describes valid regular expression patterns in Section 6.4 by accumulating recommandations:

• they SHOULD be valid for ECMA-262 with unicode enabled
• they MUST NOT be assumed as anchored
• they SHOULD conform to a subset of the regex syntax for portability (schema authors SHOULD limit themselves to the following regular expression tokens), which is:
  • individual Unicode characters eg Caractères accentués…
  • simple and character classes [abc] [a-z]
  • complemented simple character classes [^abc]
  • complemented range character classes [^a-z]
  • repetitions + * ? {x} {x,y} {x,} and their lazy …? counterparts
  • anchors ^ $, grouping (...) and alternation |

Here are some questions:

1. I understand these SHOULD as a validator is not expected to implement full ECMA-262, but only the prescribed subset, which seems reasonable enough if the spec is expected to be portable across languages. Yes?

2. The subset description seems a little fuzzy and incomplete. It could be clarified wrt the following points:

   • how are character escaped, or not, in character classes? should it assume POSIX, i.e. no escape with clever positioning? Accept \ on some characters? On all characters? How to include characters ] - ^ \ in a class?
   • can a character class be empty? Can a complemented class be empty? (I would suggest no in both cases).
   • what about wildcard .? I guess it is allowed? if it is allowed, does it reject \n? and \r? or should it assume single-line mode?
   • what about predefined character classes: \s \w \d \D \W \D, are they supported? also inside classes?
   • what about advanced character classes: \p{Letter}, I guess no? If yes, which are expected to be supported?

3. As far as the test suite is concerned, ISTM that the test suite should be limited to cases covered by the subset, which is currently not the case (it uses . and \{Letter}).

[jdesrosiers]

1. I understand these SHOULD as a validator is not expected to implement full ECMA-262, but only the prescribed subset, which seems reasonable enough if the spec is expected to be portable across languages. Yes?

The idea is that it's not reasonable to expect an implementation to implement an ECMA-262 regular expression engine if they don't have one available to them. For practical reasons, most implementations are going to use whatever they have available to them in the language they're implementing in.

The subset is a recommendation for schema authors, not a requirement for implementations. Implementers shouldn't worry about the subset. Just use what you have available that's as close to ECMA-262 RegExp in unicode mode as possible.

2. The subset description seems a little fuzzy and incomplete. It could be clarified wrt the following points:

Honestly, I think the subset should be removed from the spec. It's a moving target and we don't want to be in the business of keeping up with the evolution of every regex flavor and maintaining something like this. Also, it's a requirement on schema authors, not implementations which doesn't belong in a specification.

3. As far as the test suite is concerned, ISTM that the test suite should be limited to cases covered by the subset, which is currently not the case (it uses . and \{Letter}).

The goal is to limit tests to a subset that are as close to universally supportable in every language as possible. As far as I know, every major programming language has regex engine (either standard library or well supported third-party library) that supports both of those constructs. I think both are fair to use in tests. Keep in mind that the subset is just a recommendation for schema authors, not what implementations are expected to support or not support.

[zx80]

Thanks for the feedback.

1. I understand these SHOULD as a validator is not expected to implement full ECMA-262, but only the prescribed subset, which seems reasonable enough if the spec is expected to be portable across languages. Yes?

The idea is that it's not reasonable to expect an implementation to implement an ECMA-262 regular expression engine if they don't have one available to them.
For practical reasons, most implementations are going to use whatever they have available to them in the language they're implementing in.

Ok.

The subset is a recommendation for schema authors, not a requirement for implementations. Implementers shouldn't worry about the subset. Just use what you have available that's as close to ECMA-262 RegExp in unicode mode as possible.

Indeed, the subset is stated as being for schema authors.

However, deriving from that subset means that the schema is potentially less portable, so it makes sense to take it into account when implementing say a linter, and possibly issue warnings from an actual validator implementation. So I'm not at ease with dismissing it entirely from the implementation point of view.

2. The subset description seems a little fuzzy and incomplete. It could be clarified wrt the following points:

Honestly, I think the subset should be removed from the spec. It's a moving target and we don't want to be in the business of keeping up with the evolution of every regex flavor and maintaining something like this. Also, it's a requirement on schema authors, not implementations which doesn't belong in a specification.

I respectfully disagree: ISTM that ECMA-262 embodies javascript-only portability (aka non portability), whereas a subset aims at wide portability and is likely not to need evolutions beyond fixing ambiguities, so it seems to me that this subset does belong to a spec as it avoids an implementation dependent feature.

3. As far as the test suite is concerned, ISTM that the test suite should be limited to cases covered by the subset, which is currently not the case (it uses . and \{Letter}).

The goal is to limit tests to a subset that are as close to universally supportable in every language as possible.

Ok.

As far as I know, every major programming language has regex engine (either standard library or well supported third-party library) that supports both of those constructs. I think both are fair to use in tests. Keep in mind that the subset is just a recommendation for schema authors, not what implementations are expected to support or not support.

Ok.

However, I noticed that a trivial question such as whether . covers \r is not a universal agreement in main stream regex implementations, so the ambiguity should be removed on that point, possibly by requiring single line mode together with unicode mode. The blaze implementation makes this assumption, for instance, and it can help significantly validation performance.

Also, how to handle escapes inside a character class can change significantly between implementations, with POSIX clever positioning seeming the safest bet for achieving compatibility across implementations. Same thing goes with empty classes and complementary classes.

My opinion after your reply:

• I'm fine with keeping the tests as they are.
• I think that the regex subset should be kept in the spec, and even possibly emphasized with a MUST to make it a minimal implementation requirement.
• If it is kept, it makes sense to remove potential ambiguities.

[jdesrosiers]

it makes sense to take it into account when implementing say a linter, and possibly issue warnings from an actual validator implementation.

Good point. That would be a great thing for a linter to do, but still don't think it's practical for JSON Schema to define an official subset. I'd expect the linter implementation to do the research to determine which features are not likely to be universally supported.

I respectfully disagree: ISTM that ECMA-262 embodies javascript-only portability (aka non portability), whereas a subset aims at wide portability and is likely not to need evolutions beyond fixing ambiguities, so it seems to me that this subset does belong to a spec as it avoids an implementation dependent feature.

I completely understand your point of view. Having implementation dependent features isn't ideal, but until everyone agrees to one flavor of regular expressions, I don't think there's a practical solution here.

I noticed that a trivial question such as whether . covers \r is not a universal agreement in main stream regex implementations, so the ambiguity should be removed on that point, possibly by requiring single line mode together with unicode mode.

That's a great observation and a great suggestion! Can you make an issue on the spec repo suggesting that change. Any other suggestions that could reduce ambiguity would be welcome as well.

I think that the regex subset should be kept in the spec, and even possibly emphasized with a MUST to make it a minimal implementation requirement.

I'd be ok with it being a MUST as long as it's clear that it's a minimum requirement and that someone does the work to make sure that the subset is reasonably universal.

[zx80]

That's a great observation and a great suggestion! Can you make an issue on the spec repo suggesting that change. Any other suggestions that could reduce ambiguity would be welcome as well.

I'll try to look into that.

[RishiByte]

Hi @jdesrosiers and @zx80,

This is a really interesting discussion! I'm looking to dive deeper into contributing to the JSON Schema Test Suite and would love to help tackle the cross-language regex research mentioned here.

Since identifying a reasonably universal subset is a blocker, I can volunteer to put together a cross-language testing matrix. I can write some scripts (starting with Python and JavaScript/V8) to evaluate the specific edge cases discussed above, such as:

How . handles \r (and the impact of single-line vs. unicode modes).

Behavior around character escapes (e.g., ], -, ^) inside character classes.

Support for empty classes [] and complemented empty classes [^].

Once I have those findings, I can compile an initial list of non-portable regex features to help lay the groundwork for the linter compatibility warnings you mentioned.

Does this sound like a helpful next step? Let me know if you have a preferred format for how I should present the matrix!