Verbose examples (rules format)

[dan-tripp-siteimprove]

I think that we would benefit from a new place in the rules format to put "verbose examples". These would be examples that don't fit into our current categories (Passed / Failed / Inapplicable), and would not be counted in the tool implementation reports. They would be more obscure, and would be interesting to a smaller group of people. They would be informative nonetheless, for both ACT group members and some members of the general public.

For example, for the "Zoomed text node is not clipped with CSS overflow" rule, consider this case: zoomed text overlaps other text and does not clip. Currently this passes the ACT rule and fails the SC. I gather that overlapping text was left out of the ACT rule for now because it would be difficult to implement. I think that overlapping text would be a useful as an example somewhere, to document the difference between the ACT rule and the SC. We can't add it as a "Passed Example" because if we did, and then a keen vendor came up with an implementation that flags overlapping text and therefore matches the SC more closely, that vendor would be punished in their implementation report as having a false positive. So a new "verbose examples" section would be the appropriate home for it.

Currently there is a lot of discussion of obscure cases which lives only in our github comments and meeting minutes. This makes it difficult for us when we want to understand a decision a year or two after it was made. A new "verbose examples" section could make that easier for us.

Other names for "verbose examples" could be: "illustrative examples" or "boundary cases".

[dan-tripp-siteimprove]

Tagging @kengdoj because of our discussion of the examples for 'label in name'.

[kengdoj]

The Editor's draft ACT Rules Format 1.1 has a Background section that includes Other Resources. Would "Other Resources" be a good place to include a link to the verbose examples?

On another thought:

For example, for the "Zoomed text node is not clipped with CSS overflow" rule, consider this case: zoomed text overlaps other text and does not clip. Currently this passes the ACT rule and fails the SC. I gather that overlapping text was left out of the ACT rule for now because it would be difficult to implement. I think that overlapping text would be a useful as an example somewhere, to document the difference between the ACT rule and the SC.

Sould these be two atomic rules?

• Zoomed text node is not clipped with CSS
• Zoomed text does not overlap other text

(so the overlap example doesn't need to be included in the clipped rule)

[dan-tripp-siteimprove]

Would "Other Resources" be a good place to include a link to the verbose examples?

It could suit, but it might have some downsides. I don't know how difficult it is to add a new section to the rules format wherever we want. If it's difficult, then "Other Resources" would be a good home for "verbose examples" - at least for now. If it's not too difficult, then I'd like to set my hopes on something more purpose-built. How much hope should I have?

[dan-tripp-siteimprove]

... two atomic rules?

I'm out of my element on that one.

[Jym77]

Other cases of "boundary examples" would typically be for Image has non-empty accessible name (or similar rules) to have an example with a non-descriptive name (img src="/test-assets/shared/w3c-logo.png" alt="foo" />), showing that the rule does not care about what the name says. Such a case passes the rule but fails the SCs. Thus, it cannot be a Passed Example (as per our own design guidelines, not the ACT rules format), yet it brings value to both tool vendors and rules readers in understanding the limits of the rule (one goal of examples described in the format) and how the rule is a partial test (one of he key points highlighted in Understanding test rules.

A recent case of this creating confusion is #2145 or the discussion at https://github.com/act-rules/act-rules.github.io/pull/2101/files#r1339831735

I agree that the Background section might be a good place for these (without changing the format), especially since it wouldn't require any website generation code to change. But at the same time we don't want the size of this section to "explode" due to many examples… Just having a link to another page might be too "hidden" (and need some code update to publish these new pages).

An intermediate way could be to hide them in a <summary> element (which can be done directly in the markdown, or through some code automation like we did for the test assets). Then they might be either inside Background, or as a new subsection after the Inapplicable examples 🤔 (I think it fits the format as the format requires Passed, Failed, and Inapplicable example(s) but doesn't forbid other kind, plus these are technically Passed example…)

[dan-tripp-siteimprove]

I like the idea of a <summary> element. I don't want the size of this new section to explode either. But I think that it should be allowed to be "not very succinct".

[kengdoj]

I don't know how difficult it is to add a new section to the rules format wherever we want.

We're in the process of updating the Rules Format so if you want a new section, now is a good time to suggest it. Where would you want it? and what type of info should it contain? Why isn't Other Resources a good place for it?

[dan-tripp-siteimprove]

Where would you want it?

I think that the ideal home for "verbose examples" is somewhere very near to the existing Passed / Failed / Inapplicable examples. If, on the other hand, we put "verbose examples" under "Other Resources", then there would be some sections (such as "Accessibility Requirements Mapping" and "Input Aspects") between "verbose examples" and Passed / Failed / Inapplicable examples. I see that as a downside (I'll call it downside 1): to put these similar sections apart from each other would sub-optimal for readability.

Also, (and I'll call this downside 2), if we put "verbose examples" in "Other Resources", then they will appear higher up on the page than the Passed / Failed / Inapplicable examples. That seems wrong to me, because they're less important. So "verbose examples" should appear after Passed / Failed / Inapplicable examples, not before.

So I propose that "verbose examples" be put immediately after the Passed / Failed / Inapplicable examples, and before the next section ("Glossary").

Why isn't Other Resources a good place for it?

Also, I'll call this downside 3: I predict confusion due to the fact that the 'Other Resources' section looks like it's meant for links to "outside" resources: outside of the ACT rule in question, and maybe outside of ACT as a whole. "Verbose examples" are not that: they're part of the ACT rule in question.

what type of info should it contain?

Here's an example for "Image has non-empty accessible name":

---

Verbose Example 1

This image has a non-descriptive name. It passes the rule and fails the Success Criterion, so it cannot be a Passed Example (as per ACT design guidelines). This example shows how this rule is a partial test for the Success Criterion.

<img src="logo.png" alt="image" />

---

I am not committed to the name "verbose examples". Jean-Yves happened to use "boundary examples" above - I think that's just as good.

[daniel-montalvo]

Thanks @dan-tripp-siteimprove @Jym77 for bringing this up.

Can we get a pointer to the CG meeting where this was discussed?

[Jym77]

It hasn't been discussed yet 🙂
I think I'll make a draft PR at some point to have something concrete to talk about (I don't think it requires a change in the format).

[kengdoj]

#1458 has many examples for label in name that were discussed in a CG meeting last year(?). Sorry I can't recall a date.

[dan-tripp-siteimprove]

We discussed this in the meeting today. Minutes are here.