Validator tests are written for "author MUST" or "author MUST NOT" statements. An "author MUST" statement is testable if an HTML validator (such as AXE or validator.nu can test whether a given piece of HTML conforms to the specification. Not all "author must" statements are testable. For example, the following statement is testable:
In order for elements with a role of img to be perceivable, authors MUST provide a label using the aria-label or aria-labelledby attribute.
But the following statement is not testable, because managing focus is dynamic and cannot be done by an html validator:
Authors MUST manage focus on the following container roles...
The validator tests belong in the validator-test directory. When adding or updating a test, also supply the test results required in the README.
Ideally, we should have a test suite to test all "user agent MUST" statements, but we do not have the tooling to write all these tests.
If our change adds or changes a "user agent must" or "user agent must not" statement, please add a issue describing the test one the PR is ready for merge.
Automated tests for the ARIA IDL interface are in WPT. We have two test files:
When the IDL Interface section is updated, these tests should be update accordingly.
The tests are located in WPT's core-aam folder. They are "manual" tests in that you have to use a tool outside of the web browser and WPT test suite to inspect the accessibility API. For now, Valerie will maintain these CORE-AAM tests and no one else needs to write them.
We do not yet have tests for HTML-AAM.
We do not yet have infrastructure for AccName tests. Better AccName tests are being investigated in this issue: w3c/accname#174
When we have a new test framework for AccName test, we will create an initial set of tests and all following PRs to AccName will have a testing requirement.