AT and Browser Version management Mockups

[isaacdurazo]

AT and Browser Version management Text-Based and Visual Mockups

Introduction

This issue presents a series of mockups for how a tester can provide AT and Browser version details when starting a new Test Run and for Admins to add new AT versions to the ARIA AT App. The changes presented here take into consideration several different scenarios for each type of user. There might be edge cases that are not considered yet. Please make a comment on this issue if you identify one that's not being taken into account here.

Assistive Technology and Browser Details Modal

When a Tester or an Admin opens a Test Plan from the Test Queue, they will be prompted with a modal before landing on the Test Run page. This modal has different variations based on the different scenarios a Tester or an Admin can run into. I will first explain the general UI characteristics of this modal and will expand on the specifics of the different scenarios for each user persona afterward.

General UI Characteristics and details

• Heading: This reads: Assistive Technology and Browser Details

• Description: This reads: Before continuing, please provide details about the Assistive Technology and Browser versions you will be testing with.

• Assistive Technology Details: This area has two columns with a label and a select element each. One column is dedicated to the AT Name, which can't be modified and the other column is dedicated to the AT Version, which can be modified if needed by changing the dropdown selection.

• Browser Details: This area has two columns with a label and a select element each. One column is dedicated to the Browser Name, which can't be modified and the other column is dedicated to the Browser Version, which can be modified if needed by changing the dropdown selection.

• Actions: This area contains the buttons for the Modal. The buttons are "Cancel" and a "Save and Continue"

• Alerts: Depending on the scenario, different kinds of alerts can be triggered in different areas within the modal.

Tester Scenarios

Scenario 1 - A Tester Opens a Test Plan for the first time

When a Tester opens a Test Plan for the first time, they must choose an AT and a Browser Version before being able to continue. The variations in this scenario are:

• Assistive Technology Details: The AT Name will be selected by default because this information comes from the Test Queue; this selection can't be modified. The tester will have to manually select a version for the AT to be able to continue.

• Browser Details: The Browser Name will be selected by default because this information comes from the Test Queue; this selection can't be modified. The Browser version is automatically detected and selected in the dropdown.

• Alerts: An informational Alert in light blue color is displayed above the Browser details that reads: We have automatically detected your version of [Browser Name]

---

Scenario 2 - A Tester returns to an existing Test Run session

When a Tester comes back to continue testing and nothing has changed in their AT or Browser details. In this scenario, the AT and Browser versions remain the same assuming no new Browser version is being detected. There are no variations in this scenario and in a way is the "cleanest" scenario:

---

Scenario 3 - A Tester returns to an existing Test Run and their Browser version is different

When a Tester comes back to continue testing and their Browser version changes, the system will automatically detect it and automatically update it in the modal. The variations in this scenario are:

• Browser Details: The Browser Name will be selected by default because this information comes from the Test Queue; this selection can't be modified. The Browser version gets automatically updated and can be manually modified by the tester if needed.

• Alert: A Warning Alert in light yellow color is displayed above the Browser details that reads: We have automatically detected you are using a different version of [Browser Name] and we have updated it below. The previous version you were using was [Version Number]. This change doesn’t affect results that have already been submitted for this plan. However, results you submit during this session will be recorded with the versions specified in this form.

---

Scenario 4 - A Tester returns to an existing Test Run and their Browser is different

When a Tester comes back to continue testing and their Browser is different than what they were previously using, the system will automatically detect the change and display a warning alert within the Modal. It is important to consider that the system can't automatically update the modal for this scenario, because the Browser type is non-editable. The variations in this scenario are:

• Browser Details: The Browser Name will be selected by default because this information comes from the Test Queue; this selection can't be modified. The Browser version remains the same even though the Tester is trying to use a different browser.

• Alert: A Warning Alert in light yellow color is displayed above the Browser details that reads: We have automatically detected you are now using [X Browser Name], which is a different browser from the last one you were testing with, which was [Y Browser Name]. You can’t edit your Browser type but can continue with [X Browser Name]. Keep in mind that your test results will be recorded as if you were still using [Y Browser Name].

---

Scenario 5 - A Tester returns to an existing Test Run and selects a Browser version that's different from the one that is being automatically detected.

When a Tester comes back to continue testing and tries to select a Browser version that's different from the one the system is automatically detecting, they will be prompted with a warning alert within the Modal. The variations in this scenario are:

• Browser Details: The Browser Name will be selected by default because this information comes from the Test Queue; this selection can't be modified. In this scenario, the Browser Version has been changed by the tester, which triggers an alert because it's different than the one the system is automatically detecting.

• Alert: A Warning Alert in light yellow color is displayed above the Browser details that reads: The version of [Browser Name] you have selected is different from the one we have automatically detected, which is [Browser Version]. This change doesn’t affect results that have already been submitted for this plan. However, results you submit during this session will be recorded with the versions specified in this form.

---

Scenario 6 - A Tester returns to an existing Test Run and selects an AT version that's different from the one previously selected.

When a Tester comes back to continue testing and tries to select an AT version that's different from the one they previously selected, they will be prompted with a warning alert within the Modal. The variations in this scenario are:

• Assistive Technology Details: The AT Name will be selected by default because this information comes from the Test Queue; this selection can't be modified. In this scenario, the AT Version has been changed by the tester, which triggers an Alert

• Alert: A Warning Alert in light yellow color is displayed above the AT details that reads: The version of [AT Name] you have selected is different from the one previously selected, which was [AT Version]. This change doesn’t affect results that have already been submitted for this plan. However, results you submit during this session will be recorded with the versions specified in this form.

---

Scenario 7 - A Tester's browser version cannot be automatically detected

When the Browser version cannot be automatically detected an input field is automatically added to the modal for the tester to add the version manually. The variations in this scenario are:

• Browser Details: An input field with the label "Enter Browser Version" is automatically added underneath the Browser Name and Browser Version row. The Tester will have to manually copy and paste their Browser Version.

• Alert: A Warning Alert in light yellow color is displayed above the Browser details that reads: We could not automatically detect what version of [Browser Name] you are using. Before continuing please provide your version number.

---

Scenario 8 - A Tester tries to edit their Browser and AT Details from the Test Run Page

When a Tester clicks the Edit button in the AT/Browser details component, the Assistive Technology and Browser Details Modal is displayed. Assuming their Browser Version hasn't changed, the content of the Modal should look like the one in Scenario 2, if the version has changed, then the modal would look like the one in Scenario 1.

Admin Scenarios

Scenario 1 - Opening a Test Plan as someone else

When an Admin opens a Test plan as someone else, it is likely that their Browser version could be different than the one the tester was using. There are two warning alerts that get displayed in different areas of the Modal and in this scenario, the Browser version is not automatically updated but it is called out. The variations in this scenario are:

• Alert 1: A Warning Alert is displayed at the top of the modal, underneath the Heading that reads: You are about to review [Test Plan Name] as [Tester Username]. Your Assistive Technology and Browser might be different. Please proceed with caution.

• Alert 2: Assuming the Admin's Browser version is in fact different, a Warning Alert is displayed above the Browser details area that reads: We have automatically detected you are using [Browser Name] [Browser Version]. This version is different than what [Tester Username] was using last time, which was [Browser Version].

  We have not updated the information below. Please proceed with caution.

---

Scenario 2 - Edit AT and Browser Details with a different AT Version - Warning Modal

When an Admin clicks the Edit button in the Test Run page while looking at a Test whose results have already been recorded, they could run into the situation where their AT Version is different from the one used to record the results for that test.

In this scenario, a "Warning Modal" gets displayed before the "Assistive Technology and Browser Details Modal". Compared to a regular informational modal, a Warning Modal has a yellow decorative stripe at the top and a yellow warning icon next to the heading. This modal looks like this:

• Heading: this reads: Your AT Version is different than the one used to record this result

• Body content: You are currently running [AT Name] [AT X Version], but are editing a test result that was submitted with [AT Name] [AT Y Version].

  Do you want to update the AT version used to record this test result?

• Actions: this area contains the buttons "Cancel", "Update AT Version" and "Continue without changes"

Note: Clicking the "Update AT Version" will prompt the "Assistive Technology and Browser Details Modal"

---

Scenario 3 - Edit AT and Browser Details with a different Browser Version - Warning Modal

Similar to the previous scenario, when an Admin clicks the Edit button in the Test Run page while looking at a Test whose results have already been recorded, they could run into the situation where their Browser Version is different from the one used to record the results for that test.

In this scenario, a "Warning Modal" gets displayed as well before the "Assistive Technology and Browser Details Modal". This modal has the following variations in content compared to the previous one:

• Heading: this reads: Your Browser version is different than the one used to record this result

• Body content: You are currently using [Browser Name][Browser X Version], but are trying to edit a test result that was submitted with [Browser Name][Browser Y Version].

  Do you want to update the Browser version used to record this test result?

• Actions: this area contains the buttons "Cancel", "Update Browser Version" and "Continue without changes"

---

Scenario 4 - Edit AT and Browser Details with a different Browser - Warning Modal

In the last Admin scenario, we should consider when an Admin clicks the Edit button in the Test Run page while looking at a Test whose results have already been recorded and their Browser is different.

In this scenario, a "Warning Modal" is displayed before the "Assistive Technology and Browser Details Modal". This modal has the following variations in content compared to the previous one:

• Heading: this reads: Your Browser is different than the one used to record these test results

• Body content: You are currently using [Browser X Name][Browser Version], but are trying to edit a test result that was submitted with [Browser Y Name][Browser Version].
  You can’t change the Browser type but can make other changes. Please proceed with caution.

• Actions: this area contains the buttons "Cancel" and "Continue"

---

Test Run Page

This mockup introduces a very small change to the Test Run page and the component that displays the information about the AT and the Browser. This change allows for longer version numbers and an Edit button.

Underneath the H1 we have three containers with information about the Test Plan name, AT and Browser details, and the number of tests completed. Instead of having the AT and Browser info in one single line of text, this change proposes having each be in its own line, which improves readability for sighted users when having long version numbers. The content of this component is left aligned and has an "Edit" icon in the upper right corner where the user can open the "Assistive Technology and Browser Details Modal" to make any edits.

So instead of having:

<div>AT and Browser: NVDA 1 with Firefox 1</div>

We would have:

<div>AT: NVDA 1</div>
<div>Browser Firefox 1</div>

---

Test Reports

Historically, we have displayed the AT and Browser versions being used for a given Test Plan in all headings across the Test Reports flow. With this new working mode, we can no longer display the version information there due to the fact that a single test within a Test Plan can differ from another one in its Browser version.

In this new mockup proposal, the version numbers for the AT and Browser are removed from the headings and included as a list in a disclosure component underneath the reports table, as well as who was the tester and when it was executed. This disclosure has the label "Tun History" and the items in this list would look like this:

• Tested with JAWS 2022.2202.38 And Chrome v98.0.4758.109 by @mcking65 on March 31, 2022.
• Tested with JAWS 2022.2202.38 And Chrome v98.0.4758.109 by @jongund on March 28, 2022.

---

Test Queue - Admin View

Now that the tester is the one providing details about the AT and Browser versions they will be using for testing, there are some changes we can make to the Test Queue to simplify the Admin experience while accommodating a new feature for them to add new AT versions to make them available through the app.

At the top of the Test Queue, we currently have a button that reads "Add a Test Plan to the Queue", which opens a modal for the Admin to provide details about the Test Plan, AT, Browser, and versions for all of these. Two elements are no longer needed here, which are the AT and Browser version fields, and a new one needs to be added to be able to introduce new AT versions to the system.

This mockup proposes a different approach than the button that opens a dialog. Instead of the button, the idea is to have two disclosure components, similar to the one used for a website's FAQs, in the same area (above the Queue).

The first disclosure component reads: Manage Assistive Technology Versions; the second one: Add Test Plans to the Test Queue. These are the contents of each one when they are expanded

Manage Assistive Technology Versions

• Description: This reads: Select an Assistive Technology and manage its versions in the ARIA AT App

• AT Name and AT Versions: There are two columns, each with a label and a select element. The first one is for the AT Name and the second one for Available Versions.

  When the user selects a particular AT, the dropdown that displays the versions automatically updates itself to show only versions for the specific AT selected. Underneath the AT Version dropdown there are three links: Add a New Version, Edit, and Remove. Each of these opens a modal that will be described in detail later in this issue.

Add Test Plans to the Test Queue

• Description: Select a Test Plan and version and an Assistive Technology and Browser to add it to the Test Queue.

• Test Plan, Test Plan Version, AT, and Browser: There are four columns, each with a label and a select element. The first one is for the Test Plan Name, the second one for the Test Plan Version, the third one is for AT Name and the fourth one is for the Browser name.

  Once de user has selected an option in every column, there's a button below that gets enabled, which reads: Add Test Plan to Test Queue.

Managing Assistive Technology Versions Modals

When managing AT versions, an Admin can run into five different scenarios depending on the action they are trying to take. Based on this the contents and style might be different. Here the details

Scenario 1 - Adding a New AT Version Modal

When the Admin clicks the "Add a New Version" button to add a new version for an AT. This modal looks like this:

• Heading: this reads: Add a New Version for NVDA

• Body content: This area contains two rows with a label and an input field each. The first row has a label that reads "Version number" and the Admin should manually paste the AT Version in the input field. The second row has a label that reads "Approximate date of availability" and the admin should select an approximate release date from a date picker.

• Actions: this area contains the buttons "Cancel" and "Add Version"

---

Scenario 2 - Edit an Existing AT Version Modal

When the Admin clicks the "Edit" button to modify an AT version. This modal looks like this:

• Heading: this reads: Edit [AT Name] [AT Version]

• Body content: This area contains two rows with a label and an input field. The first row has a label that reads "Version number" and the input is automatically populated with the version that was selected to be edited. The second row has a label that reads "Approximate date of availability" and the date picker is automatically populated with the date that was previously selected. The Admin must modify the contents of the input field and the date picker to make the necessary edits.

• Actions: this area contains the buttons "Cancel" and "Save"

---

Scenario 3 - Removing an Existing AT Version - Danger Modal

When the Admin clicks the "Remove" button to delete an AT version from the system a Danger Modal is displayed. Compared to a regular informational modal, a Danger Modal has a red decorative stripe at the top, a red warning icon next to the heading, and the main call to action in red. This modal looks like this:

• Heading: this reads: Remove [AT Name] [AT Version]

• Body content: This reads: You are about to remove [AT Name] [AT Version] from the ARIA AT App

• Actions: this area contains the buttons "Cancel" and "Remove"

---

Scenario 4 - Adding an Existing AT Version

When the Admin tries to add an AT Version that is already available in the ARIA AT App. This modal looks like this:

• Heading: this reads: Existing Assistive Technology Version

• Body content: [AT Name] [AT Version] already exists in the system. Please try adding a different version.

• Actions: this area contains the button "Ok"

---

Scenario 5 - Removing an AT Version already in use - Warning Modal

When the Admin tries to remove an AT Version that is already being used by a Tester in a Test run, a Warning Modal is displayed. Compared to a regular informational modal, a Warning Modal has a yellow decorative stripe at the top and a yellow warning icon next to the heading. This modal looks like this:

• Heading: this reads: Assistive Technology Version already being used

• Body content: [AT Name] [AT Version] can’t be removed because is already being used to test the [Test Plan Name]Test Plan.

• Actions: this area contains the button "Ok"

cc @jscholes and @SamuelHShaw

[richnoah]

@mcking65

[isaacdurazo]

One thing that was brought to my attention in these Mockups that hasn't been properly captured, is that there's a case where we can have more than one tester for a given test within a Test Plan.

In the current mockups, when you look at how the details about AT/Browser and versions are represented, it doesn't take into consideration who tested it and what happens if there is more than one tester.

I will be updating these mockups later this week to display that information.

[s3ththompson]

One issue that we realized we haven't accounted for is ordering AT versions.

The system needs to understand whether a given AT version comes before or after another. Right now, these mockups only specify a human-readable string for the AT version. Since the system doesn't understand the version format within those strings, we can't rely on them for sorting.

We can think of two possible solutions:

1. The order of the AT versions is derived from their position in a list. We can't guarantee that admins will add versions in order, therefore we would need to support reordering via the UI. Unfortunately, this approach would greatly complicate the interface for adding and editing versions.
2. In addition to the human-readable string, admins would also specify a release date when adding a new AT version, via a date-picker. The system would order the AT versions by sorting their respective release dates, chronologically.

@mcking65 @jscholes do you have a perspective on which of these two approaches would be best, from an admin perspective?

[jscholes]

@s3ththompson Despite the app not having knowledge of the exact meaning behind an AT version, all of the ATs currently being targeted by the project use numeric values. For example, NVDA 2022.1, JAWS 2022.2202.38, etc. We'll need to look into the exact versioning scheme used by VoiceOver, but macOS has a version and build number that are also numeric.

Given all of this, I'd like to propose investigating a simpler third option: have the app do a string sort, respecting natural number processing.

[jscholes]

@s3ththompson One additional thing does come to mind: pre-release versions. NVDA, for instance, has alpha builds, betas, and release candidates. Some thoughts on those:

1. I don't think alpha builds are currently considered to be in scope. If they are, in the case of NVDA at least, the nightly snapshots use a Git commit hash, which is not comparable with numeric version numbers. But, if alpha builds won't be added to the system, it isn't a problem that needs to be solved.
2. For betas and RCs, we know that they should be sorted before the corresponding stable build, with betas before RCs. But a string sort will not. So, a few ideas:
   • Have a dropdown in the UI, when adding a version, that allows the admin to choose between "Beta", "Release Candidate", and "Stable". Take that metadata into account when sorting the strings. Simpler than a date field.
   • Alternatively, agree on a format for version identifiers, e.g. "Beta 1" will always go after the numeric version number. Then, have the string sort check if there is anything after the number. If there is, it's a beta or RC, so should be sorted first. As the strings would end up being sub-sorted anyway, "RC" would end up coming after "Beta", "Beta 2" would come after "Beta 1", etc., with no additional effort.

As some additional context, I'm suggesting this over dates not only because of interface complexity and the extra research implications, but also because those dates may not always be that easy to come by. For example, somebody may wish to test with a private beta of macOS, but a test admin may struggle to track down the exact release date if they don't have private beta access. Or, historic dates for previous versions may be harder to track down than more current ones. And so on.

[jscholes]

CC @sinabahram and @IsaDC on the PAC side.

[mcking65]

The alerts for when a different browser version is detected or a different AT version is specified includes the copy:

This change doesn’t impact your Test Run. Each test within a test plan is associated with an AT and Browser version.

That seems a little confusing to me. If it doesn't impact anything, the user may wonder why we are telling them about it. In fact, there is some impact.

Would the following copy be correct?

This change doesn’t affect results that have already been submitted for this plan. However, results you submit during this session will be recorded with the versions specified in this form.

[mcking65]

WRT ordering AT versions, I believe any approach that relies on version string sorting is going to be frustrating over time. There will be all kinds of situations that could lead to bizarre string machinations. We need a simple way of controlling sequence that is easy to understand, makes sequence modification easy, and is simple to implement.

I think dates meet all these criteria. To address the hesitation @jscholes had, we do not have to label the field release date. We can label it "Approximate date of availability".

[mcking65]

The test reports section says:

In this new mockup proposal, the version numbers for the AT and Browser are removed from the headings and included in the detail view of a Test Report, underneath the Test heading, which would look something like this.

<h2>
    <span>Details for test:</span>
    Navigate to a checked checkbox in interaction mode
    <span>Tested with JAWS 2022.2202.38 And Chrome v98.0.4758.109<span>
 </h2>

That is showing versions inside the heading, which I don't think we want.

BTW, I don't think we need the phrase "Details for " at the start of those headings. It makes navigating through the tests very slow. we could put " - Test Results" after the test title instead.

Just after the heading, we can mention just the most recent version used:

Latest version tested:JAWS 2022.2202.38 with Chrome v98.0.4758.109

As @s3ththompson mentioned above, there will nearly always be multiple versions with the same test results. And, we need to surface that detail. So, I recommend that after the results table for each test, we have a "Run History" disclosure that reveals a list that shows all the versions used for the run. This list would grow until some version causes the results to change.

The items in the list would look like:

• Tested with JAWS 2022.2202.38 And Chrome v98.0.4758.109 by @mcking65 on March 31, 2022.
• Tested with JAWS 2022.2202.38 And Chrome v98.0.4758.109 by @jongund on March 28, 2022.

Or, this could also be formatted as a table, which would probably be a lot easier to read. The label on the table would be:

Run history for Navigate to a checked checkbox in interaction mode

[mcking65]

@richnoah

The team has approval to move forward.

You might want to consider delivering in pieces to speed delivery of the most critical parts. We can postpone the admin parts as long as you can build the AT version list manually. We could start with the most basic scenarios, and start collecting data ASAP. Then, surface that data on the reports, then go back and address the edit cases, and then the admin cases. Just a thought. The sooner we are collecting and displaying, the sooner we can start working with AT devs.

[isaacdurazo]

Thank you all so much for the feedback and suggestions. I've captured all of them and updated both text-based and visual mockups above. The changes included are:

1. Changed copy in the Alerts from:

This change doesn’t impact your Test Run. Each test within a test plan is associated with an AT and Browser version.

to be:

This change doesn’t affect results that have already been submitted for this plan. However, results you submit during this session will be recorded with the versions specified in this form.

2. In the Test Plan's single Test page

• Remove "Details for" from the heading and added " - Test Results" after the test title instead.
• Added a disclosure underneath the table that includes the Run History as a list

3. A new AT and Browser Details Modal mockup for a scenario when a Browser might not be automatically detected by the app

4. A way for Admins to provide an approximate date of availability in addition to a version number when adding a new Assistive Technologie