GHA: Mac-R4.1's minor font discrepancy test failures

[MadhulikaTanuboddi]

For almost a week, we have been noticing that most of the tests fromMac 4.1.1 build are failing due to very subtle font styling changes. check the following shinytest snapshots

I noticed that when the build started failing, the Mac OS we are using in the GHA virtual environment is 10.15.17 whereas when it passed around 10 days ago, the virtual environment was 11.6 (I noticed that until Oct 19, with Mac 4.1 builds, we did not see the font download issues when the GHA was using Mac OS 11.6 (Big sur) --> https://github.com/rstudio/shinycoreci-apps/runs/3936390773?check_suite_focus=true)

If you notice a pattern from the above screenshots, most of the tests are failing are mostly with plots and in addition with x-axis values/text font diff. Not sure if this is a relevant information though.

It is also surprising that only Mac R4.1 is being affected by this and not the other Mac build (R4.0, R3.6, 3.5 and 3.4).

Here is just a quick log when I ranshinycoreci::fix_all_gha_branches command on my R console

Apps:
* 001-hello                           : mytest     ; macOS-4.1
* 004-mpg                             : mytest     ; macOS-4.1
* 006-tabsets                         : mytest     ; macOS-4.1
* 014-onflushed                       : mytest     ; macOS-4.1
* 026-shiny-inline                    : mytest     ; macOS-4.1
* 027-absolutely-positioned-panels    : mytest     ; macOS-4.1
* 033-conditionalpanel-demo           : mytest     ; macOS-4.1
* 050-kmeans-example                  : mytest     ; macOS-4.1
* 054-nvd3-line-chart-output          : mytest     ; macOS-4.1
* 062-submitbutton-demo               : mytest     ; macOS-4.1
* 063-superzip-example                : mytest     ; macOS-4.1, Wndws-3.5
* 108-module-output                   : mytest     ; macOS-4.1
* 117-shinythemes                     : mytest     ; Wndws-3.5
* 119-namespaced-conditionalpanel-demo: mytest     ; macOS-4.1
* 122-async-outputs                   : mytest     ; macOS-4.1
* 128-plot-dim-error                  : mytest     ; macOS-4.1
* 130-output-null                     : mytest     ; macOS-4.1
* 131-renderplot-args                 : mytest     ; macOS-4.1
* 136-plot-cache                      : mytest     ; macOS-4.1
* 137-plot-cache-key                  : mytest     ; macOS-4.1
* 143-async-plot-caching              : mytest     ; macOS-4.1
* 149-onRender                        : mytest     ; macOS-4.1
* 192-reactlog-hello                  : mytest     ; macOS-4.1
* 194-plot-cache-key                  : mytest     ; macOS-4.1
* 213-tab-panels                      : mytest     ; macOS-3.4
* 216-quosures                        : mytest     ; macOS-4.1
* 301-bs-themes                       : custom4    ; macOS-4.1
* 301-bs-themes                       : default3   ; macOS-4.1
* 301-bs-themes                       : default4   ; macOS-4.1
* 302-bootswatch-themes               : cerulean4  ; macOS-4.1
* 302-bootswatch-themes               : cosmo4     ; macOS-4.1
* 302-bootswatch-themes               : cyborg4    ; macOS-4.1
* 302-bootswatch-themes               : darkly4    ; macOS-4.1
* 302-bootswatch-themes               : journal4   ; macOS-4.1
* 302-bootswatch-themes               : litera4    ; macOS-4.1
* 302-bootswatch-themes               : lumen4     ; macOS-4.1
* 302-bootswatch-themes               : lux4       ; macOS-4.1
* 302-bootswatch-themes               : materia4   ; macOS-4.1
* 302-bootswatch-themes               : minty4     ; macOS-4.1
* 302-bootswatch-themes               : pulse4     ; macOS-4.1
* 302-bootswatch-themes               : sandstone4 ; macOS-4.1
* 302-bootswatch-themes               : simplex4   ; macOS-4.1
* 302-bootswatch-themes               : sketchy4   ; macOS-4.1
* 302-bootswatch-themes               : slate4     ; macOS-4.1
* 302-bootswatch-themes               : solar4     ; macOS-4.1
* 302-bootswatch-themes               : spacelab4  ; macOS-4.1
* 302-bootswatch-themes               : superhero4 ; macOS-4.1
* 302-bootswatch-themes               : yeti4      ; macOS-4.1

screenshare_-_2021-10-27_10_33_11_am.mp4

[MadhulikaTanuboddi]

The changes are very subtle that they could be accepted. But, I want to understand the reason for these consistent failures (i mean if they are due to the Mac OS downgrade from 11.6 to 10.15 and R4.1 with Mac?)

[MadhulikaTanuboddi]

Thanks to @schloerke. Ideally, all the times, the Mac runs should be running with 10.15. Because it is listed as the macos-latest as per https://github.com/actions/virtual-environments. So, it is surprising that few builds ago on the tests even ran on Mac 11.6. Looks like they are testing and slowly rolling-over the changes. actions/runner-images#4060

We originally thought we could set the macos-latest to 11.6 via #192. But, since it will be default latest version anyways from Nov 3, we decided to wait. That means, the test will continue to be flaky until then.

[MadhulikaTanuboddi]

Last week, we merged #192. Because, from actions/runner-images#4060, it looks like the latest version would still be 10.15 until mid-December