[general] (bug or feature?) tar_skip on parallel with retrieval = "worker"

[CorradoLanera]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

If with parallel crew computing we set retrieval = "worker" option, tar_skip signals an error if its parent was skipped as well (even if no dependencies are needed to evaluate the condition)

reprex (and expected behavior)

targets::tar_dir({
  targets::tar_script({
    tar_option_set(
      error = "continue",
      retrieval = "worker",
      controller = crew::crew_controller_local(workers = 1)
    )

    list(
      tarchetypes::tar_skip(x, 1, skip = TRUE),
      tarchetypes::tar_skip(y, x + 1, skip = TRUE)
    )
  })
  targets::tar_make()
})
#> ▶ dispatched target x
#> ■ canceled target x
#> ▶ dispatched target y
#> ✖ errored target y
#> Error: could not load dependencies of target y. invalid 'description' argument
#> 
#> ✖ errored pipeline [2.406 seconds]

targets::tar_dir({
  targets::tar_script({
    tar_option_set(
      error = "continue",
      controller = crew::crew_controller_local(workers = 1)
    )

    list(
      tarchetypes::tar_skip(x, 1, skip = TRUE),
      tarchetypes::tar_skip(y, x + 1, skip = TRUE)
    )
  })
  targets::tar_make()
})
#> ▶ dispatched target x
#> ■ canceled target x
#> ▶ dispatched target y
#> ■ canceled target y
#> ▶ ended pipeline [2.422 seconds]

^{Created on 2024-07-04 with reprex v2.1.0}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.1 (2024-06-14 ucrt)
#>  os       Windows 11 x64 (build 26120)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  English_United States.utf8
#>  ctype    English_United States.utf8
#>  tz       Europe/Berlin
#>  date     2024-07-04
#>  pandoc   3.1.11 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  ! package     * version date (UTC) lib source
#>  P backports     1.5.0   2024-05-23 [?] RSPM
#>  P base64url     1.4     2018-05-14 [?] CRAN (R 4.4.0)
#>  P callr         3.7.6   2024-03-25 [?] RSPM (R 4.4.0)
#>  P cli           3.6.3   2024-06-21 [?] RSPM
#>  P codetools     0.2-20  2024-03-31 [3] CRAN (R 4.4.1)
#>  P data.table    1.15.4  2024-03-30 [?] CRAN (R 4.4.0)
#>  P digest        0.6.36  2024-06-23 [?] CRAN (R 4.4.1)
#>  P evaluate      0.24.0  2024-06-10 [?] RSPM
#>  P fansi         1.0.6   2023-12-08 [?] RSPM (R 4.4.0)
#>  P fastmap       1.2.0   2024-05-15 [?] RSPM
#>  P fs            1.6.4   2024-04-25 [?] RSPM (R 4.4.0)
#>  P glue          1.7.0   2024-01-09 [?] RSPM (R 4.4.0)
#>  P htmltools     0.5.8.1 2024-04-04 [?] RSPM (R 4.4.0)
#>  P igraph        2.0.3   2024-03-13 [?] CRAN (R 4.4.0)
#>  P knitr         1.47    2024-05-29 [?] CRAN (R 4.4.0)
#>  P lifecycle     1.0.4   2023-11-07 [?] RSPM (R 4.4.0)
#>  P magrittr      2.0.3   2022-03-30 [?] RSPM (R 4.4.0)
#>  P pillar        1.9.0   2023-03-22 [?] RSPM (R 4.4.0)
#>  P pkgconfig     2.0.3   2019-09-22 [?] RSPM (R 4.4.0)
#>  P processx      3.8.4   2024-03-16 [?] RSPM (R 4.4.0)
#>  P ps            1.7.7   2024-07-02 [?] CRAN (R 4.4.1)
#>  P R6            2.5.1   2021-08-19 [?] RSPM (R 4.4.0)
#>  P reprex        2.1.0   2024-01-11 [?] CRAN (R 4.4.0)
#>  P rlang         1.1.4   2024-06-04 [?] CRAN (R 4.4.0)
#>  P rmarkdown     2.27    2024-05-17 [?] RSPM
#>  P rstudioapi    0.16.0  2024-03-24 [?] RSPM (R 4.4.0)
#>  P secretbase    1.0.0   2024-06-16 [?] CRAN (R 4.4.1)
#>  P sessioninfo   1.2.2   2021-12-06 [?] RSPM (R 4.4.0)
#>  P targets       1.7.1   2024-06-20 [?] CRAN (R 4.4.1)
#>  P tibble        3.2.1   2023-03-20 [?] RSPM (R 4.4.0)
#>  P tidyselect    1.2.1   2024-03-11 [?] RSPM (R 4.4.0)
#>  P utf8          1.2.4   2023-10-22 [?] RSPM (R 4.4.0)
#>  P vctrs         0.6.5   2023-12-01 [?] RSPM (R 4.4.0)
#>  P withr         3.0.0   2024-01-16 [?] RSPM (R 4.4.0)
#>  P xfun          0.45    2024-06-16 [?] RSPM
#>  P yaml          2.3.8   2023-12-11 [?] RSPM (R 4.4.0)
#> 
#>  [1] C:/Users/corra/AppData/Local/R/cache/R/renv/library/anvur.accademies.gpt-ebd9193f/windows/R-4.4/x86_64-w64-mingw32
#>  [2] C:/Users/corra/AppData/Local/R/cache/R/renv/sandbox/windows/R-4.4/x86_64-w64-mingw32/e0da0d43
#>  [3] C:/Program Files/R/R-4.4.1/library
#> 
#>  P ── Loaded and on-disk path mismatch.
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[wlandau]

This is a consequence of how cancellation works. If you look at tar_manifest(), you will see that the target calls tar_cancel() in its command.

> cat(tar_manifest(x)$command)
{
     targets::tar_cancel(TRUE)
     1
 }
> cat(tar_manifest(y)$command)
{
     targets::tar_cancel(TRUE)
     x + 1
 }

In other words, cancellation happens only after the target starts running. By contrast, the target loads dependencies before it runs, i.e. before cancellation has a chance to happen. If there is no existing value to load for the dependency, an error is thrown.

To make sure a dependency value is always available to load, you can set tar_option_set(error = "null") instead of tar_option_set(error = "continue").