Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test-macos-3 fails a lot #4301

Open
twpayne opened this issue Feb 23, 2025 · 0 comments
Open

test-macos-3 fails a lot #4301

twpayne opened this issue Feb 23, 2025 · 0 comments
Labels
help wanted Extra attention is needed internal Does not affect users

Comments

@twpayne
Copy link
Owner

twpayne commented Feb 23, 2025

Describe the bug

chezmoi's CI runs a bunch of tests. The test-macos-3 test often (~50% of the time) fails with a timeout after 10 minutes, but passes when the test is re-run. For an example failure, see https://github.com/twpayne/chezmoi/actions/runs/13487705954/job/37681029733.

To reproduce

I cannot reproduce this, but see "Additional Context" below.

Expected behavior

chezmoi's CI should pass reliably.

Output of command with the --verbose flag

n/a

Output of chezmoi doctor

n/a. This has been happening for a while.

Additional context

At this point, I have only a few observations:

  • The problem did not occur a few months ago, and I don't believe that the problem is related to the CI infrastructure. It's always the test-macos-3 job that fails with a timeout. If it were related to the CI infrastructure then I would expect all test-macos-* jobs to fail equally often.
  • Each time, the job fails at a different point. There is no consistency.
  • Looking at the logs, tests that should be really fast (i.e. take less than a second) instead take several seconds to run. Clearly, there's something wrong with this set of tests. If it were a simple "noisy neighbor" (i.e. overloaded CI) problem, then all jobs would fail equally often.
  • I've made various efforts to tweak the macOS test sharding (by reducing the last first letter of the test), but it's consistently test-macos-3 that fails.
  • Running some kind of git bisect on the test shards would likely find the problem, but it would take someone with a lot of patience to run this.

cc @KapJI because you've helped so much in the past and might have insight here.

@twpayne twpayne added internal Does not affect users help wanted Extra attention is needed labels Feb 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed internal Does not affect users
Projects
None yet
Development

No branches or pull requests

1 participant