Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flakey tests and improve the test log output #166

Merged
merged 1 commit into from
Dec 6, 2022
Merged

Fix flakey tests and improve the test log output #166

merged 1 commit into from
Dec 6, 2022

Conversation

wallrj
Copy link
Member

@wallrj wallrj commented Dec 5, 2022

I've been trying to fix the flakiness in the pull-cert-manager-approver-policy-verify tests:

The tests originally attempted to delete the test namespace after each test, but this can't work because envtest doesn't provide the controller responsible for garbage collection of namespaced resources:

@jetstack-bot jetstack-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 5, 2022
Copy link
Member Author

@wallrj wallrj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@inteon I remember you writing some code which watches until a Namespace has actually been deleted.
I think that would help here, because I think root cause of the flakiness is that the next test and its controller-runtime manager are starting up before the Namespace of the previous test has been fully deleted, and reconciling the resources associated with that previous test.

Meanwhile I will try using Eventually to periodically check that the resources have all been deleted.

@wallrj wallrj requested a review from inteon December 6, 2022 12:26
@wallrj wallrj changed the title WIP: Fix flakey tests and improve the test log output Fix flakey tests and improve the test log output Dec 6, 2022
@jetstack-bot jetstack-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 6, 2022
Don't attempt to delete test namespace because envtest doesn't support namespace deletion.

Signed-off-by: Richard Wall <[email protected]>
Copy link
Member

@inteon inteon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing the namespace issue.
All test are passing and all resources are removed between tests.
/lgtm

@jetstack-bot jetstack-bot added the lgtm Indicates that a PR is ready to be merged. label Dec 6, 2022
@jetstack-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon, wallrj

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jetstack-bot jetstack-bot merged commit a812bd5 into cert-manager:main Dec 6, 2022
@wallrj wallrj deleted the fix-flakey-tests branch December 6, 2022 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants