Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k/server/tests: reset leaders table on restart #25141

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

nvartolomei
Copy link
Contributor

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

It seems that we keep running with old leaders table after restart
otherwise and wait_for_controller_leadership succeeds before a leader is
elected in the new post-restart term.

Noticed this because test_topic_recreation_recovery keeps failing
locally for me.
@vbotbuildovich
Copy link
Collaborator

CI test results

test results on build#62133
test_id test_kind job_url test_status passed
cluster_cloud_metadata_rpfixture.cluster_cloud_metadata_rpfixture unit https://buildkite.com/redpanda/redpanda/builds/62133#01952d2b-0dbf-45f0-aa2b-d8eb6f8105d9 FAIL 0/2
cluster_cloud_metadata_rpfixture.cluster_cloud_metadata_rpfixture unit https://buildkite.com/redpanda/redpanda/builds/62133#01952d2b-0dbf-4d4d-ad60-98c12b0daac7 FAIL 0/2
gtest_cluster_cloud_metadata_rpfixture.gtest_cluster_cloud_metadata_rpfixture unit https://buildkite.com/redpanda/redpanda/builds/62133#01952d2b-0dbf-45f0-aa2b-d8eb6f8105d9 FAIL 0/2
gtest_cluster_cloud_metadata_rpfixture.gtest_cluster_cloud_metadata_rpfixture unit https://buildkite.com/redpanda/redpanda/builds/62133#01952d2b-0dbf-4d4d-ad60-98c12b0daac7 FAIL 0/2
rptest.tests.datalake.datalake_e2e_test.DatalakeE2ETests.test_topic_lifecycle.cloud_storage_type=CloudStorageType.S3.catalog_type=CatalogType.REST_HADOOP ducktape https://buildkite.com/redpanda/redpanda/builds/62133#01952d86-0530-4832-b9de-1df8caad99fc FLAKY 1/2
rptest.tests.delete_records_test.DeleteRecordsTest.test_delete_records_with_transactions.cloud_storage_enabled=True ducktape https://buildkite.com/redpanda/redpanda/builds/62133#01952d86-0530-4832-b9de-1df8caad99fc FLAKY 1/2
rptest.tests.partition_force_reconfiguration_test.PartitionForceReconfigurationTest.test_basic_reconfiguration.acks=-1.restart=False.controller_snapshots=False ducktape https://buildkite.com/redpanda/redpanda/builds/62133#01952d86-0531-4fb1-9b05-a3b39c614e0e FLAKY 1/2
rptest.tests.partition_movement_test.SIPartitionMovementTest.test_shadow_indexing.num_to_upgrade=0.cloud_storage_type=CloudStorageType.ABS ducktape https://buildkite.com/redpanda/redpanda/builds/62133#01952d8a-b46c-4f83-be41-7a646bfd24d2 FLAKY 1/3
storage_e2e_single_thread_rpunit.storage_e2e_single_thread_rpunit unit https://buildkite.com/redpanda/redpanda/builds/62133#01952d2b-0dbf-45f0-aa2b-d8eb6f8105d9 FLAKY 1/2

Comment on lines -126 to -136
void restart() {
shutdown();
app_signal = std::make_unique<::stop_signal>();
ss::smp::invoke_on_all([] {
auto& config = config::shard_local_cfg();
config.get("disable_metrics").set_value(false);
}).get();
app.initialize(proxy_config(), proxy_client_config());
app.check_environment();
app.wire_up_and_start(*app_signal, true);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@@ -280,6 +280,9 @@ class redpanda_thread_fixture {
app.initialize(proxy_config(), proxy_client_config());
app.check_environment();
app.wire_up_and_start(*app_signal, true);
app.controller->get_partition_leaders()
.invoke_on_all([](cluster::partition_leaders_table& t) { t.reset(); })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense to me. the c++ fixture is so fragile :(

cc @mmaslankaprv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants