ListGroups API KIP 848 #1267

mahajanadhitya · 2024-08-09T08:51:02Z

No description provided.

milindl

Thanks for the changes! Looks mostly okay.

Remember to add it to CHANGELOG.md also

kafka/adminapi_test.go

milindl · 2024-08-13T04:03:20Z

kafka/adminapi_test.go

+
+	listres, err := a.ListConsumerGroups(
+		ctx, SetAdminRequestTimeout(time.Second),
+		SetAdminMatchConsumerGroupTypes([]ConsumerGroupType{classicGroupType, classicGroupType}))


Suggested change

SetAdminMatchConsumerGroupTypes([]ConsumerGroupType{classicGroupType, classicGroupType}))

SetAdminMatchConsumerGroupTypes([]ConsumerGroupType{classicGroupType, classicGroupType}))

Why is the same group type included twice?

It is to test the INVALID_ARGS error code with duplicate group types.

But your test is not checking for invalid args error code below, so change that. You're checking for deadline exceeded still.

kafka/build_darwin_amd64.go

kafka/integration_test.go

milindl · 2024-08-13T04:09:23Z

kafka/integration_test.go

+		// Generating a new topic/groupID to ensure a fresh group/topic is created.
+		rand.Seed(time.Now().Unix())
+		groupID := fmt.Sprintf("%s-%d", testconf.GroupID, rand.Int())
+		topic := fmt.Sprintf("%s-%d", testconf.TopicName, rand.Int())
+		nonExistentGroupID := fmt.Sprintf("%s-nonexistent-%d", testconf.GroupID, rand.Int())


Move above the "if" block, and remove from below too, so they're not duplicated

Same for creation of adminclient, topic creation, etc.

Actually this is done voluntarily because currently in semaphore we run the whole tests with broker version < 3.8.0.0 and then when we run with higher broker version on semaphore to test the consumer group protocol we only check the case of consumer group that is why we return at the end.

Okay, so since there is nothing common between the tests, why don't you split them? I don't see why we should have an entirely different test in an if block if there's any shared code between them, it just makes it harder to read.

TestAdminClient_ListAndDescribeConsumerGroupsClassic and TestAdminClient_ListConsumerGroupsConsumer

* Upgrade Hamba to 2.24.0 (#2) (#1266) * Upgrade Hamba to 2.24.0 Also enhance CSFLE examples; minor cleanup * Fix imports * Perform Avro schema resolution if necessary (#1) (#1265) * Add missing API parts for schema registry (#1256) * add some missing API parts * fix context endpoint --------- Co-authored-by: Alexander Hein <[email protected]> Co-authored-by: Robert Yokota <[email protected]> * Regenerate SR api docs (#4) (#1268) * Fix some golint naming issues (#6) (#1272) * Fix URL for DeksByVersion (#7) (#1273) * Pinned cp-kafka version (#1270) as 7.7.0 has bug https://issues.apache.org/jira/browse/KAFKA-16310 that makes LisOffsets test fail * Single Commit --------- Co-authored-by: Robert Yokota <[email protected]> Co-authored-by: oleksandrkorzhenevskyi <[email protected]> Co-authored-by: Alexander Hein <[email protected]> Co-authored-by: Emanuele Sabellico <[email protected]>

milindl · 2024-08-21T03:14:15Z

kafka/adminapi_test.go

+	ctx, cancel := context.WithTimeout(context.Background(), expDuration)
+	defer cancel()
+
+  listres, err := a.ListConsumerGroups(


gofmt needs to be run

milindl · 2024-08-21T03:15:49Z

kafka/adminapi_test.go

@@ -464,7 +467,10 @@ func testAdminAPIsListConsumerGroups(
 		t.Fatalf("Expected ConsumerGroupTypeFromString to work for Unknown type")
 	}

-	listres, err := a.ListConsumerGroups(
+	ctx, cancel := context.WithTimeout(context.Background(), expDuration)


I think this code doesn't run. Are the tests running when you are dynamically linking librdkafka? This is a compilation error because := and not =

milindl · 2024-08-21T03:16:54Z

kafka/adminapi_test.go

+
+	listres, err := a.ListConsumerGroups(
+		ctx, SetAdminRequestTimeout(time.Second),
+		SetAdminMatchConsumerGroupTypes([]ConsumerGroupType{classicGroupType, classicGroupType}))


But your test is not checking for invalid args error code below, so change that. You're checking for deadline exceeded still.

milindl · 2024-08-21T03:21:11Z

kafka/integration_test.go

+		// Generating a new topic/groupID to ensure a fresh group/topic is created.
+		rand.Seed(time.Now().Unix())
+		groupID := fmt.Sprintf("%s-%d", testconf.GroupID, rand.Int())
+		topic := fmt.Sprintf("%s-%d", testconf.TopicName, rand.Int())
+		nonExistentGroupID := fmt.Sprintf("%s-nonexistent-%d", testconf.GroupID, rand.Int())


Okay, so since there is nothing common between the tests, why don't you split them? I don't see why we should have an entirely different test in an if block if there's any shared code between them, it just makes it harder to read.

TestAdminClient_ListAndDescribeConsumerGroupsClassic and TestAdminClient_ListConsumerGroupsConsumer

milindl

Some more comments, please run the tests

mahajanadhitya · 2024-09-06T08:36:39Z

confluent-kafka-go KIP 848 ListGroups API

Branch Name

feature/ListGroupsAPI

Build

Install the corresponding Librdkafka and build with go build -tags dynamic ./...

Unit Tests

confluent-kafka-go/kafka/adminapi_test.go via testAdminAPIsListConsumerGroups
go test -tags dynamic -timeout 30s -run ^TestAdminAPIs

Cluster

Spawn the trivup cluster in librdkafka/tests
./interactive_broker_version.py 'trunk@f6c9feea76d01a46319b0ca602d70aa855057b07' --port 9092 --kraft --conf '{"conf":["unstable.api.versions.enable=true","group.coordinator.new.enable=true","group.protocol=classic,consumer"]}'

Example

confluent-kafka-go/examples/admin_list_consumer_groups/admin_list_consumer_groups.go
go run -tags dynamic admin_list_consumer_groups.go localhost:9092 [-states <state1> <state2> ...] [-types <type1> <type2> ...]

Integration Tests

Remember to set the Environment Variable TEST_CONSUMER_GROUP_PROTOCOL to consumer/classic
confluent-kafka-go/kafka/integration_test.go via TestAdminClient_ListConsumerGroups
go test -v -tags dynamic -run ^TestIntegration -testify.m TestAdminClient_ListConsumerGroups

rebase commit (#3)

dac7607

mahajanadhitya requested review from a team as code owners August 9, 2024 08:51

milindl reviewed Aug 13, 2024

View reviewed changes

milindl reviewed Aug 21, 2024

View reviewed changes

milindl requested changes Aug 21, 2024

View reviewed changes

In sync changes

db9f908

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ListGroups API KIP 848 #1267

ListGroups API KIP 848 #1267

mahajanadhitya commented Aug 9, 2024

milindl left a comment

milindl Aug 13, 2024

mahajanadhitya Aug 16, 2024

milindl Aug 21, 2024 •

edited

Loading

milindl Aug 13, 2024

milindl Aug 13, 2024

mahajanadhitya Aug 16, 2024

milindl Aug 21, 2024

milindl Aug 21, 2024

milindl Aug 21, 2024

milindl Aug 21, 2024 •

edited

Loading

milindl Aug 21, 2024

milindl left a comment

mahajanadhitya commented Sep 6, 2024

	SetAdminMatchConsumerGroupTypes([]ConsumerGroupType{classicGroupType, classicGroupType}))
	SetAdminMatchConsumerGroupTypes([]ConsumerGroupType{classicGroupType, classicGroupType}))

ListGroups API KIP 848 #1267

Are you sure you want to change the base?

ListGroups API KIP 848 #1267

Conversation

mahajanadhitya commented Aug 9, 2024

milindl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milindl Aug 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milindl Aug 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milindl left a comment

Choose a reason for hiding this comment

mahajanadhitya commented Sep 6, 2024

confluent-kafka-go KIP 848 ListGroups API

Branch Name

Build

Unit Tests

Cluster

Example

Integration Tests

milindl Aug 21, 2024 •

edited

Loading

milindl Aug 21, 2024 •

edited

Loading