Targetconfig crd #716

aerosouund · 2025-01-24T14:41:21Z

Signed-off-by: aerosouund <[email protected]>

fjogeleit · 2025-01-24T15:04:13Z

pkg/config/resolver.go

@@ -203,6 +206,27 @@ func (r *Resolver) CustomIDGenerators() map[string]result.IDGenerator {
 	return generators
 }

+func (r *Resolver) AddTargetConfigEventHandlers() {


you should move this into a controller/client struct. The resolver is only to manage dependencies and not this kind of business logic

You are right, this is me still testing out stuff. The Informer initialization task i have in the description is to mark this exact problem

fjogeleit · 2025-01-24T15:05:33Z

pkg/kubernetes/policy_report_client.go

@@ -40,6 +40,9 @@ func (k *k8sPolicyReportClient) Stop() {
 func (k *k8sPolicyReportClient) Sync(stopper chan struct{}) error {
 	factory := metadatainformer.NewSharedInformerFactory(k.metaClient, 15*time.Minute)

+	// tcInformer := tcinformer.NewSharedInformerFactory(tcv1alpha1.New(&rest.RESTClient{}), time.Second)


I assume you wanted to try things out, don't forget to clean everything up again.

fjogeleit · 2025-01-24T15:06:38Z

pkg/target/collection.go

+	mx         *sync.Mutex
+	clients    []Client
+	targets    map[string]*Target
+	crdTargets map[string]*Target


is there a reason to separate crd targets from config targets?

I wanted to first write out all the flows for storing crd targets separate from the existing config and once everything is figured out merge them. but no, they both can be stored in the same map as i see the map key is just a random uuid

Signed-off-by: aerosouund <[email protected]>

fjogeleit · 2025-01-25T12:21:24Z

the test make target is missing. Please ensure to add it again and that all tests passing

Signed-off-by: aerosouund <[email protected]>

Signed-off-by: Ammar Yasser <[email protected]>

Signed-off-by: aerosouund <[email protected]>

fjogeleit · 2025-01-29T17:56:39Z

cmd/run.go

-			} else {
-				k8sConfig, err = rest.InClusterConfig()
-			}
+			k8sConfig, err := clientcmd.BuildConfigFromFlags("", "/Users/ammaryasser/Downloads/ips")


fjogeleit · 2025-01-29T18:00:29Z

pkg/crd/api/targetconfig/v1alpha1/targetconfig_types.go

+	// +optional
+	GCS GCSOptions `json:"gcs,omitempty"`
+
+	TargetType string `json:"targetType,omitempty"`


I don't think this is needed, you could make S3 etc. to pointers and check which property is set. You can add a schema validation that only one of this fields can be set and not multiple ones.

fjogeleit · 2025-01-29T18:16:22Z

pkg/target/factory/factory.go

+func (f *TargetFactory) CreateSingleClient(tc *v1alpha1.TargetConfig) (*target.Target, error) {
+	var t *target.Target
+	switch tc.Spec.TargetType {
+	case "s3":


as mentioned I would make S3 etc to pointers and check if they are not nil

Wouldn't this enforce priority on some level ? if the code goes: if s3 isn't nil.. else if slack isn't nil.. etc then for a config that has both (even though this is invalid) it will make it so that s3 takes precedence over whatever. with this field i can treat them all the same and only look at the field set in TargetType and also it simplifies the validation logic. if a TargetConfig is of type s3 and the s3 field is nil then its invalid. rather than having "if all the fields are not set then the config is invalid"

You can enforce that only one field is set on the schema, so the API Server will not allow multiple types to be set.
(https://github.com/kyverno/kyverno/blob/main/api/kyverno/v1/common_types.go#L99)

Maybe @JimBugwadia or @eddycharly has an opinion on the TargetType. I think it should not be more cumbersome if needed. If the user configures the s3 and has to set targetType might confuse people.

Signed-off-by: aerosouund <[email protected]>

pkg/kubernetes/policy_report_client.go

Signed-off-by: aerosouund <[email protected]>

pkg/crd/api/targetconfig/v1alpha1/doc.go

pkg/target/factory/factory.go

fjogeleit · 2025-02-10T09:46:32Z

The TargetConfig "slack-notifier" is invalid: 
* spec.slack.certificate: Required value
* spec.slack.channel: Required value
* spec.slack.headers: Required value
* spec.slack.skipTLS: Required value

They are optional and should not be required, please check also other targets to ensure that all optional values are correctly configured.

fjogeleit · 2025-02-10T09:47:08Z

Slack Notifications for a new resource are published twice

fjogeleit · 2025-02-10T10:07:56Z

Following Scenario did not work:

Start Policy Reporter
Apply new Slack TargetConfiguration
Apply new Policy
(x) Get Pushes for new violation of existing resources (got no pushes at all)

Expected:

Get pushes with new violations only

TargetConfig:

kind: TargetConfig
apiVersion: wgpolicyk8s.io/v1alpha1
metadata:
  name: slack-notifier
spec:
  targetType: slack
  slack:
    webhook: https://hooks.slack.com/...
    certificate: ""
    channel: "kyverno"
    headers: {}
    skipTLS: false
  skipExistingOnStartup: true

Signed-off-by: aerosouund <[email protected]>

pkg/listener/send_result.go

Signed-off-by: aerosouund <[email protected]>

fjogeleit · 2025-03-01T08:25:03Z

pkg/config/resolver.go

+		// if the client has skip existing preload its cache with the existing results
+		for _, client := range targets.Targets() {
+			if client.Client.SkipExistingOnStartup() {
+				client.Client.SetCache(r.resultCache.Clone())


please clean this up, as it has no affect right now

fjogeleit · 2025-03-01T08:27:10Z

pkg/config/resolver.go

+					if len(event.Targets.Clients()) < r.targetConfigClient.TargetConfigCount() && !r.targetConfigClient.HasSynced() {
+						continue
+					}
+					r.polrRestartCh <- struct{}{}


also needs cleanup, I would appreciate if we could just remove everything we do not need right now. You can keep it in a different branch for later but I do not want to role out "dead code" which could have a negative impact.

fjogeleit · 2025-03-01T08:29:47Z

pkg/listener/new_result.go

@@ -88,34 +91,8 @@ func (l *ResultListener) Listen(event report.LifecycleEvent) {
 		return


why has this listeners so many changes when we agreed to not touch this logic for now?

Signed-off-by: aerosouund <[email protected]>

aerosouund added 3 commits January 24, 2025 00:07

feat: create targetconfig clientset and crd struct

e407f8c

Signed-off-by: aerosouund <[email protected]>

modify destination package

d73cd55

Signed-off-by: aerosouund <[email protected]>

wip target config

bfcb8e1

Signed-off-by: aerosouund <[email protected]>

fjogeleit reviewed Jan 24, 2025

View reviewed changes

aerosouund added 2 commits January 24, 2025 21:11

create target config client and set it as a dependency for the resolver

0e854e8

Signed-off-by: aerosouund <[email protected]>

some fixes

ffa0339

Signed-off-by: aerosouund <[email protected]>

fjogeleit linked an issue Jan 26, 2025 that may be closed by this pull request

Feature: Allow Reconfiguration when config file changes #474

Open

aerosouund added 6 commits January 26, 2025 21:01

delete unused client package and cleanup makefile

4387ff7

Signed-off-by: aerosouund <[email protected]>

Merge branch 'main' into targetconfig-crd

d387055

Signed-off-by: Ammar Yasser <[email protected]>

change targetconfig to a config generic

d58f87c

Signed-off-by: aerosouund <[email protected]>

initialize map

135da16

Signed-off-by: aerosouund <[email protected]>

add other config types

e53777e

Signed-off-by: aerosouund <[email protected]>

use a channel to communicate target changes

1053683

Signed-off-by: aerosouund <[email protected]>

fjogeleit reviewed Jan 29, 2025

View reviewed changes

aerosouund added 11 commits February 1, 2025 22:07

use cache in target

a9f2583

Signed-off-by: aerosouund <[email protected]>

change test signatures to match codebase changes

56da453

Signed-off-by: aerosouund <[email protected]>

change webook field calles webhook to host

239435a

Signed-off-by: aerosouund <[email protected]>

change webook field calles webhook to host

e4987cb

Signed-off-by: aerosouund <[email protected]>

tc event and restart in run.go

34997d0

Signed-off-by: aerosouund <[email protected]>

remove comment

afde70e

Signed-off-by: aerosouund <[email protected]>

update makefile comment

b4b51b3

Signed-off-by: aerosouund <[email protected]>

gofumpy

befa65c

Signed-off-by: aerosouund <[email protected]>

gofumpt

edba4fa

Signed-off-by: aerosouund <[email protected]>

only restart the policy report informer once all tcs has synced

99fe42c

Signed-off-by: aerosouund <[email protected]>

remove todo

547f845

Signed-off-by: aerosouund <[email protected]>

fjogeleit reviewed Feb 7, 2025

View reviewed changes

pkg/kubernetes/policy_report_client.go Outdated Show resolved Hide resolved

aerosouund added 3 commits February 8, 2025 13:02

remove references to thew restart channel

fb2db0e

Signed-off-by: aerosouund <[email protected]>

in cluster config

4060545

Signed-off-by: aerosouund <[email protected]>

update crd

6dddc11

Signed-off-by: aerosouund <[email protected]>

fjogeleit reviewed Feb 10, 2025

View reviewed changes

pkg/crd/api/targetconfig/v1alpha1/doc.go Outdated Show resolved Hide resolved

fjogeleit reviewed Feb 10, 2025

View reviewed changes

pkg/target/factory/factory.go Outdated Show resolved Hide resolved

aerosouund added 11 commits February 10, 2025 13:30

change groupname and add missing slack target

3b1db6d

Signed-off-by: aerosouund <[email protected]>

run fmt

30f1d7a

Signed-off-by: aerosouund <[email protected]>

update group name

11a2308

Signed-off-by: aerosouund <[email protected]>

fix result sending exiting if one of the results is in the cache

02c64e1

Signed-off-by: aerosouund <[email protected]>

fix result sending for batch send clients

c5ef882

Signed-off-by: aerosouund <[email protected]>

clone cache to clients at client start time

c397c11

Signed-off-by: aerosouund <[email protected]>

fmt

c264a8d

Signed-off-by: aerosouund <[email protected]>

remove filesystem kubeconfig read

24e7e34

Signed-off-by: aerosouund <[email protected]>

fix caching logic at send

b4617df

Signed-off-by: aerosouund <[email protected]>

add tcfg to rbac

5cedec8

Signed-off-by: aerosouund <[email protected]>

return filters

edc6a28

Signed-off-by: aerosouund <[email protected]>

fjogeleit reviewed Feb 17, 2025

View reviewed changes

pkg/listener/send_result.go Outdated Show resolved Hide resolved

aerosouund added 5 commits February 17, 2025 15:35

dont use an array in validation

c74f205

Signed-off-by: aerosouund <[email protected]>

change send result tests to account for new sending logic

21fcc53

Signed-off-by: aerosouund <[email protected]>

change send result tests to account for new sending logic

ce57e0b

Signed-off-by: aerosouund <[email protected]>

add ms teams

beb8bd2

Signed-off-by: aerosouund <[email protected]>

drop the skip existing polr informer restart

3771ead

Signed-off-by: aerosouund <[email protected]>

fjogeleit reviewed Mar 1, 2025

View reviewed changes

strip the changes to new result

e68947b

Signed-off-by: aerosouund <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Targetconfig crd #716

Targetconfig crd #716

aerosouund commented Jan 24, 2025 •

edited

Loading

fjogeleit Jan 24, 2025

aerosouund Jan 24, 2025

fjogeleit Jan 24, 2025

fjogeleit Jan 24, 2025

aerosouund Jan 24, 2025

fjogeleit commented Jan 25, 2025

fjogeleit Jan 29, 2025

fjogeleit Jan 29, 2025

fjogeleit Jan 29, 2025

aerosouund Jan 30, 2025 •

edited

Loading

fjogeleit Feb 3, 2025

fjogeleit commented Feb 10, 2025

fjogeleit commented Feb 10, 2025

fjogeleit commented Feb 10, 2025

fjogeleit Mar 1, 2025

fjogeleit Mar 1, 2025

fjogeleit Mar 1, 2025

		@@ -88,34 +91,8 @@ func (l *ResultListener) Listen(event report.LifecycleEvent) {
		return

Targetconfig crd #716

Are you sure you want to change the base?

Targetconfig crd #716

Conversation

aerosouund commented Jan 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjogeleit commented Jan 25, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aerosouund Jan 30, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjogeleit commented Feb 10, 2025

fjogeleit commented Feb 10, 2025

fjogeleit commented Feb 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aerosouund commented Jan 24, 2025 •

edited

Loading

aerosouund Jan 30, 2025 •

edited

Loading