Best practice of dynamic ConcurrentReconciles

[HamzaZo]

My controller watches for created CR and starts to reconcile, in the reconcile loop I do some business logic using Terraform. If I create two CR at the same time :

• The reconcile loop get the first item from the workqueue and do the business logic which is what I expect. However, the second CR gets blocked until the first one succeeded.

Afterwards, I noticed that the default value of concurrent reconciles is worker.count=1, so I modify SetupWithManager as follows:

func (r *PostgreSqlReconciler) SetupWithManager(mgr ctrl.Manager) error {
	pred := predicate.GenerationChangedPredicate{}
	return ctrl.NewControllerManagedBy(mgr).
		For(&cloudv1alpha1.PostgreSql{}).
		WithEventFilter(pred).
		WithOptions(controller.Options{
			MaxConcurrentReconciles: 10,
		}).
		Complete(r)

But I'm wondering, what is the best practice for dynamic concurrent reconciles? and how to let my controller handles differents request at the same time without hardcoding the value of MaxConcurrentReconciles ? Also, If I create more than one replicas how to handle this ?

Any help is appreciated?
Thanks

/triage support

[k8s-ci-robot]

@HamzaZo: The label(s) triage/support cannot be applied, because the repository doesn't have them

In response to this:

My controller watches for created CR and starts to reconcile, in the reconcile loop I do some business logic using Terraform. If I create two CR at the same time :

• The reconcile loop get the first item from the workqueue and do the business logic which is what I expect. However, the second CR gets blocked until the first one succeeded.

Afterwards, I noticed that the default value of concurrent reconciles is worker.count=1, so I modify SetupWithManager as follows:

func (r *PostgreSqlReconciler) SetupWithManager(mgr ctrl.Manager) error {
  pred := predicate.GenerationChangedPredicate{}
  return ctrl.NewControllerManagedBy(mgr).
  	For(&cloudv1alpha1.PostgreSql{}).
  	WithEventFilter(pred).
  	WithOptions(controller.Options{
  		MaxConcurrentReconciles: 10,
  	}).
  	Complete(r)

But I'm wondering, what is the best practice for dynamic concurrent reconciles? and how to let my controller handles differents request at the same time without hardcoding the value of MaxConcurrentReconciles ? Also, If I create more than one replicas how to handle this ?

Any help is appreciated?
Thanks

/triage support

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[camilamacedo86]

Regard your question see that It is not possible to have two reconcile loops that handle the same object at the same time.

how to let my controller handles differents request at the same time without hardcoding the value of MaxConcurrentReconciles

By default, the MaxConcurrentReconciles will be 1 so you need to set another value if you are looking for that in the manager when the project is initialized.

It is not a k8s doc, however, might help you: http://openkruise.io/en-us/blog/blog2.html

NOTE It is not about your specific question, however, I'd like to share that regards the best practices over the controllers/reconciliation in general I'd say the main one would be to develop idempotent solutions. The idea is that when you reconcile a resource of our API(crd/cr) you are looking to do a collection of operations until it has the desired state on the cluster. I want to suggest you check doc Groups and Versions and Kinds, oh my! to understand better what APIs means and be able to design better solutions. Also, see that it’s recommended to have one controller responsible for manage each API created for the project to properly follow the design goals set by controller-runtime.

I raised an issue in the controller runtime to improve the docs with an official explanation. kubernetes-sigs/controller-runtime#1416. And then, would be nice to get your help with. Please, feel free to follow up it there.

I am closing this one as sorted out since we addressing it in controller-runtime as well. However, if you still with doubts feel free to raise new issues.