Add Helm chart for kubeflow trainer

[ChenYi015]

What this PR does / why we need it:

Close #1197

Checklist:

• Docs included if any changes are user facing

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign terrytangyuan for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[andreyvelich]

Thank you for doing this @ChenYi015!
Did you get a chance to explore Kueue script to generate Helm Charts from Kustomize manifests ?
I think, that should significantly help us to keep Kustomize and Charts in sync.

https://github.com/kubernetes-sigs/kueue/blob/main/hack/update-helm.sh

[ChenYi015]

I will take a look at the shell script, it is a bit complicated. I think it may be more easier to sync manifests templated by Helm charts to Kustomize manifests, WDYT?

[ChenYi015]

I have added a shell script hack/sync-manifests. Now one can execute make sync-manifests to sync the Kustomize manifests from the manifests templated by the Helm chart.

[andreyvelich]

Thanks for this effort @ChenYi015!
@tenzen-y @kannon92 @ahg-g @astefanutti @Electronic-Waste @kubeflow/wg-training-leads What do you think about it ?
Should we go other way around to sync Helm Charts to Kustomize Manifests ?
We can use the same approach for JobSet/Kueue if that is easier.

[andreyvelich]

cc @kubeflow/wg-manifests-leads @kubeflow/release-team @varodrig to review script of sync Kustomize + Helm automatically.

[andreyvelich]

cc @franciscojavierarceo

[andreyvelich]

@ChenYi015 Given that we created Helm Charts for JobSet, are we ready to finish this PR?
@kannon92 did we push charts to the OCI registry after this PR: kubernetes-sigs/jobset#792 ?

[kannon92]

@ChenYi015 Given that we created Helm Charts for JobSet, are we ready to finish this PR? @kannon92 did we push charts to the OCI registry after this PR: kubernetes-sigs/jobset#792 ?

Not yet. We are still working on that change. Once we have a release it should push to the registry.

[ChenYi015]

/hold for waiting jobset helm chart to be published