Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KFP-Tekton v2] Merge DAG drivers and publishers into a sub-dag controller to reduce Tekton graph complexity. #1419

Open
Tracked by #1366
Tomcli opened this issue Nov 22, 2023 · 0 comments

Comments

@Tomcli
Copy link
Member

Tomcli commented Nov 22, 2023

/kind feature

Description:
When adding driver and publisher tasks into the sub-dag, we are introducing a new layer of complexity to the Tekton graph because the driver needs to connect with all the root nodes and publisher needs to connect with all the leave node. Because driver always runs before the main sub-dag(pipelinerun in Tekton) and publisher runs after the sub-dag is done, we can have our controller handles those logic instead of embedding drivers and publishers into the Tekton pipeline graph.

This will require we implement the status-IR first as right now we don't have a way to update sub-dag and root dag pipeline status in MLMD and reflect them on the UI.

Screenshot 2023-11-22 at 9 27 59 AM

Additional information:
[Miscellaneous information that will assist in solving the issue.]

@Tomcli Tomcli changed the title [KFP-Tekton v2] Merge DAG drivers and publishers into a sub-dag controller we have to reduce Tekton graph complexity. [KFP-Tekton v2] Merge DAG drivers and publishers into a sub-dag controller to reduce Tekton graph complexity. Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant