[chore] Separate Python API into another repository (apache#12779)

Currently, our Python API code is a module in apache/dolphinscheduler codebase,
each time users change Python API code, they need to run all requests CI check
for dolphinscheduler and Python API, But if the user does only change Python
code, it could be merged if Python API CI pass and do not dependent on others CI.

Besides, we release Python API as the same version of dolphinscheduler. It is
easy for user to match Python API version. But when Python API does not change
any code, but dolphinscheduler release a bugfix version, Python API has to
release the new version to match dolphinscheduler. This happened when we
released Python API 2.0.6 and 2.0.7. 2.0.6 and 2.0.7 is bugfix version, and
Python API does not change any code, so the PyPI package is the same.

Separate Python API also makes our code more sense, we will have more
distinguished code in dolphinscheduler and Python API new repository.
Have separate issue tracker and changelog for information to users.

ref PR in other repository: apache/dolphinscheduler-sdk-python#1

see more detail in mail thread: https://lists.apache.org/thread/4z7l5l54c4d81smjlk1n8nq380p9f0oo

dolphinscheduler-python/pydolphinscheduler/.flake8 → .flake8

@@ -19,22 +19,10 @@
 max-line-length = 110
 exclude =
     .git,
-    __pycache__,
-    .pytest_cache,
-    *.egg-info,
-    docs/source/conf.py
-    old,
-    build,
-    dist,
-    htmlcov,
-    .tox,
     dist,
 ignore = 
     # It's clear and not need to add docstring
     D107,  # D107: Don't require docstrings on __init__
     D105,  # D105: Missing docstring in magic method
     # Conflict to Black
     W503   # W503: Line breaks before binary operators
-per-file-ignores =
-    */pydolphinscheduler/side/__init__.py:F401
-    */pydolphinscheduler/tasks/__init__.py:F401

.github/CODEOWNERS

@@ -38,7 +38,6 @@
 /dolphinscheduler-task-plugin/ @caishunfeng @SbloodyS @zhuangchong
 /dolphinscheduler-tools/ @caishunfeng @SbloodyS @zhongjiajie @EricGao888
 /script/ @caishunfeng @SbloodyS @zhongjiajie @EricGao888
-/dolphinscheduler-python/ @zhongjiajie
 /dolphinscheduler-ui/ @songjianet @Amy0104
 /docs/ @zhongjiajie @Tianqi-Dotes @EricGao888
 /licenses/ @kezhenxu94 @zhongjiajie

.github/actions/labeler/labeler.yml

@@ -15,9 +15,6 @@
 # limitations under the License.
 #
 
-Python:
-  - any: ['dolphinscheduler-python/**/*']
-
 backend:
   - 'dolphinscheduler-alert/**/*'
   - 'dolphinscheduler-api/**/*'
@@ -40,7 +37,6 @@ backend:
 
 document:
   - 'docs/**/*'
-  - 'dolphinscheduler-python/pydolphinscheduler/docs/**/*'
 
 CI&CD:
   - any: ['.github/**/*']

.github/workflows/unit-test.yml

@@ -23,7 +23,6 @@ on:
     paths-ignore:
       - '**/*.md'
       - 'dolphinscheduler-ui'
-      - 'dolphinscheduler-python/pydolphinscheduler'
     branches:
       - dev
 

.gitignore

@@ -50,18 +50,3 @@ dolphinscheduler-common/test
 dolphinscheduler-worker/logs
 dolphinscheduler-master/logs
 dolphinscheduler-api/logs
-
-# ------------------
-# pydolphinscheduler
-# ------------------
-# Cache
-__pycache__/
-.tox/
-
-# Build
-build/
-*egg-info/
-
-# Test coverage
-.coverage
-htmlcov/

.pre-commit-config.yaml

@@ -41,11 +41,6 @@ repos:
           'flake8-docstrings>=1.6',
           'flake8-black>=0.2',
         ]
-        # pre-commit run in the root, so we have to point out the full path of configuration
-        args: [
-          --config,
-          dolphinscheduler-python/pydolphinscheduler/.flake8
-        ]
   - repo: https://github.com/pycqa/autoflake
     rev: v1.4
     hooks:

README.md

@@ -19,10 +19,11 @@ Apache DolphinScheduler is the modern data workflow orchestration platform with
 
 The key features for DolphinScheduler are as follows:
 - Easy to deploy, we provide 4 ways to deploy, such as Standalone deployment,Cluster deployment,Docker / Kubernetes deployment and Rainbond deployment
-- Easy to use, there are 3 ways to create workflows:
+- Easy to use, there are four ways to create workflows:
   - Visually, create tasks by dragging and dropping tasks
-  - Creating workflows by PyDolphinScheduler(Python way)
-  - Creating workflows through Open API
+  - [PyDolphinScheduler](https://dolphinscheduler.apache.org/python/dev/index.html), Creating workflows via Python API, aka workflow-as-code
+  - Yaml definition, mapping yaml into workflow(have to install PyDolphinScheduler currently)
+  - Open API, Creating workflows
 
 - Highly Reliable,
 DolphinScheduler uses a decentralized multi-master and multi-worker architecture, which naturally supports horizontal scaling and high availability

docs/docs/en/contribute/release/release-post.md

@@ -1,7 +1,7 @@
 # Release Post
 
 We still have some publish task to do after we send the announcement mail, currently we have to publish Docker images to
-Docker Hub and also publish pydolphinscheduler to PyPI.
+Docker Hub.
 
 ## Publish Docker Image
 
@@ -20,11 +20,6 @@ We could reuse the main command the CI run and publish our Docker images to Dock
     -Pdocker,release
 ```
 
-## Publish pydolphinscheduler to PyPI
-
-Python API need to release to PyPI for easier download and use, you can see more detail in [Python API release](https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-python/pydolphinscheduler/RELEASE.md#to-pypi)
-to finish PyPI release.
-
 ## Get All Contributors
 
 You might need all contributors in current release when you want to publish the release news or announcement, you could

docs/docs/en/contribute/release/release-prepare.md

@@ -23,7 +23,6 @@ For example, to release `x.y.z`, the following updates are required:
   - `deploy/kubernetes/dolphinscheduler`:
     - `Chart.yaml`: `appVersion` needs to be updated to x.y.z (`version` is helm chart version，incremented and different from x.y.z)
     - `values.yaml`: `image.tag` needs to be updated to x.y.z
-  - `dolphinscheduler-python/pydolphinscheduler/setup.py`: change `version` to x.y.z
 - Version in the docs:
   - Change the placeholder `<version>`(except `pom`)  to the `x.y.z` in directory `docs`
   - Add new history version