boardwalk (feat): allow execution of playbooks in addition to tasks (#85

)

This changeset adds a `BaseJob` class, from which `TaskJob` and `PlaybookJob`
are derived. As the names suggest, these are Jobs which permit execution of
tasks -- as was previously possible when using the `Job` class -- and full
Ansible playbooks. The design features of Boardwalk, however, still exist even
when using a PlaybookJob in a Workflow; that is, under the hood,
`ansible-runner` is told to execute the targeted playbook file with a `limit` to
the host which is currently progressing through the workflow.

As such, if the playbook(s) being selected for a given Workflow do not apply to
the host(s) going through it, Ansible will skip execution of the playbook, as it
would for a standard `ansible-playbook` invocation.

As a quick overview, the new jobs would look like the following:

```python
class TestJob(TaskJob):
    def tasks(self) -> AnsibleTasksType:
        return [{"ansible.builtin.debug": {"msg": "Hello, Boardwalk!"}}]

class ShouldSucceedPlaybookExecutionTestJob(PlaybookJob):
    def playbooks(self) -> AnsibleTasksType:
        return [
            {"ansible.builtin.import_playbook": path("playbook-job-test-should-succeed.yml")},
            {"ansible.builtin.import_playbook": path("playbook-job-test-should-be-skipped.yml")},
        ]
```

A Workflow may use TaskJob, PlaybookJob, or a mix of both, as in the following
example:

```python
class ShouldSucceedMixedJobTypesWorkflow(Workflow):
    def jobs(self):
        return [
            TestJob(),
            ShouldSucceedPlaybookExecutionTestJob(),
        ]

    def exit_jobs(self):
        return TestJob()
```

And, of course, the `test/server-client/Boardwalkfile.py` contains a full
example Boardwalkfile.

Other notes
-----------
- 🧪 Test enhancements: The majority of test Workspaces in the
  `Boardwalkfile.py` have been integrated into the `pytest` test suite (see:
  `test/integration/test_workspaces.py`). No more shall the tedious cycle of
  running the trinity of commands -- `boardwalk workspace use $WORKSPACE`,
  `boardwalk init`, `boardwalk run` -- be needed, one command does them all!
  Unless of course, you _want_ to run them manually; I won't judge.
    - When running `pytest`, upon reaching these tests, the execution will
      pause, and prompt for the `BECOME password`; this is the same `sudo`
      password that _would_ be used with `boardwalk run`; the difference is now
      we write it -- via `tempfile.NamedTemporaryFile` -- to a file, and then
      set the envvar `ANSIBLE_BECOME_PASSWORD_FILE` to the path returned by the
      context manager. This results in only needing to specify your password
      _once_ if you want to run all of the test workspaces.
    - Note, however, having `make develop-server` running in a different shell
      beforehand is still needed, as is releasing any caught Workspace(s) that
      the development server may be aware of. (Having `pytest` spawn the
      development server itself is an item to complete for the future.)
    - As there is still the requirement to have the development server up, these
      tests are marked to be skipped if the `CI` envvar is found as set (and it
      is [always
      set](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables#default-environment-variables)
      in GitHub workflows).
- 📝 Logging enhancements: If you're like me, you enjoy a good logging output.
  So [`loguru`](https://github.com/delgan/loguru) has been tapped to transform
  the prior usages of Python's built-in `logger`, which looked a little
  something like this...
    ```bash
    # ... like this ...
    asullivan@MBP-NT9RPG2XV7 tmp % boardwalk run -nsc
    INFO:boardwalk.cli_run:Using workspace: ShouldFailTestWorkspace
    INFO:boardwalk.cli_run:Reading inventory to process any --limit
    INFO:boardwalk.ansible:Processing ansible-inventory
    WARNING:boardwalk.cli_run:localhost: Host started workflow but never completed. Job preconditions are ignored for this host
    [... truncated ...]

    # ... and turned it into this! ✨
    (boardwalk-py3.12) asullivan@MBP-NT9RPG2XV7 server-client % boardwalk run -nsc
    2024-10-24 15:27:52.564 | INFO     | boardwalk.cli:cli:77 - Log level is INFO
    2024-10-24 15:27:52.566 | INFO     | boardwalk.cli_run:run:125 - Using workspace: ShouldFailTestWorkspace
    2024-10-24 15:27:52.566 | INFO     | boardwalk.cli_run:filter_hosts_by_limit:375 - Reading inventory to process any --limit
    2024-10-24 15:27:52.566 | INFO     | boardwalk.ansible:ansible_inventory:191 - Processing ansible-inventory
    2024-10-24 15:27:52.933 | WARNING  | boardwalk.cli_run:check_host_preconditions_locally:442 - 127.0.0.1: Host started workflow but never completed. Job preconditions are ignored for this host
    ```
    This is admittedly a bit of a minor change, but it should aid with readability.
- 🐛 `boardwalk workspace use` tab completion - Tab completion via `click`'s
  built-in capabilities was added in
  [#81](#81), however only recently
  was it discovered that if no previous workspace was set as active -- that is,
  `.boardwalk/workspaces/active_workspaces.txt` existed -- a `NoActiveWorkspace`
  exception would be raised. (Which, is not ideal considering that if you're
  tab-completing, you might not _know what workspaces exist_ to begin with; or,
  more likely, just cannot remember their names.)

Deprecation notes
-----------------
- `Job` class - The original `Job` class is deprecated, and will be removed in a
  future release; usage of this deprecated class will emit a DeprecationWarning
  to this effect. To ease transitioning to the new differentiated classes, usage
  of the `Job` class will be treated as a `TaskJob`.

Compatibility notes
-------------------
Based on running this changeset's `pytest` session against a test instance of
`boardwalkd` running version `0.8.21`, this version should be compatible with
prior versions of the server.

Internal ticket reference: SVRENG-608

Makefile

@@ -9,19 +9,21 @@ container:
 # Cleans up temporary data that might get created during normal development
 .PHONY: clean
 clean:
-	rm -r \
-		dist \
-		src/*.egg-info \
-		src/boardwalk/__pycache__ \
-		src/boardwalkd/__pycache__ \
-		.boardwalk \
-		.boardwalkd \
-		test/boardwalk/__pycache__ \
-		test/server-client/.boardwalk \
-		test/server-client/__pycache__ \
-		.pytest_cache \
-		.ruff_cache \
-		|| :
+	set -o pipefail
+
+	@echo '[.] Removing built packages ...'
+	rm -rf dist/
+
+	@echo '[.] Cleaning __pycache__ directories ...'
+	find . -type d -name '__pycache__' | xargs rm -rf
+
+	@echo '[.] Cleaning .boardwalk and .boardwalkd directories ...'
+	find . -type d -name '.boardwalk' -or -name '.boardwalkd' | xargs rm -rf
+
+	@echo '[.] Cleaning pytest and ruff caches ...'
+	find . -type d -name '.pytest_cache' -or -name '.ruff_cache' | xargs rm -rf
+
+	@echo '[.] Removing podman images for localhost/boardwalk ...'
 	podman image rm $$(podman images 'localhost/boardwalk' --quiet) || :
 
 # Installs modules in editable mode
@@ -70,6 +72,14 @@ install-web-deps:
 	curl "https://cdn.jsdelivr.net/npm/bootstrap@$(BOOTSTRAP_VERSION)/dist/css/bootstrap.min.css" -o src/boardwalkd/static/bootstrap.min.css
 	curl "https://cdn.jsdelivr.net/npm/bootstrap@$(BOOTSTRAP_VERSION)/dist/js/bootstrap.bundle.min.js" -o src/boardwalkd/static/bootstrap.bundle.min.js
 
+# Render all d2 diagrams in ./diagrams to PNG
+.PHONY: render-d2
+render-d2:
+	for file in $$(find ./diagrams -type f -name '*.d2' -not -name '_*.d2'); \
+	do \
+		d2 $$file $${file%.*}.png; \
+	done
+
 # Runs all available tests
 .PHONY: test
 test: test-pytest test-ruff test-pyright test-semgrep