feat(pageserver): filter keys with gc-compaction

[skyzh]

Problem

Part of #8002

Close #8920

Legacy compaction (as well as gc-compaction) rely on the GC process to remove unused layer files, but this relies on many factors (i.e., key partition) to ensure data in a dropped table can be eventually removed.

In gc-compaction, we consider the keyspace information when doing the compaction process. If a key is not in the keyspace, we will skip that key and not include it in the final output.

However, this is not easy to implement because gc-compaction considers branch points (i.e., retain_lsns) and the retained keyspaces could change across different LSNs. Therefore, for now, we only remove aux v1 keys in the compaction process.

Summary of changes

• Add FilterIterator to filter out keys.
• Integrate FilterIterator with gc-compaction.
• Add collect_gc_compaction_keyspace for a spec of keyspaces that can be retained during the gc-compaction process.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[orca-security-us]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Infrastructure as Code	0   0   0   0	View in Orca
Passed	Secrets	0   0   0   0	View in Orca
Passed	Vulnerabilities	0   0   0   0	View in Orca

[github-actions]

4416 tests run: 4260 passed, 0 failed, 156 skipped (full report)

---

Flaky tests (3)

Postgres 17

• test_pg_regress[None]: release-arm64, debug-x86-64
• test_scrubber_physical_gc[4]: release-arm64

Test coverage report is not available

_{The comment gets automatically updated with the latest test results
d374220 at 2024-09-16T01:54:22.896Z :recycle:}