feat: Karapace modernization with uvicorn, FastAPI and OpenTelemetry #1039
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jjaakola-aiven
changed the title
- we create a standalone module for SR related components - we use DI to wire together the SR dependencies - we move the routers to own folder - we move the config initialization to DI and app startup
The config for backup tool is still the full Karapace config. Config for Karapace is a set of environment variables opposed to prior JSON file. Backup tool will still use the JSON format.
- we drop the `Dockerfile.dev`
- the `schema_tool` runs against avro schemas at a specific path
The REST Proxy requires still split from karapace.core modules, e.g. configuration class is there.
jjaakola-aiven
force-pushed
the
jjaakola-aiven-fastapi
branch
from
3c28958 to
34f13d5
Compare
keejon
approved these changes
Feb 13, 2025
| karapace_container = providers.Container(KarapaceContainer) | ||
| no_auth_authorizer = providers.Singleton(NoAuthAndAuthz) | ||
| http_authorizer = providers.Singleton(HTTPAuthorizer, auth_file=karapace_container.config().registry_authfile) | ||
| # http_authorizer = providers.Singleton(HTTPAuthorizer, create_http_authorizer) |
There was a problem hiding this comment.
should commented out stuff be removed?
| set_health_status_tracing_attributes(health_check_span=health_check_span, health_status=health_status) | ||
|
|
||
| # if self._auth is not None: | ||
| # resp["schema_registry_authfile_timestamp"] = self._auth.authfile_last_modified |
| telemetry_resource: Resource = Provide[TelemetryContainer.telemetry_resource], | ||
| ) -> None: | ||
| LOG.info("Setting OTel meter provider") | ||
| # metrics.set_meter_provider(MeterProvider(resource=telemetry_resource, metric_readers=[meter.get_metric_reader()])) |
| controller: KarapaceSchemaRegistryController = Depends(Provide[SchemaRegistryContainer.schema_registry_controller]), | ||
| ) -> CompatibilityLevelResponse: | ||
| subject = Subject(unquote_plus(subject)) | ||
| if authorizer and not authorizer.check_authorization(user, Operation.Read, f"Subject:{subject}"): |
There was a problem hiding this comment.
why do we have to check for authorizer here? Shouldn't this be either instance of HTTPAuthorizer or NoAuthAndAuthz?
| ) | ||
|
|
||
|
|
||
| # TODO is this needed? Is this actually the ids/schema/id/schema?? |
| # controller: KarapaceSchemaRegistryControllerDep, | ||
| # ) -> SchemasResponse: | ||
| # # TODO retrieve by id only schema | ||
| # return await controller.schemas_get() |
| # f"{new_config['advertised_protocol']}://{new_config['advertised_hostname']}:{new_config['advertised_port']}" | ||
|
|
||
| # set tags if not set | ||
| # new_config["tags"]["app"] = "Karapace" |
| # env_name, | ||
| # ) | ||
| # | ||
| # config[config_name] = parse_env_value(env_val) |
|
As agreed with @jjaakola-aiven, merging this one |
|
wow amazing! Congrats for the awesome work @jjaakola-aiven |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
About this change - What it does
Karapace modernization to use uvicorn, FastAPI and OpenTelemetry.
Why this way
The replacement of
rapuandaiohttpis required for having more feature complete framework. The FastAPI enables the use of community plugins that are easier to integrate.The new and changed code is reorganized into directories:
src/karapace/api- Schema registry FastAPI server and routerssrc/karapace/core- Schema registry core enginesrc/karapace/kafka_rest_apis- REST ProxyEntry points:
src/karapace/kafka_rest_apis/__main__.py- REST Proxy main entry pointsrc/karapace/__main__.py- Schema registry main entry pointImprovements to do later
rapuandaiohttp.src/karapace/kafka_rest_apis.