Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated installation notes #1042

Merged
merged 3 commits into from
Feb 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 60 additions & 11 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -83,15 +83,61 @@ what you need to change.

.. _`Here`: https://github.com/Aiven-Open/karapace/blob/master/karapace.config.json

Source install
--------------
Using Sources
-------------

Install
^^^^^^^

Alternatively you can do a source install using::
You can do a source install using::

pip install .

Quickstart
==========
Troubleshooting notes :
- An updated version of wheel (https://pypi.org/project/wheel/) is required.
- An updated version of 'go' and 'rust' is required
- Create and activate virtual environment (venv) to manage dependencies

Run
^^^
- Make sure kafka is running.

Start Karapace. This shout start karapace on http://localhost:8081 ::

$ karapace karapace.config.json

Verify in browser http://localhost:8081/subjects should return an array of subjects if exist or an empty array.
or with curl ::

$ curl -X GET http://localhost:8081/subjects

Start Karapace rest proxy. This shout start karapace on http://localhost:8082 ::

karapace rest-proxy-karapace.config.json

To enable authorization & authentication on the rest proxy, configure 'sasl_mechanism' in the config with values like PLAIN/OAUTHBEARER ::

sasl_mechanism = "OAUTHBEARER",
sasl_oauth_token_provider = token_provider,
security_protocol="SASL_SSL",
ssl_cafile="ca.pem",

If 'sasl_mechanism' is configured to PLAIN::

sasl_mechanism = "PLAIN",
security_protocol = "SASL_PLAIN",
sasl_plain_username = "your_username",
sasl_plain_password = "your_password"

There is a detailed section about OAuth2 authentication below.

Verify with list topics::

$ curl "http://localhost:8082/topics"


Schema Registry Api reference
=============================

To register the first version of a schema under the subject "test" using Avro schema::

Expand Down Expand Up @@ -171,6 +217,9 @@ Change compatibility requirement to FULL for the test-key subject::
--data '{"compatibility": "FULL"}' http://localhost:8081/config/test-key
{"compatibility":"FULL"}

Schema Registry Rest proxy Api reference
========================================

List topics::

$ curl "http://localhost:8082/topics"
Expand All @@ -185,18 +234,18 @@ Produce a message backed up by schema registry::
'{"value_schema": "{\"namespace\": \"example.avro\", \"type\": \"record\", \"name\": \"simple\", \"fields\": \
[{\"name\": \"name\", \"type\": \"string\"}]}", "records": [{"value": {"name": "name0"}}]}' http://localhost:8082/topics/my_topic

Create a consumer::
Create a consumer with consumer group 'avro_consumers' and consumer instance 'my_consumer' ::

$ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" -H "Accept: application/vnd.kafka.v2+json" \
--data '{"name": "my_consumer", "format": "avro", "auto.offset.reset": "earliest"}' \
http://localhost:8082/consumers/avro_consumers

Subscribe to the topic we previously published to::
Subscribe to the topic we previously created ::

$ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data '{"topics":["my_topic"]}' \
http://localhost:8082/consumers/avro_consumers/instances/my_consumer/subscription

Consume previously published message::
Consume previously produced message::

$ curl -X GET -H "Accept: application/vnd.kafka.avro.v2+json" \
http://localhost:8082/consumers/avro_consumers/instances/my_consumer/records?timeout=1000
Expand Down Expand Up @@ -596,10 +645,10 @@ Example of complete authorization file
]
}

Karapace Schema Registry access to the schemas topic
Karapace Schema Registry access to the _schemas topic
====================================================

The principal used by the Karapace Schema Registry has to have adequate access to the schemas topic (see the ``topic_name`` configuration option above).
The principal used by the Karapace Schema Registry has to have adequate access to the _schemas topic (see the ``topic_name`` configuration option above).
In addition to what is required to access the topic, as described in the Confluent Schema Registry documentation_, the unique, single-member consumer group
used by consumers in the schema registry needs ``Describe`` and ``Read`` permissions_ on the group.
These unique (per instance of the schema registry) consumer group names are prefixed by ``karapace-autogenerated-``, followed by a random string.
Expand Down Expand Up @@ -644,7 +693,7 @@ In that view the future extension of the normalization process isn't considered
Uninstall
=========

To unistall Karapace from the system you can follow the instructions described below. We would love to hear your reasons for uninstalling though. Please file an issue if you experience any problems or email us_ with feedback
To uninstall Karapace from the system, you can follow the instructions described below. We would love to hear your reasons for uninstalling though. Please file an issue if you experience any problems or email us_ with feedback

.. _`us`: mailto:[email protected]

Expand Down
8 changes: 8 additions & 0 deletions rest-proxy-karapace.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"host": "127.0.0.1",
"port": 8082,
"advertised_hostname": "localhost",
"log_level": "INFO",
"kafka_bootstrap_servers": "localhost:9092",
"karapace_rest": true
}
6 changes: 5 additions & 1 deletion website/source/about.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
About Karapace
==============

There are lots of reasons to use schemas alongside your Kafka payloads; Karapace gives you an open source solution for handling your schemas.
There are lots of reasons to use schemas alongside your Kafka payloads; Karapace gives you an open source solution for handling your schemas for the following data formats

- Json
- Avro
- Protobuf
25 changes: 25 additions & 0 deletions website/source/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,28 @@ Source install
Alternatively you can do a source install using::

pip install .

Troubleshooting notes :
- An updated version of wheel (https://pypi.org/project/wheel/) is required.
- Create and activate virtual environment (venv) to manage dependencies

Run
^^^
- Make sure kafka is running.

Start Karapace. This shout start karapace on http://localhost:8081 ::

$ karapace karapace.config.json

Verify in browser http://localhost:8081/subjects should return an array of subjects if exist or an empty array.
or with curl ::

$ curl -X GET http://localhost:8081/subjects

Start Karapace rest proxy. This shout start karapace on http://localhost:8082 ::

karapace rest-proxy-karapace.config.json

Verify with list topics::

$ curl "http://localhost:8082/topics"
12 changes: 6 additions & 6 deletions website/source/quickstart.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Quickstart examples
===================
Api References
==============

Schema Hub
----------
Schema Registry
---------------

To register the first version of a schema under the subject "test" using Avro schema::

Expand Down Expand Up @@ -83,8 +83,8 @@ Change compatibility requirement to FULL for the test-key subject::
{"compatibility":"FULL"}


REST API
---------
REST Proxy
----------

List topics::

Expand Down
Loading