diff --git a/README.rst b/README.rst index 73919ec34..e06643756 100644 --- a/README.rst +++ b/README.rst @@ -83,15 +83,61 @@ what you need to change. .. _`Here`: https://github.com/Aiven-Open/karapace/blob/master/karapace.config.json -Source install --------------- +Using Sources +------------- + +Install +^^^^^^^ -Alternatively you can do a source install using:: +You can do a source install using:: pip install . -Quickstart -========== +Troubleshooting notes : +- An updated version of wheel (https://pypi.org/project/wheel/) is required. +- An updated version of 'go' and 'rust' is required +- Create and activate virtual environment (venv) to manage dependencies + +Run +^^^ +- Make sure kafka is running. + +Start Karapace. This shout start karapace on http://localhost:8081 :: + + $ karapace karapace.config.json + +Verify in browser http://localhost:8081/subjects should return an array of subjects if exist or an empty array. +or with curl :: + + $ curl -X GET http://localhost:8081/subjects + +Start Karapace rest proxy. This shout start karapace on http://localhost:8082 :: + + karapace rest-proxy-karapace.config.json + +To enable authorization & authentication on the rest proxy, configure 'sasl_mechanism' in the config with values like PLAIN/OAUTHBEARER :: + + sasl_mechanism = "OAUTHBEARER", + sasl_oauth_token_provider = token_provider, + security_protocol="SASL_SSL", + ssl_cafile="ca.pem", + +If 'sasl_mechanism' is configured to PLAIN:: + + sasl_mechanism = "PLAIN", + security_protocol = "SASL_PLAIN", + sasl_plain_username = "your_username", + sasl_plain_password = "your_password" + +There is a detailed section about OAuth2 authentication below. + +Verify with list topics:: + + $ curl "http://localhost:8082/topics" + + +Schema Registry Api reference +============================= To register the first version of a schema under the subject "test" using Avro schema:: @@ -171,6 +217,9 @@ Change compatibility requirement to FULL for the test-key subject:: --data '{"compatibility": "FULL"}' http://localhost:8081/config/test-key {"compatibility":"FULL"} +Schema Registry Rest proxy Api reference +======================================== + List topics:: $ curl "http://localhost:8082/topics" @@ -185,18 +234,18 @@ Produce a message backed up by schema registry:: '{"value_schema": "{\"namespace\": \"example.avro\", \"type\": \"record\", \"name\": \"simple\", \"fields\": \ [{\"name\": \"name\", \"type\": \"string\"}]}", "records": [{"value": {"name": "name0"}}]}' http://localhost:8082/topics/my_topic -Create a consumer:: +Create a consumer with consumer group 'avro_consumers' and consumer instance 'my_consumer' :: $ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" -H "Accept: application/vnd.kafka.v2+json" \ --data '{"name": "my_consumer", "format": "avro", "auto.offset.reset": "earliest"}' \ http://localhost:8082/consumers/avro_consumers -Subscribe to the topic we previously published to:: +Subscribe to the topic we previously created :: $ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data '{"topics":["my_topic"]}' \ http://localhost:8082/consumers/avro_consumers/instances/my_consumer/subscription -Consume previously published message:: +Consume previously produced message:: $ curl -X GET -H "Accept: application/vnd.kafka.avro.v2+json" \ http://localhost:8082/consumers/avro_consumers/instances/my_consumer/records?timeout=1000 @@ -596,10 +645,10 @@ Example of complete authorization file ] } -Karapace Schema Registry access to the schemas topic +Karapace Schema Registry access to the _schemas topic ==================================================== -The principal used by the Karapace Schema Registry has to have adequate access to the schemas topic (see the ``topic_name`` configuration option above). +The principal used by the Karapace Schema Registry has to have adequate access to the _schemas topic (see the ``topic_name`` configuration option above). In addition to what is required to access the topic, as described in the Confluent Schema Registry documentation_, the unique, single-member consumer group used by consumers in the schema registry needs ``Describe`` and ``Read`` permissions_ on the group. These unique (per instance of the schema registry) consumer group names are prefixed by ``karapace-autogenerated-``, followed by a random string. @@ -644,7 +693,7 @@ In that view the future extension of the normalization process isn't considered Uninstall ========= -To unistall Karapace from the system you can follow the instructions described below. We would love to hear your reasons for uninstalling though. Please file an issue if you experience any problems or email us_ with feedback +To uninstall Karapace from the system, you can follow the instructions described below. We would love to hear your reasons for uninstalling though. Please file an issue if you experience any problems or email us_ with feedback .. _`us`: mailto:opensource@aiven.io diff --git a/rest-proxy-karapace.config.json b/rest-proxy-karapace.config.json new file mode 100644 index 000000000..a4a4ddded --- /dev/null +++ b/rest-proxy-karapace.config.json @@ -0,0 +1,8 @@ +{ + "host": "127.0.0.1", + "port": 8082, + "advertised_hostname": "localhost", + "log_level": "INFO", + "kafka_bootstrap_servers": "localhost:9092", + "karapace_rest": true +} diff --git a/website/source/about.rst b/website/source/about.rst index ab491583a..026975101 100644 --- a/website/source/about.rst +++ b/website/source/about.rst @@ -1,4 +1,8 @@ About Karapace ============== -There are lots of reasons to use schemas alongside your Kafka payloads; Karapace gives you an open source solution for handling your schemas. +There are lots of reasons to use schemas alongside your Kafka payloads; Karapace gives you an open source solution for handling your schemas for the following data formats + +- Json +- Avro +- Protobuf diff --git a/website/source/install.rst b/website/source/install.rst index ac6d5459f..d3b3eef73 100644 --- a/website/source/install.rst +++ b/website/source/install.rst @@ -38,3 +38,28 @@ Source install Alternatively you can do a source install using:: pip install . + +Troubleshooting notes : +- An updated version of wheel (https://pypi.org/project/wheel/) is required. +- Create and activate virtual environment (venv) to manage dependencies + +Run +^^^ +- Make sure kafka is running. + +Start Karapace. This shout start karapace on http://localhost:8081 :: + + $ karapace karapace.config.json + +Verify in browser http://localhost:8081/subjects should return an array of subjects if exist or an empty array. +or with curl :: + + $ curl -X GET http://localhost:8081/subjects + +Start Karapace rest proxy. This shout start karapace on http://localhost:8082 :: + + karapace rest-proxy-karapace.config.json + +Verify with list topics:: + + $ curl "http://localhost:8082/topics" diff --git a/website/source/quickstart.rst b/website/source/quickstart.rst index f640e68d2..18c091ce6 100644 --- a/website/source/quickstart.rst +++ b/website/source/quickstart.rst @@ -1,8 +1,8 @@ -Quickstart examples -=================== +Api References +============== -Schema Hub ----------- +Schema Registry +--------------- To register the first version of a schema under the subject "test" using Avro schema:: @@ -83,8 +83,8 @@ Change compatibility requirement to FULL for the test-key subject:: {"compatibility":"FULL"} -REST API ---------- +REST Proxy +---------- List topics::