# Kafka Ease
Automating the creation of topics and ACLs is a common task for Kafka administrators. This repository contains a set of utilities to automate these tasks.
- Create a topic with a given number of partitions and replication factor
- Create ACLs for a given topic
[](https://github.com/rdomenzain/kafka-ease/actions/workflows/main.yml)
## Contents
- [Kafka Ease](#kafka-ease)
- [Contents](#contents)
- [Getting started](#getting-started)
- [Installation](#installation)
- [requirements.txt](#requirementstxt)
- [Pip install](#pip-install)
- [How to use](#how-to-use)
- [File format](#file-format)
- [Validate configuration](#validate-configuration)
- [Apply configuration](#apply-configuration)
- [Help](#help)
- [Changelog](#changelog)
## Getting started
### Installation
#### requirements.txt
To install the library from the GitLab package registry, write the following lines in the file `requirements.txt`:
```text
kafka-ease==[VERSION]
```
#### Pip install
To install the library, run the following command:
```bash
pip install kafka-ease==[VERSION]
```
Check [PEP 440](https://www.python.org/dev/peps/pep-0440/) for version schemes and version specifiers.
## How to use
### File format
The file format is YAML or JSON. The file must contain a list of topics and a list of ACLs.
```yaml
# acl.yaml
topics:
- name: topic.name
num_partitions: 1
replication_factor: 1
cleanup_policy: delete
retention_days: 7
acls:
- resource_type: TOPIC
resource_name: topic.name
principal: User:kafka-user
host: "*"
operation: READ
permission_type: ALLOW
pattern_type: LITERAL
```
```json
{
"topics": [
{
"name": "topic.name",
"num_partitions": 1,
"replication_factor": 1,
"cleanup_policy": "delete",
"retention_days": 7
}
],
"acls": [
{
"resource_type": "TOPIC",
"resource_name": "topic.name",
"principal": "User:kafka-user",
"host": "*",
"operation": "READ",
"permission_type": "ALLOW",
"pattern_type": "LITERAL"
}
]
}
```
- Topics
- **name**: Topic name
- **num_partitions**: Number of partitions
- **replication_factor**: Replication factor
- **cleanup_policy**: Cleanup policy (`delete`, `compact`)
- **retention_days**: Delete retention days
- ACLs
- **resource_type**: Resource type (`UNKNOWN`, `ANY`, `CLUSTER`, `TOPIC`, `DELEGATION_TOKEN`, `GROUP`, `TRANSACTIONAL_ID`)
- **resource_name**: Resource name
- **principal**: Principal (`User:`, `Group:`)
- **host**: Host (`*`)
- **operation**: Operation (`ANY`, `ALL`, `READ`, `WRITE`, `CREATE`, `DELETE`, `ALTER`, `DESCRIBE`, `CLUSTER_ACTION`, `DESCRIBE_CONFIGS`, `ALTER_CONFIGS`, `IDEMPOTENT_WRITE`)
- **permission_type**: Permission type (`ANY`, `DENY`, `ALLOW`)
- **pattern_type**: Pattern type (`ANY`, `MATCH`, `LITERAL`, `PREFIXED`)
### Validate configuration
```shell
kafka-ease apply -f acl.yaml --only-validate
```
Expected output:
```text
Validating file acl.yaml...
File: acl.yaml
YAML file detected
2 topics found.
2 ACLs found.
File acl.yaml is valid.
```
### Apply configuration
```shell
kafka-ease apply -f acl.yaml \
--kafka-brokers kafka:9093 \
--security-protocol SASL_PLAINTEXT \
--sasl-mechanism SCRAM-SHA-256 \
--sasl-username kafka-admin \
--sasl-password SECRET
```
Expected output:
```text
Applying file...
Kafka brokers: kafka:9093
Security protocol: SASL_PLAINTEXT
SASL mechanism: SCRAM-SHA-256
SASL username: kafka-admin
SASL password: ***************
File: acl.yaml
YAML file detected
1 topics found.
1 ACLs found.
Topic topic.name updated
Removing old ACLs 3
ACL User:kafka-user synced
File synced successfully.
```
### Help
```shell
kafka-ease apply --help
```
Expected output:
```text
Usage: kafka-ease apply [OPTIONS]
Apply configuration file with topics and ACL to Kafka.
Options:
--kafka-brokers TEXT Kafka server to connect to use.
--security-protocol [PLAINTEXT|SASL_PLAINTEXT]
Kafka Security protocol to use.
--sasl-mechanism [PLAIN|SCRAM-SHA-256|SCRAM-SHA-512]
SASL mechanism to use.
--sasl-username TEXT SASL username to use.
--sasl-password TEXT SASL password to use.
-f, --file TEXT File path to validate
--only-validate Only validate the file
--help Show this message and exit.
```
## Changelog
- 1.0.0 (2023-09-20)
> Initial release of Kafka Ease
- 1.0.1 (2023-09-20)
> Fix packaging error
- 1.0.2 (2023-09-20)
> Fix minor bugs
Raw data
{
"_id": null,
"home_page": "https://github.com/rdomenzain/kafka-ease",
"name": "kafka-ease",
"maintainer": "Ricardo Domenzain",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "richydomenzain@gmail.com",
"keywords": "kafka,kafka-topics,kafka-acls,kafka-administration,kafka-automation",
"author": "Ricardo Domenzain",
"author_email": "richydomenzain@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/8a/b0/5d35fe661c943636ebcaa36d9dd8052a72985474490e396689fc8700463c/kafka-ease-1.0.2.tar.gz",
"platform": null,
"description": "# Kafka Ease\n\nAutomating the creation of topics and ACLs is a common task for Kafka administrators. This repository contains a set of utilities to automate these tasks.\n\n- Create a topic with a given number of partitions and replication factor\n- Create ACLs for a given topic\n\n\n\n[](https://github.com/rdomenzain/kafka-ease/actions/workflows/main.yml)\n\n## Contents\n\n- [Kafka Ease](#kafka-ease)\n - [Contents](#contents)\n - [Getting started](#getting-started)\n - [Installation](#installation)\n - [requirements.txt](#requirementstxt)\n - [Pip install](#pip-install)\n - [How to use](#how-to-use)\n - [File format](#file-format)\n - [Validate configuration](#validate-configuration)\n - [Apply configuration](#apply-configuration)\n - [Help](#help)\n - [Changelog](#changelog)\n\n## Getting started\n\n### Installation\n\n#### requirements.txt\n\nTo install the library from the GitLab package registry, write the following lines in the file `requirements.txt`:\n\n```text\nkafka-ease==[VERSION]\n```\n\n#### Pip install\n\nTo install the library, run the following command:\n\n```bash\npip install kafka-ease==[VERSION]\n```\n\nCheck [PEP 440](https://www.python.org/dev/peps/pep-0440/) for version schemes and version specifiers.\n\n## How to use\n\n### File format\n\nThe file format is YAML or JSON. The file must contain a list of topics and a list of ACLs.\n\n```yaml\n# acl.yaml\ntopics:\n - name: topic.name\n num_partitions: 1\n replication_factor: 1\n cleanup_policy: delete\n retention_days: 7\n\nacls:\n - resource_type: TOPIC\n resource_name: topic.name\n principal: User:kafka-user\n host: \"*\"\n operation: READ\n permission_type: ALLOW\n pattern_type: LITERAL\n```\n\n```json\n{\n \"topics\": [\n {\n \"name\": \"topic.name\",\n \"num_partitions\": 1,\n \"replication_factor\": 1,\n \"cleanup_policy\": \"delete\",\n \"retention_days\": 7\n }\n ],\n \"acls\": [\n {\n \"resource_type\": \"TOPIC\",\n \"resource_name\": \"topic.name\",\n \"principal\": \"User:kafka-user\",\n \"host\": \"*\",\n \"operation\": \"READ\",\n \"permission_type\": \"ALLOW\",\n \"pattern_type\": \"LITERAL\"\n }\n ]\n}\n```\n\n- Topics\n - **name**: Topic name\n - **num_partitions**: Number of partitions\n - **replication_factor**: Replication factor\n - **cleanup_policy**: Cleanup policy (`delete`, `compact`)\n - **retention_days**: Delete retention days\n\n- ACLs\n - **resource_type**: Resource type (`UNKNOWN`, `ANY`, `CLUSTER`, `TOPIC`, `DELEGATION_TOKEN`, `GROUP`, `TRANSACTIONAL_ID`)\n - **resource_name**: Resource name\n - **principal**: Principal (`User:`, `Group:`)\n - **host**: Host (`*`)\n - **operation**: Operation (`ANY`, `ALL`, `READ`, `WRITE`, `CREATE`, `DELETE`, `ALTER`, `DESCRIBE`, `CLUSTER_ACTION`, `DESCRIBE_CONFIGS`, `ALTER_CONFIGS`, `IDEMPOTENT_WRITE`)\n - **permission_type**: Permission type (`ANY`, `DENY`, `ALLOW`)\n - **pattern_type**: Pattern type (`ANY`, `MATCH`, `LITERAL`, `PREFIXED`)\n\n### Validate configuration\n\n```shell\nkafka-ease apply -f acl.yaml --only-validate\n```\n\nExpected output:\n\n```text\nValidating file acl.yaml...\nFile: acl.yaml\nYAML file detected\n2 topics found.\n2 ACLs found.\nFile acl.yaml is valid.\n```\n\n### Apply configuration\n\n```shell\nkafka-ease apply -f acl.yaml \\\n --kafka-brokers kafka:9093 \\\n --security-protocol SASL_PLAINTEXT \\ \n --sasl-mechanism SCRAM-SHA-256 \\\n --sasl-username kafka-admin \\\n --sasl-password SECRET\n```\n\nExpected output:\n\n```text\nApplying file...\nKafka brokers: kafka:9093\nSecurity protocol: SASL_PLAINTEXT\nSASL mechanism: SCRAM-SHA-256\nSASL username: kafka-admin\nSASL password: ***************\nFile: acl.yaml\nYAML file detected\n1 topics found.\n1 ACLs found.\nTopic topic.name updated\nRemoving old ACLs 3\nACL User:kafka-user synced\nFile synced successfully.\n```\n\n### Help\n\n```shell\nkafka-ease apply --help\n```\n\nExpected output:\n\n```text\nUsage: kafka-ease apply [OPTIONS]\n\n Apply configuration file with topics and ACL to Kafka.\n\nOptions:\n --kafka-brokers TEXT Kafka server to connect to use.\n --security-protocol [PLAINTEXT|SASL_PLAINTEXT]\n Kafka Security protocol to use.\n --sasl-mechanism [PLAIN|SCRAM-SHA-256|SCRAM-SHA-512]\n SASL mechanism to use.\n --sasl-username TEXT SASL username to use.\n --sasl-password TEXT SASL password to use.\n -f, --file TEXT File path to validate\n --only-validate Only validate the file\n --help Show this message and exit.\n```\n\n## Changelog\n\n- 1.0.0 (2023-09-20)\n > Initial release of Kafka Ease\n- 1.0.1 (2023-09-20)\n > Fix packaging error\n- 1.0.2 (2023-09-20)\n > Fix minor bugs\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "Automating the creation of topics and ACLs is a common task for Kafka administrators. This repository contains a set of utilities to automate these tasks.",
"version": "1.0.2",
"project_urls": {
"Bug Tracker": "https://github.com/rdomenzain/kafka-ease/issues",
"Changelog": "https://github.com/rdomenzain/kafka-ease/releases",
"Homepage": "https://github.com/rdomenzain/kafka-ease"
},
"split_keywords": [
"kafka",
"kafka-topics",
"kafka-acls",
"kafka-administration",
"kafka-automation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a43cecc7ff065916d38acb9b3aa458bacaf98f4ab442e2102b1ecb2ac8508c8b",
"md5": "02b6a158a2babe6062c2a34a648f970d",
"sha256": "3da58034abf260dbbe1f6b5a5f7ffa3e812dab4c29e543605e0bfa60254208c9"
},
"downloads": -1,
"filename": "kafka_ease-1.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "02b6a158a2babe6062c2a34a648f970d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 21202,
"upload_time": "2023-09-20T21:11:23",
"upload_time_iso_8601": "2023-09-20T21:11:23.269033Z",
"url": "https://files.pythonhosted.org/packages/a4/3c/ecc7ff065916d38acb9b3aa458bacaf98f4ab442e2102b1ecb2ac8508c8b/kafka_ease-1.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8ab05d35fe661c943636ebcaa36d9dd8052a72985474490e396689fc8700463c",
"md5": "d9097118e4a7ac5953075b04d378edc2",
"sha256": "59882daca65ce88700087443862b62d4839a1cbb347b69e881c9b7169cd8af69"
},
"downloads": -1,
"filename": "kafka-ease-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "d9097118e4a7ac5953075b04d378edc2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 28090,
"upload_time": "2023-09-20T21:11:24",
"upload_time_iso_8601": "2023-09-20T21:11:24.784484Z",
"url": "https://files.pythonhosted.org/packages/8a/b0/5d35fe661c943636ebcaa36d9dd8052a72985474490e396689fc8700463c/kafka-ease-1.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-20 21:11:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rdomenzain",
"github_project": "kafka-ease",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "kafka-ease"
}
JSON
