# cmem-plugin-kafka
Send and receive messages from Apache Kafka.
[![eccenca Corporate Memory][cmem-shield]][cmem-link]
This is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com). You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line client like this:
```
cmemc admin workspace python install cmem-plugin-kafka
```
[](https://github.com/eccenca/cmem-plugin-kafka/actions) [](https://pypi.org/project/cmem-plugin-kafka) [](https://pypi.org/project/cmem-plugin-kafka)
[![poetry][poetry-shield]][poetry-link] [![ruff][ruff-shield]][ruff-link] [![mypy][mypy-shield]][mypy-link] [![copier][copier-shield]][copier]
[cmem-link]: https://documentation.eccenca.com
[cmem-shield]: https://img.shields.io/endpoint?url=https://dev.documentation.eccenca.com/badge.json
[poetry-link]: https://python-poetry.org/
[poetry-shield]: https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json
[ruff-link]: https://docs.astral.sh/ruff/
[ruff-shield]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json&label=Code%20Style
[mypy-link]: https://mypy-lang.org/
[mypy-shield]: https://www.mypy-lang.org/static/mypy_badge.svg
[copier]: https://copier.readthedocs.io/
[copier-shield]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/copier-org/copier/master/img/badge/badge-grayscale-inverted-border-purple.json
## Plugin supported message format
### XML dataset format
An example XML document is shown below. This document will be sent as two messages
to the configured topic. Each message is created as a proper XML document.
```xml
<?xml version="1.0" encoding="utf-8"?>
<KafkaMessages>
<Message>
<PurchaseOrder OrderDate="1996-04-06">
<ShipTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>9200</zip>
</ShipTo>
<BillTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>2381</zip>
</BillTo>
</PurchaseOrder>
</Message>
<Message key="1234">
<SingleTagHere>
.
.
.
</SingleTagHere>
</Message>
</KafkaMessages>
```
Producer plugin generates 2 messages with below content
```xml
<?xml version="1.0" encoding="utf-8"?>
<PurchaseOrder OrderDate="1996-04-06">
<ShipTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>9200</zip>
</ShipTo>
<BillTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>2381</zip>
</BillTo>
</PurchaseOrder>
```
```xml
<?xml version="1.0" encoding="utf-8"?>
<SingleTagHere>
.
.
.
</SingleTagHere>
```
### JSON Dataset format
An example JSON document is shown below. This document will be sent as two messages
to the configured topic. Each message is created as a proper JSON document.
```json
[
{
"message": {
"key": "818432-942813-832642-453478",
"headers": {
"type": "ADD"
},
"content": {
"location": [
"Leipzig"
],
"obstacle": {
"name": "Iron Bars",
"order": "1"
}
}
}
},
{
"message": {
"key": "887428-119918-570674-866526",
"headers": {
"type": "REMOVE"
},
"content": {
"comments": "We can pass any json payload here."
}
}
}
]
```
Producer plugin generates 2 messages with below content
```json
{
"location": [
"Leipzig"
],
"obstacle": {
"name": "Iron Bars",
"order": "1"
}
}
```
```json
{
"comments": "We can pass any json payload here."
}
```
### Entities format
Random values plugin entities will generate below format JSON document.
```json
{
"schema": {
"type_uri": "https://example.org/vocab/RandomValueRow"
},
"entity": {
"uri": "urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f",
"values": {
"<https://example.org/vocab/RandomValuePath0>": [
"a8o4Ocsb6RZClFRUZU3b2w"
],
"<https://example.org/vocab/RandomValuePath1>": [
"RTICRU7JcTUVn94decelPg"
],
"<https://example.org/vocab/RandomValuePath2>": [
"A9r-969NjAlX0DNWftxKoA"
],
"<https://example.org/vocab/RandomValuePath3>": [
"FygWRy1UJ4-IzIim1qukJA"
],
"<https://example.org/vocab/RandomValuePath4>": [
"AJcbn-LJEs-Dif96xu2eww"
]
}
}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/eccenca/cmem-plugin-kafka",
"name": "cmem-plugin-kafka",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.13",
"maintainer_email": null,
"keywords": "eccenca Corporate Memory, plugin, kafka, kafka-producer, kafka-consumer",
"author": "eccenca GmbH",
"author_email": "cmempy-developer@eccenca.com",
"download_url": "https://files.pythonhosted.org/packages/fe/f4/d353710804b96585950a8500d08ea08f50f93c5538fcb29dab46f785a712/cmem_plugin_kafka-3.4.0.tar.gz",
"platform": null,
"description": "# cmem-plugin-kafka\n\nSend and receive messages from Apache Kafka.\n\n[![eccenca Corporate Memory][cmem-shield]][cmem-link]\n\nThis is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com). You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line client like this:\n\n```\ncmemc admin workspace python install cmem-plugin-kafka\n```\n[](https://github.com/eccenca/cmem-plugin-kafka/actions) [](https://pypi.org/project/cmem-plugin-kafka) [](https://pypi.org/project/cmem-plugin-kafka)\n[![poetry][poetry-shield]][poetry-link] [![ruff][ruff-shield]][ruff-link] [![mypy][mypy-shield]][mypy-link] [![copier][copier-shield]][copier] \n\n[cmem-link]: https://documentation.eccenca.com\n[cmem-shield]: https://img.shields.io/endpoint?url=https://dev.documentation.eccenca.com/badge.json\n[poetry-link]: https://python-poetry.org/\n[poetry-shield]: https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json\n[ruff-link]: https://docs.astral.sh/ruff/\n[ruff-shield]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json&label=Code%20Style\n[mypy-link]: https://mypy-lang.org/\n[mypy-shield]: https://www.mypy-lang.org/static/mypy_badge.svg\n[copier]: https://copier.readthedocs.io/\n[copier-shield]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/copier-org/copier/master/img/badge/badge-grayscale-inverted-border-purple.json\n\n## Plugin supported message format\n\n### XML dataset format\n\nAn example XML document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper XML document.\n\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<KafkaMessages>\n <Message>\n <PurchaseOrder OrderDate=\"1996-04-06\">\n <ShipTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>9200</zip>\n </ShipTo>\n <BillTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>2381</zip>\n </BillTo>\n </PurchaseOrder>\n </Message>\n <Message key=\"1234\">\n <SingleTagHere>\n .\n .\n .\n </SingleTagHere>\n </Message>\n</KafkaMessages>\n```\nProducer plugin generates 2 messages with below content\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<PurchaseOrder OrderDate=\"1996-04-06\">\n <ShipTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>9200</zip>\n </ShipTo>\n <BillTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>2381</zip>\n </BillTo>\n</PurchaseOrder>\n```\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<SingleTagHere>\n .\n .\n .\n</SingleTagHere>\n```\n### JSON Dataset format\n\nAn example JSON document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper JSON document.\n\n```json\n[\n {\n \"message\": {\n \"key\": \"818432-942813-832642-453478\",\n \"headers\": {\n \"type\": \"ADD\"\n },\n \"content\": {\n \"location\": [\n \"Leipzig\"\n ],\n \"obstacle\": {\n \"name\": \"Iron Bars\",\n \"order\": \"1\"\n }\n }\n }\n },\n {\n \"message\": {\n \"key\": \"887428-119918-570674-866526\",\n \"headers\": {\n \"type\": \"REMOVE\"\n },\n \"content\": {\n \"comments\": \"We can pass any json payload here.\"\n }\n }\n }\n]\n```\nProducer plugin generates 2 messages with below content\n```json\n{\n \"location\": [\n \"Leipzig\"\n ],\n \"obstacle\": {\n \"name\": \"Iron Bars\",\n \"order\": \"1\"\n }\n}\n```\n```json\n{\n \"comments\": \"We can pass any json payload here.\"\n}\n```\n### Entities format\n\nRandom values plugin entities will generate below format JSON document.\n\n```json\n{\n \"schema\": {\n \"type_uri\": \"https://example.org/vocab/RandomValueRow\"\n },\n \"entity\": {\n \"uri\": \"urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f\",\n \"values\": {\n \"<https://example.org/vocab/RandomValuePath0>\": [\n \"a8o4Ocsb6RZClFRUZU3b2w\"\n ],\n \"<https://example.org/vocab/RandomValuePath1>\": [\n \"RTICRU7JcTUVn94decelPg\"\n ],\n \"<https://example.org/vocab/RandomValuePath2>\": [\n \"A9r-969NjAlX0DNWftxKoA\"\n ],\n \"<https://example.org/vocab/RandomValuePath3>\": [\n \"FygWRy1UJ4-IzIim1qukJA\"\n ],\n \"<https://example.org/vocab/RandomValuePath4>\": [\n \"AJcbn-LJEs-Dif96xu2eww\"\n ]\n }\n }\n}\n```",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Send and receive messages from Apache Kafka.",
"version": "3.4.0",
"project_urls": {
"Homepage": "https://github.com/eccenca/cmem-plugin-kafka"
},
"split_keywords": [
"eccenca corporate memory",
" plugin",
" kafka",
" kafka-producer",
" kafka-consumer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ae3386a5f8edcfa4e3c82f790524e602fa944c0926ccc2f2f9628bad3b429632",
"md5": "11e2ea852cee78a5be1101b1021f02e6",
"sha256": "4becfae5986ae8a4342c66a53f8b0f2606430d34c4be8697513e23b73b5300db"
},
"downloads": -1,
"filename": "cmem_plugin_kafka-3.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "11e2ea852cee78a5be1101b1021f02e6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.13",
"size": 24566,
"upload_time": "2025-10-20T12:50:36",
"upload_time_iso_8601": "2025-10-20T12:50:36.314344Z",
"url": "https://files.pythonhosted.org/packages/ae/33/86a5f8edcfa4e3c82f790524e602fa944c0926ccc2f2f9628bad3b429632/cmem_plugin_kafka-3.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fef4d353710804b96585950a8500d08ea08f50f93c5538fcb29dab46f785a712",
"md5": "8be55a49b5c732398e3645c5bc934e11",
"sha256": "15793c20f30b24799efbe409580d9a4470208173179f0a8c0166d21fc0c409cd"
},
"downloads": -1,
"filename": "cmem_plugin_kafka-3.4.0.tar.gz",
"has_sig": false,
"md5_digest": "8be55a49b5c732398e3645c5bc934e11",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.13",
"size": 21547,
"upload_time": "2025-10-20T12:50:37",
"upload_time_iso_8601": "2025-10-20T12:50:37.328987Z",
"url": "https://files.pythonhosted.org/packages/fe/f4/d353710804b96585950a8500d08ea08f50f93c5538fcb29dab46f785a712/cmem_plugin_kafka-3.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-20 12:50:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eccenca",
"github_project": "cmem-plugin-kafka",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "cmem-plugin-kafka"
}
JSON
