# cmem-plugin-kafka
Send and receive messages from Apache Kafka.
This is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com).
You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line
clients like this:
```
cmemc admin workspace python install cmem-plugin-kafka
```
## Plugin supported message format
### XML dataset format
An example XML document is shown below. This document will be sent as two messages
to the configured topic. Each message is created as a proper XML document.
```xml
<?xml version="1.0" encoding="utf-8"?>
<KafkaMessages>
<Message>
<PurchaseOrder OrderDate="1996-04-06">
<ShipTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>9200</zip>
</ShipTo>
<BillTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>2381</zip>
</BillTo>
</PurchaseOrder>
</Message>
<Message key="1234">
<SingleTagHere>
.
.
.
</SingleTagHere>
</Message>
</KafkaMessages>
```
Producer plugin generates 2 messages with below content
```xml
<?xml version="1.0" encoding="utf-8"?>
<PurchaseOrder OrderDate="1996-04-06">
<ShipTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>9200</zip>
</ShipTo>
<BillTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>2381</zip>
</BillTo>
</PurchaseOrder>
```
```xml
<?xml version="1.0" encoding="utf-8"?>
<SingleTagHere>
.
.
.
</SingleTagHere>
```
### JSON Dataset format
An example JSON document is shown below. This document will be sent as two messages
to the configured topic. Each message is created as a proper JSON document.
```json
[
{
"message": {
"key": "818432-942813-832642-453478",
"headers": {
"type": "ADD"
},
"content": {
"location": [
"Leipzig"
],
"obstacle": {
"name": "Iron Bars",
"order": "1"
}
}
}
},
{
"message": {
"key": "887428-119918-570674-866526",
"headers": {
"type": "REMOVE"
},
"content": {
"comments": "We can pass any json payload here."
}
}
}
]
```
Producer plugin generates 2 messages with below content
```json
{
"location": [
"Leipzig"
],
"obstacle": {
"name": "Iron Bars",
"order": "1"
}
}
```
```json
{
"comments": "We can pass any json payload here."
}
```
### Entities format
Random values plugin entities will generate below format JSON document.
```json
{
"schema": {
"type_uri": "https://example.org/vocab/RandomValueRow"
},
"entity": {
"uri": "urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f",
"values": {
"<https://example.org/vocab/RandomValuePath0>": [
"a8o4Ocsb6RZClFRUZU3b2w"
],
"<https://example.org/vocab/RandomValuePath1>": [
"RTICRU7JcTUVn94decelPg"
],
"<https://example.org/vocab/RandomValuePath2>": [
"A9r-969NjAlX0DNWftxKoA"
],
"<https://example.org/vocab/RandomValuePath3>": [
"FygWRy1UJ4-IzIim1qukJA"
],
"<https://example.org/vocab/RandomValuePath4>": [
"AJcbn-LJEs-Dif96xu2eww"
]
}
}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/eccenca/cmem-plugin-kafka",
"name": "cmem-plugin-kafka",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.11,<4.0",
"maintainer_email": "",
"keywords": "eccenca Corporate Memory,plugin,kafka,kafka-producer,kafka-consumer",
"author": "eccenca GmbH",
"author_email": "cmempy-developer@eccenca.com",
"download_url": "https://files.pythonhosted.org/packages/37/2a/2b107dcd5f3d8fa53ca5f7123dcb31fd3df900ebd9b0b9337d95c68c2a39/cmem_plugin_kafka-3.1.0.tar.gz",
"platform": null,
"description": "# cmem-plugin-kafka\n\nSend and receive messages from Apache Kafka.\n\nThis is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com).\n\nYou can install it with the [cmemc](https://eccenca.com/go/cmemc) command line\nclients like this:\n\n```\ncmemc admin workspace python install cmem-plugin-kafka\n```\n\n## Plugin supported message format\n\n### XML dataset format\n\nAn example XML document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper XML document.\n\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<KafkaMessages>\n <Message>\n <PurchaseOrder OrderDate=\"1996-04-06\">\n <ShipTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>9200</zip>\n </ShipTo>\n <BillTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>2381</zip>\n </BillTo>\n </PurchaseOrder>\n </Message>\n <Message key=\"1234\">\n <SingleTagHere>\n .\n .\n .\n </SingleTagHere>\n </Message>\n</KafkaMessages>\n```\nProducer plugin generates 2 messages with below content\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<PurchaseOrder OrderDate=\"1996-04-06\">\n <ShipTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>9200</zip>\n </ShipTo>\n <BillTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>2381</zip>\n </BillTo>\n</PurchaseOrder>\n```\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<SingleTagHere>\n .\n .\n .\n</SingleTagHere>\n```\n### JSON Dataset format\n\nAn example JSON document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper JSON document.\n\n```json\n[\n {\n \"message\": {\n \"key\": \"818432-942813-832642-453478\",\n \"headers\": {\n \"type\": \"ADD\"\n },\n \"content\": {\n \"location\": [\n \"Leipzig\"\n ],\n \"obstacle\": {\n \"name\": \"Iron Bars\",\n \"order\": \"1\"\n }\n }\n }\n },\n {\n \"message\": {\n \"key\": \"887428-119918-570674-866526\",\n \"headers\": {\n \"type\": \"REMOVE\"\n },\n \"content\": {\n \"comments\": \"We can pass any json payload here.\"\n }\n }\n }\n]\n```\nProducer plugin generates 2 messages with below content\n```json\n{\n \"location\": [\n \"Leipzig\"\n ],\n \"obstacle\": {\n \"name\": \"Iron Bars\",\n \"order\": \"1\"\n }\n}\n```\n```json\n{\n \"comments\": \"We can pass any json payload here.\"\n}\n```\n### Entities format\n\nRandom values plugin entities will generate below format JSON document.\n\n```json\n{\n \"schema\": {\n \"type_uri\": \"https://example.org/vocab/RandomValueRow\"\n },\n \"entity\": {\n \"uri\": \"urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f\",\n \"values\": {\n \"<https://example.org/vocab/RandomValuePath0>\": [\n \"a8o4Ocsb6RZClFRUZU3b2w\"\n ],\n \"<https://example.org/vocab/RandomValuePath1>\": [\n \"RTICRU7JcTUVn94decelPg\"\n ],\n \"<https://example.org/vocab/RandomValuePath2>\": [\n \"A9r-969NjAlX0DNWftxKoA\"\n ],\n \"<https://example.org/vocab/RandomValuePath3>\": [\n \"FygWRy1UJ4-IzIim1qukJA\"\n ],\n \"<https://example.org/vocab/RandomValuePath4>\": [\n \"AJcbn-LJEs-Dif96xu2eww\"\n ]\n }\n }\n}\n```",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Send and receive messages from Apache Kafka.",
"version": "3.1.0",
"project_urls": {
"Homepage": "https://github.com/eccenca/cmem-plugin-kafka"
},
"split_keywords": [
"eccenca corporate memory",
"plugin",
"kafka",
"kafka-producer",
"kafka-consumer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9c3e77d4aff4b2631c64951cbe2d4a60d8fe6bdbb9ba8f214d9a333ffe57a0ba",
"md5": "54289dfc81c178df5815d4e5bab71a20",
"sha256": "f72c9bb2d0ae58c587f2fc010ec941c3be1f53f3c2388059778d7d310696c27d"
},
"downloads": -1,
"filename": "cmem_plugin_kafka-3.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "54289dfc81c178df5815d4e5bab71a20",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11,<4.0",
"size": 23709,
"upload_time": "2023-12-30T12:50:30",
"upload_time_iso_8601": "2023-12-30T12:50:30.975943Z",
"url": "https://files.pythonhosted.org/packages/9c/3e/77d4aff4b2631c64951cbe2d4a60d8fe6bdbb9ba8f214d9a333ffe57a0ba/cmem_plugin_kafka-3.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "372a2b107dcd5f3d8fa53ca5f7123dcb31fd3df900ebd9b0b9337d95c68c2a39",
"md5": "81f130c36c261065704027bd3a9f2125",
"sha256": "098126ebe47c3148445068c48fe617c2b880ee9c86ef0a38ed250277527d6262"
},
"downloads": -1,
"filename": "cmem_plugin_kafka-3.1.0.tar.gz",
"has_sig": false,
"md5_digest": "81f130c36c261065704027bd3a9f2125",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11,<4.0",
"size": 20681,
"upload_time": "2023-12-30T12:50:32",
"upload_time_iso_8601": "2023-12-30T12:50:32.720265Z",
"url": "https://files.pythonhosted.org/packages/37/2a/2b107dcd5f3d8fa53ca5f7123dcb31fd3df900ebd9b0b9337d95c68c2a39/cmem_plugin_kafka-3.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-30 12:50:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eccenca",
"github_project": "cmem-plugin-kafka",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "cmem-plugin-kafka"
}
JSON
