# cmem-plugin-kafka
Send and receive messages from Apache Kafka.
This is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com).
You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line
clients like this:
```
cmemc admin workspace python install cmem-plugin-kafka
```
## Plugin supported message format
### XML dataset format
An example XML document is shown below. This document will be sent as two messages
to the configured topic. Each message is created as a proper XML document.
```xml
<?xml version="1.0" encoding="utf-8"?>
<KafkaMessages>
<Message>
<PurchaseOrder OrderDate="1996-04-06">
<ShipTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>9200</zip>
</ShipTo>
<BillTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>2381</zip>
</BillTo>
</PurchaseOrder>
</Message>
<Message key="1234">
<SingleTagHere>
.
.
.
</SingleTagHere>
</Message>
</KafkaMessages>
```
Producer plugin generates 2 messages with below content
```xml
<?xml version="1.0" encoding="utf-8"?>
<PurchaseOrder OrderDate="1996-04-06">
<ShipTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>9200</zip>
</ShipTo>
<BillTo country="string">
<name>string</name>
<street>string</street>
<city>string</city>
<state>string</state>
<zip>2381</zip>
</BillTo>
</PurchaseOrder>
```
```xml
<?xml version="1.0" encoding="utf-8"?>
<SingleTagHere>
.
.
.
</SingleTagHere>
```
### JSON Dataset format
An example JSON document is shown below. This document will be sent as two messages
to the configured topic. Each message is created as a proper JSON document.
```json
[
{
"message": {
"key": "818432-942813-832642-453478",
"headers": {
"type": "ADD"
},
"content": {
"location": [
"Leipzig"
],
"obstacle": {
"name": "Iron Bars",
"order": "1"
}
}
}
},
{
"message": {
"key": "887428-119918-570674-866526",
"headers": {
"type": "REMOVE"
},
"content": {
"comments": "We can pass any json payload here."
}
}
}
]
```
Producer plugin generates 2 messages with below content
```json
{
"location": [
"Leipzig"
],
"obstacle": {
"name": "Iron Bars",
"order": "1"
}
}
```
```json
{
"comments": "We can pass any json payload here."
}
```
### Entities format
Random values plugin entities will generate below format JSON document.
```json
{
"schema": {
"type_uri": "https://example.org/vocab/RandomValueRow"
},
"entity": {
"uri": "urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f",
"values": {
"<https://example.org/vocab/RandomValuePath0>": [
"a8o4Ocsb6RZClFRUZU3b2w"
],
"<https://example.org/vocab/RandomValuePath1>": [
"RTICRU7JcTUVn94decelPg"
],
"<https://example.org/vocab/RandomValuePath2>": [
"A9r-969NjAlX0DNWftxKoA"
],
"<https://example.org/vocab/RandomValuePath3>": [
"FygWRy1UJ4-IzIim1qukJA"
],
"<https://example.org/vocab/RandomValuePath4>": [
"AJcbn-LJEs-Dif96xu2eww"
]
}
}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/eccenca/cmem-plugin-kafka",
"name": "cmem-plugin-kafka",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "eccenca Corporate Memory, plugin, kafka, kafka-producer, kafka-consumer",
"author": "eccenca GmbH",
"author_email": "cmempy-developer@eccenca.com",
"download_url": "https://files.pythonhosted.org/packages/f5/f4/e3e05cd4adfd1f9cf5ef40749539203c6445e1f232ad63b7b8d1d0bc78c4/cmem_plugin_kafka-3.2.0.tar.gz",
"platform": null,
"description": "# cmem-plugin-kafka\n\nSend and receive messages from Apache Kafka.\n\nThis is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com).\n\nYou can install it with the [cmemc](https://eccenca.com/go/cmemc) command line\nclients like this:\n\n```\ncmemc admin workspace python install cmem-plugin-kafka\n```\n\n## Plugin supported message format\n\n### XML dataset format\n\nAn example XML document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper XML document.\n\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<KafkaMessages>\n <Message>\n <PurchaseOrder OrderDate=\"1996-04-06\">\n <ShipTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>9200</zip>\n </ShipTo>\n <BillTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>2381</zip>\n </BillTo>\n </PurchaseOrder>\n </Message>\n <Message key=\"1234\">\n <SingleTagHere>\n .\n .\n .\n </SingleTagHere>\n </Message>\n</KafkaMessages>\n```\nProducer plugin generates 2 messages with below content\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<PurchaseOrder OrderDate=\"1996-04-06\">\n <ShipTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>9200</zip>\n </ShipTo>\n <BillTo country=\"string\">\n <name>string</name>\n <street>string</street>\n <city>string</city>\n <state>string</state>\n <zip>2381</zip>\n </BillTo>\n</PurchaseOrder>\n```\n```xml\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<SingleTagHere>\n .\n .\n .\n</SingleTagHere>\n```\n### JSON Dataset format\n\nAn example JSON document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper JSON document.\n\n```json\n[\n {\n \"message\": {\n \"key\": \"818432-942813-832642-453478\",\n \"headers\": {\n \"type\": \"ADD\"\n },\n \"content\": {\n \"location\": [\n \"Leipzig\"\n ],\n \"obstacle\": {\n \"name\": \"Iron Bars\",\n \"order\": \"1\"\n }\n }\n }\n },\n {\n \"message\": {\n \"key\": \"887428-119918-570674-866526\",\n \"headers\": {\n \"type\": \"REMOVE\"\n },\n \"content\": {\n \"comments\": \"We can pass any json payload here.\"\n }\n }\n }\n]\n```\nProducer plugin generates 2 messages with below content\n```json\n{\n \"location\": [\n \"Leipzig\"\n ],\n \"obstacle\": {\n \"name\": \"Iron Bars\",\n \"order\": \"1\"\n }\n}\n```\n```json\n{\n \"comments\": \"We can pass any json payload here.\"\n}\n```\n### Entities format\n\nRandom values plugin entities will generate below format JSON document.\n\n```json\n{\n \"schema\": {\n \"type_uri\": \"https://example.org/vocab/RandomValueRow\"\n },\n \"entity\": {\n \"uri\": \"urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f\",\n \"values\": {\n \"<https://example.org/vocab/RandomValuePath0>\": [\n \"a8o4Ocsb6RZClFRUZU3b2w\"\n ],\n \"<https://example.org/vocab/RandomValuePath1>\": [\n \"RTICRU7JcTUVn94decelPg\"\n ],\n \"<https://example.org/vocab/RandomValuePath2>\": [\n \"A9r-969NjAlX0DNWftxKoA\"\n ],\n \"<https://example.org/vocab/RandomValuePath3>\": [\n \"FygWRy1UJ4-IzIim1qukJA\"\n ],\n \"<https://example.org/vocab/RandomValuePath4>\": [\n \"AJcbn-LJEs-Dif96xu2eww\"\n ]\n }\n }\n}\n```",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Send and receive messages from Apache Kafka.",
"version": "3.2.0",
"project_urls": {
"Homepage": "https://github.com/eccenca/cmem-plugin-kafka"
},
"split_keywords": [
"eccenca corporate memory",
" plugin",
" kafka",
" kafka-producer",
" kafka-consumer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e820a508730b3b1896e707a7f1de94a0a83ce3cb8c79498496a5917904a8b461",
"md5": "f789fbc61543622997187e6586200d49",
"sha256": "3e9e21981c85b3d0cb12822333e87f955063fea1ff0e973d3306416dd80f1576"
},
"downloads": -1,
"filename": "cmem_plugin_kafka-3.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f789fbc61543622997187e6586200d49",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 24132,
"upload_time": "2024-06-07T13:15:27",
"upload_time_iso_8601": "2024-06-07T13:15:27.295929Z",
"url": "https://files.pythonhosted.org/packages/e8/20/a508730b3b1896e707a7f1de94a0a83ce3cb8c79498496a5917904a8b461/cmem_plugin_kafka-3.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f5f4e3e05cd4adfd1f9cf5ef40749539203c6445e1f232ad63b7b8d1d0bc78c4",
"md5": "8400018a890a783942625079a94f7d40",
"sha256": "54422128c63006b5f2dd7e3a96bda01e89166ba6f4913f451831d37ed82ffb59"
},
"downloads": -1,
"filename": "cmem_plugin_kafka-3.2.0.tar.gz",
"has_sig": false,
"md5_digest": "8400018a890a783942625079a94f7d40",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 21199,
"upload_time": "2024-06-07T13:15:30",
"upload_time_iso_8601": "2024-06-07T13:15:30.004627Z",
"url": "https://files.pythonhosted.org/packages/f5/f4/e3e05cd4adfd1f9cf5ef40749539203c6445e1f232ad63b7b8d1d0bc78c4/cmem_plugin_kafka-3.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-07 13:15:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eccenca",
"github_project": "cmem-plugin-kafka",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "cmem-plugin-kafka"
}
JSON
