# EventHive
Network PubSub and Async Message Passing for Humans
## What is that thing?
`eventhive` is a Python package that enables Python code to communicate using
Publisher/Subscriber model uniformly, be it in the same process or in different hosts!
Supports external PubSub backends (like Redis), Service Discovery (using [python-zeroconf](https://github.com/python-zeroconf/python-zeroconf/)) and application level Message Signing and Encryption (using [Python Native AES](https://github.com/boppreh/aes)).
## What does it do?
Leverages Python reflection magic and the excelent API created by [@dzervas](https://github.com/dzervas) in [`hooker`](https://github.com/satori-ng/hooker) for event-based programming and expands it to network level event-passing, effectively creating a PubSub-based RPC (Remote Procedure Calling) framework.
It can be used in IoT Development, to Games, to Kubernetes and microservices!
## How to Use
### Worker
* Create a minimal `eventhive` YAML configuration for the Worker:
```yaml
connectors:
my-hive:
pubsub_type: fastapi
input_channel: 'worker'
init:
host: 127.0.0.1
port: 8085
endpoint: /pubsub
```
* Register an `eventhive` event:
```python
eventhive.EVENTS.append("worker/action")
```
* Create a function with a single argument and annotate it with the `eventhive` event:
```python
@eventhive.hook("worker/action")
def work(arg): print("Working with: %s" arg)
```
* Start `eventhive`
```python
eventhive.init()
```
### Queen
* Create an `eventhive` YAML configuration for the Queen. Also add a PubSub server in the mix:
```yaml
connectors:
my-hive:
pubsub_type: fastapi
input_channel: 'queen'
init:
host: 127.0.0.1
port: 8085
endpoint: /pubsub
servers:
my-hive:
pubsub_type: fastapi
create: always
broadcast: false
init:
host: 0.0.0.0
port: 8085
endpoint: /pubsub
```
* Register an `eventhive` event implemented by the worker:
```python
eventhive.EVENTS.append("my-hive/worker/action")
```
* Start `eventhive`
```python
eventhive.init()
```
* Call the registered event with a `dict`
```python
eventhive.EVENTS.call("my-hive/worker/action", {"param1":1, "param2":2})
```
The Worker will print:
```
Working with: {'param1':1, 'param2':2}
```
## What with the `/` in the Event names?
A convention exists in `eventhive` so it can be used as a PubSub for event-based programming
and network message passing engine at the same time. This convention is based on *Event* names.
The *Events* are split in 3 categories following a naming convention:
### Strictly Local Events
Like [`bee_stuff`](https://github.com/operatorequals/eventhive/blob/master/examples/single_process.py#L3) in the single-process example. These Events cannot be called from the network.
They have to be defined (`eventhive.append("bee_stuff")`), implemented (`@eventhive.hook("bee_stuff")`) and
called (`eventhive.EVENTS["bee_stuff"]("arg")`) in the same process.
### Network Accessible Events
Like Worker Bee's [`worker-bee/work`](https://github.com/operatorequals/eventhive/blob/master/examples/different_hosts/worker.py#L40). These Events have to be defined and implemented by the same process,
but they can be called either from the same process (`eventhive.EVENTS["worker-bee/work"]({"random":"dict"})`), or
by any other `eventhive` process in the network as a *Remote Event*.
### Remote Events
Like Queen Bee's [`my-beehive/worker-bee/work`](https://github.com/operatorequals/eventhive/blob/master/examples/different_hosts/boss.py#L33). These Events are not implemented in the process they are
defined (`eventhive.append("my-beehive/worker-bee/work")`). Calling these Events
(`eventhive.EVENTS["my-beehive/worker-bee/work"]({"random":"dict"})`) informs `eventhive` that they
have to travel over `my-beehive` network and get published to the `my-beehive/worker-bee/work` channel.
Other `eventhive` processes that are connected to `my-beehive` and have `input_channel: worker-bee` in their configuration, pick up these Events, ditch the *Network Name* (`my-beehive/worker-bee/work` becomes `worker-bee/work`) and consume them like *Local Events*.
## Message Examples
`eventhive` events are simple JSON messages that can have metadata attached or not.
In the examples below metadata will be available under the key `__eventhive__`. Disabling metadata or changing the key name
can be configured through the [YAML configuration](https://github.com/operatorequals/eventhive/blob/master/eventhive/config.py#L20).
### No `secret` set - plaintext messages
```python
eventhive.EVENTS.call("my-hive/worker/action", {"param1":1, "param2":2})
```
```json
{"param1": 1, "param2": 2, "__eventhive__": {"time": 1680610165.4884348, "version": "0.3.2", "event": "my-hive/worker/action", "id": "d891d47f-64a4-4ecc-88ab-69249da31154"}}
```
`eventhive` metadata can be used from a hooked function for accessing event creation time, version checking and duplicate message checking.
Yet, it is planned to move some of these functionalities to the library.
Turning off metadata (i.e `eventhive.remove_metadata: true`) can save bandwidth, e.g in very limited IoT networks.
### With `secret` set
Setting a [`secret`](https://github.com/operatorequals/eventhive/blob/master/eventhive/config.py#L63) in a `connector` in the YAML configuration automatically enables AES Encryption and Message Signing. *This is totally transparent to the hooked function*, which will continue to accept events like the one described above.
Yet, the messages travelling through the network will look as below:
```json
{"iv": "2SH5S8J75afKxr76osauow==", "ciphertext": "JFJvaixp9/8oiYaxrCS+yA6TkCKlX85g0qG0GlZa8eaQTuXf1Ot33yiIIr7Y+fsFTzL7kzOtFbaq1uO6QH54N9oyeWUi7rDelQi2HNZGYRJCqwtwAbFX4+D8IgBGqYkYGPKuiUZCLRvPArPmaMh8PpMrq4/nEOGf0ivyS9hKEVb9KSrm4+VedAfBMQfpxP3Z/cm/jpj2sKDb9rfcjWATEcQToQ/U4PP40mUGeDpKbmyTAxGLdGAp3jDcghkAM76nDAXmTpuP0PN7YpSp/3cRAiweXxuBszIdeLuoUBOa"}
```
The `ciphertext` encapsulates messages like the one shown above, plus the `__sign__` key, which is used to verify the plaintext content and is stripped from the final message.
In case decryption or signature verification fails, the event gets dropped and does not arrive to the hooked functions.
Finally, if `secret` is set for a connector, it is impossible to accept non-encrypted, non-signed messages.
## The CLI Tool
A CLI tool is part of the `eventhive` package to make it possible to test YAML configurations, without writing application code (and adding moving parts to the test).
The output of the tool is the last `eventhive` message that was received in JSON, or `{"no":"output"}` if no message is received.
It uses YAML configuration and also can template YAML files, using the similar named arguments provided through CLI, as shown below:
```bash
eventhive-cli --network my-hive --receiver worker --secret m1s3cr3t --config fastapi-template.yaml
```
`fastapi-template.yaml`:
```yaml
connectors:
{network}:
pubsub_type: fastapi
input_channel: '{receiver}'
secret: '{secret}'
init:
host: 127.0.0.1
port: 8085
endpoint: /pubsub
```
### CLI Usage
```bash
$ eventhive-cli -h
usage: eventhive-cli [-h] [--network NETWORK] [--receiver RECEIVER] [--event EVENT] [--json-data JSON_DATA] [--target-event TARGET_EVENT] [--json-fallback JSON_FALLBACK]
[--config CONFIG] [--timeout TIMEOUT] [--secret SECRET] [--debug] [--verbose]
The Swiss-Army knife for Eventhive
optional arguments:
-h, --help show this help message and exit
--network NETWORK The Eventhive Network of Events (<here>/*/*) (default: eventhive)
--receiver RECEIVER The Eventhive Channel of Events (*/<here>/*) (default: cli)
--event EVENT The Eventhive Event Name (*/*/<here>) (default: do)
--json-data JSON_DATA
If set - the JSON will be sent to "<network>/<receiver>/<event>" (default: False)
--target-event TARGET_EVENT
If set - the JSON defined in "--json-data" will be published on it instead of "<network>/<receiver>/<event>" (default: None)
--json-fallback JSON_FALLBACK
The JSON to print to STDOUT if no data is received in "<network>/<receiver>/<event>" (default: {"no":"output"})
--config CONFIG The Configuration file to be used by Eventhive (can include formatting) (default: .eventhive-config.yaml)
--timeout TIMEOUT Time to run before exiting and printing to STDOUT (default: 6)
--secret SECRET A secret string used for signing and encryption of messages (default: None)
--debug, -d When set - full DEBUG logs will be printed (default: 30)
--verbose, -v When set - INFO logs will be printed (default: None)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/operatorequals/eventhive",
"name": "eventhive",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "pubsub,microservice,iot,rpc",
"author": "attr: eventhive.__meta__.__author__",
"author_email": "\"john.torakis@gmail.com\"",
"download_url": "https://files.pythonhosted.org/packages/54/0e/b50b70ea083d8a63e330eea7a07aeecd8286b6ab9ae364b49b1a167e5e51/eventhive-0.5.0.tar.gz",
"platform": null,
"description": "# EventHive\n\nNetwork PubSub and Async Message Passing for Humans\n\n\n\n\n## What is that thing?\n\n`eventhive` is a Python package that enables Python code to communicate using\nPublisher/Subscriber model uniformly, be it in the same process or in different hosts!\n\nSupports external PubSub backends (like Redis), Service Discovery (using [python-zeroconf](https://github.com/python-zeroconf/python-zeroconf/)) and application level Message Signing and Encryption (using [Python Native AES](https://github.com/boppreh/aes)).\n\n\n## What does it do?\n\nLeverages Python reflection magic and the excelent API created by [@dzervas](https://github.com/dzervas) in [`hooker`](https://github.com/satori-ng/hooker) for event-based programming and expands it to network level event-passing, effectively creating a PubSub-based RPC (Remote Procedure Calling) framework.\n\nIt can be used in IoT Development, to Games, to Kubernetes and microservices!\n\n## How to Use\n\n### Worker\n\n* Create a minimal `eventhive` YAML configuration for the Worker:\n ```yaml\n connectors:\n my-hive:\n pubsub_type: fastapi\n input_channel: 'worker'\n init:\n host: 127.0.0.1\n port: 8085\n endpoint: /pubsub\n ```\n\n* Register an `eventhive` event:\n ```python\n eventhive.EVENTS.append(\"worker/action\")\n ```\n\n* Create a function with a single argument and annotate it with the `eventhive` event:\n ```python\n @eventhive.hook(\"worker/action\")\n def work(arg): print(\"Working with: %s\" arg)\n ```\n\n* Start `eventhive`\n ```python\n eventhive.init()\n ```\n\n### Queen\n\n* Create an `eventhive` YAML configuration for the Queen. Also add a PubSub server in the mix:\n ```yaml\n connectors:\n my-hive:\n pubsub_type: fastapi\n input_channel: 'queen'\n init:\n host: 127.0.0.1\n port: 8085\n endpoint: /pubsub\n\n servers:\n my-hive:\n pubsub_type: fastapi\n create: always\n broadcast: false\n init:\n host: 0.0.0.0\n port: 8085\n endpoint: /pubsub\n ```\n\n* Register an `eventhive` event implemented by the worker:\n ```python\n eventhive.EVENTS.append(\"my-hive/worker/action\")\n ```\n\n* Start `eventhive`\n ```python\n eventhive.init()\n ```\n\n* Call the registered event with a `dict`\n ```python\n eventhive.EVENTS.call(\"my-hive/worker/action\", {\"param1\":1, \"param2\":2})\n ```\n\n The Worker will print:\n ```\n Working with: {'param1':1, 'param2':2}\n ```\n\n## What with the `/` in the Event names?\n\nA convention exists in `eventhive` so it can be used as a PubSub for event-based programming\nand network message passing engine at the same time. This convention is based on *Event* names.\n\nThe *Events* are split in 3 categories following a naming convention:\n\n### Strictly Local Events\nLike [`bee_stuff`](https://github.com/operatorequals/eventhive/blob/master/examples/single_process.py#L3) in the single-process example. These Events cannot be called from the network.\nThey have to be defined (`eventhive.append(\"bee_stuff\")`), implemented (`@eventhive.hook(\"bee_stuff\")`) and\ncalled (`eventhive.EVENTS[\"bee_stuff\"](\"arg\")`) in the same process.\n\n### Network Accessible Events\nLike Worker Bee's [`worker-bee/work`](https://github.com/operatorequals/eventhive/blob/master/examples/different_hosts/worker.py#L40). These Events have to be defined and implemented by the same process,\nbut they can be called either from the same process (`eventhive.EVENTS[\"worker-bee/work\"]({\"random\":\"dict\"})`), or\nby any other `eventhive` process in the network as a *Remote Event*.\n\n### Remote Events\nLike Queen Bee's [`my-beehive/worker-bee/work`](https://github.com/operatorequals/eventhive/blob/master/examples/different_hosts/boss.py#L33). These Events are not implemented in the process they are\ndefined (`eventhive.append(\"my-beehive/worker-bee/work\")`). Calling these Events\n(`eventhive.EVENTS[\"my-beehive/worker-bee/work\"]({\"random\":\"dict\"})`) informs `eventhive` that they\nhave to travel over `my-beehive` network and get published to the `my-beehive/worker-bee/work` channel.\n\nOther `eventhive` processes that are connected to `my-beehive` and have `input_channel: worker-bee` in their configuration, pick up these Events, ditch the *Network Name* (`my-beehive/worker-bee/work` becomes `worker-bee/work`) and consume them like *Local Events*.\n\n\n## Message Examples\n\n`eventhive` events are simple JSON messages that can have metadata attached or not.\nIn the examples below metadata will be available under the key `__eventhive__`. Disabling metadata or changing the key name\ncan be configured through the [YAML configuration](https://github.com/operatorequals/eventhive/blob/master/eventhive/config.py#L20).\n\n### No `secret` set - plaintext messages\n\n```python\neventhive.EVENTS.call(\"my-hive/worker/action\", {\"param1\":1, \"param2\":2})\n```\n\n```json\n{\"param1\": 1, \"param2\": 2, \"__eventhive__\": {\"time\": 1680610165.4884348, \"version\": \"0.3.2\", \"event\": \"my-hive/worker/action\", \"id\": \"d891d47f-64a4-4ecc-88ab-69249da31154\"}}\n```\n\n`eventhive` metadata can be used from a hooked function for accessing event creation time, version checking and duplicate message checking.\nYet, it is planned to move some of these functionalities to the library.\n\nTurning off metadata (i.e `eventhive.remove_metadata: true`) can save bandwidth, e.g in very limited IoT networks.\n\n### With `secret` set\n\nSetting a [`secret`](https://github.com/operatorequals/eventhive/blob/master/eventhive/config.py#L63) in a `connector` in the YAML configuration automatically enables AES Encryption and Message Signing. *This is totally transparent to the hooked function*, which will continue to accept events like the one described above.\nYet, the messages travelling through the network will look as below:\n\n```json\n{\"iv\": \"2SH5S8J75afKxr76osauow==\", \"ciphertext\": \"JFJvaixp9/8oiYaxrCS+yA6TkCKlX85g0qG0GlZa8eaQTuXf1Ot33yiIIr7Y+fsFTzL7kzOtFbaq1uO6QH54N9oyeWUi7rDelQi2HNZGYRJCqwtwAbFX4+D8IgBGqYkYGPKuiUZCLRvPArPmaMh8PpMrq4/nEOGf0ivyS9hKEVb9KSrm4+VedAfBMQfpxP3Z/cm/jpj2sKDb9rfcjWATEcQToQ/U4PP40mUGeDpKbmyTAxGLdGAp3jDcghkAM76nDAXmTpuP0PN7YpSp/3cRAiweXxuBszIdeLuoUBOa\"}\n```\n\nThe `ciphertext` encapsulates messages like the one shown above, plus the `__sign__` key, which is used to verify the plaintext content and is stripped from the final message.\n\nIn case decryption or signature verification fails, the event gets dropped and does not arrive to the hooked functions.\n\nFinally, if `secret` is set for a connector, it is impossible to accept non-encrypted, non-signed messages.\n\n## The CLI Tool\n\nA CLI tool is part of the `eventhive` package to make it possible to test YAML configurations, without writing application code (and adding moving parts to the test).\n\nThe output of the tool is the last `eventhive` message that was received in JSON, or `{\"no\":\"output\"}` if no message is received.\n\nIt uses YAML configuration and also can template YAML files, using the similar named arguments provided through CLI, as shown below:\n\n```bash\neventhive-cli --network my-hive --receiver worker --secret m1s3cr3t --config fastapi-template.yaml\n```\n\n`fastapi-template.yaml`:\n```yaml\nconnectors:\n {network}:\n pubsub_type: fastapi\n input_channel: '{receiver}'\n secret: '{secret}'\n init:\n host: 127.0.0.1\n port: 8085\n endpoint: /pubsub\n```\n\n\n### CLI Usage\n\n```bash\n$ eventhive-cli -h\nusage: eventhive-cli [-h] [--network NETWORK] [--receiver RECEIVER] [--event EVENT] [--json-data JSON_DATA] [--target-event TARGET_EVENT] [--json-fallback JSON_FALLBACK]\n [--config CONFIG] [--timeout TIMEOUT] [--secret SECRET] [--debug] [--verbose]\n\nThe Swiss-Army knife for Eventhive\n\noptional arguments:\n -h, --help show this help message and exit\n --network NETWORK The Eventhive Network of Events (<here>/*/*) (default: eventhive)\n --receiver RECEIVER The Eventhive Channel of Events (*/<here>/*) (default: cli)\n --event EVENT The Eventhive Event Name (*/*/<here>) (default: do)\n --json-data JSON_DATA\n If set - the JSON will be sent to \"<network>/<receiver>/<event>\" (default: False)\n --target-event TARGET_EVENT\n If set - the JSON defined in \"--json-data\" will be published on it instead of \"<network>/<receiver>/<event>\" (default: None)\n --json-fallback JSON_FALLBACK\n The JSON to print to STDOUT if no data is received in \"<network>/<receiver>/<event>\" (default: {\"no\":\"output\"})\n --config CONFIG The Configuration file to be used by Eventhive (can include formatting) (default: .eventhive-config.yaml)\n --timeout TIMEOUT Time to run before exiting and printing to STDOUT (default: 6)\n --secret SECRET A secret string used for signing and encryption of messages (default: None)\n --debug, -d When set - full DEBUG logs will be printed (default: 30)\n --verbose, -v When set - INFO logs will be printed (default: None)\n\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Network PubSub and Async Message Passing for Humans",
"version": "0.5.0",
"split_keywords": [
"pubsub",
"microservice",
"iot",
"rpc"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "27dfcb6136ed188b43127c043b8c34822545f2b8f46359c93076d70d1b39a90c",
"md5": "92f720a38528cdf014a2d89405f643ec",
"sha256": "face84070195ca600df1f82d0015cc66654f38175d14d015497952146a86f123"
},
"downloads": -1,
"filename": "eventhive-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "92f720a38528cdf014a2d89405f643ec",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 27935,
"upload_time": "2023-04-13T11:31:30",
"upload_time_iso_8601": "2023-04-13T11:31:30.163676Z",
"url": "https://files.pythonhosted.org/packages/27/df/cb6136ed188b43127c043b8c34822545f2b8f46359c93076d70d1b39a90c/eventhive-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "540eb50b70ea083d8a63e330eea7a07aeecd8286b6ab9ae364b49b1a167e5e51",
"md5": "db83d823844f1f035d8e754bf6a35881",
"sha256": "a61bd0c3ee1658a4d17f3ee1c581989f882ef29d4002ae41b91b8fff1e6215b3"
},
"downloads": -1,
"filename": "eventhive-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "db83d823844f1f035d8e754bf6a35881",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 24961,
"upload_time": "2023-04-13T11:31:31",
"upload_time_iso_8601": "2023-04-13T11:31:31.438866Z",
"url": "https://files.pythonhosted.org/packages/54/0e/b50b70ea083d8a63e330eea7a07aeecd8286b6ab9ae364b49b1a167e5e51/eventhive-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-13 11:31:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "operatorequals",
"github_project": "eventhive",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "eventhive"
}
JSON
