# signal-slot
[](https://github.com/alex-petrenko/signal-slot/actions/workflows/test-ci.yml)
[](https://pepy.tech/project/signal-slot-mp)
[<img src="https://img.shields.io/discord/987232982798598164?label=discord">](https://discord.gg/BCfHWaSMkr)
Qt-like event loops, signals and slots for communication across threads and processes in Python.
### Installation
```bash
pip install signal-slot-mp
```
Linux, macOS, and Windows are supported.
### Overview
`signal-slot` enables a parallel programming paradigm inspired by Qt's signals and slots, but in Python.
The main idea can be summarized as follows:
* Application is a collection of `EventLoop`s. Each `EventLoop` is an infinite loop that occupies a thread or a process.
* Logic of the system is implemented in `EventLoopObject`s that live on `EventLoop`s. Each `EventLoop` can support multiple `EventLoopObject`s.
* `EventLoopObject`s can emit signals. A signal "message" contains a name of the signal
and the payload (arbitrary data).
* Components can also connect to signals emitted by other components by specifying a `slot` function to be called when the signal is received
by the EventLoop.
### Usage example
```python
import time
import datetime
from signal_slot.signal_slot import EventLoop, EventLoopObject, EventLoopProcess, Timer, signal
now = datetime.datetime.now
# classes derived from EventLoopObject define signals and slots (actually any method can be a slot)
class A(EventLoopObject):
@signal
def signal_a(self):
...
def on_signal_b(self, msg: str):
print(f"{now()} {self.object_id} received signal_b: {msg}")
time.sleep(1)
self.signal_a.emit("hello from A", 42)
class B(EventLoopObject):
@signal
def signal_b(self):
...
def on_signal_a(self, msg: str, other_data: int):
print(f"{now()} {self.object_id} received signal_a: {msg} {other_data}")
time.sleep(1)
self.signal_b.emit("hello from B")
# create main event loop and object of type A
main_event_loop = EventLoop("main_loop")
a = A(main_event_loop, "object: a")
# create a background process with a separate event loop and object b that lives on that event loop
bg_process = EventLoopProcess(unique_process_name="background_process")
b = B(bg_process.event_loop, "object: b")
# connect signals and slots
a.signal_a.connect(b.on_signal_a)
b.signal_b.connect(a.on_signal_b)
# emit signal from a to kick off the communication
a.signal_a.emit("Initial hello from A", 1337)
# create a timer that will stop our system after 10 seconds
stop_timer = Timer(main_event_loop, 10.0, single_shot=True)
stop_timer.start()
# connect the stop method of the event loop to the timeout signal of the timer
stop_timer.timeout.connect(main_event_loop.stop)
stop_timer.timeout.connect(bg_process.stop) # stops the event loop of the background process
# start the background process
bg_process.start()
# start the main event loop
main_event_loop.exec()
# if we get here, the main event loop has stopped
# wait for the background process to finish
bg_process.join()
print(f"{now()} Done!")
```
The output should roughly look like this:
```bash
2022-11-30 01:51:58.943425 object: b received signal_a: Initial hello from A 1337
2022-11-30 01:51:59.944957 object: a received signal_b: hello from B
2022-11-30 01:52:00.945852 object: b received signal_a: hello from A 42
2022-11-30 01:52:01.947599 object: a received signal_b: hello from B
2022-11-30 01:52:02.949214 object: b received signal_a: hello from A 42
2022-11-30 01:52:03.950762 object: a received signal_b: hello from B
2022-11-30 01:52:04.952419 object: b received signal_a: hello from A 42
2022-11-30 01:52:05.953596 object: a received signal_b: hello from B
2022-11-30 01:52:06.954918 object: b received signal_a: hello from A 42
2022-11-30 01:52:07.956701 object: a received signal_b: hello from B
2022-11-30 01:52:08.957755 object: b received signal_a: hello from A 42
2022-11-30 01:52:09.963144 Done!
```
### Implementation details
* There's not argument validation for signals and slots. If you connect a slot to a signal with a different signature,
it will fail at runtime. This can also be used to your advantage by allowing to propagate arbitrary data as
payload with appropriate runtime checks.
* It is currently impossible to connect a slot to a signal if emitter and receiver objects belong to event loops
already running in different processes (although it should be possible to implement this feature).
Connect signals to slots during system initialization.
* Signal-slot mechanism in the current implementation can't implement a message passing protocol where
only a single copy of the signal is received by the subscribers. Signals are always delivered to all connected slots.
Use a FIFO multiprocessing queue if you want only one receiver to receive the signal.
### Multiprocessing queues
At the core of the signal-slot mechanism are the queues that are used to pass messages between processes.
Python provides a default implementation `multiprocessing.Queue`, which turns out to be rather slow.
By default we use a custom queue implementation written in C++ using POSIX API that is significantly faster:
https://github.com/alex-petrenko/faster-fifo.
### Contributing
Local installation for development:
```bash
pip install -e .[dev]
```
Automatic code formatting:
```bash
make format && make check-codestyle
```
Run tests:
```bash
make test
```
### Recent releases
##### v1.0.5
* Windows support (do not require POSIX-only faster-fifo on Windows)
##### v1.0.4
* Use updated version of faster-fifo
##### v1.0.3
* Improved logging
##### v1.0.2
* Catching queue.Full exception to handle situations where receiver event loop process is killed
##### v1.0.1
* Added signal_slot.configure_logger() function to configure a custom logger
##### v1.0.0
* First PyPI version
### Footnote
Originally designed for Sample Factory 2.0, a high-throughput asynchronous RL codebase https://github.com/alex-petrenko/sample-factory.
Distributed under MIT License (see LICENSE), feel free to use for any purpose, commercial or not, at your own risk.
Raw data
{
"_id": null,
"home_page": "https://github.com/alex-petrenko/signal-slot",
"name": "signal-slot-mp",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "asynchronous multithreading multiprocessing queue faster-fifo signal slot event loop",
"author": "Aleksei Petrenko",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/3f/2a/86bcc1df3132b7b3105be124299946cd053851593d6a8ab34a822a1ec65f/signal-slot-mp-1.0.5.tar.gz",
"platform": null,
"description": "# signal-slot\n\n[](https://github.com/alex-petrenko/signal-slot/actions/workflows/test-ci.yml)\n[](https://pepy.tech/project/signal-slot-mp)\n[<img src=\"https://img.shields.io/discord/987232982798598164?label=discord\">](https://discord.gg/BCfHWaSMkr)\n\nQt-like event loops, signals and slots for communication across threads and processes in Python.\n\n### Installation\n\n```bash\npip install signal-slot-mp\n```\n\nLinux, macOS, and Windows are supported.\n\n### Overview\n\n`signal-slot` enables a parallel programming paradigm inspired by Qt's signals and slots, but in Python.\n\nThe main idea can be summarized as follows:\n\n* Application is a collection of `EventLoop`s. Each `EventLoop` is an infinite loop that occupies a thread or a process.\n* Logic of the system is implemented in `EventLoopObject`s that live on `EventLoop`s. Each `EventLoop` can support multiple `EventLoopObject`s.\n* `EventLoopObject`s can emit signals. A signal \"message\" contains a name of the signal\nand the payload (arbitrary data).\n* Components can also connect to signals emitted by other components by specifying a `slot` function to be called when the signal is received\nby the EventLoop.\n\n### Usage example\n\n```python\nimport time\nimport datetime\nfrom signal_slot.signal_slot import EventLoop, EventLoopObject, EventLoopProcess, Timer, signal\n\nnow = datetime.datetime.now\n\n# classes derived from EventLoopObject define signals and slots (actually any method can be a slot)\nclass A(EventLoopObject):\n @signal\n def signal_a(self):\n ...\n\n def on_signal_b(self, msg: str):\n print(f\"{now()} {self.object_id} received signal_b: {msg}\")\n time.sleep(1)\n self.signal_a.emit(\"hello from A\", 42)\n\nclass B(EventLoopObject):\n @signal\n def signal_b(self):\n ...\n\n def on_signal_a(self, msg: str, other_data: int):\n print(f\"{now()} {self.object_id} received signal_a: {msg} {other_data}\")\n time.sleep(1)\n self.signal_b.emit(\"hello from B\")\n\n# create main event loop and object of type A\nmain_event_loop = EventLoop(\"main_loop\")\na = A(main_event_loop, \"object: a\")\n\n# create a background process with a separate event loop and object b that lives on that event loop\nbg_process = EventLoopProcess(unique_process_name=\"background_process\")\nb = B(bg_process.event_loop, \"object: b\")\n\n# connect signals and slots\na.signal_a.connect(b.on_signal_a)\nb.signal_b.connect(a.on_signal_b)\n\n# emit signal from a to kick off the communication\na.signal_a.emit(\"Initial hello from A\", 1337)\n\n# create a timer that will stop our system after 10 seconds\nstop_timer = Timer(main_event_loop, 10.0, single_shot=True)\nstop_timer.start()\n\n# connect the stop method of the event loop to the timeout signal of the timer\nstop_timer.timeout.connect(main_event_loop.stop)\nstop_timer.timeout.connect(bg_process.stop) # stops the event loop of the background process\n\n# start the background process\nbg_process.start()\n\n# start the main event loop\nmain_event_loop.exec()\n\n# if we get here, the main event loop has stopped\n# wait for the background process to finish\nbg_process.join()\n\nprint(f\"{now()} Done!\")\n```\n\nThe output should roughly look like this:\n\n```bash\n2022-11-30 01:51:58.943425 object: b received signal_a: Initial hello from A 1337\n2022-11-30 01:51:59.944957 object: a received signal_b: hello from B\n2022-11-30 01:52:00.945852 object: b received signal_a: hello from A 42\n2022-11-30 01:52:01.947599 object: a received signal_b: hello from B\n2022-11-30 01:52:02.949214 object: b received signal_a: hello from A 42\n2022-11-30 01:52:03.950762 object: a received signal_b: hello from B\n2022-11-30 01:52:04.952419 object: b received signal_a: hello from A 42\n2022-11-30 01:52:05.953596 object: a received signal_b: hello from B\n2022-11-30 01:52:06.954918 object: b received signal_a: hello from A 42\n2022-11-30 01:52:07.956701 object: a received signal_b: hello from B\n2022-11-30 01:52:08.957755 object: b received signal_a: hello from A 42\n2022-11-30 01:52:09.963144 Done!\n```\n\n### Implementation details\n\n* There's not argument validation for signals and slots. If you connect a slot to a signal with a different signature,\nit will fail at runtime. This can also be used to your advantage by allowing to propagate arbitrary data as\npayload with appropriate runtime checks.\n* It is currently impossible to connect a slot to a signal if emitter and receiver objects belong to event loops\nalready running in different processes (although it should be possible to implement this feature).\nConnect signals to slots during system initialization.\n* Signal-slot mechanism in the current implementation can't implement a message passing protocol where\nonly a single copy of the signal is received by the subscribers. Signals are always delivered to all connected slots.\nUse a FIFO multiprocessing queue if you want only one receiver to receive the signal.\n\n### Multiprocessing queues\n\nAt the core of the signal-slot mechanism are the queues that are used to pass messages between processes.\nPython provides a default implementation `multiprocessing.Queue`, which turns out to be rather slow.\n\nBy default we use a custom queue implementation written in C++ using POSIX API that is significantly faster:\nhttps://github.com/alex-petrenko/faster-fifo.\n\n### Contributing\n\nLocal installation for development:\n\n```bash\npip install -e .[dev]\n```\n\nAutomatic code formatting:\n\n```bash\nmake format && make check-codestyle\n```\n\nRun tests:\n\n```bash\nmake test\n```\n\n### Recent releases\n\n##### v1.0.5\n* Windows support (do not require POSIX-only faster-fifo on Windows)\n\n##### v1.0.4\n* Use updated version of faster-fifo \n\n##### v1.0.3\n* Improved logging\n\n##### v1.0.2\n* Catching queue.Full exception to handle situations where receiver event loop process is killed\n\n##### v1.0.1\n* Added signal_slot.configure_logger() function to configure a custom logger\n\n##### v1.0.0\n* First PyPI version\n\n### Footnote\n\nOriginally designed for Sample Factory 2.0, a high-throughput asynchronous RL codebase https://github.com/alex-petrenko/sample-factory.\nDistributed under MIT License (see LICENSE), feel free to use for any purpose, commercial or not, at your own risk.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Fast and compact framework for communication between threads and processes in Python using event loops, signals and slots.",
"version": "1.0.5",
"project_urls": {
"Github": "https://github.com/alex-petrenko/signal-slot",
"Homepage": "https://github.com/alex-petrenko/signal-slot",
"Sample Factory": "https://github.com/alex-petrenko/sample-factory"
},
"split_keywords": [
"asynchronous",
"multithreading",
"multiprocessing",
"queue",
"faster-fifo",
"signal",
"slot",
"event",
"loop"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f5d4f886f72762a9fe465edf952662747fb595e2b4ab2f9fe948bbc27ba41055",
"md5": "d3b2fc5b32d3be4a5b86d3ed42e6a382",
"sha256": "aa069ccaf408561271d2c455d60afd01066bb015a9c5b8af1d350a6722c56c68"
},
"downloads": -1,
"filename": "signal_slot_mp-1.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d3b2fc5b32d3be4a5b86d3ed42e6a382",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 13723,
"upload_time": "2023-06-19T04:32:11",
"upload_time_iso_8601": "2023-06-19T04:32:11.559580Z",
"url": "https://files.pythonhosted.org/packages/f5/d4/f886f72762a9fe465edf952662747fb595e2b4ab2f9fe948bbc27ba41055/signal_slot_mp-1.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3f2a86bcc1df3132b7b3105be124299946cd053851593d6a8ab34a822a1ec65f",
"md5": "138c33c76525e01014f093d5914e46c6",
"sha256": "61c99044ac1ffbc01e5d6edebaa6294bb1346efe54cf195f01cfc898a95dd915"
},
"downloads": -1,
"filename": "signal-slot-mp-1.0.5.tar.gz",
"has_sig": false,
"md5_digest": "138c33c76525e01014f093d5914e46c6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 14523,
"upload_time": "2023-06-19T04:32:13",
"upload_time_iso_8601": "2023-06-19T04:32:13.185412Z",
"url": "https://files.pythonhosted.org/packages/3f/2a/86bcc1df3132b7b3105be124299946cd053851593d6a8ab34a822a1ec65f/signal-slot-mp-1.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-19 04:32:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "alex-petrenko",
"github_project": "signal-slot",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "signal-slot-mp"
}
JSON
