callosum


Namecallosum JSON
Version 1.0.3 PyPI version JSON
download
home_page
SummaryCallosum RPC Library
upload_time2024-03-04 10:16:19
maintainerJoongi Kim
docs_urlNone
authorLablup Inc.
requires_python>=3.11
licenseMIT
keywords asyncio rpc pubsub messaging
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            Callosum
========

An RPC Transport Library

It provides an asynchronous multi-channel order-preserving message and data
streaming transport for *upper* RPC layers (e.g., Apache Thrift) by wrapping
*lower* transport implementations (e.g., ZeroMQ).

It aims to follow the latest coding style and conventions in Python asyncio.

*Corpus callosum* is a bundle of neuron fibers that connects two cerebral
hemispheres of a human brain.

Prerequisite
------------

Python 3.11 or higher.

Features
--------

* RPC
  - Native timeout and cancellation support
  - Explicit server-to-client error propagation including stringified tracebacks
  - Order preserving based on user-defined keys while keeping executions asynchronous
* Streaming
  - Broadcast & shared pipelines
* Optional client authentication and encrypted communication
  - Currently supported for only ZeroMQ with its CURVE library
* Optional message compression using [snappy](https://pypi.org/project/python-snappy/)
* Replacible and combinable lower/upper layers (ZeroMQ/Redis + JSON/msgpack/Thrift)

Planned features
----------------

* Managed streaming (with acks)
* Tunneling to bundle other channels and generic network traffic in a single connection
* Bidirectional RPC
* Chunked transfer of large messages

Installation
------------

To install the core:

```console
$ pip install -U pip setuptools
$ pip install callosum
```

You may add extra dependencies like:

```console
$ pip install 'callosum[zeromq,redis,thrift,snappy]'
```

Examples
--------

Please check out [the examples directory.](https://github.com/lablup/callosum/tree/master/examples)

Development
-----------

Create a virtual environment or an isolated Python environment using your favorite tool.

Inside it, run editable installation as follows:

```console
$ pip install -U pip setuptools
$ pip install -U -r requirements/dev.txt
```

# Changes

<!--
    You should *NOT* be adding new change log entries to this file, this
    file is managed by towncrier. You *may* edit previous change logs to
    fix problems like typo corrections or such.

    To add a new change log entry, please refer
    https://pip.pypa.io/en/latest/development/contributing/#news-entries

    We named the news folder "changes".

    WARNING: Don't drop the last line!
-->

<!-- towncrier release notes start -->

## 1.0.3 (2024-03-04)

### Fixes
* Relax yarl version requirements to avoid potential dependency conflicts in the downstream Backend.AI platform ([#28](https://github.com/lablup/callosum/issues/28))


## 1.0.2 (2024-02-29)

### Fixes
* Stabilize ZMQ RPC auth and malformed packet handling ([#27](https://github.com/lablup/callosum/issues/27))

### Miscellaneous
* Update dependencies including pyzmq ('&gt;=23' &rarr; '&gt;=25.1.1' to support Python 3.12) and development tools including ruff to replace the black formatter ([#26](https://github.com/lablup/callosum/issues/26))


## v1.0.1 (2023-09-18)

### Fixes
* Prevent leaking secret keys in the logs and allow infinite timeouts on connection pings ([#24](https://github.com/lablup/callosum/issues/24))


## v1.0.0 (2023-09-14)

### Breaking change
* Now it requires Python 3.11 to work!

### Features
* Implement the full version of secure, encrypted RPC communication based on ZeroMQ's ZAP protocol using CURVE keypairs ([#21](https://github.com/lablup/callosum/issues/21))


## v0.9.10 (2022-02-17)

### Fixes
* Fix pyzmq attribute error on Ubuntu 20.04 at aarch64, which is built using older libzeromq without some socket monitoring event constants, by loading the constant declarations dynamically ([#20](https://github.com/lablup/callosum/issues/20))


## v0.9.9 (2021-10-05)

### Fixes
* Add explicit `task_done()` call to the output queue of `rpc.Peer` instances to avoid potential missing wakeup on joined coroutines ([#16](https://github.com/lablup/callosum/issues/16))


## v0.9.8 (2021-10-05)

### Features
* lower.zeromq: Add a transport option to attach monitors for logging underlying socket events ([#17](https://github.com/lablup/callosum/issues/17))

### Fixes
* Improve `zsock_opts` handling when explicitly specified by the library users, as previously it caused argument errors in binders and connectors ([#18](https://github.com/lablup/callosum/issues/18))


## v0.9.7 (2020-12-22)

### Features
* Provide `repr()` of exceptions in RPC user/internal errors for better application-level error logging ([#15](https://github.com/lablup/callosum/issues/15))


## v0.9.6 (2020-06-05)

### Features
* `upper.rpc`: Support server-side cancellation and propagation to clients by adding the `CANCELLED` RPC message type ([#14](https://github.com/lablup/callosum/issues/14))

### Deprecations
* Drop use of aiojobs in favor of native semaphores for limiting the task concurrency and less clutters on job scheduling semantics ([#14](https://github.com/lablup/callosum/issues/14))

### Fixes
* Stability updates for the RPC layer: ([#14](https://github.com/lablup/callosum/issues/14))
  - Fix wrong message sequence calcuation with `SEQ_BITS` and clarify the roles of `cleanup()` and `cancel()` methods in the schedulers.
  - Now we use the exit-ordered scheduler by default.


## v0.9.5 (2020-05-12)

### Fixes
* lower.zeromq: Use destroy() for zmq context termination to improve stability and shutdown open socekts cleanly ([#13](https://github.com/lablup/callosum/issues/13))


## v0.9.4 (2020-04-10)

### Fixes
* Fix a race condition due to overlapping RPC message sequence IDs by separating server/client message sequence IDs [(#12)](https://github.com/lablup/callosum/issues/12)

### Miscellaneous
* Adopt towncrier for changelog management [(#11)](https://github.com/lablup/callosum/issues/11)


## v0.9.3 (2020-03-20)

* FIX: Mitigate bogus KeyError when cleaning up task futures in RPC servers that has caused event
  loop spinning.


## v0.9.2 (2020-02-28)

* MAINTENANCE: Update dependencies and only specify the minimum versions since
  Callosum is a library.


## v0.9.1 (2020-01-05)

* FIX: wrong typing of `RPCMessage.body` field

* IMPROVE: Add `debug_rpc` option to `rpc.Peer` for logging exceptions in RPC
  scheduler and user-defined handlers explicitly.

* Update dependencies and remove unused ones.


## v0.9.0 (2019-12-06)

* First public release with a working RPC based on ZeroMQ DEALER/ROUTER sockets.


## 2018-05-02

* Started the project.

            

Raw data

            {
    "_id": null,
    "home_page": "",
    "name": "callosum",
    "maintainer": "Joongi Kim",
    "docs_url": null,
    "requires_python": ">=3.11",
    "maintainer_email": "joongi@lablup.com",
    "keywords": "asyncio,rpc,pubsub,messaging",
    "author": "Lablup Inc.",
    "author_email": "devops@lablup.com",
    "download_url": "https://files.pythonhosted.org/packages/c6/87/af504e56c3f43d73e22385fd9b05f3c63ac08885f3958ecd03443a17761f/callosum-1.0.3.tar.gz",
    "platform": "POSIX",
    "description": "Callosum\n========\n\nAn RPC Transport Library\n\nIt provides an asynchronous multi-channel order-preserving message and data\nstreaming transport for *upper* RPC layers (e.g., Apache Thrift) by wrapping\n*lower* transport implementations (e.g., ZeroMQ).\n\nIt aims to follow the latest coding style and conventions in Python asyncio.\n\n*Corpus callosum* is a bundle of neuron fibers that connects two cerebral\nhemispheres of a human brain.\n\nPrerequisite\n------------\n\nPython 3.11 or higher.\n\nFeatures\n--------\n\n* RPC\n  - Native timeout and cancellation support\n  - Explicit server-to-client error propagation including stringified tracebacks\n  - Order preserving based on user-defined keys while keeping executions asynchronous\n* Streaming\n  - Broadcast & shared pipelines\n* Optional client authentication and encrypted communication\n  - Currently supported for only ZeroMQ with its CURVE library\n* Optional message compression using [snappy](https://pypi.org/project/python-snappy/)\n* Replacible and combinable lower/upper layers (ZeroMQ/Redis + JSON/msgpack/Thrift)\n\nPlanned features\n----------------\n\n* Managed streaming (with acks)\n* Tunneling to bundle other channels and generic network traffic in a single connection\n* Bidirectional RPC\n* Chunked transfer of large messages\n\nInstallation\n------------\n\nTo install the core:\n\n```console\n$ pip install -U pip setuptools\n$ pip install callosum\n```\n\nYou may add extra dependencies like:\n\n```console\n$ pip install 'callosum[zeromq,redis,thrift,snappy]'\n```\n\nExamples\n--------\n\nPlease check out [the examples directory.](https://github.com/lablup/callosum/tree/master/examples)\n\nDevelopment\n-----------\n\nCreate a virtual environment or an isolated Python environment using your favorite tool.\n\nInside it, run editable installation as follows:\n\n```console\n$ pip install -U pip setuptools\n$ pip install -U -r requirements/dev.txt\n```\n\n# Changes\n\n<!--\n    You should *NOT* be adding new change log entries to this file, this\n    file is managed by towncrier. You *may* edit previous change logs to\n    fix problems like typo corrections or such.\n\n    To add a new change log entry, please refer\n    https://pip.pypa.io/en/latest/development/contributing/#news-entries\n\n    We named the news folder \"changes\".\n\n    WARNING: Don't drop the last line!\n-->\n\n<!-- towncrier release notes start -->\n\n## 1.0.3 (2024-03-04)\n\n### Fixes\n* Relax yarl version requirements to avoid potential dependency conflicts in the downstream Backend.AI platform ([#28](https://github.com/lablup/callosum/issues/28))\n\n\n## 1.0.2 (2024-02-29)\n\n### Fixes\n* Stabilize ZMQ RPC auth and malformed packet handling ([#27](https://github.com/lablup/callosum/issues/27))\n\n### Miscellaneous\n* Update dependencies including pyzmq ('&gt;=23' &rarr; '&gt;=25.1.1' to support Python 3.12) and development tools including ruff to replace the black formatter ([#26](https://github.com/lablup/callosum/issues/26))\n\n\n## v1.0.1 (2023-09-18)\n\n### Fixes\n* Prevent leaking secret keys in the logs and allow infinite timeouts on connection pings ([#24](https://github.com/lablup/callosum/issues/24))\n\n\n## v1.0.0 (2023-09-14)\n\n### Breaking change\n* Now it requires Python 3.11 to work!\n\n### Features\n* Implement the full version of secure, encrypted RPC communication based on ZeroMQ's ZAP protocol using CURVE keypairs ([#21](https://github.com/lablup/callosum/issues/21))\n\n\n## v0.9.10 (2022-02-17)\n\n### Fixes\n* Fix pyzmq attribute error on Ubuntu 20.04 at aarch64, which is built using older libzeromq without some socket monitoring event constants, by loading the constant declarations dynamically ([#20](https://github.com/lablup/callosum/issues/20))\n\n\n## v0.9.9 (2021-10-05)\n\n### Fixes\n* Add explicit `task_done()` call to the output queue of `rpc.Peer` instances to avoid potential missing wakeup on joined coroutines ([#16](https://github.com/lablup/callosum/issues/16))\n\n\n## v0.9.8 (2021-10-05)\n\n### Features\n* lower.zeromq: Add a transport option to attach monitors for logging underlying socket events ([#17](https://github.com/lablup/callosum/issues/17))\n\n### Fixes\n* Improve `zsock_opts` handling when explicitly specified by the library users, as previously it caused argument errors in binders and connectors ([#18](https://github.com/lablup/callosum/issues/18))\n\n\n## v0.9.7 (2020-12-22)\n\n### Features\n* Provide `repr()` of exceptions in RPC user/internal errors for better application-level error logging ([#15](https://github.com/lablup/callosum/issues/15))\n\n\n## v0.9.6 (2020-06-05)\n\n### Features\n* `upper.rpc`: Support server-side cancellation and propagation to clients by adding the `CANCELLED` RPC message type ([#14](https://github.com/lablup/callosum/issues/14))\n\n### Deprecations\n* Drop use of aiojobs in favor of native semaphores for limiting the task concurrency and less clutters on job scheduling semantics ([#14](https://github.com/lablup/callosum/issues/14))\n\n### Fixes\n* Stability updates for the RPC layer: ([#14](https://github.com/lablup/callosum/issues/14))\n  - Fix wrong message sequence calcuation with `SEQ_BITS` and clarify the roles of `cleanup()` and `cancel()` methods in the schedulers.\n  - Now we use the exit-ordered scheduler by default.\n\n\n## v0.9.5 (2020-05-12)\n\n### Fixes\n* lower.zeromq: Use destroy() for zmq context termination to improve stability and shutdown open socekts cleanly ([#13](https://github.com/lablup/callosum/issues/13))\n\n\n## v0.9.4 (2020-04-10)\n\n### Fixes\n* Fix a race condition due to overlapping RPC message sequence IDs by separating server/client message sequence IDs [(#12)](https://github.com/lablup/callosum/issues/12)\n\n### Miscellaneous\n* Adopt towncrier for changelog management [(#11)](https://github.com/lablup/callosum/issues/11)\n\n\n## v0.9.3 (2020-03-20)\n\n* FIX: Mitigate bogus KeyError when cleaning up task futures in RPC servers that has caused event\n  loop spinning.\n\n\n## v0.9.2 (2020-02-28)\n\n* MAINTENANCE: Update dependencies and only specify the minimum versions since\n  Callosum is a library.\n\n\n## v0.9.1 (2020-01-05)\n\n* FIX: wrong typing of `RPCMessage.body` field\n\n* IMPROVE: Add `debug_rpc` option to `rpc.Peer` for logging exceptions in RPC\n  scheduler and user-defined handlers explicitly.\n\n* Update dependencies and remove unused ones.\n\n\n## v0.9.0 (2019-12-06)\n\n* First public release with a working RPC based on ZeroMQ DEALER/ROUTER sockets.\n\n\n## 2018-05-02\n\n* Started the project.\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Callosum RPC Library",
    "version": "1.0.3",
    "project_urls": {
        "Changelog": "https://github.com/lablup/callosum/blob/main/CHANGES.md",
        "Download": "https://pypi.org/project/callosum",
        "Homepage": "https://github.com/lablup/callosum",
        "Issues": "https://github.com/lablup/callosum/issues",
        "Repository": "https://github.com/lablup/callosum"
    },
    "split_keywords": [
        "asyncio",
        "rpc",
        "pubsub",
        "messaging"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d734be5c545880a3fb21c74a828dde06c9c281a8769902a4f1bed373816f772d",
                "md5": "b2f8f8b7ac76b439951bbab729e5b47b",
                "sha256": "2dd92ab8d86408df943510539f64d40844d755dc53799ed6a16b2312a77b9557"
            },
            "downloads": -1,
            "filename": "callosum-1.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b2f8f8b7ac76b439951bbab729e5b47b",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.11",
            "size": 33019,
            "upload_time": "2024-03-04T10:16:18",
            "upload_time_iso_8601": "2024-03-04T10:16:18.107788Z",
            "url": "https://files.pythonhosted.org/packages/d7/34/be5c545880a3fb21c74a828dde06c9c281a8769902a4f1bed373816f772d/callosum-1.0.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c687af504e56c3f43d73e22385fd9b05f3c63ac08885f3958ecd03443a17761f",
                "md5": "1e93e88bd237bdddcd73d3fed2c085b9",
                "sha256": "381b62718accbc777c05eb8d6ae2fd98ab68fca2b5842de7ef94fdf7d8de5a38"
            },
            "downloads": -1,
            "filename": "callosum-1.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "1e93e88bd237bdddcd73d3fed2c085b9",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.11",
            "size": 42176,
            "upload_time": "2024-03-04T10:16:19",
            "upload_time_iso_8601": "2024-03-04T10:16:19.496387Z",
            "url": "https://files.pythonhosted.org/packages/c6/87/af504e56c3f43d73e22385fd9b05f3c63ac08885f3958ecd03443a17761f/callosum-1.0.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-03-04 10:16:19",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "lablup",
    "github_project": "callosum",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "callosum"
}
        
Elapsed time: 3.17110s