# TNO PET Lab - secure Multi-Party Computation (MPC) - Communication
The TNO PET Lab consists of generic software components, procedures, and
functionalities developed and maintained on a regular basis to facilitate and
aid in the development of PET solutions. The lab is a cross-project initiative
allowing us to integrate and reuse previously developed PET functionalities to
boost the development of new protocols and solutions.
The package tno.mpc.communication is part of the TNO Python Toolbox.
_Limitations in (end-)use: the content of this repository may solely be used for
applications that comply with international export control laws._
_This implementation of cryptographic software has not been audited. Use at your
own risk._
## Documentation
Documentation of the tno.mpc.communication package can be found
[here](https://docs.pet.tno.nl/mpc/communication/4.8.1).
## Install
Easily install the tno.mpc.communication package using pip:
```console
$ python -m pip install tno.mpc.communication
```
Note: The package specifies several optional dependency groups:
- `gmpy`: Adds support for sending various `gmpy2` types
- `tests`: Includes all optional libraries required to run the full test suite
- `tls`: Required if SSL is needed
- `bitarray`: Adds support for sending `bitarray` types
- `numpy`: Adds support for sending `numpy` types
- `pandas`: Adds support for sending `pandas` types
See [sending, receiving messages](#sending-receiving-messages) for more
information on the supported third party types. Optional dependencies can be
installed by specifying their names in brackets after the package name, e.g.
when using `pip install`, use `pip install tno.mpc.communication[extra1,extra2]`
to install the groups `extra1` and `extra2`.
## Usage
The communication module uses `async` functions for sending and receiving. If
you are familiar with the async module, you can skip to the `Pools` section.
### Async explanation
When `async` functions are called, they return what is called a _coroutine_.
This is a special kind of object, because it is basically a promise that the
code will be run and a result will be given when the coroutine is given to a
so-called _event loop_. For example, see the following
```python
import asyncio
async def add(a: int, b: int) -> int:
return a + b
def main():
a,b = 1, 2
coroutine_object = add(a, b) # This is now a coroutine object of type Awaitable[int]
event_loop = asyncio.get_event_loop() # This is the event loop that will run the coroutine
result = event_loop.run_until_complete(coroutine_object) # This call starts the coroutine in the event loop
print(result) # this prints 3
if __name__ == "__main__":
main()
```
As you can see from the example, the async methods are defined using
`async def`, which tells python that it should return a coroutine. We saw how we
can call an async function from a regular function using the event loop. _Note
that you should never redefine the event loop and always retrieve the event loop
as done in the example_ (unless you know what you are doing). We can also call
async functions from other async functions using the `await` statement, as is
shown in the following example.
```python
import asyncio
async def add_four_numbers(first: int, second: int, third: int, fourth: int) -> int:
first_second = await add(first, second) # This is blocking, so the function will wait until this is done
third_fourth_coroutine = add(third, fourth) # This is non-blocking, so the code will continue while the add(third,fourth) code starts running
# we can do some other stuff here
print("I am a print statement")
third_fourth = await third_fourth_coroutine # we wait until the add(third,fourth) is done
result = await add(first_second, third_fourth)
# here it is important to use await for the result, because then an integer is produced and given
# to the return statement instead of a coroutine
return result
async def add(a: int, b: int) -> int:
return a + b
def main():
a, b, c, d = 1, 2, 3, 4
coroutine_object = add_four_numbers(a, b, c, d) # This is now a coroutine object of type Awaitable[int]
event_loop = asyncio.get_event_loop() # This is the event loop that will run the coroutine
result = event_loop.run_until_complete(coroutine_object) # This call starts the coroutine in the event loop
print(result) # this prints 10
if __name__ == "__main__":
main()
```
Note that the type of the `coroutine_object` in the `main` function is an
`Awaitable[int]`. This refers to the fact that the result can be awaited (inside
an `async` function) and will return an integer once that is done.
### Pools
A `Pool` represents a network. A Pool contains a server, which listens for
incoming messages from other parties in the network, and clients for each other
party in the network. These clients are called upon when we want to send or
receive messages.
It is also possible to use and initialize the pool without taking care of the
event loop yourself, in that case the template below can be ignored and the
examples can be used as one would regularly do. (An event loop is however still
needed when using the `await` keyword or when calling an `async` function.)
### Template
Below you can find a template for using `Pool`. Alternatively, you could create
the pool in the `main` logic and give it as a parameter to the `async_main`
function.
```python
import asyncio
from tno.mpc.communication import Pool
async def async_main():
pool = Pool()
# ...
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(async_main())
```
### Pool initialization
The following logic works both in regular functions and `async` functions.
#### Without SSL/TLS (do not use in production)
The following snippet will start a HTTP server and define its clients. Clients
are configured on both the sending and the receiving side. The sending side
needs to know who to send a message to. The receiving side needs to know who it
receives a message from for further handling.
By default the `Pool` object uses the origin IP and port to identify the client.
However, a more secure and robust identification through SSL/TLS certificates is
also supported and described in section
[With SSL/TLS (SSL/TLS certificate as client identifier)](#with-ssltls-ssltls-certificate-as-client-identifier).
```python
from tno.mpc.communication import Pool
pool = Pool()
pool.add_http_server() # default port=80
pool.add_http_client("Client 1", "192.168.0.101") # default port=80
pool.add_http_client("Client 2", "192.168.0.102", port=1234)
```
#### With SSL/TLS
A more secure connection can be achieved by using SSL/TLS. A `Pool` object can
be initialized with paths to key, certificate and CA certificate files that are
passed as arguments to a
[`ssl.SSLContext`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext)
object. More information on the expected files can be found in the
`Pool.__init__` docstring and the
[`ssl` documentation](https://docs.python.org/3/library/ssl.html#certificates).
```python
from tno.mpc.communication import Pool
pool = Pool(key="path/to/keyfile", cert="path/to/certfile", ca_cert="path/to/cafile")
pool.add_http_server() # default port=443
pool.add_http_client("Client 1", "192.168.0.101") # default port=443
pool.add_http_client("Client 2", "192.168.0.102", port=1234)
```
We do not pose constraints on the certificates that you use in the protocol.
However, your organisation most likely poses minimal security requirements on
the certificates used. As such we do not advocate a method for generating
certificates but rather suggest to contact your system administrator for
obtaining certificates.
#### With SSL/TLS (SSL/TLS certificate as client identifier)
This approach does not use the origin of a message (HTTP request) as identifier
of a party, but rather the SSL/TLS certificate of that party. This requires a
priori exchange of the certificates, but is more robust to more complex (docker)
network stacks, proxies, port forwarding, load balancers, IP spoofing, etc.
More specifically, we assume that a certificate has a unique combination of
issuer Common Name and S/N and use these components to create a HTTP client
identifier. Our assumption is based on the fact that we trust the issuer (TSL
assumption) and that the issuer is supposed to hand out end-user certificates
with different serial numbers.
```python
from tno.mpc.communication import Pool
pool = Pool(key="path/to/own/keyfile", cert="path/to/own/certfile", ca_cert="path/to/cafile")
pool.add_http_server() # default port=443
pool.add_http_client("Client 1", "192.168.0.101", port=1234, cert="path/to/client/certfile")
```
Additional dependencies are required in order to load and compare certificates.
These can be installed by installing this package with the `tls` extra, e.g.
`pip install tno.mpc.communication[tls]`.
#### Adding clients
HTTP clients are identified by an address. The address can be an IP address, but
hostnames are also supported. For example, when communicating between two docker
containers on the same network, the address that is provided to
`pool.add_http_client` can either be the IP address of the client container or
the name of the client container.
### Sending, receiving messages
The library supports sending the following objects through the send and receive
methods:
- strings
- byte strings
- integers
- floats
- enum (partially, see [Serializing `Enum`](#serializing-enum))
- (nested) lists/tuples/dictionaries/numpy arrays containing any of the above.
Combinations of these as well.
Under the hood [`ormsgpack`](https://pypi.org/project/ormsgpack) is used,
additional options can be activated using the `option` parameter (see,
https://github.com/aviramha/ormsgpack#option).
Messages can be sent both synchronously and asynchronously. If you do not know
which one to use, use the synchronous methods with `await`.
```python
# Client 0
await pool.send("Client 1", "Hello!") # Synchronous send message (blocking)
pool.asend("Client 1", "Hello!") # Asynchronous send message (non-blocking, schedule send task)
# Client 1
res = await pool.recv("Client 0") # Receive message synchronously (blocking)
res = pool.arecv("Client 0") # Receive message asynchronously (non-blocking, returns Future if message did not arrive yet)
```
### Custom message IDs
```python
# Client 0
await pool.send("Client 1", "Hello!", "Message ID 1")
# Client 1
res = await pool.recv("Client 0", "Message ID 1")
```
### Custom serialization logic
It is also possible to define serialization logic in custom classes and load the
logic into the commmunication module. An example is given below. We elaborate on
the requirements for such classes after the example.
```python
class SomeClass:
def serialize(self, **kwargs: Any) -> Dict[str, Any]:
# serialization logic that returns a dictionary
@staticmethod
def deserialize(obj: Dict[str, Any], **kwargs: Any) -> 'SomeClass':
# deserialization logic that turns the dictionary produced
# by serialize back into an object of type SomeClass
```
The class needs to contain a `serialize` method and a `deserialize` method. The
type annotation is necessary and validated by the communication module. Next to
this, the `**kwargs` argument is also necessary to allow for nested
(de)serialization that makes use of additional optional keyword arguments. It is
not necessary to use any of these optional keyword arguments. If one does not
make use of the `**kwargs` and also does not make a call to a subsequent
`Serialization.serialize()` or `Serialization.deserialize()`, it is advised to
write `**_kwargs: Any` instead of `**kwargs: Any`.
To add this logic to the communication module, you have to run the following
command at the start of your script. The `check_annotiations` parameter
determines whether the type hints of the serialization code and the presence of
a `**kwargs` parameter are checked. You should only change this to False _if you
are exactly sure of what you are doing_.
```python
from tno.mpc.communication import Serialization
if __name__ == "__main__":
Serialization.set_serialization_logic(SomeClass, check_annotations=True)
```
### Serializing `Enum`
The `Serialization` module can serialize an `Enum` member; however, only the
value is serialized. The simplest way to work around this limitation is to
convert the deserialized object into an `Enum` member:
```python
from enum import Enum, auto
class TestEnum(Enum):
A = auto()
B = auto()
enum_obj = TestEnum.B
# Client 0
await pool.send("Client 1", enum_obj)
# Client 1
res = await pool.recv("Client 0") # 2 <class 'int'>
enum_res = TestEnum(res) # TestEnum.B <enum 'TestEnum'>
```
## Example code
Below is a very minimal example of how to use the library. It consists of two
instances, Alice and Bob, who greet each other. Here, Alice runs on localhost
and uses port 61001 for sending/receiving. Bob also runs on localhost, but uses
port 61002.
`alice.py`
```python
import asyncio
from tno.mpc.communication import Pool
async def async_main():
# Create the pool for Alice.
# Alice listens on port 61001 and adds Bob as client.
pool = Pool()
pool.add_http_server(addr="127.0.0.1", port=61001)
pool.add_http_client("Bob", addr="127.0.0.1", port=61002)
# Alice sends a message to Bob and waits for a reply.
# She prints the reply and shuts down the pool
await pool.send("Bob", "Hello Bob! This is Alice speaking.")
reply = await pool.recv("Bob")
print(reply)
await pool.shutdown()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(async_main())
```
`bob.py`
```python
import asyncio
from tno.mpc.communication import Pool
async def async_main():
# Create the pool for Bob.
# Bob listens on port 61002 and adds Alice as client.
pool = Pool()
pool.add_http_server(addr="127.0.0.1", port=61002)
pool.add_http_client("Alice", addr="127.0.0.1", port=61001)
# Bob waits for a message from Alice and prints it.
# He replies and shuts down his pool instance.
message = await pool.recv("Alice")
print(message)
await pool.send("Alice", "Hello back to you, Alice!")
await pool.shutdown()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(async_main())
```
To run this example, run each of the files in a separate terminal window. Note
that if `alice.py` is started prior to `bob.py`, it will throw a
`ClientConnectorError`. Namely, Alice tries to send a message to port 61002,
which has not been opened by Bob yet. After starting `bob.py`, the error
disappears.
The outputs in the two terminals will be something similar to the following:
```bash
>>> python alice.py
2022-07-07 09:36:20,220 - tno.mpc.communication.httphandlers - INFO - Serving on 127.0.0.1:61001
2022-07-07 09:36:20,230 - tno.mpc.communication.httphandlers - INFO - Received message from 127.0.0.1:61002
Hello back to you, Bob!
2022-07-07 09:36:20,232 - tno.mpc.communication.httphandlers - INFO - HTTPServer: Shutting down server task
2022-07-07 09:36:20,232 - tno.mpc.communication.httphandlers - INFO - Server 127.0.0.1:61001 shutdown
```
```bash
>>> python bob.py
2022-07-07 09:36:16,915 - tno.mpc.communication.httphandlers - INFO - Serving on 127.0.0.1:61002
2022-07-07 09:36:20,223 - tno.mpc.communication.httphandlers - INFO - Received message from 127.0.0.1:61001
Hello Bob! This is Alice speaking.
2022-07-07 09:36:20,232 - tno.mpc.communication.httphandlers - INFO - HTTPServer: Shutting down server task
2022-07-07 09:36:20,256 - tno.mpc.communication.httphandlers - INFO - Server 127.0.0.1:61002 shutdown
```
## Test fixtures
The `tno.mpc.communication` package exports several pytest fixtures as pytest
plugins to facilitate the user in testing with pool objects. The fixtures take
care of all configuration and clean-up of the pool objects so that you don't
have to worry about that.
Usage:
```py
# test_my_module.py
import pytest
from typing import Callable
from tno.mpc.communication import Pool
def test_with_two_pools(http_pool_duo: tuple[Pool, Pool]) -> None:
sender, receiver = http_pool_duo
# ... your code
def test_with_three_pools(http_pool_trio: tuple[Pool, Pool, Pool]) -> None:
alice, bob, charlie = http_pool_trio
# ... your code
@pytest.mark.parameterize("n_players", (2, 3, 4))
def test_with_variable_pools(
n_players: int,
http_pool_group_factory: Callable[[int], tuple[Pool, ...]],
) -> None:
pools = http_pool_group_factory(n_players)
# ... your code
```
### Fixture scope
The scope of the fixtures can be set dynamically through the
`--fixture-pool-scope`
[option to pytest](https://docs.pytest.org/en/7.1.x/reference/reference.html#configuration-options).
_Note that this will also change the scope of the global `event_loop` fixture
that is provided by `pytest-asyncio`._ By default, in line with
`pytest_asyncio`, the scope of all our fixtures is `"function"`. We advise to
configure a larger scope (e.g. `"session"`, `"package"` or `"module"`) when
possible to reduce test set-up and teardown time.
Our fixtures pass `True` to the `port_reuse` argument of `aiohttp.web.TCPSite`.
Their
[documentation](https://docs.aiohttp.org/en/stable/web_reference.html?highlight=reuse_port#aiohttp.web.TCPSite)
states that this option is not supported on Windows (outside of WSL). If you
experience any issues, please disable the plugin by adding
`-p no:pytest_tno.tno.mpc.communication.pytest_pool_fixtures` to your pytest
configuration. Note that without `port_reuse` the tests may crash, as the test
may try to bind to ports which may not have been freed by the operating system.
For more reliable testing, run the tests on a WSL / Linux platform.
Raw data
{
"_id": null,
"home_page": "",
"name": "tno.mpc.communication",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "TNO PET Lab <petlab@tno.nl>",
"keywords": "TNO,MPC,multi-party computation,communication",
"author": "",
"author_email": "TNO PET Lab <petlab@tno.nl>",
"download_url": "https://files.pythonhosted.org/packages/95/d9/81ec88a4570ef642cfec991e30010f4379fd2b49a0f4b3128f0f073e1b8d/tno.mpc.communication-4.8.1.tar.gz",
"platform": "any",
"description": "# TNO PET Lab - secure Multi-Party Computation (MPC) - Communication\n\nThe TNO PET Lab consists of generic software components, procedures, and\nfunctionalities developed and maintained on a regular basis to facilitate and\naid in the development of PET solutions. The lab is a cross-project initiative\nallowing us to integrate and reuse previously developed PET functionalities to\nboost the development of new protocols and solutions.\n\nThe package tno.mpc.communication is part of the TNO Python Toolbox.\n\n_Limitations in (end-)use: the content of this repository may solely be used for\napplications that comply with international export control laws._ \n_This implementation of cryptographic software has not been audited. Use at your\nown risk._\n\n## Documentation\n\nDocumentation of the tno.mpc.communication package can be found\n[here](https://docs.pet.tno.nl/mpc/communication/4.8.1).\n\n## Install\n\nEasily install the tno.mpc.communication package using pip:\n\n```console\n$ python -m pip install tno.mpc.communication\n```\n\nNote: The package specifies several optional dependency groups:\n\n- `gmpy`: Adds support for sending various `gmpy2` types\n- `tests`: Includes all optional libraries required to run the full test suite\n- `tls`: Required if SSL is needed\n- `bitarray`: Adds support for sending `bitarray` types\n- `numpy`: Adds support for sending `numpy` types\n- `pandas`: Adds support for sending `pandas` types\n\nSee [sending, receiving messages](#sending-receiving-messages) for more\ninformation on the supported third party types. Optional dependencies can be\ninstalled by specifying their names in brackets after the package name, e.g.\nwhen using `pip install`, use `pip install tno.mpc.communication[extra1,extra2]`\nto install the groups `extra1` and `extra2`.\n\n## Usage\n\nThe communication module uses `async` functions for sending and receiving. If\nyou are familiar with the async module, you can skip to the `Pools` section.\n\n### Async explanation\n\nWhen `async` functions are called, they return what is called a _coroutine_.\nThis is a special kind of object, because it is basically a promise that the\ncode will be run and a result will be given when the coroutine is given to a\nso-called _event loop_. For example, see the following\n\n```python\nimport asyncio\n\nasync def add(a: int, b: int) -> int:\n return a + b\n\ndef main():\n a,b = 1, 2\n coroutine_object = add(a, b) # This is now a coroutine object of type Awaitable[int]\n event_loop = asyncio.get_event_loop() # This is the event loop that will run the coroutine\n result = event_loop.run_until_complete(coroutine_object) # This call starts the coroutine in the event loop\n print(result) # this prints 3\n\nif __name__ == \"__main__\":\n main()\n```\n\nAs you can see from the example, the async methods are defined using\n`async def`, which tells python that it should return a coroutine. We saw how we\ncan call an async function from a regular function using the event loop. _Note\nthat you should never redefine the event loop and always retrieve the event loop\nas done in the example_ (unless you know what you are doing). We can also call\nasync functions from other async functions using the `await` statement, as is\nshown in the following example.\n\n```python\nimport asyncio\n\nasync def add_four_numbers(first: int, second: int, third: int, fourth: int) -> int:\n first_second = await add(first, second) # This is blocking, so the function will wait until this is done\n third_fourth_coroutine = add(third, fourth) # This is non-blocking, so the code will continue while the add(third,fourth) code starts running\n # we can do some other stuff here\n print(\"I am a print statement\")\n third_fourth = await third_fourth_coroutine # we wait until the add(third,fourth) is done\n result = await add(first_second, third_fourth)\n # here it is important to use await for the result, because then an integer is produced and given\n # to the return statement instead of a coroutine\n return result\n\nasync def add(a: int, b: int) -> int:\n return a + b\n\ndef main():\n a, b, c, d = 1, 2, 3, 4\n coroutine_object = add_four_numbers(a, b, c, d) # This is now a coroutine object of type Awaitable[int]\n event_loop = asyncio.get_event_loop() # This is the event loop that will run the coroutine\n result = event_loop.run_until_complete(coroutine_object) # This call starts the coroutine in the event loop\n print(result) # this prints 10\n\nif __name__ == \"__main__\":\n main()\n```\n\nNote that the type of the `coroutine_object` in the `main` function is an\n`Awaitable[int]`. This refers to the fact that the result can be awaited (inside\nan `async` function) and will return an integer once that is done.\n\n### Pools\n\nA `Pool` represents a network. A Pool contains a server, which listens for\nincoming messages from other parties in the network, and clients for each other\nparty in the network. These clients are called upon when we want to send or\nreceive messages.\n\nIt is also possible to use and initialize the pool without taking care of the\nevent loop yourself, in that case the template below can be ignored and the\nexamples can be used as one would regularly do. (An event loop is however still\nneeded when using the `await` keyword or when calling an `async` function.)\n\n### Template\n\nBelow you can find a template for using `Pool`. Alternatively, you could create\nthe pool in the `main` logic and give it as a parameter to the `async_main`\nfunction.\n\n```python\nimport asyncio\n\nfrom tno.mpc.communication import Pool\n\nasync def async_main():\n pool = Pool()\n # ...\n\nif __name__ == \"__main__\":\n loop = asyncio.get_event_loop()\n loop.run_until_complete(async_main())\n```\n\n### Pool initialization\n\nThe following logic works both in regular functions and `async` functions.\n\n#### Without SSL/TLS (do not use in production)\n\nThe following snippet will start a HTTP server and define its clients. Clients\nare configured on both the sending and the receiving side. The sending side\nneeds to know who to send a message to. The receiving side needs to know who it\nreceives a message from for further handling.\n\nBy default the `Pool` object uses the origin IP and port to identify the client.\nHowever, a more secure and robust identification through SSL/TLS certificates is\nalso supported and described in section\n[With SSL/TLS (SSL/TLS certificate as client identifier)](#with-ssltls-ssltls-certificate-as-client-identifier).\n\n```python\nfrom tno.mpc.communication import Pool\n\npool = Pool()\npool.add_http_server() # default port=80\npool.add_http_client(\"Client 1\", \"192.168.0.101\") # default port=80\npool.add_http_client(\"Client 2\", \"192.168.0.102\", port=1234)\n```\n\n#### With SSL/TLS\n\nA more secure connection can be achieved by using SSL/TLS. A `Pool` object can\nbe initialized with paths to key, certificate and CA certificate files that are\npassed as arguments to a\n[`ssl.SSLContext`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext)\nobject. More information on the expected files can be found in the\n`Pool.__init__` docstring and the\n[`ssl` documentation](https://docs.python.org/3/library/ssl.html#certificates).\n\n```python\nfrom tno.mpc.communication import Pool\n\npool = Pool(key=\"path/to/keyfile\", cert=\"path/to/certfile\", ca_cert=\"path/to/cafile\")\npool.add_http_server() # default port=443\npool.add_http_client(\"Client 1\", \"192.168.0.101\") # default port=443\npool.add_http_client(\"Client 2\", \"192.168.0.102\", port=1234)\n```\n\nWe do not pose constraints on the certificates that you use in the protocol.\nHowever, your organisation most likely poses minimal security requirements on\nthe certificates used. As such we do not advocate a method for generating\ncertificates but rather suggest to contact your system administrator for\nobtaining certificates.\n\n#### With SSL/TLS (SSL/TLS certificate as client identifier)\n\nThis approach does not use the origin of a message (HTTP request) as identifier\nof a party, but rather the SSL/TLS certificate of that party. This requires a\npriori exchange of the certificates, but is more robust to more complex (docker)\nnetwork stacks, proxies, port forwarding, load balancers, IP spoofing, etc.\n\nMore specifically, we assume that a certificate has a unique combination of\nissuer Common Name and S/N and use these components to create a HTTP client\nidentifier. Our assumption is based on the fact that we trust the issuer (TSL\nassumption) and that the issuer is supposed to hand out end-user certificates\nwith different serial numbers.\n\n```python\nfrom tno.mpc.communication import Pool\n\npool = Pool(key=\"path/to/own/keyfile\", cert=\"path/to/own/certfile\", ca_cert=\"path/to/cafile\")\npool.add_http_server() # default port=443\npool.add_http_client(\"Client 1\", \"192.168.0.101\", port=1234, cert=\"path/to/client/certfile\")\n```\n\nAdditional dependencies are required in order to load and compare certificates.\nThese can be installed by installing this package with the `tls` extra, e.g.\n`pip install tno.mpc.communication[tls]`.\n\n#### Adding clients\n\nHTTP clients are identified by an address. The address can be an IP address, but\nhostnames are also supported. For example, when communicating between two docker\ncontainers on the same network, the address that is provided to\n`pool.add_http_client` can either be the IP address of the client container or\nthe name of the client container.\n\n### Sending, receiving messages\n\nThe library supports sending the following objects through the send and receive\nmethods:\n\n- strings\n- byte strings\n- integers\n- floats\n- enum (partially, see [Serializing `Enum`](#serializing-enum))\n- (nested) lists/tuples/dictionaries/numpy arrays containing any of the above.\n Combinations of these as well.\n\nUnder the hood [`ormsgpack`](https://pypi.org/project/ormsgpack) is used,\nadditional options can be activated using the `option` parameter (see,\nhttps://github.com/aviramha/ormsgpack#option).\n\nMessages can be sent both synchronously and asynchronously. If you do not know\nwhich one to use, use the synchronous methods with `await`.\n\n```python\n# Client 0\nawait pool.send(\"Client 1\", \"Hello!\") # Synchronous send message (blocking)\npool.asend(\"Client 1\", \"Hello!\") # Asynchronous send message (non-blocking, schedule send task)\n\n# Client 1\nres = await pool.recv(\"Client 0\") # Receive message synchronously (blocking)\nres = pool.arecv(\"Client 0\") # Receive message asynchronously (non-blocking, returns Future if message did not arrive yet)\n```\n\n### Custom message IDs\n\n```python\n# Client 0\nawait pool.send(\"Client 1\", \"Hello!\", \"Message ID 1\")\n\n# Client 1\nres = await pool.recv(\"Client 0\", \"Message ID 1\")\n```\n\n### Custom serialization logic\n\nIt is also possible to define serialization logic in custom classes and load the\nlogic into the commmunication module. An example is given below. We elaborate on\nthe requirements for such classes after the example.\n\n```python\nclass SomeClass:\n\n def serialize(self, **kwargs: Any) -> Dict[str, Any]:\n # serialization logic that returns a dictionary\n\n @staticmethod\n def deserialize(obj: Dict[str, Any], **kwargs: Any) -> 'SomeClass':\n # deserialization logic that turns the dictionary produced\n # by serialize back into an object of type SomeClass\n```\n\nThe class needs to contain a `serialize` method and a `deserialize` method. The\ntype annotation is necessary and validated by the communication module. Next to\nthis, the `**kwargs` argument is also necessary to allow for nested\n(de)serialization that makes use of additional optional keyword arguments. It is\nnot necessary to use any of these optional keyword arguments. If one does not\nmake use of the `**kwargs` and also does not make a call to a subsequent\n`Serialization.serialize()` or `Serialization.deserialize()`, it is advised to\nwrite `**_kwargs: Any` instead of `**kwargs: Any`.\n\nTo add this logic to the communication module, you have to run the following\ncommand at the start of your script. The `check_annotiations` parameter\ndetermines whether the type hints of the serialization code and the presence of\na `**kwargs` parameter are checked. You should only change this to False _if you\nare exactly sure of what you are doing_.\n\n```python\nfrom tno.mpc.communication import Serialization\n\nif __name__ == \"__main__\":\n Serialization.set_serialization_logic(SomeClass, check_annotations=True)\n```\n\n### Serializing `Enum`\n\nThe `Serialization` module can serialize an `Enum` member; however, only the\nvalue is serialized. The simplest way to work around this limitation is to\nconvert the deserialized object into an `Enum` member:\n\n```python\nfrom enum import Enum, auto\n\n\nclass TestEnum(Enum):\n A = auto()\n B = auto()\n\nenum_obj = TestEnum.B\n\n# Client 0\nawait pool.send(\"Client 1\", enum_obj)\n\n# Client 1\nres = await pool.recv(\"Client 0\") # 2 <class 'int'>\nenum_res = TestEnum(res) # TestEnum.B <enum 'TestEnum'>\n```\n\n## Example code\n\nBelow is a very minimal example of how to use the library. It consists of two\ninstances, Alice and Bob, who greet each other. Here, Alice runs on localhost\nand uses port 61001 for sending/receiving. Bob also runs on localhost, but uses\nport 61002.\n\n`alice.py`\n\n```python\nimport asyncio\n\nfrom tno.mpc.communication import Pool\n\n\nasync def async_main():\n # Create the pool for Alice.\n # Alice listens on port 61001 and adds Bob as client.\n pool = Pool()\n pool.add_http_server(addr=\"127.0.0.1\", port=61001)\n pool.add_http_client(\"Bob\", addr=\"127.0.0.1\", port=61002)\n\n # Alice sends a message to Bob and waits for a reply.\n # She prints the reply and shuts down the pool\n await pool.send(\"Bob\", \"Hello Bob! This is Alice speaking.\")\n reply = await pool.recv(\"Bob\")\n print(reply)\n await pool.shutdown()\n\n\nif __name__ == \"__main__\":\n loop = asyncio.get_event_loop()\n loop.run_until_complete(async_main())\n```\n\n`bob.py`\n\n```python\nimport asyncio\n\nfrom tno.mpc.communication import Pool\n\n\nasync def async_main():\n # Create the pool for Bob.\n # Bob listens on port 61002 and adds Alice as client.\n pool = Pool()\n pool.add_http_server(addr=\"127.0.0.1\", port=61002)\n pool.add_http_client(\"Alice\", addr=\"127.0.0.1\", port=61001)\n\n # Bob waits for a message from Alice and prints it.\n # He replies and shuts down his pool instance.\n message = await pool.recv(\"Alice\")\n print(message)\n await pool.send(\"Alice\", \"Hello back to you, Alice!\")\n await pool.shutdown()\n\n\nif __name__ == \"__main__\":\n loop = asyncio.get_event_loop()\n loop.run_until_complete(async_main())\n```\n\nTo run this example, run each of the files in a separate terminal window. Note\nthat if `alice.py` is started prior to `bob.py`, it will throw a\n`ClientConnectorError`. Namely, Alice tries to send a message to port 61002,\nwhich has not been opened by Bob yet. After starting `bob.py`, the error\ndisappears.\n\nThe outputs in the two terminals will be something similar to the following:\n\n```bash\n>>> python alice.py\n2022-07-07 09:36:20,220 - tno.mpc.communication.httphandlers - INFO - Serving on 127.0.0.1:61001\n2022-07-07 09:36:20,230 - tno.mpc.communication.httphandlers - INFO - Received message from 127.0.0.1:61002\nHello back to you, Bob!\n2022-07-07 09:36:20,232 - tno.mpc.communication.httphandlers - INFO - HTTPServer: Shutting down server task\n2022-07-07 09:36:20,232 - tno.mpc.communication.httphandlers - INFO - Server 127.0.0.1:61001 shutdown\n```\n\n```bash\n>>> python bob.py\n2022-07-07 09:36:16,915 - tno.mpc.communication.httphandlers - INFO - Serving on 127.0.0.1:61002\n2022-07-07 09:36:20,223 - tno.mpc.communication.httphandlers - INFO - Received message from 127.0.0.1:61001\nHello Bob! This is Alice speaking.\n2022-07-07 09:36:20,232 - tno.mpc.communication.httphandlers - INFO - HTTPServer: Shutting down server task\n2022-07-07 09:36:20,256 - tno.mpc.communication.httphandlers - INFO - Server 127.0.0.1:61002 shutdown\n```\n\n## Test fixtures\n\nThe `tno.mpc.communication` package exports several pytest fixtures as pytest\nplugins to facilitate the user in testing with pool objects. The fixtures take\ncare of all configuration and clean-up of the pool objects so that you don't\nhave to worry about that.\n\nUsage:\n\n```py\n# test_my_module.py\nimport pytest\nfrom typing import Callable\nfrom tno.mpc.communication import Pool\n\ndef test_with_two_pools(http_pool_duo: tuple[Pool, Pool]) -> None:\n sender, receiver = http_pool_duo\n # ... your code\n\ndef test_with_three_pools(http_pool_trio: tuple[Pool, Pool, Pool]) -> None:\n alice, bob, charlie = http_pool_trio\n # ... your code\n\n@pytest.mark.parameterize(\"n_players\", (2, 3, 4))\ndef test_with_variable_pools(\n n_players: int,\n http_pool_group_factory: Callable[[int], tuple[Pool, ...]],\n) -> None:\n pools = http_pool_group_factory(n_players)\n # ... your code\n```\n\n### Fixture scope\n\nThe scope of the fixtures can be set dynamically through the\n`--fixture-pool-scope`\n[option to pytest](https://docs.pytest.org/en/7.1.x/reference/reference.html#configuration-options).\n_Note that this will also change the scope of the global `event_loop` fixture\nthat is provided by `pytest-asyncio`._ By default, in line with\n`pytest_asyncio`, the scope of all our fixtures is `\"function\"`. We advise to\nconfigure a larger scope (e.g. `\"session\"`, `\"package\"` or `\"module\"`) when\npossible to reduce test set-up and teardown time.\n\nOur fixtures pass `True` to the `port_reuse` argument of `aiohttp.web.TCPSite`.\nTheir\n[documentation](https://docs.aiohttp.org/en/stable/web_reference.html?highlight=reuse_port#aiohttp.web.TCPSite)\nstates that this option is not supported on Windows (outside of WSL). If you\nexperience any issues, please disable the plugin by adding\n`-p no:pytest_tno.tno.mpc.communication.pytest_pool_fixtures` to your pytest\nconfiguration. Note that without `port_reuse` the tests may crash, as the test\nmay try to bind to ports which may not have been freed by the operating system.\nFor more reliable testing, run the tests on a WSL / Linux platform.\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "MPC Communication module",
"version": "4.8.1",
"project_urls": {
"Documentation": "https://docs.pet.tno.nl/mpc/communication/4.8.1",
"Homepage": "https://pet.tno.nl/",
"Source": "https://github.com/TNO-MPC/communication"
},
"split_keywords": [
"tno",
"mpc",
"multi-party computation",
"communication"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4475240d8973ca07780357e671ed61f2152a9d2c814a8f1074ac5dbca14b943a",
"md5": "37ae1e7b07f328595e9ffac1d8b7192f",
"sha256": "14a476c1fc35f3baa776cd60509f68000b1dc3cb55706affe37c65f6d23d8b01"
},
"downloads": -1,
"filename": "tno.mpc.communication-4.8.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "37ae1e7b07f328595e9ffac1d8b7192f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 81477,
"upload_time": "2023-08-03T09:49:34",
"upload_time_iso_8601": "2023-08-03T09:49:34.098814Z",
"url": "https://files.pythonhosted.org/packages/44/75/240d8973ca07780357e671ed61f2152a9d2c814a8f1074ac5dbca14b943a/tno.mpc.communication-4.8.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "95d981ec88a4570ef642cfec991e30010f4379fd2b49a0f4b3128f0f073e1b8d",
"md5": "dca56fb3adf658138cb0666fd4d275c2",
"sha256": "444448d79f7814bcb650727a2910d16169f12dd2ab3d9a037d453da332664435"
},
"downloads": -1,
"filename": "tno.mpc.communication-4.8.1.tar.gz",
"has_sig": false,
"md5_digest": "dca56fb3adf658138cb0666fd4d275c2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 69977,
"upload_time": "2023-08-03T09:49:36",
"upload_time_iso_8601": "2023-08-03T09:49:36.204643Z",
"url": "https://files.pythonhosted.org/packages/95/d9/81ec88a4570ef642cfec991e30010f4379fd2b49a0f4b3128f0f073e1b8d/tno.mpc.communication-4.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-03 09:49:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "TNO-MPC",
"github_project": "communication",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "tno.mpc.communication"
}
JSON
