| Name | moat-lib-cmd JSON |
| Version |
0.3.3
JSON |
| download |
| home_page | None |
| Summary | A simple command/stream multiplexer |
| upload_time | 2024-11-18 18:57:50 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | The MIT License (MIT) Copyright (c) 2016 Chris von Csefalvay Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
moat
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
========================
The MoaT-Command library
========================
Rationale
=========
MoaT contains some components which require a possibly bidirectional stream
of asynchronous messaging, including request/reply interactions and data
streaming.
This library supports such interactions.
Prerequisites
=============
The library requires a reliable underlying transport. MoaT uses CBOR, but
any reliable, non-reordering messsage stream that can encode basic Python
data structures (plus whatever objects you send/receive) works.
The MoaT-Cmd library does not itself call the transport. Instead it affords
basic async methods to iterate on messages to send, and to feed incoming
lower-level data in.
Usage
=====
.. code-block:: python
from moat.util import packer, stream_unpacker
async def handle_command(msg):
if msg.cmd[0] == "Start":
return "OK starting"
if msg.cmd[0] == "gimme data":
async with msg.stream_w("Start") as st:
for i in range(10):
await st.send(i+msg.data["x"])
return "OK I'm done"
if msg.cmd[0] == "alive":
async with msg.stream_r("Start") as st:
async for data in st:
print("We got", data)
return "OK nice"
raise ValueError(f"Unknown: {msg !r}")
async with Transport(handle_command) as tr, anyio.create_task_group() as tg:
decoder = stream_unpacker(cbor=True)
def reader():
# receive messages from channel
async for msg in channel.receive():
decoder.feed(msg)
for m in decoder:
await tr.msg_in(m)
def sender():
# send messages to channel
while True:
msg = await tr.msg_out()
await channel.send(packer(msg))
def request():
# streaming data in
msg = await tr.cmd("Start", x=123)
print("Start", msg)
async with tr.stream_r("gimme data") as st:
print("They are starting", st.msg)
async for msg in st:
print("I got", msg)
print("They are done", st.msg)
# may be None if they didn't send a stream
def int_stream():
# streaming data out
async with tr.stream_w("alive") as st:
print("They replied", st.msg)
i = 0
while i < 100:
await st.send(i)
i += 1
anyio.sleep(1/10)
st.msg = "The end."
print("I am done", st.msg)
tg.start_soon(reader)
tg.start_soon(sender)
tg.start_soon(handler)
tg.start_soon(request)
tg.start_soon(int_stream)
Specification
=============
All MoaT-Cmd messages are non-empty lists whose first element is a
small integer, identifying a sub-channel. Messages that don't match this
description MAY be used for out-of-band communication.
A transport that enforces message boundaries MAY send each message without
the leading array mark byte(s).
MoaT-Cmd messaging is simple by design and basically consists of a command
(sent from A to B) followed by a reply (sent from B to A). Both directions
may independently indicate that more, streamed data will follow. The first
and last message of a streamed command or reply are considered to be
out-of-band.
The side opening a sub-channel uses non-negative integers as channel ID.
Replies carry the ID's bitwise-negated value. Thus the ID spaces of both
directions are separate.
IDs are allocated with the first message on a sub-channel. They MUST NOT be
reused until final messages have been exchanged. Exactly one final message
MUST be sent in both directions.
Message format
++++++++++++++
A Moat-Cmd message consist of a preferably-small signed integer, plus a
variable and usually non-empty amount of data.
The integer is interpreted as follows.
* Bit 0: if set, the message starts or continues a data stream; if clear,
the message is the final message for this subchannel and direction.
* Bit 1: Error/Warning.
If bit 0 is set, the message is a warning or similar information and
SHOULD be attached to the following command or reply. Otherwise it is an
error.
All other bits contain the message ID, left-shifted by two bits. This
scheme allows for five concurrent messages per direction before encoding to
two bytes is required.
Negative integers signal that the ID has been allocated by that message's
recipient. They are inverted bit-wise, i.e. ``(-1-id)``. Thus an ID of zero
is legal. The bits described above are not affected by his inversion. Thus
a command with ID=1 (no streaming, no error) is sent with an initial
integer of 4; the reply uses -5.
Streaming
+++++++++
Data streams are inherently bidirectional. The command's semantics SHOULD
specify which side of a stream is supposed to send data. Error -2 will be
sent (once) if a streamed item is received that won't be handled.
Streaming may start when both sides have exchanged initial messages.
Sending a stream SHOULD NOT commence before the initial command has been
replied to.
Messages with both the streaming and error bits set carry out-of-band data
while the stream is open, e.g. advising the recipient of data loss.
Otherwise they MAY be delivered as warnings or similar out-of-band data.
Conceptally, these messages are attached to the command or reply that
immediately follows them.
For both directions, the initial and final message are assumed to be
out-of-band data. This also applies to warnings.
Flow Control
------------
For the most part: None. MoaT-Cmd is mostly used for monitoring events or
enumerating small data sets.
However, *if* a stream's recipient has limited buffer space and sends a
command that might trigger a nontrivial amount of messages, it MAY send a
specific warning (i.e. a message with both Error and Streaming bits set)
before its initial command or reply. This warning MUST consist of a single
non-negative integer that advises the sender of the number of streamed
messages it may transmit.
During stream transmission, the recipient then SHOULD periodically send some
more (positive) integers to signal the availability of more buffer space.
It MUST send such a message if the counter is zero (after space becomes
available of course) and more messages are expected.
The initial flow control messages SHOULD be sent before the initial command
or reply, but MAY be deferred until later.
A receiver SHOULD start flow control sufficiently early, but that isn't
always feasible. It MUST notify the remote side (error -5, below) if an
incoming message gets dropped due to resource exhaustion; likewise, the API
is required to notify the local side.
Error handling
++++++++++++++
The exact semantics of error messages are application specific.
Error messages with the streaming bit clear terminate the command.
They should be treated as fatal.
Error messages with the streaming bit set are either flow control
messages (see above) or warnings.
Known errors
------------
* -1: Unspecified
The ``.stop()`` API method was called.
This message MAY be sent as a warning.
Usage: assume that a sender reads and transmits a block of ten
measurements each second. If a "stop" warning arrives, the sender should
complete the current block before terminating, while a "stop" error
forces the current transmission to end immediately.
* -2: Can't receive this stream
Sent if a command isn't prepared to receive a streamed reply.
* -3: Cancel
The sender's or receiver's task is cancelled: the work is no longer
required / performed.
This message SHOULD NOT be transmitted as a warning;
that would be pointless.
* -4: No Commands
The sender on this side doesn't process commands at all.
* -5: Data loss
An incoming message was dropped due to resource exhaustion (full queue).
This message SHOULD be sent as a warning, but MAY be interpreted as a
hard error by its receiver.
* -6: Must stream
Sent if a command isn't prepared to handle a non-streamed request or
reply.
* -11 …: No Command
The command is not recognized.
The error number encodes the command's position for a hierarchical lookup
at the destination, i.e. if the command is ("foo","bahr","baz") and "foo"
doesn't know about "bahr", the error is -12.
Examples
========
.. note::
Legend:
* D: direction / sign of message ID
* S: Streaming
* E: Error
= = = ====
S E D Data
= = = ====
- - + Hello
- - - You too
= = = ====
= = = ====
S E D Data
= = = ====
- - + Hello again
- * - Meh. you already said that
= = = ====
= = = ====
S E D Data
= = = ====
* - + gimme some data
* - - OK here they are
* - - ONE
* - - TWO
* * - Missed some
* - - FIVE
- - + Oops? better stop
* - - SIX
- - - stopped
= = = ====
= = = ====
S E D Data
= = = ====
* - + I want to send some data
* - - OK send them
* - + FOO
- - - Nonono I don't want those after all
* - + BAR
- * + OK OK I'll stop
= = = ====
= = = ====
S E D Data
= = = ====
* - + gimme some more data
* - - OK here they are
* - - NINE
* - - TEN
- * - oops I crashed
- - + *sigh*
= = = ====
= = = ====
S E D Data
= = = ====
* - + Let's talk
* - - OK
* - + *voice data* …
* - - *also voice data* …
- - + hanging up
- - - oh well
= = = ====
= = = ====
S E D Data
= = = ====
* * + 2
* - + gimme your database
* - - OK here they are
* - - A
* * + 1
* - - BB
* * + 1
* - - CCC
* - - DDDD
* * + 5
* - - EEEEE
* - - FFFFFF
* - - GGGGGGG
- - - that's all
- - + thx
= = = ====
Raw data
{
"_id": null,
"home_page": null,
"name": "moat-lib-cmd",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Matthias Urlichs <matthias@urlichs.de>",
"keywords": "MoaT",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/73/0a/d2ac442986d9c9e45511939038023b5ebc7ea5563e241fef068359e605d5/moat_lib_cmd-0.3.3.tar.gz",
"platform": null,
"description": "========================\nThe MoaT-Command library\n========================\n\nRationale\n=========\n\nMoaT contains some components which require a possibly bidirectional stream\nof asynchronous messaging, including request/reply interactions and data\nstreaming.\n\nThis library supports such interactions.\n\nPrerequisites\n=============\n\nThe library requires a reliable underlying transport. MoaT uses CBOR, but\nany reliable, non-reordering messsage stream that can encode basic Python\ndata structures (plus whatever objects you send/receive) works.\n\nThe MoaT-Cmd library does not itself call the transport. Instead it affords\nbasic async methods to iterate on messages to send, and to feed incoming\nlower-level data in.\n\n\nUsage\n=====\n\n.. code-block:: python\n\n from moat.util import packer, stream_unpacker\n\n async def handle_command(msg):\n if msg.cmd[0] == \"Start\":\n return \"OK starting\"\n\n if msg.cmd[0] == \"gimme data\":\n async with msg.stream_w(\"Start\") as st:\n for i in range(10):\n await st.send(i+msg.data[\"x\"])\n return \"OK I'm done\"\n\n if msg.cmd[0] == \"alive\":\n async with msg.stream_r(\"Start\") as st:\n async for data in st:\n print(\"We got\", data)\n return \"OK nice\"\n\n raise ValueError(f\"Unknown: {msg !r}\")\n \n async with Transport(handle_command) as tr, anyio.create_task_group() as tg:\n decoder = stream_unpacker(cbor=True)\n\n def reader():\n # receive messages from channel\n async for msg in channel.receive():\n decoder.feed(msg)\n for m in decoder:\n await tr.msg_in(m)\n\n def sender():\n # send messages to channel\n while True:\n msg = await tr.msg_out()\n await channel.send(packer(msg))\n\n def request():\n # streaming data in\n msg = await tr.cmd(\"Start\", x=123)\n print(\"Start\", msg)\n async with tr.stream_r(\"gimme data\") as st:\n print(\"They are starting\", st.msg)\n async for msg in st:\n print(\"I got\", msg)\n print(\"They are done\", st.msg)\n # may be None if they didn't send a stream\n\n def int_stream():\n # streaming data out\n async with tr.stream_w(\"alive\") as st:\n print(\"They replied\", st.msg)\n i = 0\n while i < 100:\n await st.send(i)\n i += 1\n anyio.sleep(1/10)\n st.msg = \"The end.\"\n print(\"I am done\", st.msg)\n \n \n tg.start_soon(reader)\n tg.start_soon(sender)\n tg.start_soon(handler)\n\n tg.start_soon(request)\n tg.start_soon(int_stream)\n\n\nSpecification\n=============\n\nAll MoaT-Cmd messages are non-empty lists whose first element is a\nsmall integer, identifying a sub-channel. Messages that don't match this\ndescription MAY be used for out-of-band communication.\n\nA transport that enforces message boundaries MAY send each message without\nthe leading array mark byte(s).\n\nMoaT-Cmd messaging is simple by design and basically consists of a command\n(sent from A to B) followed by a reply (sent from B to A). Both directions\nmay independently indicate that more, streamed data will follow. The first\nand last message of a streamed command or reply are considered to be\nout-of-band.\n\nThe side opening a sub-channel uses non-negative integers as channel ID.\nReplies carry the ID's bitwise-negated value. Thus the ID spaces of both\ndirections are separate.\n\nIDs are allocated with the first message on a sub-channel. They MUST NOT be\nreused until final messages have been exchanged. Exactly one final message\nMUST be sent in both directions.\n\n\nMessage format\n++++++++++++++\n\nA Moat-Cmd message consist of a preferably-small signed integer, plus a\nvariable and usually non-empty amount of data.\n\nThe integer is interpreted as follows.\n\n* Bit 0: if set, the message starts or continues a data stream; if clear,\n the message is the final message for this subchannel and direction.\n\n* Bit 1: Error/Warning.\n If bit 0 is set, the message is a warning or similar information and\n SHOULD be attached to the following command or reply. Otherwise it is an\n error.\n\nAll other bits contain the message ID, left-shifted by two bits. This\nscheme allows for five concurrent messages per direction before encoding to\ntwo bytes is required.\n\nNegative integers signal that the ID has been allocated by that message's\nrecipient. They are inverted bit-wise, i.e. ``(-1-id)``. Thus an ID of zero\nis legal. The bits described above are not affected by his inversion. Thus\na command with ID=1 (no streaming, no error) is sent with an initial\ninteger of 4; the reply uses -5.\n\n\nStreaming\n+++++++++\n\nData streams are inherently bidirectional. The command's semantics SHOULD\nspecify which side of a stream is supposed to send data. Error -2 will be\nsent (once) if a streamed item is received that won't be handled.\n\nStreaming may start when both sides have exchanged initial messages.\nSending a stream SHOULD NOT commence before the initial command has been\nreplied to.\n\nMessages with both the streaming and error bits set carry out-of-band data\nwhile the stream is open, e.g. advising the recipient of data loss.\nOtherwise they MAY be delivered as warnings or similar out-of-band data.\nConceptally, these messages are attached to the command or reply that\nimmediately follows them.\n\nFor both directions, the initial and final message are assumed to be\nout-of-band data. This also applies to warnings.\n\nFlow Control\n------------\n\nFor the most part: None. MoaT-Cmd is mostly used for monitoring events or\nenumerating small data sets.\n\nHowever, *if* a stream's recipient has limited buffer space and sends a\ncommand that might trigger a nontrivial amount of messages, it MAY send a\nspecific warning (i.e. a message with both Error and Streaming bits set)\nbefore its initial command or reply. This warning MUST consist of a single\nnon-negative integer that advises the sender of the number of streamed\nmessages it may transmit.\n\nDuring stream transmission, the recipient then SHOULD periodically send some\nmore (positive) integers to signal the availability of more buffer space.\nIt MUST send such a message if the counter is zero (after space becomes\navailable of course) and more messages are expected.\n\nThe initial flow control messages SHOULD be sent before the initial command\nor reply, but MAY be deferred until later.\n\nA receiver SHOULD start flow control sufficiently early, but that isn't\nalways feasible. It MUST notify the remote side (error -5, below) if an\nincoming message gets dropped due to resource exhaustion; likewise, the API\nis required to notify the local side.\n\nError handling\n++++++++++++++\n\nThe exact semantics of error messages are application specific.\n\nError messages with the streaming bit clear terminate the command.\nThey should be treated as fatal.\n\nError messages with the streaming bit set are either flow control\nmessages (see above) or warnings.\n\n\nKnown errors\n------------\n\n* -1: Unspecified\n\n The ``.stop()`` API method was called.\n\n This message MAY be sent as a warning.\n\n Usage: assume that a sender reads and transmits a block of ten\n measurements each second. If a \"stop\" warning arrives, the sender should\n complete the current block before terminating, while a \"stop\" error\n forces the current transmission to end immediately.\n\n* -2: Can't receive this stream\n\n Sent if a command isn't prepared to receive a streamed reply.\n\n* -3: Cancel\n\n The sender's or receiver's task is cancelled: the work is no longer\n required / performed.\n\n This message SHOULD NOT be transmitted as a warning;\n that would be pointless.\n\n* -4: No Commands\n\n The sender on this side doesn't process commands at all.\n\n* -5: Data loss\n\n An incoming message was dropped due to resource exhaustion (full queue).\n\n This message SHOULD be sent as a warning, but MAY be interpreted as a\n hard error by its receiver.\n\n* -6: Must stream\n\n Sent if a command isn't prepared to handle a non-streamed request or\n reply.\n\n\n* -11 \u2026: No Command\n\n The command is not recognized.\n\n The error number encodes the command's position for a hierarchical lookup\n at the destination, i.e. if the command is (\"foo\",\"bahr\",\"baz\") and \"foo\"\n doesn't know about \"bahr\", the error is -12.\n\n\nExamples\n========\n\n.. note::\n\n Legend:\n * D: direction / sign of message ID\n * S: Streaming\n * E: Error\n\n= = = ====\nS E D Data\n= = = ====\n- - + Hello\n- - - You too\n= = = ====\n\n= = = ====\nS E D Data\n= = = ====\n- - + Hello again\n- * - Meh. you already said that\n= = = ====\n\n= = = ====\nS E D Data\n= = = ====\n* - + gimme some data\n* - - OK here they are\n* - - ONE\n* - - TWO\n* * - Missed some\n* - - FIVE\n- - + Oops? better stop\n* - - SIX\n- - - stopped\n= = = ====\n\n= = = ====\nS E D Data\n= = = ====\n* - + I want to send some data\n* - - OK send them\n* - + FOO\n- - - Nonono I don't want those after all\n* - + BAR\n- * + OK OK I'll stop\n= = = ====\n\n= = = ====\nS E D Data\n= = = ====\n* - + gimme some more data\n* - - OK here they are\n* - - NINE\n* - - TEN\n- * - oops I crashed\n- - + *sigh*\n= = = ====\n\n= = = ====\nS E D Data\n= = = ====\n* - + Let's talk\n* - - OK\n* - + *voice data* \u2026\n* - - *also voice data* \u2026\n- - + hanging up\n- - - oh well\n= = = ====\n\n= = = ====\nS E D Data\n= = = ====\n* * + 2\n* - + gimme your database\n* - - OK here they are\n* - - A\n* * + 1\n* - - BB\n* * + 1\n* - - CCC\n* - - DDDD\n* * + 5\n* - - EEEEE\n* - - FFFFFF\n* - - GGGGGGG\n- - - that's all\n- - + thx\n= = = ====\n\n",
"bugtrack_url": null,
"license": "The MIT License (MIT) Copyright (c) 2016 Chris von Csefalvay Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "A simple command/stream multiplexer",
"version": "0.3.3",
"project_urls": {
"homepage": "https://m-o-a-t.org",
"repository": "https://github.com/M-o-a-T/moat-lib-cmd"
},
"split_keywords": [
"moat"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "aeba817f03b004bf8b5fdc1f3ee2ea62b36377a76bd51425625b3cc7e3bf297f",
"md5": "5b476dc8ea3ce3a82d4efd7d71a39dfc",
"sha256": "3f8ab8d02f0a27ad70266b519b5207cab67d77dc8126b88b40f9d9c43a92d5cc"
},
"downloads": -1,
"filename": "moat_lib_cmd-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5b476dc8ea3ce3a82d4efd7d71a39dfc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 12871,
"upload_time": "2024-11-18T18:57:48",
"upload_time_iso_8601": "2024-11-18T18:57:48.430159Z",
"url": "https://files.pythonhosted.org/packages/ae/ba/817f03b004bf8b5fdc1f3ee2ea62b36377a76bd51425625b3cc7e3bf297f/moat_lib_cmd-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "730ad2ac442986d9c9e45511939038023b5ebc7ea5563e241fef068359e605d5",
"md5": "e551fdca22c33120c8935e4615cda037",
"sha256": "7a68a32490befd35a76bb549184b286834e34a188d5ef2969fa121a7df612390"
},
"downloads": -1,
"filename": "moat_lib_cmd-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "e551fdca22c33120c8935e4615cda037",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 18762,
"upload_time": "2024-11-18T18:57:50",
"upload_time_iso_8601": "2024-11-18T18:57:50.216914Z",
"url": "https://files.pythonhosted.org/packages/73/0a/d2ac442986d9c9e45511939038023b5ebc7ea5563e241fef068359e605d5/moat_lib_cmd-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 18:57:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "M-o-a-T",
"github_project": "moat-lib-cmd",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "moat-lib-cmd"
}
