====================
Utilities for Python
====================
This is a Python "util" submodule with code that I habitually use in so
many projects that I decided to split it off into its own git archive.
Feel free to extend and embrace. Patches welcome.
Features
========
A few. Read the code and the subpackages' documentation; I didn't get
around to documenting all of that yet.
License
=======
No restrictions. CC0 / public domain / MIT / GPLv3 / whatever, as long as
you don't want anybody (i.e. the author(s)) to pay for the effects of any
bugs, features or "features" which this code might or might not have.
Please contribute any enhancements.
Dependencies
============
A few. Notable:
* anyio, version 3. Required for `ValueEvent`, `spawn` (obviously) and
`as_service`.
* ruyaml. This is a fork of ruamel.yaml which is a fork of pyyaml.
Fixes some bugs and has a more reasonable API. Required for
`yload`/`yprint`/`yformat`.
* msgpack, obviously required for `pack`/`unpack` and `MsgReader`/`MsgWriter`.
* asyncclick, required for `main_`/`wrap_main`.
MoaT does not depend on:
* cbor2. Our implementation is shared with a version running on
MicroPython and thus needs to be minimal. Also we want to support
efficient async streaming.
CBOR Tags
=========
This package uses the following CBOR tags.
The author presumes that the proposed tags 202 and 203 generally useful
beyond the confines of this specification and has submitted them to IANA.
Path
----
=============== =============================
Tag 202
Data Item array
Semantics mark array as object path
Reference https://github.com/M-o-a-T/moat-util/
Contact Matthias Urlichs <matthias@urlichs.de>
=============== =============================
A Path is a sequence of object accessors, i.e. a way to reference a
possibly-deeply nested object. These typically include strings
(object members, map keys) and numbers (array indices).
A recipient can use this tag to distinguish a sequence of lookups from
a tuple that's directly used as a map key. (Python allows this.)
Also, a path is typically entered and displayed as a string with dots or
slashes as separators rather than an explicit list, i.e. ``foo:0.bar``
(MoaT's representation – indicating that the zero is an integer, not a string)
or ``foo/0/bar`` (file system, MQTT topic), instead of an explicit array
like ``["foo", 0, "bar"]``.
The array SHOULD include only strings and non-negative integers.
The list of allowed characters in the string(s) is application dependent.
Object Proxy
------------
=============== =============================
Tag 203
Data Item string, integer, array
Semantics reference a well-known or unencodable object
Reference https://github.com/M-o-a-T/moat-util/
Contact Matthias Urlichs <matthias@urlichs.de>
=============== =============================
A Proxy refers to an object that cannot be encoded in CBOR. In a messaging
system, a sender may cache the object and replace it with a proxy instead
of throwing an error. The recipient can subsequently refer to the object
using the same Proxy tag when it sends a message back.
When the proxy's content is an array, it SHOULD consist of two elements:
a string or integer that uniquely identifies the origin of the proxy object,
and a string or integer which the originator can use to recover the
original.
An API to release auto-generated proxies is recommended but out of scope of
this specification.
Object Constructor
------------------
=============== =============================
Tag 27
Data Item array
Semantics build an object from a class
Reference http://cbor.schmorp.de/generic-object
Contact Marc A. Lehmann
=============== =============================
This tag is already specified and included here for MoaT-specific usage
details.
MoaT assumes a Proxy (referencing the class of the object in question) as
the array's first entry; tag 203 MAY be omitted.
The remaining array elements mirror Python's object serialization scheme:
an array of positional arguments, a map of key/value arguments, an array of
items to add/append to the object, and a map of attributes to set. Trailing
elements may be omitted if empty.
File Identifier
---------------
=============== =============================
Tag 1299145044
Data Item array
Semantics MoaT file identifier / details
Reference https://github.com/M-o-a-T/moat-util/
Contact Matthias Urlichs <matthias@urlichs.de>
=============== =============================
Files with MoaT-compatible messages start with an array that is wrapped with
tag 1299145044 (0x4d6f6154, "MoaT"), inside tag 55799 (CBOR).
The array has two elements. The first contains a string that describes the
file's contents. The string MUST be at least 24 bytes long (pad with spaces
if necessary), for the benefit of the "file" utility. It is free-format,
meant to be shown to humans, and MUST be ignored by programs that read the
file.
The second array member is a map that describes the file. Programs that read
it should use the map's contents to determine how to interpret it, or
to extract metadata (e.g. range of record, file creation date, etc.).
This way, ``file`` can show basic data about the file, using these magic entries:
0 string/3b \xd9\xd9\xf7 CBOR
>3 string/5b \xdaMoaT MoaT file
>>8 string/2b \x82\x78
>>>10 pstring >\0 %s
>>8 string/2b \x82\x79
>>>10 pstring/H >\0 %s
Shorter file type strings would require 24 additional entries in ``file``'s
magic pattern file (as it cannot mask the high bits of a string's length
field), which seems excessive.
End of file marker
------------------
=============== =============================
Tag 1298493254
Data Item map
Semantics MoaT end-of-file marker
Reference https://github.com/M-o-a-T/moat-util/
Contact Matthias Urlichs <matthias@urlichs.de>
=============== =============================
This tag ("MeoF") is the last tag written to a file before it's closed. Its
content describes e.g. why the file has ended (timeout, interrupt, restart …)
and which file will continue the content (if applicable).
When this tag is not the last CBOR data item in a file, it MUST be followed
with a tag 55799+1299145044 with matching continuation IDs ("cont") in its
map part. MoaT uses this element to verify that multiple files have been
concatenated correctly.
Raw data
{
"_id": null,
"home_page": null,
"name": "moat-util",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "MoaT",
"author": null,
"author_email": "Matthias Urlichs <matthias@urlichs.de>",
"download_url": "https://files.pythonhosted.org/packages/b6/cc/8185ff2efee48463f1c1d6073d443f07652bbbb69771b4d12026f618f802/moat_util-0.54.1.tar.gz",
"platform": null,
"description": "====================\nUtilities for Python\n====================\n\nThis is a Python \"util\" submodule with code that I habitually use in so\nmany projects that I decided to split it off into its own git archive.\n\nFeel free to extend and embrace. Patches welcome.\n\n\nFeatures\n========\n\nA few. Read the code and the subpackages' documentation; I didn't get\naround to documenting all of that yet.\n\n\nLicense\n=======\n\nNo restrictions. CC0 / public domain / MIT / GPLv3 / whatever, as long as\nyou don't want anybody (i.e. the author(s)) to pay for the effects of any\nbugs, features or \"features\" which this code might or might not have.\n\nPlease contribute any enhancements.\n\n\nDependencies\n============\n\nA few. Notable:\n\n* anyio, version 3. Required for `ValueEvent`, `spawn` (obviously) and\n `as_service`.\n\n* ruyaml. This is a fork of ruamel.yaml which is a fork of pyyaml.\n Fixes some bugs and has a more reasonable API. Required for\n `yload`/`yprint`/`yformat`.\n\n* msgpack, obviously required for `pack`/`unpack` and `MsgReader`/`MsgWriter`.\n\n* asyncclick, required for `main_`/`wrap_main`.\n\nMoaT does not depend on:\n\n* cbor2. Our implementation is shared with a version running on\n MicroPython and thus needs to be minimal. Also we want to support\n efficient async streaming.\n\n\nCBOR Tags\n=========\n\nThis package uses the following CBOR tags.\n\nThe author presumes that the proposed tags 202 and 203 generally useful\nbeyond the confines of this specification and has submitted them to IANA.\n\n\nPath\n----\n\n =============== =============================\n Tag 202\n Data Item array\n Semantics mark array as object path\n Reference https://github.com/M-o-a-T/moat-util/\n Contact Matthias Urlichs <matthias@urlichs.de>\n =============== =============================\n\nA Path is a sequence of object accessors, i.e. a way to reference a\npossibly-deeply nested object. These typically include strings\n(object members, map keys) and numbers (array indices).\n\nA recipient can use this tag to distinguish a sequence of lookups from\na tuple that's directly used as a map key. (Python allows this.)\n\nAlso, a path is typically entered and displayed as a string with dots or\nslashes as separators rather than an explicit list, i.e. ``foo:0.bar``\n(MoaT's representation \u2013 indicating that the zero is an integer, not a string)\nor ``foo/0/bar`` (file system, MQTT topic), instead of an explicit array\nlike ``[\"foo\", 0, \"bar\"]``.\n\nThe array SHOULD include only strings and non-negative integers.\nThe list of allowed characters in the string(s) is application dependent.\n\n\nObject Proxy\n------------\n\n =============== =============================\n Tag 203\n Data Item string, integer, array\n Semantics reference a well-known or unencodable object\n Reference https://github.com/M-o-a-T/moat-util/\n Contact Matthias Urlichs <matthias@urlichs.de>\n =============== =============================\n\nA Proxy refers to an object that cannot be encoded in CBOR. In a messaging\nsystem, a sender may cache the object and replace it with a proxy instead\nof throwing an error. The recipient can subsequently refer to the object\nusing the same Proxy tag when it sends a message back.\n\nWhen the proxy's content is an array, it SHOULD consist of two elements:\na string or integer that uniquely identifies the origin of the proxy object,\nand a string or integer which the originator can use to recover the\noriginal.\n\nAn API to release auto-generated proxies is recommended but out of scope of\nthis specification.\n\n\nObject Constructor\n------------------\n\n =============== =============================\n Tag 27\n Data Item array\n Semantics build an object from a class\n Reference http://cbor.schmorp.de/generic-object\n Contact Marc A. Lehmann\n =============== =============================\n\nThis tag is already specified and included here for MoaT-specific usage\ndetails.\n\nMoaT assumes a Proxy (referencing the class of the object in question) as\nthe array's first entry; tag 203 MAY be omitted.\n\nThe remaining array elements mirror Python's object serialization scheme:\nan array of positional arguments, a map of key/value arguments, an array of\nitems to add/append to the object, and a map of attributes to set. Trailing\nelements may be omitted if empty.\n\n\nFile Identifier\n---------------\n\n =============== =============================\n Tag 1299145044\n Data Item array\n Semantics MoaT file identifier / details\n Reference https://github.com/M-o-a-T/moat-util/\n Contact Matthias Urlichs <matthias@urlichs.de>\n =============== =============================\n\nFiles with MoaT-compatible messages start with an array that is wrapped with\ntag 1299145044 (0x4d6f6154, \"MoaT\"), inside tag 55799 (CBOR).\n\nThe array has two elements. The first contains a string that describes the\nfile's contents. The string MUST be at least 24 bytes long (pad with spaces\nif necessary), for the benefit of the \"file\" utility. It is free-format,\nmeant to be shown to humans, and MUST be ignored by programs that read the\nfile.\n\nThe second array member is a map that describes the file. Programs that read\nit should use the map's contents to determine how to interpret it, or\nto extract metadata (e.g. range of record, file creation date, etc.).\n\nThis way, ``file`` can show basic data about the file, using these magic entries:\n\n 0 string/3b \\xd9\\xd9\\xf7 CBOR\n >3 string/5b \\xdaMoaT MoaT file\n >>8 string/2b \\x82\\x78 \n >>>10 pstring >\\0 %s\n >>8 string/2b \\x82\\x79 \n >>>10 pstring/H >\\0 %s\n\nShorter file type strings would require 24 additional entries in ``file``'s\nmagic pattern file (as it cannot mask the high bits of a string's length\nfield), which seems excessive.\n\nEnd of file marker\n------------------\n\n =============== =============================\n Tag 1298493254\n Data Item map\n Semantics MoaT end-of-file marker\n Reference https://github.com/M-o-a-T/moat-util/\n Contact Matthias Urlichs <matthias@urlichs.de>\n =============== =============================\n\nThis tag (\"MeoF\") is the last tag written to a file before it's closed. Its\ncontent describes e.g. why the file has ended (timeout, interrupt, restart \u2026)\nand which file will continue the content (if applicable).\n\nWhen this tag is not the last CBOR data item in a file, it MUST be followed\nwith a tag 55799+1299145044 with matching continuation IDs (\"cont\") in its\nmap part. MoaT uses this element to verify that multiple files have been\nconcatenated correctly.\n\n",
"bugtrack_url": null,
"license": "The code in this repository is part of the MoaT project. The licensing information in the moat-main repository at https://github.com/M-o-a-T/moat-main is applicable. ",
"summary": "Utility code for the rest of MoaT",
"version": "0.54.1",
"project_urls": {
"homepage": "https://m-o-a-t.org",
"repository": "https://github.com/M-o-a-T/moat-util"
},
"split_keywords": [
"moat"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8f0bae8eaca69d67599cd0fe16d54b8840afbfa11a5c7918badee33fb35f4861",
"md5": "09daa90395e4c0d07fe913a7961ba441",
"sha256": "8a7fbaf5896a08b6ec192e42082728a6988b64c60512f315335a35733a5020f4"
},
"downloads": -1,
"filename": "moat_util-0.54.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "09daa90395e4c0d07fe913a7961ba441",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 56100,
"upload_time": "2024-11-19T05:31:16",
"upload_time_iso_8601": "2024-11-19T05:31:16.353006Z",
"url": "https://files.pythonhosted.org/packages/8f/0b/ae8eaca69d67599cd0fe16d54b8840afbfa11a5c7918badee33fb35f4861/moat_util-0.54.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b6cc8185ff2efee48463f1c1d6073d443f07652bbbb69771b4d12026f618f802",
"md5": "5df044a46b2896eb66fea24300c187c2",
"sha256": "3addde20cff1d98b6b7fc88735780b83b54021ba8d17a724de7894af57b15f0c"
},
"downloads": -1,
"filename": "moat_util-0.54.1.tar.gz",
"has_sig": false,
"md5_digest": "5df044a46b2896eb66fea24300c187c2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 58852,
"upload_time": "2024-11-19T05:31:18",
"upload_time_iso_8601": "2024-11-19T05:31:18.271012Z",
"url": "https://files.pythonhosted.org/packages/b6/cc/8185ff2efee48463f1c1d6073d443f07652bbbb69771b4d12026f618f802/moat_util-0.54.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-19 05:31:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "M-o-a-T",
"github_project": "moat-util",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "moat-util"
}
JSON
