| Name | pystructtype JSON |
| Version |
0.3.0
JSON |
| download |
| home_page | None |
| Summary | Leverage Python Types to Define C-Struct Interfaces |
| upload_time | 2025-02-22 03:54:43 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.13 |
| license | None |
| keywords |
cstruct
struct
type
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# PyStructTypes
Leverage Python Types to Define C-Struct Interfaces
# Reasoning
I made this project for 2 reasons:
1. I wanted to see if I could leverage the typing system to effectively automatically
decode and encode c-type structs in python.
2. Build a tool to do this for a separate project I am working on.
I am aware of other very similar c-type struct to python class libraries available,
but I wanted to try something new so here we are.
This may or may not end up being super useful, as there are quite a few bits of
hacky metaprogramming to get the type system to play nicely for what I want, but
perhaps over time it can be cleaned up and made more useful.
# StructDataclass
The `StructDataclass` class is based off of the `Dataclass` class, and thus
is used in a similar fashion.
# Basic Structs
Basic structs can mostly be copied over 1:1
```c
struct MyStruct {
int16_t myNum;
char myLetter;
};
```
```python
@struct_dataclass
class MyStruct(StructDataclass):
myNum: int16_t
myLetter: char_t
s = MyStruct()
s.decode([4, 2, 65])
# MyStruct(myNum=1026, myLetter=b"A")
s.decode([4, 2, 65], little_endian=True)
# MyStruct(myNum=516, myLetter=b"A")
# We can modify the class values and encode the data to send back
s.myNum = 2562
s.encode()
# [10, 2, 65]
```
For arrays of basic elements, you need to Annotate them with
the `TypeMeta` object, and set their type to `list[_type_]`.
```c
struct MyStruct {
uint8_t myInts[4];
uint16_t myBiggerInts[2];
};
```
```python
@struct_dataclass
class MyStruct(StructDataclass):
myInts: Annotated[list[uint8_t], TypeMeta(size=4)]
myBiggerInts: Annotated[list[uint16_t], TypeMeta(size=2)]
s = MyStruct()
s.decode([0, 64, 128, 255, 16, 0, 255, 255])
# MyStruct(myInts=[0, 64, 128, 255], myBiggerInts=[4096, 65535])
```
You can also set defaults for both basic types and lists.
All values will default to 0 or the initialized value for the chosen class if no
specific value is set.
List defaults will set all items in the list to the same value. Currently
setting a complete default list for all values is not implemented.
```c
struct MyStruct {
uint8_t myInt = 5;
uint8_t myInts[2];
};
```
```python
@struct_dataclass
class MyStruct(StructDataclass):
myInt: uint8_t = 5
myInts: Annnotated[list[uint8_t], TypeMeta(size=2, default=1)]
s = MyStruct()
# MyStruct(myInt=5, myInts=[1, 1])
s.decode([10, 5, 6])
# MyStruct(myInt=10, myInts=[5, 6])
```
# String / char[] Type
Defining c-string types is a little different. Instead of using
`size` in the `TypeMeta`, we need to instead use `chunk_size`.
This is because the way the struct format is defined for c-strings needs
to know how big the string data is expected to be so that it can put the
whole string in a single variable.
The `chunk_size` is also introduced to allow for `char[][]` for converting
a list of strings.
```c
struct MyStruct {
char myStr[3];
char myStrList[2][3];
};
```
```python
@struct_dataclass
class MyStruct(StructDataclass):
myStr: Annotated[string_t, TypeMeta[str](chunk_size=3)]
myStrList: Annotated[list[string_t], TypeMeta[str](size=2, chunk_size=3)]
s = MyStruct()
s.decode([65, 66, 67, 68, 69, 70, 71, 72, 73])
# MyStruct(myStr=b"ABC", myStrList=[b"DEF", b"GHI"])
```
If you instead try to define this as a list of `char_t` types,
you would only be able to end up with
`MyStruct(myStr=[b"A", b"B", b"C"], myStrList=[b"D", b"E", b"F", b"G", b"H", b"I"])`
# The Bits Abstraction
This library includes a `bits` abstraction to map bits to variables for easier access.
One example of this is converting a C enum like so:
```c
enum ConfigFlags {
lights_flag = 1 << 0,
platform_flag = 1 << 1,
};
#pragma pack(push, 1)
```
```python
@bits(uint8_t, {"lights_flag": 0, "platform_flag": 1})
class FlagsType(BitsType): ...
f = FlagsType()
f.decode([3])
# FlagsType(lights_flag=True, platform_flag=True)
f.decode([2])
# FlagsType(lights_flag=False, platform_flag=True)
f.decode([1])
# FlagsType(lights_flag=True, platform_flag=False)
```
# Custom StructDataclass Processing and Extensions
There may be times when you want to make the python class do
cool fun python class type of stuff with the data structure.
We can extend the class functions `_decode` and `_encode` to
handle this processing.
In this example, lets say you want to be able to read/write the
class object as a list, using `__getitem__` and `__setitem__` as well
as keeping the data in a different data structure than what the
c struct defines.
```c
struct MyStruct {
uint8_t enabledSensors[5];
};
```
```python
@struct_dataclass
class EnabledSensors(StructDataclass):
# We can define the actual data we are ingesting here
# This mirrors the `uint8_t enabledSensors[5]` data
_raw: Annotated[list[uint8_t], TypeMeta(size=5)]
# We use this to store the data in the way we actually want
_data: list[list[bool]] = field(default_factory=list)
def _decode(self, data: list[int]) -> None:
# First call the super function. This will store the raw values into `_raw`
super()._decode(data)
# Erase everything in self._data to remove any old data
self._data = []
# 2 Panels are packed into a single uint8_t, the left most 4 bits for the first
# and the right most 4 bits for the second
for bitlist in (list(map(bool, map(int, format(_byte, "#010b")[2:]))) for _byte in self._raw):
self._data.append(bitlist[0:4])
self._data.append(bitlist[4:])
# Remove the last item in self._data as there are only 9 panels
del self._data[-1]
def _encode(self) -> list[int]:
# Modify self._raw with updated values from self._data
for idx, items in enumerate(list_chunks(self._data, 2)):
# Last chunk
if len(items) == 1:
items.append([False, False, False, False])
self._raw[idx] = sum(v << i for i, v in enumerate(list(itertools.chain.from_iterable(items))[::-1]))
# Run the super function to return the encoded data from self._raw()
return super()._encode()
def __getitem__(self, index: int) -> list[bool]:
# This lets us access the data with square brackets
# ex. `config.enabled_sensors[Panel.UP][Sensor.RIGHT]`
return self._data[index]
def __setitem__(self, index: int, value: list[bool]) -> None:
# Only use this to set a complete set for a panel
# ex. `config.enabled_sensors[Panel.UP] = [True, True, False, True]`
if len(value) != 4 or not all(isinstance(x, bool) for x in value):
raise Exception("must set all 4 items at once")
s = EnabledSensors()
s.decode([15, 15, 15, 15, 0])
# The `self._data` here would look like:
# [
# [False, False, False, False],
# [True, True, True, True],
# [False, False, False, False],
# [True, True, True, True],
# [False, False, False, False],
# [True, True, True, True],
# [False, False, False, False],
# [True, True, True, True],
# [False, False, False, False],
# ]
# With the get/set functioned defined, we can access the data
# with square accessors.
# s[1][2] == True
```
# StructDataclass is Composable
You can use StructDataclasses in other StructDataclasses to create more complex
structs.
```c
struct RGB {
uint8_t r;
uint8_t g;
uint8_t b;
};
struct LEDS {
RGB lights[3];
};
```
```python
@struct_dataclass
class RGB(StructDataclass):
r: uint8_t
g: uint8_t
b: uint8_t
@struct_dataclass
class LEDS(StructDataclass):
lights: Annotated[list[RGB], TypeMeta(size=3])]
l = LEDS()
l.decode([1, 2, 3, 4, 5, 6, 7, 8, 9])
# LEDS(lights=[RGB(r=1, g=2, b=3), RGB(r=4, g=5, b=6), RGB(r=7, g=8, b=9)])
```
# Future Updates
- Bitfield: Similar to the `Bits` abstraction. An easy way to define bitfields
- Potentially more ways to define bits (dicts/lists/etc).
- Potentially allowing list defaults to be entire pre-defined lists.
- ???
# Examples
You can see a more fully fledged example in the `test/examples.py` file.
Raw data
{
"_id": null,
"home_page": null,
"name": "pystructtype",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.13",
"maintainer_email": null,
"keywords": "cstruct, struct, type",
"author": null,
"author_email": "fchorney <github@djsbx.com>",
"download_url": "https://files.pythonhosted.org/packages/6d/80/1883f6a32bcad1e6d34a375cacf934e261685eff6afd44dc95143950a0ce/pystructtype-0.3.0.tar.gz",
"platform": null,
"description": "# PyStructTypes\n\nLeverage Python Types to Define C-Struct Interfaces\n\n\n# Reasoning\n\nI made this project for 2 reasons:\n1. I wanted to see if I could leverage the typing system to effectively automatically\ndecode and encode c-type structs in python.\n2. Build a tool to do this for a separate project I am working on.\n\nI am aware of other very similar c-type struct to python class libraries available,\nbut I wanted to try something new so here we are. \n\nThis may or may not end up being super useful, as there are quite a few bits of \nhacky metaprogramming to get the type system to play nicely for what I want, but \nperhaps over time it can be cleaned up and made more useful.\n\n# StructDataclass\n\nThe `StructDataclass` class is based off of the `Dataclass` class, and thus\nis used in a similar fashion.\n\n# Basic Structs\n\nBasic structs can mostly be copied over 1:1\n\n```c\nstruct MyStruct {\n int16_t myNum;\n char myLetter;\n};\n```\n\n```python\n@struct_dataclass\nclass MyStruct(StructDataclass):\n myNum: int16_t\n myLetter: char_t\n\ns = MyStruct()\ns.decode([4, 2, 65])\n# MyStruct(myNum=1026, myLetter=b\"A\")\ns.decode([4, 2, 65], little_endian=True)\n# MyStruct(myNum=516, myLetter=b\"A\")\n\n# We can modify the class values and encode the data to send back\ns.myNum = 2562\ns.encode()\n# [10, 2, 65]\n```\n\nFor arrays of basic elements, you need to Annotate them with\nthe `TypeMeta` object, and set their type to `list[_type_]`.\n\n```c\nstruct MyStruct {\n uint8_t myInts[4];\n uint16_t myBiggerInts[2];\n};\n```\n```python\n@struct_dataclass\nclass MyStruct(StructDataclass):\n myInts: Annotated[list[uint8_t], TypeMeta(size=4)]\n myBiggerInts: Annotated[list[uint16_t], TypeMeta(size=2)]\n\ns = MyStruct()\ns.decode([0, 64, 128, 255, 16, 0, 255, 255])\n# MyStruct(myInts=[0, 64, 128, 255], myBiggerInts=[4096, 65535])\n```\n\nYou can also set defaults for both basic types and lists.\n\nAll values will default to 0 or the initialized value for the chosen class if no\nspecific value is set.\n\nList defaults will set all items in the list to the same value. Currently\nsetting a complete default list for all values is not implemented.\n\n```c\nstruct MyStruct {\n uint8_t myInt = 5;\n uint8_t myInts[2];\n};\n```\n\n```python\n@struct_dataclass\nclass MyStruct(StructDataclass):\n myInt: uint8_t = 5\n myInts: Annnotated[list[uint8_t], TypeMeta(size=2, default=1)]\n\ns = MyStruct()\n# MyStruct(myInt=5, myInts=[1, 1])\ns.decode([10, 5, 6])\n# MyStruct(myInt=10, myInts=[5, 6])\n```\n\n# String / char[] Type\n\nDefining c-string types is a little different. Instead of using\n`size` in the `TypeMeta`, we need to instead use `chunk_size`.\n\nThis is because the way the struct format is defined for c-strings needs\nto know how big the string data is expected to be so that it can put the\nwhole string in a single variable. \n\nThe `chunk_size` is also introduced to allow for `char[][]` for converting\na list of strings.\n\n```c\nstruct MyStruct {\n char myStr[3];\n char myStrList[2][3];\n};\n```\n```python\n@struct_dataclass\nclass MyStruct(StructDataclass):\n myStr: Annotated[string_t, TypeMeta[str](chunk_size=3)]\n myStrList: Annotated[list[string_t], TypeMeta[str](size=2, chunk_size=3)]\n\n\ns = MyStruct()\ns.decode([65, 66, 67, 68, 69, 70, 71, 72, 73])\n# MyStruct(myStr=b\"ABC\", myStrList=[b\"DEF\", b\"GHI\"])\n```\n\nIf you instead try to define this as a list of `char_t` types,\nyou would only be able to end up with \n`MyStruct(myStr=[b\"A\", b\"B\", b\"C\"], myStrList=[b\"D\", b\"E\", b\"F\", b\"G\", b\"H\", b\"I\"])`\n\n# The Bits Abstraction\n\nThis library includes a `bits` abstraction to map bits to variables for easier access.\n\nOne example of this is converting a C enum like so:\n\n```c\nenum ConfigFlags {\n lights_flag = 1 << 0,\n platform_flag = 1 << 1,\n};\n#pragma pack(push, 1)\n```\n\n```python\n@bits(uint8_t, {\"lights_flag\": 0, \"platform_flag\": 1})\nclass FlagsType(BitsType): ...\n\nf = FlagsType()\nf.decode([3])\n# FlagsType(lights_flag=True, platform_flag=True)\nf.decode([2])\n# FlagsType(lights_flag=False, platform_flag=True)\nf.decode([1])\n# FlagsType(lights_flag=True, platform_flag=False)\n```\n\n# Custom StructDataclass Processing and Extensions\n\nThere may be times when you want to make the python class do \ncool fun python class type of stuff with the data structure. \nWe can extend the class functions `_decode` and `_encode` to\nhandle this processing.\n\nIn this example, lets say you want to be able to read/write the\nclass object as a list, using `__getitem__` and `__setitem__` as well\nas keeping the data in a different data structure than what the \nc struct defines.\n\n```c\nstruct MyStruct {\n uint8_t enabledSensors[5];\n};\n```\n\n```python\n@struct_dataclass\nclass EnabledSensors(StructDataclass):\n # We can define the actual data we are ingesting here\n # This mirrors the `uint8_t enabledSensors[5]` data\n _raw: Annotated[list[uint8_t], TypeMeta(size=5)]\n\n # We use this to store the data in the way we actually want\n _data: list[list[bool]] = field(default_factory=list)\n\n def _decode(self, data: list[int]) -> None:\n # First call the super function. This will store the raw values into `_raw`\n super()._decode(data)\n\n # Erase everything in self._data to remove any old data\n self._data = []\n\n # 2 Panels are packed into a single uint8_t, the left most 4 bits for the first\n # and the right most 4 bits for the second\n for bitlist in (list(map(bool, map(int, format(_byte, \"#010b\")[2:]))) for _byte in self._raw):\n self._data.append(bitlist[0:4])\n self._data.append(bitlist[4:])\n\n # Remove the last item in self._data as there are only 9 panels\n del self._data[-1]\n\n def _encode(self) -> list[int]:\n # Modify self._raw with updated values from self._data\n for idx, items in enumerate(list_chunks(self._data, 2)):\n # Last chunk\n if len(items) == 1:\n items.append([False, False, False, False])\n self._raw[idx] = sum(v << i for i, v in enumerate(list(itertools.chain.from_iterable(items))[::-1]))\n \n # Run the super function to return the encoded data from self._raw()\n return super()._encode()\n\n def __getitem__(self, index: int) -> list[bool]:\n # This lets us access the data with square brackets\n # ex. `config.enabled_sensors[Panel.UP][Sensor.RIGHT]`\n return self._data[index]\n\n def __setitem__(self, index: int, value: list[bool]) -> None:\n # Only use this to set a complete set for a panel\n # ex. `config.enabled_sensors[Panel.UP] = [True, True, False, True]`\n if len(value) != 4 or not all(isinstance(x, bool) for x in value):\n raise Exception(\"must set all 4 items at once\")\n\ns = EnabledSensors()\ns.decode([15, 15, 15, 15, 0])\n\n# The `self._data` here would look like:\n# [\n# [False, False, False, False],\n# [True, True, True, True],\n# [False, False, False, False],\n# [True, True, True, True],\n# [False, False, False, False],\n# [True, True, True, True],\n# [False, False, False, False],\n# [True, True, True, True],\n# [False, False, False, False],\n# ]\n\n# With the get/set functioned defined, we can access the data\n# with square accessors.\n# s[1][2] == True \n```\n\n# StructDataclass is Composable\n\nYou can use StructDataclasses in other StructDataclasses to create more complex\nstructs.\n\n```c \nstruct RGB {\n uint8_t r;\n uint8_t g;\n uint8_t b;\n};\n\nstruct LEDS {\n RGB lights[3];\n};\n```\n\n```python\n@struct_dataclass\nclass RGB(StructDataclass):\n r: uint8_t\n g: uint8_t\n b: uint8_t\n\n@struct_dataclass\nclass LEDS(StructDataclass):\n lights: Annotated[list[RGB], TypeMeta(size=3])]\n\nl = LEDS()\nl.decode([1, 2, 3, 4, 5, 6, 7, 8, 9])\n# LEDS(lights=[RGB(r=1, g=2, b=3), RGB(r=4, g=5, b=6), RGB(r=7, g=8, b=9)])\n```\n\n# Future Updates\n\n- Bitfield: Similar to the `Bits` abstraction. An easy way to define bitfields\n- Potentially more ways to define bits (dicts/lists/etc).\n- Potentially allowing list defaults to be entire pre-defined lists.\n- ???\n\n# Examples\n\nYou can see a more fully fledged example in the `test/examples.py` file. ",
"bugtrack_url": null,
"license": null,
"summary": "Leverage Python Types to Define C-Struct Interfaces",
"version": "0.3.0",
"project_urls": {
"Documentation": "https://pystructtype.readthedocs.io/en/latest/",
"Homepage": "https://github.com/fchorney/pystructtype",
"Issues": "https://github.com/fchorney/pystructtype/issues",
"Repository": "https://github.com/fchorney/pystructtype"
},
"split_keywords": [
"cstruct",
" struct",
" type"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "626f75812d1da2e9ec2c3d01ae27165e9bc5e36cc665c0212deb6ed53438d73d",
"md5": "dd32beb0e07a38e9d9753a3ffe76770b",
"sha256": "7a8c430e07c71d003190e998cacc48f4b91d6643d97b37ae9d308cdcac084fb1"
},
"downloads": -1,
"filename": "pystructtype-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "dd32beb0e07a38e9d9753a3ffe76770b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.13",
"size": 14591,
"upload_time": "2025-02-22T03:54:41",
"upload_time_iso_8601": "2025-02-22T03:54:41.464527Z",
"url": "https://files.pythonhosted.org/packages/62/6f/75812d1da2e9ec2c3d01ae27165e9bc5e36cc665c0212deb6ed53438d73d/pystructtype-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "6d801883f6a32bcad1e6d34a375cacf934e261685eff6afd44dc95143950a0ce",
"md5": "5b33ccbeb405e846ad8e9ea1dfca1cff",
"sha256": "d29703aaeb27f6127b0ff9b6761d9932540460f30d2a1f783f447d22c126fce9"
},
"downloads": -1,
"filename": "pystructtype-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "5b33ccbeb405e846ad8e9ea1dfca1cff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.13",
"size": 35876,
"upload_time": "2025-02-22T03:54:43",
"upload_time_iso_8601": "2025-02-22T03:54:43.857192Z",
"url": "https://files.pythonhosted.org/packages/6d/80/1883f6a32bcad1e6d34a375cacf934e261685eff6afd44dc95143950a0ce/pystructtype-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-22 03:54:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fchorney",
"github_project": "pystructtype",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pystructtype"
}
