# Caterpillar - 🐛
[](https://www.python.org/downloads/)
](https://img.shields.io/github/v/release/MatrixEditor/caterpillar.svg?logo=github&label=Latest+Version)
[](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-sphinx.yml)
[](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-test.yml)
Caterpillar is a Python 3.12+ library to pack and unpack structurized binary data. It
enhances the capabilities of [Python Struct](https://docs.python.org/3/library/struct.html)
by enabling direct class declaration. More information about the different configuration
options will be added in the future. Documentation is [here >](https://matrixeditor.github.io/caterpillar/).
*Caterpillar* is able to:
* Pack and unpack data just from processing Python class definitions (including support for powerful bitfields, c++-like templates and c-like unions!),
* apply a wide range of data types (with endianess and architecture configuration),
* dynamically adapt structs based on their inheritance layout,
* reduce the used memory space using `__slots__`,
* allowing you to place conditional statements into class definitions,
* insert proper types into the class definition to support documentation and
* it helps you to create cleaner and more compact code.
* You can even extend Caterpillar and write your parsing logic in C or C++!!
> [!NOTE]
> Python 3.14 breaks `with` statements in class definitions since `__annotations__` are added at the end
> of a class definition. Therefore, `Digest` and conditional statements **ARE NOT SUPPORTED** using the `with` syntax in Python 3.14+.
> As of version `2.4.5` the `Digest` class has a counterpart (`DigestField`), which can be used to manually specify a digest without
> the need of a `ẁith` statement.
## Give me some code!
```python
@bitfield(order=LittleEndian)
class Header:
version : 4 # 4bit integer
valid : 1 # 1bit flag (boolean)
ident : (8, CharFactory) # 8bit char
# automatic alignment to 16bits
@struct(order=LittleEndian)
class Format:
magic : b"ITS MAGIC" # Supports string and byte constants directly
header : Header
a : uint8 # Primitive data types
b : int32
length : uint8 # String fields with computed lengths
name : String(this.length) # -> you can also use Prefixed(uint8)
_hash_begin : DigestField.begin("hash", Md5_Algo) # custom actions, e.g. for hashes
names : CString[uint8::] # Sequences with prefixed, computed lengths
hash : Md5_Field("hash", verify=True) = None # automatic hash creation and verification + default value
# Instantiation (keyword-only arguments, magic is auto-inferred):
obj = Format(
header=Header(version=2, valid=True, ident="F"),
a=1,
b=2,
length=3,
name="foo",
names=["a", "b"]
)
# Packing the object, reads as 'PACK obj FROM Format'
# objects of struct classes can be packed right away
blob = pack(obj, Format)
# results in: b'ITS MAGIC0*\x01\x02\x00\x00\x00\x03foo\x02a\x00b\x00)\x9a...'
# Unpacking the binary data, reads as 'UNPACK Format FROM blob'
obj2 = unpack(Format, blob)
```
This library offers extensive functionality beyond basic struct handling. For further details
on its powerful features, explore the official [documentation](https://matrixeditor.github.io/caterpillar/),
[examples](./examples/), and [test cases](./test/).
## Installation
> [!NOTE]
> As of Caterpillar v2.1.2 it is possible to install the library without the need of
> compiling the C extension.
### PIP installation (Python-only)
```bash
pip install caterpillar-py
```
### Python-only installation
```bash
pip install "caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar"
```
### Installation + C-extension
```bash
pip install "caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar/#subdirectory=src/ccaterpillar"
```
## Starting Point
Please visit the [Documentation](https://matrixeditor.github.io/caterpillar/), it contains a complete tutorial on how to use this library.
## Other Approaches
A list of similar approaches to parsing structured binary data with Python can be taken from below:
* [construct](https://github.com/construct/construct)
* [kaitai_struct](https://github.com/kaitai-io/kaitai_struct)
* [hachoir](https://hachoir.readthedocs.io/en/latest/)
* [mrcrowbar](https://github.com/moralrecordings/mrcrowbar)
The documentation also provides a [Comparison](https://matrixeditor.github.io/caterpillar/reference/introduction.html#comparison)
to these approaches.
## License
Distributed under the GNU General Public License (V3). See [License](LICENSE) for more information.
Raw data
{
"_id": null,
"home_page": null,
"name": "caterpillar-py",
"maintainer": "MatrixEditor",
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "MatrixEditor",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# Caterpillar - \ud83d\udc1b\n\n[](https://www.python.org/downloads/)\n](https://img.shields.io/github/v/release/MatrixEditor/caterpillar.svg?logo=github&label=Latest+Version)\n[](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-sphinx.yml)\n[](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-test.yml)\n\n\n\n\nCaterpillar is a Python 3.12+ library to pack and unpack structurized binary data. It\nenhances the capabilities of [Python Struct](https://docs.python.org/3/library/struct.html)\nby enabling direct class declaration. More information about the different configuration\noptions will be added in the future. Documentation is [here >](https://matrixeditor.github.io/caterpillar/).\n\n*Caterpillar* is able to:\n\n* Pack and unpack data just from processing Python class definitions (including support for powerful bitfields, c++-like templates and c-like unions!),\n* apply a wide range of data types (with endianess and architecture configuration),\n* dynamically adapt structs based on their inheritance layout,\n* reduce the used memory space using `__slots__`,\n* allowing you to place conditional statements into class definitions,\n* insert proper types into the class definition to support documentation and\n* it helps you to create cleaner and more compact code.\n* You can even extend Caterpillar and write your parsing logic in C or C++!!\n\n> [!NOTE]\n> Python 3.14 breaks `with` statements in class definitions since `__annotations__` are added at the end\n> of a class definition. Therefore, `Digest` and conditional statements **ARE NOT SUPPORTED** using the `with` syntax in Python 3.14+.\n> As of version `2.4.5` the `Digest` class has a counterpart (`DigestField`), which can be used to manually specify a digest without\n> the need of a `\u1e81ith` statement.\n\n\n## Give me some code!\n\n```python\n@bitfield(order=LittleEndian)\nclass Header:\n version : 4 # 4bit integer\n valid : 1 # 1bit flag (boolean)\n ident : (8, CharFactory) # 8bit char\n # automatic alignment to 16bits\n\n@struct(order=LittleEndian)\nclass Format:\n magic : b\"ITS MAGIC\" # Supports string and byte constants directly\n header : Header\n a : uint8 # Primitive data types\n b : int32\n length : uint8 # String fields with computed lengths\n name : String(this.length) # -> you can also use Prefixed(uint8)\n\n _hash_begin : DigestField.begin(\"hash\", Md5_Algo) # custom actions, e.g. for hashes\n names : CString[uint8::] # Sequences with prefixed, computed lengths\n hash : Md5_Field(\"hash\", verify=True) = None # automatic hash creation and verification + default value\n\n# Instantiation (keyword-only arguments, magic is auto-inferred):\nobj = Format(\n header=Header(version=2, valid=True, ident=\"F\"),\n a=1,\n b=2,\n length=3,\n name=\"foo\",\n names=[\"a\", \"b\"]\n)\n# Packing the object, reads as 'PACK obj FROM Format'\n# objects of struct classes can be packed right away\nblob = pack(obj, Format)\n# results in: b'ITS MAGIC0*\\x01\\x02\\x00\\x00\\x00\\x03foo\\x02a\\x00b\\x00)\\x9a...'\n\n# Unpacking the binary data, reads as 'UNPACK Format FROM blob'\nobj2 = unpack(Format, blob)\n```\n\nThis library offers extensive functionality beyond basic struct handling. For further details\non its powerful features, explore the official [documentation](https://matrixeditor.github.io/caterpillar/),\n[examples](./examples/), and [test cases](./test/).\n\n## Installation\n\n> [!NOTE]\n> As of Caterpillar v2.1.2 it is possible to install the library without the need of\n> compiling the C extension.\n\n### PIP installation (Python-only)\n\n```bash\npip install caterpillar-py\n```\n\n### Python-only installation\n\n```bash\npip install \"caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar\"\n```\n\n### Installation + C-extension\n\n```bash\npip install \"caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar/#subdirectory=src/ccaterpillar\"\n```\n\n\n## Starting Point\n\nPlease visit the [Documentation](https://matrixeditor.github.io/caterpillar/), it contains a complete tutorial on how to use this library.\n\n## Other Approaches\n\nA list of similar approaches to parsing structured binary data with Python can be taken from below:\n\n* [construct](https://github.com/construct/construct)\n* [kaitai_struct](https://github.com/kaitai-io/kaitai_struct)\n* [hachoir](https://hachoir.readthedocs.io/en/latest/)\n* [mrcrowbar](https://github.com/moralrecordings/mrcrowbar)\n\nThe documentation also provides a [Comparison](https://matrixeditor.github.io/caterpillar/reference/introduction.html#comparison)\nto these approaches.\n\n## License\n\nDistributed under the GNU General Public License (V3). See [License](LICENSE) for more information.\n",
"bugtrack_url": null,
"license": null,
"summary": "Library to pack and unpack structurized binary data.",
"version": "2.6.0",
"project_urls": {
"API-Docs": "https://matrixeditor.github.io/caterpillar",
"Homepage": "https://github.com/MatrixEditor/caterpillar"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1bbe7752ec08113339c7541b0643e4243833ad7dd4a1006d36faa7f6e95d4a6f",
"md5": "2ba1d388089e5407f37efc8c55038a2c",
"sha256": "64553201a42bfbcf46da177086f93fa942e0223adcb940aef1ce39d5b99c59eb"
},
"downloads": -1,
"filename": "caterpillar_py-2.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2ba1d388089e5407f37efc8c55038a2c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 134314,
"upload_time": "2025-08-13T17:12:10",
"upload_time_iso_8601": "2025-08-13T17:12:10.475525Z",
"url": "https://files.pythonhosted.org/packages/1b/be/7752ec08113339c7541b0643e4243833ad7dd4a1006d36faa7f6e95d4a6f/caterpillar_py-2.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-13 17:12:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MatrixEditor",
"github_project": "caterpillar",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "cryptography",
"specs": []
},
{
"name": "lzallright",
"specs": []
}
],
"tox": true,
"lcname": "caterpillar-py"
}
JSON
