| Name | robotframework-construct JSON |
| Version |
0.3.1
JSON |
| download |
| home_page | None |
| Summary | Infrastructure to use construct Structs in robotframework |
| upload_time | 2024-12-17 18:02:55 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10.0 |
| license | None |
| keywords |
robotframework
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# robotframework-construct
## I am in a hurry, let's jump-start with an example!
[git](https://git-scm.com/) (a version control system) and [uv](https://github.com/astral-sh/uv) (a tool for managing Python virtual environments) need to be preinstalled to run the examples.
```bash
git clone https://github.com/MarketSquare/robotframework-construct.git
cd robotframework-construct
uv sync --extra test --dev
uv run xonsh tasks/baseQC.xsh
```
Some examples, such as [USB HID](./atests/HIDKeyboard/) and [this](./atests/nfc_nci/), require specific hardware to function. For the [this](./atests/nfc_nci/) example, STMicroelectronics hardware and firmware is required. For the HID example, a USB Keyboard on a Linux machine is sufficient.
## What is robotframework-construct?
robotframework-construct is a [Robot Framework](https://robotframework.org) keyword library powered by [construct](https://construct.readthedocs.io/en/latest/).
[construct](https://construct.readthedocs.io/en/latest/) is a declarative and symmetrical parser and builder for binary data.
Aiming for :rocket: speed, :white_check_mark: reliability, and :microscope: visibility.
Your binary data becomes as accessible as numbers and strings are in Robot Framework.
Check out the documentation [here](https://marketsquare.github.io/robotframework-construct/)
### Licensing
robotframework-construct is an opensource keyword library licensed under Apache-2.0, leveraging the [construct](https://construct.readthedocs.io/en/latest/) library licensed under MIT.
## Use cases
- Beautifully access registers, for both reading and writing.
- Test your production construct specification against a reference implementation of the same protocol.
- Test your production binary parser/generator against a construct implementation of your binary format.
- Use your construct declarations to:
- Craft intentionally corrupted data.
- Fuzz test your binary parsers.
- While the network is an interesting area for robotframework-construct, other interfaces (UART/SPI/I2C) are considered first-class citizen.
## Features
Check out the full documentation at [robotframework-construct](https://marketsquare.github.io/robotframework-construct/).
### From construct, declaration not implementation of a parser/generator
This is a real-world usable declaration of the bson protocol.
```python
from construct import Struct, Int8ul, Int32sl, Int64sl, Int64ul, Float64l, Const, Array, Byte, GreedyBytes, CString, Prefixed, Switch, LazyBound, Pass, GreedyRange, Rebuild, this
_bson_element = Struct(
"type" / Int8ul,
"name" / CString("utf8"),
"value" / Switch(this.type, {
1: Float64l,
2: Prefixed(Int32sl, CString("utf8")),
3: LazyBound(lambda: document),
4: LazyBound(lambda: document),
5: Prefixed(Int32sl, GreedyBytes),
6: Pass,
7: Array(12, Byte),
8: Int8ul,
9: Int64sl,
10: Pass,
11: Struct("pattern" / CString("utf8"), "options" / CString("utf8")),
12: Struct("namespace" / CString("utf8"), "id" / Array(12, Byte)),
13: Prefixed(Int32sl, CString("utf8")),
14: Prefixed(Int32sl, CString("utf8")),
15: Struct("code" / Prefixed(Int32sl, CString("utf8")), "scope" / LazyBound(lambda: document)),
16: Int32sl,
17: Int64ul,
18: Int64sl,
19: Array(16, Byte),
-1: Pass,
127: Pass,
})
)
_e_list = GreedyRange(_bson_element)
def _calc_size(this):
return len(_e_list.build(this["elements"]))+5
document = Struct(
document = Struct(
"size" / Rebuild(Int32sl, _calc_size),
"elements" / _e_list,
"EOO" / Const(b"\x00")
)
```
This can be readily and directly derived from the [bson specification](https://bsonspec.org/spec.html). AI can assist in this process efficiently. This is because the mapping between the specification and the declaration is a very direct and straightforward task, making it easy to supervise the process and verify the result.
Using AI to generate a parser+generator would result in a larger volume of code to be verified, and the verification is harder.
### Checking and modifying binary data
There are keywords with embedded parameters that allow checking and modifying binary data in a Robot Framework way
A checking example
and a modifying example
**Note:** This is very natural in the Robot Framework environment. If multiple elements need to be checked, these checks should be organized in keywords.
### Observing the binary data
The built and parsed binary data is easily accessible. This helps with trust issues and makes it easier to read digital analyzer or oscilloscope screens. Also, a name to identify what definition is doing the parsing/generating may be provided.
A building example:
A parsing example:
### Breaking out of the ecosystems
The highly valuable building/parsing infrastructure does not depend on robotframework, and in the case of the parsing part, it also does not depend on Python.
The Structs can be transformed into kaitai. Kaitai is a DSL that can be transformed into parsers in 10+ languages and counting, you find further details [here](https://kaitai.io/).
Keep in mind that some limitations apply to these transformations.
For reference: [./tasks/breakoutCpp.xsh], which is a script that demonstrates how to transform Construct declarations into a C++ parser using the Kaitai DSL.
## Relationships in the Ecosystem
The number of dependencies is kept low, with no transitive dependencies.
This is important as it keeps coordination feasible. Construct is well-developed and not expected to change significantly soon. Robot Framework releases major updates annually, but these are well-managed and communicated.
### [Construct](https://github.com/construct/construct)
All parsing and generating capabilities come from Construct. No additional parsing/generating code is added; the only code added interfaces Construct with Robot Framework. The way Construct objects are created remains unchanged.
Construct has no non-optional dependencies.
### [Robot Framework](https://robotframework.org/)
This project connects Construct with Robot Framework. Only official APIs are used, and this project depends entirely on Robot Framework.
Robot Framework has no non-optional dependencies.
### [Rammbock](https://github.com/MarketSquare/Rammbock)
Rammbock inspired this project, as it was one of the reasons I started using Robot Framework.
Instead of maintaining Rammbock, we chose to integrate Construct.
#### Reasoning
Both Rammbock and Construct have limited engineering resources, but Construct is currently better supported. Construct also collaborates with Kaitai, engaging communities in C#, C++, and other ecosystems.
Using Construct provides a clear separation between parsing/generating logic and interface code, enabling expansion into other ecosystems.
## Installation
The robotframework-construct keyword library is hosted on pypi and can be installed like any pypi hosted python dependency with pip.
```
pip install robotframework-construct
```
## Limitations
Construct declarations must be written in `.py` files. There are no plans to integrate the Construct DSL into Robot Framework.
This eases the breaking out of the robot-framework and Python ecosystems.
## Quality Control Measures
Tested examples and acceptance tests using Robot Framework are provided. Unit tests are not a priority.
### Mutation Testing
Since this project primarily consists of interface code, it is crucial to catch user errors and produce clear error messages. Mutation testing ensures that all code paths and error messages are tested, supporting efforts to make errors informative.
## Project To-Do List
- [x] Parsing functionality demonstrated with an in-memory BSON object.
- [x] Parsing functionality demonstrated with a BSON file.
- [x] Generating functionality demonstrated with an in-memory BSON object.
- [x] Generating functionality demonstrated with a binary file.
- [x] Register read/write demonstrated with a mockup register.
- [x] Receive/transmit network example using DNS.
- [x] Reflector tool to allow to implement servers using clients.
- [x] Upload wheel to pypi.
- [x] Increase test coverage (Mutant killing) of the reflector
- [x] Segmentise mutation testing to speedup
- [x] Comment and document the real world example with the USB HID keyboard
- [x] Add a second real world example with binary interface to Readme
- [x] Have libdoc documentation online
- [x] Have libdoc documentation online for all keywords, not only the central ones
- [ ] User guide and tutorials/Article for (https://medium.com/@RobotFramework/).
- [x] Example on how to breakout of the python ecosystem
- [x] Midway review with Robot Framework Foundation.
- [ ] Final review with Robot Framework Foundation.
Raw data
{
"_id": null,
"home_page": null,
"name": "robotframework-construct",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10.0",
"maintainer_email": null,
"keywords": "robotframework",
"author": null,
"author_email": "Franz Haas <franz.dominik.haas@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/a4/48/2de099dfcd127c0b1e5a4ff97320bd11273dd17a26dc9c88b5515f9921b4/robotframework_construct-0.3.1.tar.gz",
"platform": null,
"description": "\n\n\n\n\n\n\n# robotframework-construct\n\n## I am in a hurry, let's jump-start with an example!\n[git](https://git-scm.com/) (a version control system) and [uv](https://github.com/astral-sh/uv) (a tool for managing Python virtual environments) need to be preinstalled to run the examples.\n\n```bash\ngit clone https://github.com/MarketSquare/robotframework-construct.git\ncd robotframework-construct\nuv sync --extra test --dev\nuv run xonsh tasks/baseQC.xsh\n```\n\nSome examples, such as [USB HID](./atests/HIDKeyboard/) and [this](./atests/nfc_nci/), require specific hardware to function. For the [this](./atests/nfc_nci/) example, STMicroelectronics hardware and firmware is required. For the HID example, a USB Keyboard on a Linux machine is sufficient.\n\n## What is robotframework-construct?\nrobotframework-construct is a [Robot Framework](https://robotframework.org) keyword library powered by [construct](https://construct.readthedocs.io/en/latest/).\n\n[construct](https://construct.readthedocs.io/en/latest/) is a declarative and symmetrical parser and builder for binary data.\n\nAiming for :rocket: speed, :white_check_mark: reliability, and :microscope: visibility.\n\nYour binary data becomes as accessible as numbers and strings are in Robot Framework.\n\nCheck out the documentation [here](https://marketsquare.github.io/robotframework-construct/)\n### Licensing\nrobotframework-construct is an opensource keyword library licensed under Apache-2.0, leveraging the [construct](https://construct.readthedocs.io/en/latest/) library licensed under MIT.\n\n## Use cases\n\n- Beautifully access registers, for both reading and writing.\n- Test your production construct specification against a reference implementation of the same protocol.\n- Test your production binary parser/generator against a construct implementation of your binary format.\n- Use your construct declarations to:\n - Craft intentionally corrupted data.\n - Fuzz test your binary parsers.\n- While the network is an interesting area for robotframework-construct, other interfaces (UART/SPI/I2C) are considered first-class citizen.\n\n## Features\n\nCheck out the full documentation at [robotframework-construct](https://marketsquare.github.io/robotframework-construct/).\n\n### From construct, declaration not implementation of a parser/generator\n\nThis is a real-world usable declaration of the bson protocol.\n\n```python\nfrom construct import Struct, Int8ul, Int32sl, Int64sl, Int64ul, Float64l, Const, Array, Byte, GreedyBytes, CString, Prefixed, Switch, LazyBound, Pass, GreedyRange, Rebuild, this\n\n\n_bson_element = Struct(\n \"type\" / Int8ul,\n \"name\" / CString(\"utf8\"),\n \"value\" / Switch(this.type, {\n 1: Float64l,\n 2: Prefixed(Int32sl, CString(\"utf8\")),\n 3: LazyBound(lambda: document),\n 4: LazyBound(lambda: document),\n 5: Prefixed(Int32sl, GreedyBytes),\n 6: Pass,\n 7: Array(12, Byte),\n 8: Int8ul,\n 9: Int64sl,\n 10: Pass,\n 11: Struct(\"pattern\" / CString(\"utf8\"), \"options\" / CString(\"utf8\")),\n 12: Struct(\"namespace\" / CString(\"utf8\"), \"id\" / Array(12, Byte)),\n 13: Prefixed(Int32sl, CString(\"utf8\")),\n 14: Prefixed(Int32sl, CString(\"utf8\")),\n 15: Struct(\"code\" / Prefixed(Int32sl, CString(\"utf8\")), \"scope\" / LazyBound(lambda: document)),\n 16: Int32sl,\n 17: Int64ul,\n 18: Int64sl,\n 19: Array(16, Byte),\n -1: Pass,\n 127: Pass,\n })\n)\n_e_list = GreedyRange(_bson_element)\n\ndef _calc_size(this):\n return len(_e_list.build(this[\"elements\"]))+5\n\ndocument = Struct(\ndocument = Struct(\n \"size\" / Rebuild(Int32sl, _calc_size),\n \"elements\" / _e_list,\n \"EOO\" / Const(b\"\\x00\")\n)\n```\n\nThis can be readily and directly derived from the [bson specification](https://bsonspec.org/spec.html). AI can assist in this process efficiently. This is because the mapping between the specification and the declaration is a very direct and straightforward task, making it easy to supervise the process and verify the result.\n\nUsing AI to generate a parser+generator would result in a larger volume of code to be verified, and the verification is harder.\n\n### Checking and modifying binary data\nThere are keywords with embedded parameters that allow checking and modifying binary data in a Robot Framework way\n\nA checking example\n\n\nand a modifying example\n\n\n**Note:** This is very natural in the Robot Framework environment. If multiple elements need to be checked, these checks should be organized in keywords.\n\n### Observing the binary data\nThe built and parsed binary data is easily accessible. This helps with trust issues and makes it easier to read digital analyzer or oscilloscope screens. Also, a name to identify what definition is doing the parsing/generating may be provided.\n\nA building example:\n\n\n\nA parsing example:\n\n\n\n### Breaking out of the ecosystems\nThe highly valuable building/parsing infrastructure does not depend on robotframework, and in the case of the parsing part, it also does not depend on Python.\nThe Structs can be transformed into kaitai. Kaitai is a DSL that can be transformed into parsers in 10+ languages and counting, you find further details [here](https://kaitai.io/).\n\nKeep in mind that some limitations apply to these transformations.\n\nFor reference: [./tasks/breakoutCpp.xsh], which is a script that demonstrates how to transform Construct declarations into a C++ parser using the Kaitai DSL.\n\n## Relationships in the Ecosystem\n\nThe number of dependencies is kept low, with no transitive dependencies.\n\nThis is important as it keeps coordination feasible. Construct is well-developed and not expected to change significantly soon. Robot Framework releases major updates annually, but these are well-managed and communicated.\n\n### [Construct](https://github.com/construct/construct)\n\nAll parsing and generating capabilities come from Construct. No additional parsing/generating code is added; the only code added interfaces Construct with Robot Framework. The way Construct objects are created remains unchanged.\n\nConstruct has no non-optional dependencies.\n\n### [Robot Framework](https://robotframework.org/)\n\nThis project connects Construct with Robot Framework. Only official APIs are used, and this project depends entirely on Robot Framework.\n\nRobot Framework has no non-optional dependencies.\n\n### [Rammbock](https://github.com/MarketSquare/Rammbock)\n\nRammbock inspired this project, as it was one of the reasons I started using Robot Framework.\n\nInstead of maintaining Rammbock, we chose to integrate Construct.\n\n#### Reasoning\n\nBoth Rammbock and Construct have limited engineering resources, but Construct is currently better supported. Construct also collaborates with Kaitai, engaging communities in C#, C++, and other ecosystems.\n\nUsing Construct provides a clear separation between parsing/generating logic and interface code, enabling expansion into other ecosystems.\n\n## Installation\n\nThe robotframework-construct keyword library is hosted on pypi and can be installed like any pypi hosted python dependency with pip.\n\n```\npip install robotframework-construct\n```\n\n## Limitations\n\nConstruct declarations must be written in `.py` files. There are no plans to integrate the Construct DSL into Robot Framework.\n\nThis eases the breaking out of the robot-framework and Python ecosystems.\n\n## Quality Control Measures\n\nTested examples and acceptance tests using Robot Framework are provided. Unit tests are not a priority.\n\n### Mutation Testing\n\nSince this project primarily consists of interface code, it is crucial to catch user errors and produce clear error messages. Mutation testing ensures that all code paths and error messages are tested, supporting efforts to make errors informative.\n\n## Project To-Do List\n\n- [x] Parsing functionality demonstrated with an in-memory BSON object.\n- [x] Parsing functionality demonstrated with a BSON file.\n- [x] Generating functionality demonstrated with an in-memory BSON object.\n- [x] Generating functionality demonstrated with a binary file.\n- [x] Register read/write demonstrated with a mockup register.\n- [x] Receive/transmit network example using DNS.\n- [x] Reflector tool to allow to implement servers using clients.\n- [x] Upload wheel to pypi.\n- [x] Increase test coverage (Mutant killing) of the reflector\n- [x] Segmentise mutation testing to speedup\n- [x] Comment and document the real world example with the USB HID keyboard\n- [x] Add a second real world example with binary interface to Readme\n- [x] Have libdoc documentation online\n- [x] Have libdoc documentation online for all keywords, not only the central ones\n- [ ] User guide and tutorials/Article for (https://medium.com/@RobotFramework/).\n- [x] Example on how to breakout of the python ecosystem\n- [x] Midway review with Robot Framework Foundation.\n- [ ] Final review with Robot Framework Foundation.\n",
"bugtrack_url": null,
"license": null,
"summary": "Infrastructure to use construct Structs in robotframework",
"version": "0.3.1",
"project_urls": {
"Documentation": "https://marketsquare.github.io/robotframework-construct",
"Release_Notes": "https://marketsquare.github.io/robotframework-construct/release_notes.md",
"Repository": "https://github.com/MarketSquare/robotframework-construct"
},
"split_keywords": [
"robotframework"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "25e90d24c029a7f7a3ce02bc14763f520029b621fb237ddf6f629a602c923648",
"md5": "73ddd1ec414e784d6e76cd5ef6a6dff9",
"sha256": "635e1f04254606a182215f607b10f7fb0d1cf8383f2f264e7363e3cb0e685ff0"
},
"downloads": -1,
"filename": "robotframework_construct-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "73ddd1ec414e784d6e76cd5ef6a6dff9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10.0",
"size": 15713,
"upload_time": "2024-12-17T18:02:50",
"upload_time_iso_8601": "2024-12-17T18:02:50.397021Z",
"url": "https://files.pythonhosted.org/packages/25/e9/0d24c029a7f7a3ce02bc14763f520029b621fb237ddf6f629a602c923648/robotframework_construct-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a4482de099dfcd127c0b1e5a4ff97320bd11273dd17a26dc9c88b5515f9921b4",
"md5": "f33ed23d7cb687a497cac842a79a1bf4",
"sha256": "36ea25f208619a783cbd77112e55914c72e461448f89baf699c549b16195d634"
},
"downloads": -1,
"filename": "robotframework_construct-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "f33ed23d7cb687a497cac842a79a1bf4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10.0",
"size": 354243,
"upload_time": "2024-12-17T18:02:55",
"upload_time_iso_8601": "2024-12-17T18:02:55.052158Z",
"url": "https://files.pythonhosted.org/packages/a4/48/2de099dfcd127c0b1e5a4ff97320bd11273dd17a26dc9c88b5515f9921b4/robotframework_construct-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-17 18:02:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MarketSquare",
"github_project": "robotframework-construct",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "robotframework-construct"
}
