| Name | blark JSON |
| Version |
0.8.1
JSON |
| download |
| home_page | |
| Summary | Beckhoff TwinCAT IEC 61131-3 parsing tools |
| upload_time | 2023-11-21 23:18:42 |
| maintainer | |
| docs_url | None |
| author | SLAC National Accelerator Laboratory |
| requires_python | >=3.6 |
| license | GPL |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Beckhoff TwinCAT IEC 61131-3 Lark-based Structured Text Tools
Or for short, blark. B(eckhoff)-lark. It sounded good in my head, at least.
## The Grammar
The [grammar](blark/iec.lark) uses Lark's Earley parser algorithm.
The grammar itself is not perfect. It may not reliably parse your source code
or produce useful Python instances just yet.
See [issues](https://github.com/klauer/blark/issues) for further details.
As a fun side project, blark isn't at the top of my priority list. For
an idea of where the project is going, see the issues list.
## Requirements
* [lark](https://github.com/lark-parser/lark) (for grammar-based parsing)
* [lxml](https://github.com/lxml/lxml) (for parsing TwinCAT projects)
## Capabilities
* TwinCAT source code file parsing (``*.TcPOU`` and others)
* TwinCAT project and solution loading
* ``lark.Tree`` generation of any supported source code
* Python dataclasses of supported source code, with introspection and code refactoring
### Works-in-progress
* Sphinx API documentation generation (a new Sphinx domain)
* Code reformatting
* "Dependency store" - recursively parse and inspect project dependencies
* Summary generation - a layer on top of dataclasses to summarize source code details
* Rewriting source code directly in TwinCAT source code files
## Installation
Installation is quick with Pip.
```bash
pip install --upgrade blark
```
### Quickstart (pip / virtualenv with venv)
1. Set up an environment using venv:
```bash
$ python -m venv blark_venv
$ source blark_venv/bin/activate
```
2. Install the library with pip:
```bash
$ python -m pip install blark
```
### Quickstart (Conda)
1. Set up an environment using conda:
```bash
$ conda create -n blark-env -c conda-forge python=3.10 pip blark
$ conda activate blark-env
```
2. Install the library from conda:
```bash
$ conda install blark
```
### Development install
If you run into issues or wish to run an unreleased version of blark, you may
install directly from this repository like so:
```bash
$ python -m pip install git+https://github.com/klauer/blark
```
## Sample runs
Run the parser or experimental formatter utility. Current supported file types
include those from TwinCAT3 projects ( ``.tsproj``, ``.sln``, ``.TcPOU``,
``.TcGVL``) and plain-text ``.st`` files.
```bash
$ blark parse --print-tree blark/tests/POUs/F_SetStateParams.TcPOU
function_declaration
None
F_SetStateParams
indirect_simple_specification
None
simple_specification BOOL
input_declarations
None
var1_init_decl
var1_list
... (clipped) ...
```
To interact with the Python dataclasses directly, make sure IPython is
installed first and then try:
```
$ blark parse --interactive blark/tests/POUs/F_SetStateParams.TcPOU
# Assuming IPython is installed, the following prompt will come up:
In [1]: results[0].identifier
Out[1]: 'F_SetStateParams/declaration'
In [2]: results[1].identifier
Out[2]: 'F_SetStateParams/implementation'
```
Dump out a parsed and reformatted set of source code:
```bash
$ blark format blark/tests/source/array_of_objects.st
{attribute 'hide'}
METHOD prv_Detection : BOOL
VAR_IN_OUT
currentChannel : ARRAY [APhase..CPhase] OF class_baseVector(SIZEOF(vector_t), 0);
END_VAR
END_METHOD
```
blark supports rewriting TwinCAT source code files directly as well:
```bash
$ blark format blark/tests/POUs/F_SetStateParams.TcPOU
<TcPlcObject Version="1.1.0.1" ProductVersion="3.1.4024.0">
<POU Name="F_SetStateParams" Id="{f9611d23-4bb5-422d-9f11-2cc94e61fc9e}" SpecialFunc="None">
<Declaration><![CDATA[FUNCTION F_SetStateParams : BOOL
VAR_INPUT
nStateRef : UDINT;
rPosition : REAL;
rTolerance : REAL;
stBeamParams : ST_BeamParams;
... (clipped) ...
```
It is also possible to parse the source code into a tokenized ``SourceCode``
tree which supports code introspection and rewriting:
```python
In [1]: import blark
In [2]: parsed = blark.parse_source_code(
...: """
...: PROGRAM ProgramName
...: VAR_INPUT
...: iValue : INT;
...: END_VAR
...: VAR_ACCESS
...: AccessName : SymbolicVariable : TypeName READ_WRITE;
...: END_VAR
...: iValue := iValue + 1;
...: END_PROGRAM
...: """
...: )
# Access the lark Tree here:
In [3]: parsed.tree.data
Out[3]: Token('RULE', 'iec_source')
# Or the transformed information:
In [3]: transformed = parsed.transform()
In [4]: program = transformed.items[0]
In [5]: program.declarations[0].items[0].variables[0].name
Out[5]: Token('IDENTIFIER', 'iValue')
```
The supported starting grammar rules for the reusable parser include:
```
"iec_source"
"action"
"data_type_declaration"
"function_block_method_declaration"
"function_block_property_declaration"
"function_block_type_declaration"
"function_declaration"
"global_var_declarations"
"program_declaration"
"statement_list"
```
Other starting rules remain possible for advanced users, however a new parser
must be created in that scenario and transformations are not supported.
Additionally, please note that you should avoid creating parsers on-the-fly as
there is a startup cost to re-parsing the grammar. Utilize the provided parser
from ``blark.get_parser()`` whenever possible.
```
In [1]: import blark
In [2]: parser = blark.new_parser(start=["any_integer"])
In [3]: Tree('hex_integer', [Token('HEX_STRING', '1010')])
```
## Adding Test Cases
Presently, test cases are provided in two forms. Within the `blark/tests/`
directory there are `POUs/` and `source/` directories.
TwinCAT source code files belong in ``blark/tests/POUs``.
Plain-text source code files (e.g., ``.st`` files) belong in
``blark/tests/source``.
Feel free to contribute your own test cases and we'll do our best to ensure
that blark parses them (and continues to parse them) without issue.
## Acknowledgements
Originally based on Volker Birk's IEC 61131-3 grammar
[iec2xml](https://fdik.org/iec2xml/) (GitHub fork
[here](https://github.com/klauer/iec2xml)) and [A Syntactic
Specification for the Programming Languages of theIEC 61131-3
Standard](https://www.researchgate.net/publication/228971719_A_syntactic_specification_for_the_programming_languages_of_the_IEC_61131-3_standard)
by Flor Narciso et al. Many aspects of the grammar have been added to,
modified, and in cases entirely rewritten to better support lark grammars and
transformers.
Special thanks to the blark contributors:
- @engineerjoe440
## Related, Similar, or Alternative Projects
There are a number of similar, or related projects that are available.
- ["MATIEC"](https://github.com/nucleron/matiec) - another IEC 61131-3 Structured
Text parser which supports IEC 61131-3 second edition, without classes,
namespaces and other fancy features. An updated version is also
[available on Github](https://github.com/sm1820/matiec)
- [OpenPLC Runtime Version 3](https://github.com/thiagoralves/OpenPLC_v3) -
As stated by the project:
> OpenPLC is an open-source Programmable Logic Controller that is based on easy to use software. Our focus is to provide a low cost industrial solution for automation and research. OpenPLC has been used in many research papers as a framework for industrial cyber security research, given that it is the only controller to provide the entire source code.
- [RuSTy](https://github.com/PLC-lang/rusty)
[documentation](https://plc-lang.github.io/rusty/intro_1.html) - Structured text
compiler written in Rust. As stated by the project:
> RuSTy is a structured text (ST) compiler written in Rust. RuSTy utilizes the LLVM framework to compile eventually to native code.
- [IEC Checker](https://github.com/jubnzv/iec-checker) - Static analysis tool
for IEC 61131-3 logic. As described by the maintainer:
> iec-checker has the ability to parse ST source code and dump AST and CFG to JSON format, so you can process it with your language of choice.
- [TcBlack](https://github.com/Roald87/TcBlack) - Python black-like code formatter for TwinCAT code.
Raw data
{
"_id": null,
"home_page": "",
"name": "blark",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "SLAC National Accelerator Laboratory",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/54/0d/355eebe9ff16335f542d0e52c0245c4425c3c3e52fe4a1f08da686d25f8c/blark-0.8.1.tar.gz",
"platform": null,
"description": "# Beckhoff TwinCAT IEC 61131-3 Lark-based Structured Text Tools\n\nOr for short, blark. B(eckhoff)-lark. It sounded good in my head, at least.\n\n## The Grammar\n\nThe [grammar](blark/iec.lark) uses Lark's Earley parser algorithm.\n\nThe grammar itself is not perfect. It may not reliably parse your source code\nor produce useful Python instances just yet.\n\nSee [issues](https://github.com/klauer/blark/issues) for further details.\n\nAs a fun side project, blark isn't at the top of my priority list. For\nan idea of where the project is going, see the issues list.\n\n## Requirements\n\n* [lark](https://github.com/lark-parser/lark) (for grammar-based parsing)\n* [lxml](https://github.com/lxml/lxml) (for parsing TwinCAT projects)\n\n## Capabilities\n\n* TwinCAT source code file parsing (``*.TcPOU`` and others)\n* TwinCAT project and solution loading\n* ``lark.Tree`` generation of any supported source code\n* Python dataclasses of supported source code, with introspection and code refactoring\n\n### Works-in-progress\n\n* Sphinx API documentation generation (a new Sphinx domain)\n* Code reformatting\n* \"Dependency store\" - recursively parse and inspect project dependencies\n* Summary generation - a layer on top of dataclasses to summarize source code details\n* Rewriting source code directly in TwinCAT source code files\n\n## Installation\n\nInstallation is quick with Pip.\n\n```bash\npip install --upgrade blark\n```\n\n### Quickstart (pip / virtualenv with venv)\n\n1. Set up an environment using venv:\n ```bash\n $ python -m venv blark_venv\n $ source blark_venv/bin/activate\n ```\n2. Install the library with pip:\n ```bash\n $ python -m pip install blark\n ```\n\n### Quickstart (Conda)\n\n1. Set up an environment using conda:\n ```bash\n $ conda create -n blark-env -c conda-forge python=3.10 pip blark\n $ conda activate blark-env\n ```\n2. Install the library from conda:\n ```bash\n $ conda install blark\n ```\n\n### Development install\n\nIf you run into issues or wish to run an unreleased version of blark, you may\ninstall directly from this repository like so:\n```bash\n$ python -m pip install git+https://github.com/klauer/blark\n```\n\n## Sample runs\n\nRun the parser or experimental formatter utility. Current supported file types\ninclude those from TwinCAT3 projects ( ``.tsproj``, ``.sln``, ``.TcPOU``,\n``.TcGVL``) and plain-text ``.st`` files.\n\n```bash\n$ blark parse --print-tree blark/tests/POUs/F_SetStateParams.TcPOU\nfunction_declaration\n None\n F_SetStateParams\n indirect_simple_specification\n None\n simple_specification BOOL\n input_declarations\n None\n var1_init_decl\n var1_list\n... (clipped) ...\n```\n\nTo interact with the Python dataclasses directly, make sure IPython is\ninstalled first and then try:\n\n```\n$ blark parse --interactive blark/tests/POUs/F_SetStateParams.TcPOU\n# Assuming IPython is installed, the following prompt will come up:\n\nIn [1]: results[0].identifier\nOut[1]: 'F_SetStateParams/declaration'\n\nIn [2]: results[1].identifier\nOut[2]: 'F_SetStateParams/implementation'\n```\n\nDump out a parsed and reformatted set of source code:\n\n```bash\n$ blark format blark/tests/source/array_of_objects.st\n{attribute 'hide'}\nMETHOD prv_Detection : BOOL\n VAR_IN_OUT\n currentChannel : ARRAY [APhase..CPhase] OF class_baseVector(SIZEOF(vector_t), 0);\n END_VAR\nEND_METHOD\n```\n\nblark supports rewriting TwinCAT source code files directly as well:\n\n```bash\n$ blark format blark/tests/POUs/F_SetStateParams.TcPOU\n\n<TcPlcObject Version=\"1.1.0.1\" ProductVersion=\"3.1.4024.0\">\n <POU Name=\"F_SetStateParams\" Id=\"{f9611d23-4bb5-422d-9f11-2cc94e61fc9e}\" SpecialFunc=\"None\">\n <Declaration><![CDATA[FUNCTION F_SetStateParams : BOOL\n VAR_INPUT\n nStateRef : UDINT;\n rPosition : REAL;\n rTolerance : REAL;\n stBeamParams : ST_BeamParams;\n\n... (clipped) ...\n```\n\nIt is also possible to parse the source code into a tokenized ``SourceCode``\ntree which supports code introspection and rewriting:\n\n```python\nIn [1]: import blark\n\nIn [2]: parsed = blark.parse_source_code(\n ...: \"\"\"\n ...: PROGRAM ProgramName\n ...: VAR_INPUT\n ...: iValue : INT;\n ...: END_VAR\n ...: VAR_ACCESS\n ...: AccessName : SymbolicVariable : TypeName READ_WRITE;\n ...: END_VAR\n ...: iValue := iValue + 1;\n ...: END_PROGRAM\n ...: \"\"\"\n ...: )\n\n# Access the lark Tree here:\nIn [3]: parsed.tree.data\nOut[3]: Token('RULE', 'iec_source')\n\n# Or the transformed information:\nIn [3]: transformed = parsed.transform()\n\nIn [4]: program = transformed.items[0]\n\nIn [5]: program.declarations[0].items[0].variables[0].name\nOut[5]: Token('IDENTIFIER', 'iValue')\n```\n\nThe supported starting grammar rules for the reusable parser include:\n\n```\n\"iec_source\"\n\"action\"\n\"data_type_declaration\"\n\"function_block_method_declaration\"\n\"function_block_property_declaration\"\n\"function_block_type_declaration\"\n\"function_declaration\"\n\"global_var_declarations\"\n\"program_declaration\"\n\"statement_list\"\n```\n\nOther starting rules remain possible for advanced users, however a new parser\nmust be created in that scenario and transformations are not supported.\n\nAdditionally, please note that you should avoid creating parsers on-the-fly as\nthere is a startup cost to re-parsing the grammar. Utilize the provided parser\nfrom ``blark.get_parser()`` whenever possible.\n\n```\nIn [1]: import blark\n\nIn [2]: parser = blark.new_parser(start=[\"any_integer\"])\n\nIn [3]: Tree('hex_integer', [Token('HEX_STRING', '1010')])\n```\n\n## Adding Test Cases\n\nPresently, test cases are provided in two forms. Within the `blark/tests/`\ndirectory there are `POUs/` and `source/` directories.\n\nTwinCAT source code files belong in ``blark/tests/POUs``.\nPlain-text source code files (e.g., ``.st`` files) belong in\n``blark/tests/source``.\n\nFeel free to contribute your own test cases and we'll do our best to ensure\nthat blark parses them (and continues to parse them) without issue.\n\n## Acknowledgements\n\nOriginally based on Volker Birk's IEC 61131-3 grammar\n[iec2xml](https://fdik.org/iec2xml/) (GitHub fork\n[here](https://github.com/klauer/iec2xml)) and [A Syntactic\nSpecification for the Programming Languages of theIEC 61131-3\nStandard](https://www.researchgate.net/publication/228971719_A_syntactic_specification_for_the_programming_languages_of_the_IEC_61131-3_standard)\nby Flor Narciso et al. Many aspects of the grammar have been added to,\nmodified, and in cases entirely rewritten to better support lark grammars and\ntransformers.\n\nSpecial thanks to the blark contributors:\n\n- @engineerjoe440\n\n## Related, Similar, or Alternative Projects\n\nThere are a number of similar, or related projects that are available.\n\n- [\"MATIEC\"](https://github.com/nucleron/matiec) - another IEC 61131-3 Structured\nText parser which supports IEC 61131-3 second edition, without classes,\nnamespaces and other fancy features. An updated version is also\n[available on Github](https://github.com/sm1820/matiec)\n- [OpenPLC Runtime Version 3](https://github.com/thiagoralves/OpenPLC_v3) -\nAs stated by the project:\n > OpenPLC is an open-source Programmable Logic Controller that is based on easy to use software. Our focus is to provide a low cost industrial solution for automation and research. OpenPLC has been used in many research papers as a framework for industrial cyber security research, given that it is the only controller to provide the entire source code.\n- [RuSTy](https://github.com/PLC-lang/rusty)\n[documentation](https://plc-lang.github.io/rusty/intro_1.html) - Structured text\ncompiler written in Rust. As stated by the project:\n > RuSTy is a structured text (ST) compiler written in Rust. RuSTy utilizes the LLVM framework to compile eventually to native code.\n- [IEC Checker](https://github.com/jubnzv/iec-checker) - Static analysis tool\nfor IEC 61131-3 logic. As described by the maintainer:\n > iec-checker has the ability to parse ST source code and dump AST and CFG to JSON format, so you can process it with your language of choice.\n- [TcBlack](https://github.com/Roald87/TcBlack) - Python black-like code formatter for TwinCAT code.\n",
"bugtrack_url": null,
"license": "GPL",
"summary": "Beckhoff TwinCAT IEC 61131-3 parsing tools",
"version": "0.8.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "01db8d30d8979ec01a89a9f65cfc5cff1bd307ee920383bc4a3ccf86f9e03606",
"md5": "342a56e4b572a643c593658f1f46fa69",
"sha256": "5b1df248459a2f12a57d09046742b9eb89aa3f7bdb55dfc14a6b0ee773b31aa8"
},
"downloads": -1,
"filename": "blark-0.8.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "342a56e4b572a643c593658f1f46fa69",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 127656,
"upload_time": "2023-11-21T23:27:40",
"upload_time_iso_8601": "2023-11-21T23:27:40.326105Z",
"url": "https://files.pythonhosted.org/packages/01/db/8d30d8979ec01a89a9f65cfc5cff1bd307ee920383bc4a3ccf86f9e03606/blark-0.8.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "540d355eebe9ff16335f542d0e52c0245c4425c3c3e52fe4a1f08da686d25f8c",
"md5": "2a5631670697e70ecf2a5540df04d4e4",
"sha256": "76a8c9438a2afeb2f121b44571ec95608de9d037920d95c3f27a0be94b64c699"
},
"downloads": -1,
"filename": "blark-0.8.1.tar.gz",
"has_sig": false,
"md5_digest": "2a5631670697e70ecf2a5540df04d4e4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 133852,
"upload_time": "2023-11-21T23:18:42",
"upload_time_iso_8601": "2023-11-21T23:18:42.603953Z",
"url": "https://files.pythonhosted.org/packages/54/0d/355eebe9ff16335f542d0e52c0245c4425c3c3e52fe4a1f08da686d25f8c/blark-0.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-21 23:18:42",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "blark"
}
