# "Fat Token" Parser and Modularity Checker for C Code
This library provides a parser for C code that extracts tokens and checks the modularity of the code based on the rules defined in the [Modularity](MODULARITY.md) document.
The core concept of the library is the "Fat Token" which represents a higher-level abstraction of elements in C code rather than a simple sequence of characters. A Fat Token can represent items such as "words", "comments", "strings", "expressions within parentheses", etc.
With this this higher-level representation, the library enables a layered approach to parsing, focusing on high level structures without getting bogged down in unnecessary details. The parser extracts Fat Tokens from the C code and groups them into "statements" representing logical units like function definitions, struct declarations, variable declarations, etc.
For example, a function definition might be represented as a sequence of: "comment", "word", "word", "expression in parentheses", and "expression in curly braces". We can recognize this as a function definition without needing to parse the internal details of the function body or arguments list.
This approach makes it possible to perform a variety of analyses on C code or other languages with clear syntax rules.
Implementation details are described in the [Implementation](IMPL.md) document.
## Installation
```bash
$ pip install layercparse
```
## Usage
```python
import os
from layercparse import *
def main():
# setLogLevel(LogLevel.WARNING)
setRootPath(os.path.realpath(sys.argv[1]))
setModules([
Module("module1"),
Module("module2", fileAliases=["m2"], sourceAliases = ["mod2", "m2"]),
])
code = Codebase()
code.scanFiles(get_files(), twopass=True, multithread=True)
AccessCheck(code).checkAccess(multithread=True)
return not workspace.errors
if __name__ == "__main__":
# When using multithreaded processing, it's criticak to check __name__
# rather than doing things directly in the global scope.
sys.exit(main())
```
## Links
* This project on GitHub: [layercparse](https://github.com/ershov/layercparse).
Raw data
{
"_id": null,
"home_page": "https://github.com/ershov/layercparse",
"name": "layercparse",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Yury Ershov",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# \"Fat Token\" Parser and Modularity Checker for C Code\n\nThis library provides a parser for C code that extracts tokens and checks the modularity of the code based on the rules defined in the [Modularity](MODULARITY.md) document.\n\nThe core concept of the library is the \"Fat Token\" which represents a higher-level abstraction of elements in C code rather than a simple sequence of characters. A Fat Token can represent items such as \"words\", \"comments\", \"strings\", \"expressions within parentheses\", etc.\n\nWith this this higher-level representation, the library enables a layered approach to parsing, focusing on high level structures without getting bogged down in unnecessary details. The parser extracts Fat Tokens from the C code and groups them into \"statements\" representing logical units like function definitions, struct declarations, variable declarations, etc.\n\nFor example, a function definition might be represented as a sequence of: \"comment\", \"word\", \"word\", \"expression in parentheses\", and \"expression in curly braces\". We can recognize this as a function definition without needing to parse the internal details of the function body or arguments list.\n\nThis approach makes it possible to perform a variety of analyses on C code or other languages with clear syntax rules.\n\nImplementation details are described in the [Implementation](IMPL.md) document.\n\n## Installation\n\n```bash\n$ pip install layercparse\n```\n\n## Usage\n\n```python\nimport os\nfrom layercparse import *\n\ndef main():\n # setLogLevel(LogLevel.WARNING)\n setRootPath(os.path.realpath(sys.argv[1]))\n setModules([\n Module(\"module1\"),\n Module(\"module2\", fileAliases=[\"m2\"], sourceAliases = [\"mod2\", \"m2\"]),\n ])\n\n code = Codebase()\n code.scanFiles(get_files(), twopass=True, multithread=True)\n AccessCheck(code).checkAccess(multithread=True)\n\n return not workspace.errors\n\nif __name__ == \"__main__\":\n # When using multithreaded processing, it's criticak to check __name__\n # rather than doing things directly in the global scope.\n sys.exit(main())\n```\n\n## Links\n\n* This project on GitHub: [layercparse](https://github.com/ershov/layercparse).\n",
"bugtrack_url": null,
"license": "GPL3",
"summary": "Layered C parser",
"version": "0.4.3",
"project_urls": {
"Homepage": "https://github.com/ershov/layercparse"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b2304f1bb96eb844232bcf5ecde9dd5eb8ca62364290257d2536888967f68f50",
"md5": "45cc19a583981321034a35e45b5a4e09",
"sha256": "0bfdd3994e84cbecfa29e5d865e1c1fedae0d233ddb9b62173d4a9eab0b3cffb"
},
"downloads": -1,
"filename": "layercparse-0.4.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "45cc19a583981321034a35e45b5a4e09",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 57517,
"upload_time": "2024-12-01T22:29:24",
"upload_time_iso_8601": "2024-12-01T22:29:24.898581Z",
"url": "https://files.pythonhosted.org/packages/b2/30/4f1bb96eb844232bcf5ecde9dd5eb8ca62364290257d2536888967f68f50/layercparse-0.4.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-01 22:29:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ershov",
"github_project": "layercparse",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "layercparse"
}
JSON
