| Name | pybind11-weaver JSON |
| Version |
0.1.2
JSON |
| download |
| home_page | |
| Summary | A fully customizable pybind11 generator to help generate code for exsiting c/c++ library. |
| upload_time | 2023-10-30 08:22:56 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | |
| license | MIT |
| keywords |
pybind11
generator
codegen
gen
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Pybind11 Weaver: Python Binding Code Generator
**Pybind11 Weaver** is a powerful code generator designed to automate the generation of pybind11 code from C++ header
files. It streamlines the process of creating Python bindings, enabling users to focus on writing critical pybind11 code
and offloading the tedious work to Pybind11 Weaver.
This tool takes a [sample.h](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/sample.h) file
and transforms it into
a [sample_binding.cc.inc](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/sample_binding.cc.inc)
file using [cfg.yaml](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/cfg.yaml) as a guide.
Following the binding with a single line `auto update_guard = DeclFn(m);`
in [binding.cc](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/binding.cc), all elements
from the header file become accessible in Python as demonstrated in
this [example](https://github.com/edimetia3d/pybind11_weaver/blob/main/test/sample_test/launch_module.py).
A more pragmatic example is available in [pylibclang](https://github.com/edimetia3d/pylibclang), a comprehensive Python
wrapper for libclang that uses Pybind11 Weaver to generate the binding code.
1. Its practicality stems from the fact that **Pybind11 Weaver operates on it** as well. Indeed, Pybind11 Weaver is
self-hosted and generates the binding code for its own use.
2. Approximately 30k lines of C++ code are generated from a mere 10 lines
of [cfg.yaml](https://github.com/edimetia3d/pylibclang/blob/master/c_src/cfg.yaml).
3. Some [binding code](https://github.com/edimetia3d/pylibclang/blob/master/c_src/binding.cc) is manually crafted to
handle special cases and integrates seamlessly with the generated code.
[pylibtooling](https://github.com/edimetia3d/pylibtooling) is a much more advanced example that uses Pybind11 Weaver to
generate the binding code for [libtooling](https://clang.llvm.org/docs/LibTooling.html), and will be used to demonstrate
the capabilities of Pybind11 Weaver when working with large C++ only libraries.
## Key Features
1. **Highly Customizable:** While the default configuration is super simple and suitable for most cases, it allows for
high customization.
2. **Ease of Use:** As a pure Python package, a simple `pip install` gets it ready to work.
3. **Versatility:** All generated code is under your control, you can easily modify/enhance/disable any part of
generated
code, and all generated code will work with your hand-written code seamlessly.
4. **Structure Preservation:** It retains the module structure of the original C++ code.
## Features & Roadmap
- [x] Binding for Enum
- [x] Binding for Namespace (as submodule)
- [x] Binding for Function, with support of function overloading
- [x] Binding for C style function pointer (usually used as callback functions)
- [x] Binding for opaque pointer and pointer to incomplete type
- [ ] Binding for Operator overloading
- [x] Binding for Class method, method overloading, static method, static method overloading, constructor, constructor
overloading, class field
- [x] Trampoline class for virtual function
- [x] Binding for concreate template instance, that includes: implicit(explicit) class(struct) template instantiation,
full class(struct) template specialization, extern function template instance declaration.
- [x] Support class inheritance hierarchy
- [x] Auto ignore symbols by : Linkage (e.g. `static`), visibility (e.g. `visibility=hidden`), member access
control (e.g. `private`, `protected`)
- [x] Docstring generation from c++ doxygen style comment
- [x] Namespace hierarchy to Python module hierarchy
- [x] Dynamic update/disable binding by API call.
- [x] Static update/disable binding by define macro (Mainly used to disable wrong binding code to avoid compilation
error)
- [ ] Auto snake case
## Background & Recommendations
This project originated from an internal project aimed at creating a Python binding for a **LARGE** developing C++
library. This posed significant challenges:
1. The C++ library interface contained a vast number of classes, functions, and enums. Creating bindings for all these
elements was not only **tedious** but also **error-prone**.
2. Because the C++ library was under active development, staying updated with daily additions and frequent code
modifications was a **maintenance challenge**.
3. Some aspects of the C++ library, due to historical reasons, were incompatible with Python conventions, necessitating
**hand-written binding codes**.
4. The sheer size of the library added to the complexity, making it difficult to develop a generator smart enough to
handle everything, hence the need for manual binding code writing.
In light of these challenges, I designed Pybind11 Weaver as a tool to generate the majority of the binding code,
leaving users to handcraft the remaining parts as needed. If this approach suits your needs, this tool will be a
valuable asset.
**Typical workflow:**
Though most features should work out of the box, the more your API looks like "C With Class", the higher chance Pybind11
Weaver will do all the work for you. If you use too many advanced C++ features, you may need to write some binding
code by yourself.
1. Create a `cfg.yaml` file, mainly to tell the generator which files to parse.
2. Use Pybind11 Weaver to generate files, like `pybind11-weaver --config cfg.yaml`.
3. Create a `binding.cc`, include the generated files, and call the binding code.
4. Disable some generated binding code by define some macro, if there is any compilation error.
5. Add some custom code to replace part of the generated code, or adding some new binding that generator had not
exported.
6. Compile all code into a pybind11 module.
7. Optionally, use [pybind11-stubgen](https://github.com/sizmailov/pybind11-stubgen) to generate `.pyi` stub files,
enhancing readability for both humans and MYPY in a static way.
8. Test the module in Python, find bugs, and go to step 5 to fix them.
Also, if you encountered too many problems, you are welcome to open an issue at github, or create a PR to fix it.
## Installation
### Via PYPI
```bash
python3 -m pip install pybind11-weaver
```
### From Source
* To install from source:
```bash
git clone https://github.com/edimetia3d/pybind11_weaver
python3 -m pip install $(pwd)/pybind11_weaver/
```
* To run from source (Editable/Develop Mode):
```bash
git clone https://github.com/edimetia3d/pybind11_weaver
python3 -m pip install -e $(pwd)/pybind11_weaver/ -v --config-settings editable_mode=compat
```
## How it works
The Pybind11 Weaver operates under the hood by utilizing [libclang](https://clang.llvm.org/), a library that parses C++
header files. This enables us to obtain all APIs from the header file, which are then used to generate the binding code
on your behalf.
Notably, only header files are required, as we need declarations, not definitions. However, to ensure accurate parsing
of the code, some compiler flags, especially for macros, are necessary.
The code generated is structured into a `struct`:
1. During the construction of the struct, it creates some Pybind11 objects, such as `pybind11::class_`
or `pybind11::enum_`.
2. When the `Update()` API is invoked, the Pybind11 object experiences an update.
The use of a struct permits us to:
* Separate the processes of object creation and updates, ensuring that Pybind11 consistently acknowledges all exported
classes, which aids in the generation of accurate documentation.
* Increase the readability of the generated code, making it simpler to debug.
* Simplify customization, as you can easily inherit the struct and override or reimplement necessary elements.
Raw data
{
"_id": null,
"home_page": "",
"name": "pybind11-weaver",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "pybind11,generator,codegen,gen",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/40/88/378138bfcb2efd7b752dfee2a8dec03bdb3a5cbe8f94c4d9f15b1c39ef6e/pybind11_weaver-0.1.2.tar.gz",
"platform": null,
"description": "# Pybind11 Weaver: Python Binding Code Generator\n\n**Pybind11 Weaver** is a powerful code generator designed to automate the generation of pybind11 code from C++ header\nfiles. It streamlines the process of creating Python bindings, enabling users to focus on writing critical pybind11 code\nand offloading the tedious work to Pybind11 Weaver.\n\nThis tool takes a [sample.h](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/sample.h) file\nand transforms it into\na [sample_binding.cc.inc](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/sample_binding.cc.inc)\nfile using [cfg.yaml](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/cfg.yaml) as a guide.\nFollowing the binding with a single line `auto update_guard = DeclFn(m);`\nin [binding.cc](https://github.com/edimetia3d/pybind11_weaver/blob/main/sample/all_feature/binding.cc), all elements\nfrom the header file become accessible in Python as demonstrated in\nthis [example](https://github.com/edimetia3d/pybind11_weaver/blob/main/test/sample_test/launch_module.py).\n\nA more pragmatic example is available in [pylibclang](https://github.com/edimetia3d/pylibclang), a comprehensive Python\nwrapper for libclang that uses Pybind11 Weaver to generate the binding code.\n\n1. Its practicality stems from the fact that **Pybind11 Weaver operates on it** as well. Indeed, Pybind11 Weaver is\n self-hosted and generates the binding code for its own use.\n2. Approximately 30k lines of C++ code are generated from a mere 10 lines\n of [cfg.yaml](https://github.com/edimetia3d/pylibclang/blob/master/c_src/cfg.yaml).\n3. Some [binding code](https://github.com/edimetia3d/pylibclang/blob/master/c_src/binding.cc) is manually crafted to\n handle special cases and integrates seamlessly with the generated code.\n\n[pylibtooling](https://github.com/edimetia3d/pylibtooling) is a much more advanced example that uses Pybind11 Weaver to\ngenerate the binding code for [libtooling](https://clang.llvm.org/docs/LibTooling.html), and will be used to demonstrate\nthe capabilities of Pybind11 Weaver when working with large C++ only libraries.\n\n## Key Features\n\n1. **Highly Customizable:** While the default configuration is super simple and suitable for most cases, it allows for\n high customization.\n2. **Ease of Use:** As a pure Python package, a simple `pip install` gets it ready to work.\n3. **Versatility:** All generated code is under your control, you can easily modify/enhance/disable any part of\n generated\n code, and all generated code will work with your hand-written code seamlessly.\n4. **Structure Preservation:** It retains the module structure of the original C++ code.\n\n## Features & Roadmap\n\n- [x] Binding for Enum\n- [x] Binding for Namespace (as submodule)\n- [x] Binding for Function, with support of function overloading\n- [x] Binding for C style function pointer (usually used as callback functions)\n- [x] Binding for opaque pointer and pointer to incomplete type\n- [ ] Binding for Operator overloading\n- [x] Binding for Class method, method overloading, static method, static method overloading, constructor, constructor\n overloading, class field\n- [x] Trampoline class for virtual function\n- [x] Binding for concreate template instance, that includes: implicit(explicit) class(struct) template instantiation,\n full class(struct) template specialization, extern function template instance declaration.\n- [x] Support class inheritance hierarchy\n- [x] Auto ignore symbols by : Linkage (e.g. `static`), visibility (e.g. `visibility=hidden`), member access\n control (e.g. `private`, `protected`)\n- [x] Docstring generation from c++ doxygen style comment\n- [x] Namespace hierarchy to Python module hierarchy\n- [x] Dynamic update/disable binding by API call.\n- [x] Static update/disable binding by define macro (Mainly used to disable wrong binding code to avoid compilation\n error)\n- [ ] Auto snake case\n\n## Background & Recommendations\n\nThis project originated from an internal project aimed at creating a Python binding for a **LARGE** developing C++\nlibrary. This posed significant challenges:\n\n1. The C++ library interface contained a vast number of classes, functions, and enums. Creating bindings for all these\n elements was not only **tedious** but also **error-prone**.\n2. Because the C++ library was under active development, staying updated with daily additions and frequent code\n modifications was a **maintenance challenge**.\n3. Some aspects of the C++ library, due to historical reasons, were incompatible with Python conventions, necessitating\n **hand-written binding codes**.\n4. The sheer size of the library added to the complexity, making it difficult to develop a generator smart enough to\n handle everything, hence the need for manual binding code writing.\n\nIn light of these challenges, I designed Pybind11 Weaver as a tool to generate the majority of the binding code,\nleaving users to handcraft the remaining parts as needed. If this approach suits your needs, this tool will be a\nvaluable asset.\n\n**Typical workflow:**\n\nThough most features should work out of the box, the more your API looks like \"C With Class\", the higher chance Pybind11\nWeaver will do all the work for you. If you use too many advanced C++ features, you may need to write some binding\ncode by yourself.\n\n1. Create a `cfg.yaml` file, mainly to tell the generator which files to parse.\n2. Use Pybind11 Weaver to generate files, like `pybind11-weaver --config cfg.yaml`.\n3. Create a `binding.cc`, include the generated files, and call the binding code.\n4. Disable some generated binding code by define some macro, if there is any compilation error.\n5. Add some custom code to replace part of the generated code, or adding some new binding that generator had not\n exported.\n6. Compile all code into a pybind11 module.\n7. Optionally, use [pybind11-stubgen](https://github.com/sizmailov/pybind11-stubgen) to generate `.pyi` stub files,\n enhancing readability for both humans and MYPY in a static way.\n8. Test the module in Python, find bugs, and go to step 5 to fix them.\n\nAlso, if you encountered too many problems, you are welcome to open an issue at github, or create a PR to fix it.\n\n## Installation\n\n### Via PYPI\n\n```bash\npython3 -m pip install pybind11-weaver\n```\n\n### From Source\n\n* To install from source:\n\n```bash\ngit clone https://github.com/edimetia3d/pybind11_weaver\npython3 -m pip install $(pwd)/pybind11_weaver/\n```\n\n* To run from source (Editable/Develop Mode):\n\n```bash\ngit clone https://github.com/edimetia3d/pybind11_weaver\npython3 -m pip install -e $(pwd)/pybind11_weaver/ -v --config-settings editable_mode=compat\n```\n\n## How it works\n\nThe Pybind11 Weaver operates under the hood by utilizing [libclang](https://clang.llvm.org/), a library that parses C++\nheader files. This enables us to obtain all APIs from the header file, which are then used to generate the binding code\non your behalf.\n\nNotably, only header files are required, as we need declarations, not definitions. However, to ensure accurate parsing\nof the code, some compiler flags, especially for macros, are necessary.\n\nThe code generated is structured into a `struct`:\n\n1. During the construction of the struct, it creates some Pybind11 objects, such as `pybind11::class_`\n or `pybind11::enum_`.\n2. When the `Update()` API is invoked, the Pybind11 object experiences an update.\n\nThe use of a struct permits us to:\n\n* Separate the processes of object creation and updates, ensuring that Pybind11 consistently acknowledges all exported\n classes, which aids in the generation of accurate documentation.\n* Increase the readability of the generated code, making it simpler to debug.\n* Simplify customization, as you can easily inherit the struct and override or reimplement necessary elements.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A fully customizable pybind11 generator to help generate code for exsiting c/c++ library.",
"version": "0.1.2",
"project_urls": {
"Changelog": "https://github.com/edimetia3d/pybind11_weaver/releases",
"Documentation": "https://github.com/edimetia3d/pybind11_weaver",
"Homepage": "https://github.com/edimetia3d/pybind11_weaver",
"Repository": "https://github.com/edimetia3d/pybind11_weaver"
},
"split_keywords": [
"pybind11",
"generator",
"codegen",
"gen"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "04ad07892293d1e2c2a5230a14fc4e9869f168a8849ff14f259130d5738a5123",
"md5": "074f6b0c7b1eac63e972bb26d8147f45",
"sha256": "ffdeeacfba896f0048e784daee92373e4cb2cf890720e73c37fb6d33bb92bc1c"
},
"downloads": -1,
"filename": "pybind11_weaver-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "074f6b0c7b1eac63e972bb26d8147f45",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 34376,
"upload_time": "2023-10-30T08:22:54",
"upload_time_iso_8601": "2023-10-30T08:22:54.487411Z",
"url": "https://files.pythonhosted.org/packages/04/ad/07892293d1e2c2a5230a14fc4e9869f168a8849ff14f259130d5738a5123/pybind11_weaver-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4088378138bfcb2efd7b752dfee2a8dec03bdb3a5cbe8f94c4d9f15b1c39ef6e",
"md5": "4267df88dbf02fe776a07093dff8bc65",
"sha256": "9e4e9b91066e507264f121be352bd0ba0bafc1b7f32c6bc04a528795154118ae"
},
"downloads": -1,
"filename": "pybind11_weaver-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "4267df88dbf02fe776a07093dff8bc65",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 45134,
"upload_time": "2023-10-30T08:22:56",
"upload_time_iso_8601": "2023-10-30T08:22:56.454952Z",
"url": "https://files.pythonhosted.org/packages/40/88/378138bfcb2efd7b752dfee2a8dec03bdb3a5cbe8f94c4d9f15b1c39ef6e/pybind11_weaver-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-30 08:22:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "edimetia3d",
"github_project": "pybind11_weaver",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "pybind11-weaver"
}
