# cmake-build-extension
[][pypi]
[][pypi]
[][pypi]
[][pypi]
[][pypi]
[][ci_cd]
[pypi]: https://pypi.org/project/cmake-build-extension/
[ci_cd]: https://github.com/diegoferigo/cmake-build-extension/actions/workflows/python.yml
**Setuptools extension to build and package CMake projects.**
This project aims to simplify the integration of C++ projects based on CMake with Python packaging tools.
CMake provides out-of-the-box support to either [SWIG][swig] and [pybind11][pybind11],
that are two among the most used projects to create Python bindings from C++ sources.
[swig]: https://github.com/swig/swig
[pybind11]: https://github.com/pybind/pybind11
If you have any experience with these hybrid projects, you know the challenges to make packaging right!
This project takes inspiration from pre-existing examples
([`pybind/cmake_example`][pybind11_example], among many others)
and provides a simple, flexible, and reusable setuptools extension with the following features:
- Bridge between CMake projects and Python packaging.
- Configure and build the CMake project from `setup.py`.
- Install the CMake project in the resulting Python package.
- Allow passing custom CMake options.
- Allow creating a top-level `__init__.py`.
- Expose C++ executables to the Python environment.
- Provide a context manager to import reliably CPython modules on all major OSs.
- Disable the C++ extension in editable installations (requiring to manually call CMake to install the C++ project).
[pybind11_example]: https://github.com/pybind/cmake_example
Have a look to the [example](example/) for an overview of what can be done with this extension.
It shows how to create SWIG and pybind11 bindings for a project composed by a small C++ library with NumPy support
and an executable.
### Advanced features
1. This extension supports creating packages [PEP517] and [PEP518] compliant ([more details][pyproject]).
2. If the CMake project [exports the targets][export_targets], downstream projects can:
1. Extend their `CMAKE_MODULE_PATH` with the root of your installed Python package,
that could be obtained with:
```bash
python -c "import <pkg>, pathlib; print(pathlib.Path(<pkg>.__file__).parent)"
```
and consume the exported CMake targets.
1. Use `cmake-build-extension` with the `cmake_depends_on` option and link against the exported CMake targets
during the downstream packaging.
Note that the second feature allows distributing C++ dependencies through PyPI.
The resulting package structure is similar to other projects like [pybind11][pybind11_pypi] and [CasADi][casadi_pypi].
Be aware that ensuring ABI compatibility could be problematic in edge cases,
and the usage of a proper [compatible release pattern][pep440] (`~=`) could be necessary.
[pep440]: https://www.python.org/dev/peps/pep-0440/#compatible-release
[pep517]: https://www.python.org/dev/peps/pep-0517/
[pep518]: https://www.python.org/dev/peps/pep-0518/
[pyproject]: https://snarky.ca/what-the-heck-is-pyproject-toml/
[export_targets]: https://cmake.org/cmake/help/git-stage/guide/importing-exporting/index.html
[pybind11_pypi]: https://pypi.org/project/pybind11/
[casadi_pypi]: https://pypi.org/project/casadi/
## Installation
From PyPI:
```bash
pip install cmake-build-extension
```
From the repository:
```bash
pip install git+https://github.com/diegoferigo/cmake-build-extension
```
## Usage
Once both CMake project and `setup.py|setup.cfg|pyproject.toml` of your hybrid package are correctly configured
to use the resources provided by cmake-build-extension, the following commands can be used:
```bash
# ============
# Create sdist
# ============
# Calling setup.py
python setup.py sdist
# Using pypa/build
python -m build --sdist
# ============
# Create wheel
# ============
# Calling setup.py
python setup.py bdist_wheel
# Using pip
pip wheel -w dist/ .
# Using pypa/build
python -m build --wheel
# ==========================================================
# Create wheel or install package passing additional options
# ==========================================================
# Calling setup.py
python setup.py {bdist_wheel|install} build_ext -D"BAR=Foo;VAR=TRUE"
# Using pip
pip {wheel|install} --global-option="build_ext" --global-option="-DBAR=Foo;VAR=TRUE" .
# Using pypa/build (only wheel creation)
python -m build --wheel "-C--global-option=build_ext" "-C--global-option=-DBAR=Foo;VAR=TRUE"
```
## Caveats
### `manylinux*` support
This extension, beyond packaging the hybrid C++ / Python project,
also allows the inclusion of the exported CMake targets in the resulting wheel.
This result depends on how the CMake project is configured,
and whether the [exported targets][exp_imp_wiki] are installed together with the other files.
[exp_imp_wiki]: https://gitlab.kitware.com/cmake/community/-/wikis/doc/tutorials/Exporting-and-Importing-Targets
Such hybrid packages can then be uploaded to PyPI.
Though, on GNU/Linux, the generated wheel is not compliant by default with any [`manylinux*`][manylinux] standard.
Tools such [auditwheel][auditwheel] exist to fix these wheels, but they require running on selected distributions.
Luckily, other projects like [cibuildwheel][cibuildwheel] greatly simplify the process in CI.
[manylinux]: https://github.com/pypa/manylinux
[auditwheel]: https://github.com/pypa/auditwheel
[cibuildwheel]: https://github.com/joerick/cibuildwheel
This being said, `manylinux*` guidelines could still work against you.
In fact, wheels supporting `manylinux2010|manylinux2014` are built [with gcc4][pep599_manylinux2014]
that does not support the new C++11 ABIs.
In few words, this means that the exported libraries bundled in the wheel cannot
be imported in a downstream project using relatively new C++ standards!
For more details visit [robotology/idyntree#776](https://github.com/robotology/idyntree/issues/776).
[pep599_manylinux2014]: https://www.python.org/dev/peps/pep-0599/#the-manylinux2014-policy
Luckily, the situation changed thanks to the finalization of [PEP600][pep600], i.e. `manylinuxX_YY` :tada:
If you build a PEP600 compliant wheel (nowadays compatible with most of the commonly used distributions),
your exported CMake project bundled in the wheel can be successfully imported downstream.
If you want to support this use case, make sure to produce and distribute wheels compliant with PEP600.
[pep600]: https://www.python.org/dev/peps/pep-0600/
### Loading CPython modules in Windows
Python 3.8 [changed][changelog_3_8] how DLL are resolved.
By default, modules that could be imported in Python 3.7 stopped working, and using the new
[`os.add_dll_directory`][add_dll_directory] is now necessary.
In order to streamline the process, `cmake-build-extension` implements a context manager that can be used
to import reliably the bindings module:
```python
import cmake_build_extension
with cmake_build_extension.build_extension_env():
from . import bindings
```
It will take care to temporarily fix the search path.
For more details, refer to [#8][windows_import_issue] and [#12][windows_import_pr].
[changelog_3_8]: https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew
[add_dll_directory]: https://docs.python.org/3/library/os.html#os.add_dll_directory
[windows_import_issue]: https://github.com/diegoferigo/cmake-build-extension/issues/8
[windows_import_pr]: https://github.com/diegoferigo/cmake-build-extension/pull/12
### `setup.py|setup.cfg|pyproject.toml` files in subfolder
Sometimes hybrid projects are C++ centric, and keeping these files in the top-level folder is not desirable.
In this setup, however, problems occur if the main `CMakeLists.txt` is kept in the top-level folder
(see [pypa/build#322][sdist_issue]).
To solve this problem, `cmake-build-extension` provides custom commands to create source distribution.
You can use one of the following custom `sdist` options in `setup.py`:
```python
setuptools.setup(
cmdclass=dict(
# [...]
# Pack the whole git folder:
sdist=cmake_build_extension.GitSdistFolder,
# Pack only the git tree:
sdist=cmake_build_extension.GitSdistTree,
# Or, inherit from cmake_build_extension.sdist_command.GitSdistABC and
# make your own custom sdist including only the files you are interested
),
)
```
[sdist_issue]: https://github.com/pypa/build/issues/322
## Downstream projects
If the provided example is not enough complex, find here below a list of projects using `cmake-build-extension`:
- [`robotology/idyntree`](https://github.com/robotology/idyntree/)
- [`robotology/yarp`](https://github.com/robotology/yarp/)
- [`robotology/ycm`](https://github.com/robotology/ycm/)
- [`diegoferigo/gazebo-yarp-synchronizer`](https://github.com/diegoferigo/gazebo-yarp-synchronizer)
- [`robotology/gym-ignition@scenario`](https://github.com/robotology/gym-ignition/tree/devel/scenario)
- [`dic-iit/gazebo-scenario-plugins`](https://github.com/dic-iit/gazebo-scenario-plugins/)
- [`dic-iit/bipedal-locomotion-framework`](https://github.com/dic-iit/bipedal-locomotion-framework)
- [`artivis/manif`](https://github.com/artivis/manif)
## Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
## License
[MIT](https://choosealicense.com/licenses/mit/)
Raw data
{
"_id": null,
"home_page": "https://github.com/claudioperez/amoeba-build",
"name": "amoeba-build",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "setuptools,extension,cmake,build,package,pypi",
"author": "Diego Ferigo",
"author_email": "dgferigo@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/19/17/c5cea468b888742e1b786d47790bcb1853675ea2ee4f7cf776212d3a5c12/amoeba-build-0.0.9.tar.gz",
"platform": "any",
"description": "# cmake-build-extension\n\n[][pypi]\n[][pypi]\n[][pypi]\n[][pypi]\n[][pypi]\n[][ci_cd]\n\n[pypi]: https://pypi.org/project/cmake-build-extension/\n[ci_cd]: https://github.com/diegoferigo/cmake-build-extension/actions/workflows/python.yml\n\n**Setuptools extension to build and package CMake projects.**\n\nThis project aims to simplify the integration of C++ projects based on CMake with Python packaging tools.\nCMake provides out-of-the-box support to either [SWIG][swig] and [pybind11][pybind11],\nthat are two among the most used projects to create Python bindings from C++ sources.\n\n[swig]: https://github.com/swig/swig\n[pybind11]: https://github.com/pybind/pybind11\n\nIf you have any experience with these hybrid projects, you know the challenges to make packaging right!\nThis project takes inspiration from pre-existing examples \n([`pybind/cmake_example`][pybind11_example], among many others)\nand provides a simple, flexible, and reusable setuptools extension with the following features:\n\n- Bridge between CMake projects and Python packaging.\n- Configure and build the CMake project from `setup.py`. \n- Install the CMake project in the resulting Python package.\n- Allow passing custom CMake options.\n- Allow creating a top-level `__init__.py`.\n- Expose C++ executables to the Python environment.\n- Provide a context manager to import reliably CPython modules on all major OSs.\n- Disable the C++ extension in editable installations (requiring to manually call CMake to install the C++ project).\n\n[pybind11_example]: https://github.com/pybind/cmake_example\n\nHave a look to the [example](example/) for an overview of what can be done with this extension.\nIt shows how to create SWIG and pybind11 bindings for a project composed by a small C++ library with NumPy support \nand an executable. \n\n### Advanced features\n\n1. This extension supports creating packages [PEP517] and [PEP518] compliant ([more details][pyproject]).\n2. If the CMake project [exports the targets][export_targets], downstream projects can:\n 1. Extend their `CMAKE_MODULE_PATH` with the root of your installed Python package,\n that could be obtained with:\n ```bash\n python -c \"import <pkg>, pathlib; print(pathlib.Path(<pkg>.__file__).parent)\"\n ```\n and consume the exported CMake targets.\n 1. Use `cmake-build-extension` with the `cmake_depends_on` option and link against the exported CMake targets\n during the downstream packaging.\n\nNote that the second feature allows distributing C++ dependencies through PyPI.\nThe resulting package structure is similar to other projects like [pybind11][pybind11_pypi] and [CasADi][casadi_pypi].\nBe aware that ensuring ABI compatibility could be problematic in edge cases, \nand the usage of a proper [compatible release pattern][pep440] (`~=`) could be necessary.\n\n[pep440]: https://www.python.org/dev/peps/pep-0440/#compatible-release\n[pep517]: https://www.python.org/dev/peps/pep-0517/\n[pep518]: https://www.python.org/dev/peps/pep-0518/\n\n[pyproject]: https://snarky.ca/what-the-heck-is-pyproject-toml/\n[export_targets]: https://cmake.org/cmake/help/git-stage/guide/importing-exporting/index.html\n\n[pybind11_pypi]: https://pypi.org/project/pybind11/\n[casadi_pypi]: https://pypi.org/project/casadi/\n\n## Installation\n\nFrom PyPI:\n\n```bash\npip install cmake-build-extension\n```\n\nFrom the repository:\n\n```bash\npip install git+https://github.com/diegoferigo/cmake-build-extension\n```\n\n## Usage\n\nOnce both CMake project and `setup.py|setup.cfg|pyproject.toml` of your hybrid package are correctly configured\nto use the resources provided by cmake-build-extension, the following commands can be used:\n\n```bash\n# ============\n# Create sdist\n# ============\n\n# Calling setup.py\npython setup.py sdist\n\n# Using pypa/build\npython -m build --sdist\n\n# ============\n# Create wheel\n# ============\n\n# Calling setup.py\npython setup.py bdist_wheel\n\n# Using pip\npip wheel -w dist/ .\n\n# Using pypa/build\npython -m build --wheel\n\n# ==========================================================\n# Create wheel or install package passing additional options\n# ==========================================================\n\n# Calling setup.py\npython setup.py {bdist_wheel|install} build_ext -D\"BAR=Foo;VAR=TRUE\"\n\n# Using pip\npip {wheel|install} --global-option=\"build_ext\" --global-option=\"-DBAR=Foo;VAR=TRUE\" .\n\n# Using pypa/build (only wheel creation)\npython -m build --wheel \"-C--global-option=build_ext\" \"-C--global-option=-DBAR=Foo;VAR=TRUE\"\n```\n\n## Caveats\n\n### `manylinux*` support\n\nThis extension, beyond packaging the hybrid C++ / Python project, \nalso allows the inclusion of the exported CMake targets in the resulting wheel.\nThis result depends on how the CMake project is configured, \nand whether the [exported targets][exp_imp_wiki] are installed together with the other files.\n\n[exp_imp_wiki]: https://gitlab.kitware.com/cmake/community/-/wikis/doc/tutorials/Exporting-and-Importing-Targets\n\nSuch hybrid packages can then be uploaded to PyPI. \nThough, on GNU/Linux, the generated wheel is not compliant by default with any [`manylinux*`][manylinux] standard.\nTools such [auditwheel][auditwheel] exist to fix these wheels, but they require running on selected distributions.\nLuckily, other projects like [cibuildwheel][cibuildwheel] greatly simplify the process in CI.\n\n[manylinux]: https://github.com/pypa/manylinux\n[auditwheel]: https://github.com/pypa/auditwheel\n[cibuildwheel]: https://github.com/joerick/cibuildwheel\n\nThis being said, `manylinux*` guidelines could still work against you.\nIn fact, wheels supporting `manylinux2010|manylinux2014` are built [with gcc4][pep599_manylinux2014] \nthat does not support the new C++11 ABIs.\nIn few words, this means that the exported libraries bundled in the wheel cannot \nbe imported in a downstream project using relatively new C++ standards!\nFor more details visit [robotology/idyntree#776](https://github.com/robotology/idyntree/issues/776). \n\n[pep599_manylinux2014]: https://www.python.org/dev/peps/pep-0599/#the-manylinux2014-policy\n\nLuckily, the situation changed thanks to the finalization of [PEP600][pep600], i.e. `manylinuxX_YY` :tada: \nIf you build a PEP600 compliant wheel (nowadays compatible with most of the commonly used distributions), \nyour exported CMake project bundled in the wheel can be successfully imported downstream.\nIf you want to support this use case, make sure to produce and distribute wheels compliant with PEP600.\n\n[pep600]: https://www.python.org/dev/peps/pep-0600/\n\n### Loading CPython modules in Windows\n\nPython 3.8 [changed][changelog_3_8] how DLL are resolved.\nBy default, modules that could be imported in Python 3.7 stopped working, and using the new \n[`os.add_dll_directory`][add_dll_directory] is now necessary.\n\nIn order to streamline the process, `cmake-build-extension` implements a context manager that can be used \nto import reliably the bindings module:\n\n```python\nimport cmake_build_extension\n\nwith cmake_build_extension.build_extension_env():\n from . import bindings\n```\n\nIt will take care to temporarily fix the search path.\n\nFor more details, refer to [#8][windows_import_issue] and [#12][windows_import_pr].\n\n[changelog_3_8]: https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew\n[add_dll_directory]: https://docs.python.org/3/library/os.html#os.add_dll_directory\n[windows_import_issue]: https://github.com/diegoferigo/cmake-build-extension/issues/8\n[windows_import_pr]: https://github.com/diegoferigo/cmake-build-extension/pull/12\n\n### `setup.py|setup.cfg|pyproject.toml` files in subfolder\n\nSometimes hybrid projects are C++ centric, and keeping these files in the top-level folder is not desirable.\nIn this setup, however, problems occur if the main `CMakeLists.txt` is kept in the top-level folder \n(see [pypa/build#322][sdist_issue]).\nTo solve this problem, `cmake-build-extension` provides custom commands to create source distribution.\nYou can use one of the following custom `sdist` options in `setup.py`:\n\n```python\nsetuptools.setup(\n cmdclass=dict(\n # [...]\n # Pack the whole git folder:\n sdist=cmake_build_extension.GitSdistFolder,\n # Pack only the git tree:\n sdist=cmake_build_extension.GitSdistTree,\n # Or, inherit from cmake_build_extension.sdist_command.GitSdistABC and\n # make your own custom sdist including only the files you are interested\n ),\n)\n```\n\n[sdist_issue]: https://github.com/pypa/build/issues/322\n\n## Downstream projects\n\nIf the provided example is not enough complex, find here below a list of projects using `cmake-build-extension`: \n\n- [`robotology/idyntree`](https://github.com/robotology/idyntree/)\n- [`robotology/yarp`](https://github.com/robotology/yarp/)\n- [`robotology/ycm`](https://github.com/robotology/ycm/)\n- [`diegoferigo/gazebo-yarp-synchronizer`](https://github.com/diegoferigo/gazebo-yarp-synchronizer)\n- [`robotology/gym-ignition@scenario`](https://github.com/robotology/gym-ignition/tree/devel/scenario)\n- [`dic-iit/gazebo-scenario-plugins`](https://github.com/dic-iit/gazebo-scenario-plugins/)\n- [`dic-iit/bipedal-locomotion-framework`](https://github.com/dic-iit/bipedal-locomotion-framework)\n- [`artivis/manif`](https://github.com/artivis/manif)\n\n## Contributing\n\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Setuptools extension to build and package CMake projects.",
"version": "0.0.9",
"split_keywords": [
"setuptools",
"extension",
"cmake",
"build",
"package",
"pypi"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "e81e18c114623ba1eec4490e719da779",
"sha256": "360754cfc96dfc550774db51276183ef51ab491ec64a73e6c08d9b95521c3968"
},
"downloads": -1,
"filename": "amoeba-build-0.0.9.tar.gz",
"has_sig": false,
"md5_digest": "e81e18c114623ba1eec4490e719da779",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 38069,
"upload_time": "2022-08-19T18:21:01",
"upload_time_iso_8601": "2022-08-19T18:21:01.762515Z",
"url": "https://files.pythonhosted.org/packages/19/17/c5cea468b888742e1b786d47790bcb1853675ea2ee4f7cf776212d3a5c12/amoeba-build-0.0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-08-19 18:21:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "claudioperez",
"github_project": "amoeba-build",
"lcname": "amoeba-build"
}
JSON
