| Name | pygo-tools JSON |
| Version |
0.1.8
JSON |
| download |
| home_page | None |
| Summary | Simplify Python-Go integration for Libraries with Precompiled Extensions |
| upload_time | 2024-04-09 20:49:06 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | <4.0,>=3.10 |
| license | MIT NON-AI License |
| keywords |
python
golang
cffi
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
## Python Golang Tools
This project aims to bridge the gap between Python and Golang, enabling developers to easily create high performance Python
libraries with precompiled Go extensions. `pygo-tools` wraps the `setup` function from `setuptools` and handles the process
of compiling your library's golang source before building the wheel file. `pygo-tools` also patches the resulting extension,
removing the need to manually configure `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` before runtime.
### Installation
```shell
pip install pygo-tools
```
### Example Projects
- [hello world with `setup.py` and `config.json`][hello-v1]
- [hello world with `pyproject.toml`][hello-v2]
### Walkthrough for Hello World with `setup.py` and `config.json`
#### Project Structure
```text
|-- Makefile
|-- config.json
|-- example
| |-- __init__.py
| |-- go
| | |-- Makefile
| | `-- main.go
| |-- lib -> go/local
| `-- wrapper.py
|-- MANIFEST.in
|-- setup.py
`-- test.py
```
#### Project Config
`pygo-tools` looks for `config.json` at the root of your project. The config contains metadata like the python package name,
golang library, python extension, and `C` functon signatures for the underlying library.
```json
{
"package": "example",
"extension": "_example",
"library": "hello",
"signatures": [
"char* Hello(char* message, int count);"
]
}
```
#### Go
```go
// example/go/main.go
package main
import "C"
import "fmt"
func hello(message string, count int) string {
return fmt.Sprintf("hello %s %d", message, count)
}
//export Hello
func Hello(message *C.char, count C.int) *C.char {
result := hello(C.GoString(message), int(count))
return C.CString(result)
}
func main() {}
```
```makefile
# example/go/Makefile
all: clean build
build:
go build -buildmode=c-shared -o local/libhello.so main.go
clean:
@rm -rf local
```
#### Python
```python
# example/wrapper.py
from _example import ffi, lib
from dataclasses import dataclass
@dataclass
class ExtensionAdapter:
@staticmethod
def hello(message: str, count: int = 1) -> str:
params = ffi.new('char[]', message.encode()), ffi.cast('int', count)
result = lib.Hello(*params)
return ffi.string(result).decode()
```
```python
# example/__init__.py
from .wrapper import ExtensionAdapter
```
```python
# test.py
from example import ExtensionAdapter
adapter = ExtensionAdapter()
print(adapter.hello(message='world', count=4))
```
```python
# setup.py
from pygo_tools import setup
from setuptools import find_packages
setup(
name='example',
version='0.1.0',
packages=find_packages(),
include_package_data=True
)
```
```text
# MANIFEST.in
include example/go/*.go
include example/lib/*.so
```
```makefile
all: clean build post_build
build:
python -m build -n --wheel
unzip -l dist/*.whl
clean:
@rm -rf build dist *.egg-info
@rm -rf */go/local
post_build:
@rm -rf build *.egg-info
```
#### Build
The `Makefile` at the project level runs `python -m build -n --wheel` in order to create the wheel file under the `dist` folder.
The binary distribution should include your python and golang source code, along with the compiled library and extension.
```text
- example/lib/libhello.so
- _example.abi3.so
```
#### Testing
Install the wheel file in your virtual environment and run `python test.py` to check that things are working end to end locally.
#### Distributing
Binary distributions with compiled code should be built for each platform that you want to support during runtime. [`cibuildwheel`]
can be used for this. However, you can also create a `Dockerfile` and `docker-compose.yaml` to build wheels for `linux/arm64`
and `linux/amd64`:
```dockerfile
# Dockerfile
ARG PYTHON_VERSION=3.12
FROM public.ecr.aws/sam/build-python${PYTHON_VERSION}
RUN dnf install -y golang make
RUN pip install -U pip setuptools
RUN pip install pygo-tools
COPY example example/
COPY setup.py config.json MANIFEST.in ./
RUN python -m build -n --wheel
ENTRYPOINT ["/bin/sh"]
```
```yaml
# docker-compose.yaml
version: '3'
services:
builder:
build: .
platform: linux/arm64
volumes:
- ./out:/var/task/out:rw
entrypoint: ["/bin/sh", "-c"]
command: ["cp dist/* out/"]
```
### Related Links
- [cffi]
- [cibuildwheel]
[cffi]: https://cffi.readthedocs.io
[cibuildwheel]: https://cibuildwheel.readthedocs.io
[hello-v1]: https://github.com/rkhullar/python-libraries/tree/main/examples/pygo-tools/hello-v1
[hello-v2]: https://github.com/rkhullar/python-libraries/tree/main/examples/pygo-tools/hello-v2
Raw data
{
"_id": null,
"home_page": null,
"name": "pygo-tools",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "python, golang, cffi",
"author": null,
"author_email": "Rajan Khullar <rkhullar03@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/4d/27/7ee0ca64577b9fe579261ed1e52b9a805c9767a7f590786a1c9887535386/pygo-tools-0.1.8.tar.gz",
"platform": null,
"description": "## Python Golang Tools\n\nThis project aims to bridge the gap between Python and Golang, enabling developers to easily create high performance Python\nlibraries with precompiled Go extensions. `pygo-tools` wraps the `setup` function from `setuptools` and handles the process\nof compiling your library's golang source before building the wheel file. `pygo-tools` also patches the resulting extension,\nremoving the need to manually configure `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` before runtime.\n\n### Installation\n```shell\npip install pygo-tools\n```\n\n### Example Projects\n- [hello world with `setup.py` and `config.json`][hello-v1]\n- [hello world with `pyproject.toml`][hello-v2]\n\n### Walkthrough for Hello World with `setup.py` and `config.json`\n#### Project Structure\n```text\n|-- Makefile\n|-- config.json\n|-- example\n| |-- __init__.py\n| |-- go\n| | |-- Makefile\n| | `-- main.go\n| |-- lib -> go/local\n| `-- wrapper.py\n|-- MANIFEST.in\n|-- setup.py\n`-- test.py\n```\n\n#### Project Config\n`pygo-tools` looks for `config.json` at the root of your project. The config contains metadata like the python package name,\ngolang library, python extension, and `C` functon signatures for the underlying library.\n```json\n{\n \"package\": \"example\",\n \"extension\": \"_example\",\n \"library\": \"hello\",\n \"signatures\": [\n \"char* Hello(char* message, int count);\"\n ]\n}\n```\n\n#### Go\n```go\n// example/go/main.go\n\npackage main\n\nimport \"C\"\nimport \"fmt\"\n\nfunc hello(message string, count int) string {\n\treturn fmt.Sprintf(\"hello %s %d\", message, count)\n}\n\n//export Hello\nfunc Hello(message *C.char, count C.int) *C.char {\n\tresult := hello(C.GoString(message), int(count))\n\treturn C.CString(result)\n}\n\nfunc main() {}\n```\n\n```makefile\n# example/go/Makefile\n\nall: clean build\n\nbuild:\n\tgo build -buildmode=c-shared -o local/libhello.so main.go\n\nclean:\n\t@rm -rf local\n```\n\n#### Python\n```python\n# example/wrapper.py\n\nfrom _example import ffi, lib\nfrom dataclasses import dataclass\n\n\n@dataclass\nclass ExtensionAdapter:\n\n @staticmethod\n def hello(message: str, count: int = 1) -> str:\n params = ffi.new('char[]', message.encode()), ffi.cast('int', count)\n result = lib.Hello(*params)\n return ffi.string(result).decode()\n```\n\n```python\n# example/__init__.py\nfrom .wrapper import ExtensionAdapter\n```\n\n```python\n# test.py\n\nfrom example import ExtensionAdapter\n\nadapter = ExtensionAdapter()\nprint(adapter.hello(message='world', count=4))\n```\n\n```python\n# setup.py\n\nfrom pygo_tools import setup\nfrom setuptools import find_packages\n\nsetup(\n name='example',\n version='0.1.0',\n packages=find_packages(),\n include_package_data=True\n)\n```\n\n```text\n# MANIFEST.in\ninclude example/go/*.go\ninclude example/lib/*.so\n```\n\n```makefile\nall: clean build post_build\n\nbuild:\n\tpython -m build -n --wheel\n\tunzip -l dist/*.whl\n\nclean:\n\t@rm -rf build dist *.egg-info\n\t@rm -rf */go/local\n\npost_build:\n\t@rm -rf build *.egg-info\n```\n\n#### Build\nThe `Makefile` at the project level runs `python -m build -n --wheel` in order to create the wheel file under the `dist` folder.\nThe binary distribution should include your python and golang source code, along with the compiled library and extension.\n```text\n- example/lib/libhello.so\n- _example.abi3.so\n```\n\n#### Testing\nInstall the wheel file in your virtual environment and run `python test.py` to check that things are working end to end locally.\n\n#### Distributing\nBinary distributions with compiled code should be built for each platform that you want to support during runtime. [`cibuildwheel`]\ncan be used for this. However, you can also create a `Dockerfile` and `docker-compose.yaml` to build wheels for `linux/arm64`\nand `linux/amd64`:\n\n```dockerfile\n# Dockerfile\nARG PYTHON_VERSION=3.12\nFROM public.ecr.aws/sam/build-python${PYTHON_VERSION}\nRUN dnf install -y golang make\nRUN pip install -U pip setuptools\nRUN pip install pygo-tools\nCOPY example example/\nCOPY setup.py config.json MANIFEST.in ./\nRUN python -m build -n --wheel\nENTRYPOINT [\"/bin/sh\"]\n```\n\n```yaml\n# docker-compose.yaml\nversion: '3'\nservices:\n builder:\n build: .\n platform: linux/arm64\n volumes:\n - ./out:/var/task/out:rw\n entrypoint: [\"/bin/sh\", \"-c\"]\n command: [\"cp dist/* out/\"]\n```\n\n### Related Links\n- [cffi]\n- [cibuildwheel]\n\n[cffi]: https://cffi.readthedocs.io\n[cibuildwheel]: https://cibuildwheel.readthedocs.io\n[hello-v1]: https://github.com/rkhullar/python-libraries/tree/main/examples/pygo-tools/hello-v1\n[hello-v2]: https://github.com/rkhullar/python-libraries/tree/main/examples/pygo-tools/hello-v2\n",
"bugtrack_url": null,
"license": "MIT NON-AI License",
"summary": "Simplify Python-Go integration for Libraries with Precompiled Extensions",
"version": "0.1.8",
"project_urls": {
"Repository": "https://github.com/rkhullar/python-libraries/tree/main/pygo-tools"
},
"split_keywords": [
"python",
" golang",
" cffi"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "54eca68df2df8b8cc02ce60d4573424bdf93dd0590ae5cad85dd6d4a4fe33b8e",
"md5": "7db2d1ac1562031ac33c47fa336f0c7f",
"sha256": "688bc2b95b8b107c8cfac5f10906f8769349f18e75dffaf746cf79b5e3cea968"
},
"downloads": -1,
"filename": "pygo_tools-0.1.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7db2d1ac1562031ac33c47fa336f0c7f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 8432,
"upload_time": "2024-04-09T20:49:05",
"upload_time_iso_8601": "2024-04-09T20:49:05.249861Z",
"url": "https://files.pythonhosted.org/packages/54/ec/a68df2df8b8cc02ce60d4573424bdf93dd0590ae5cad85dd6d4a4fe33b8e/pygo_tools-0.1.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4d277ee0ca64577b9fe579261ed1e52b9a805c9767a7f590786a1c9887535386",
"md5": "d825b7df220c048257fe11d6582ea98f",
"sha256": "4707aaa145f4c1c60a683dba36e62b5999d761a4f9c0976d5a9899f0b8765b52"
},
"downloads": -1,
"filename": "pygo-tools-0.1.8.tar.gz",
"has_sig": false,
"md5_digest": "d825b7df220c048257fe11d6582ea98f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 8482,
"upload_time": "2024-04-09T20:49:06",
"upload_time_iso_8601": "2024-04-09T20:49:06.855148Z",
"url": "https://files.pythonhosted.org/packages/4d/27/7ee0ca64577b9fe579261ed1e52b9a805c9767a7f590786a1c9887535386/pygo-tools-0.1.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-09 20:49:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rkhullar",
"github_project": "python-libraries",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pygo-tools"
}
