# Flux Python Bindings
> 🐍️ You called me?
Hello! You've found the flux Python bindings, an experiment to build and deploy
Flux to Pypi without needing to store code alongside Flux. The goal of
this experiment is to test them separately. This was originally developed
at [vsoch/flux-python](https://github.com/vsoch/flux-python) and has
been ported here to automate more officially. The following guides might be useful
to you:
- ⭐️ [Flux Framework Documentation](https://flux-framework.readthedocs.io)
- ⭐️ [Flux Projects](https://flux-framework.org)
- ⭐️ [Tutorials](https://flux-framework.readthedocs.io/en/latest/tutorials/index.html)
- ⭐️ [Installing Flux Python on an LLNL System](https://github.com/flux-framework/flux-python/blob/main/LLNL.md)
## Helper Install
If you don't want to figure out your flux version and install with pip directory (per instructions in [manual install](#manual-install))
we provide a helper script [install-flux-python.sh](script/install-flux-python.sh) that will:
1. Determine that Flux and Python are installed with pip
2. Parse the version of Flux on your PATH
3. Derive the version and run the command for you!
You'll first want to make sure that flux is on your path, as is a `python3` executable with pip installed
as a module alongside it. And you can run it as follows:
```bash
wget https://raw.githubusercontent.com/flux-framework/flux-python/main/script/install-flux-python.sh
chmod +x ./install-flux-python.sh
./install-flux-python.sh
```
If a version is missing on pypi or you otherwise have a bug, please open an issue here.
## Manual Install
Since we need to link to Flux libraries, you are advised to install flux and flux-security in the same location
that will be discovered via the executable "flux," so typically `/usr` or `/usr/local`. You will need `wheel` installed:
```bash
$ pip install wheel
```
You can do either:
```bash
# Find the flux version on your system
flux --version
# Install the bindings that match that version
pip install flux-python==0.48.0
```
And if you are having trouble with the WCI cache, you can also wget the [file directly from pypi](https://pypi.org/project/flux-python/#history)
and install the .tar.gz directly.
```bash
wget https://files.pythonhosted.org/packages/25/fb/02951d80e44a19db291f0e7370d4e7d82c0c1b17709a37913881f958dff7/flux-python-0.48.0rc0.tar.gz
pip install flux-python-0.48.0rc0.tar.gz
```
If you install to a local (personal) location (e.g., `$HOME/.local`) you'll need to add this to your `PYTHONPATH`
```bash
# The directory "flux" is under the site-packages here
export PYTHONPATH=$HOME/.local/lib/python3.7/site-packages
```
You can then do some basic testing.
```bash
$ flux start --test-size=4
```
**Ensure your PYTHONPATH is correct here** And import flux.
```bash
$ ipython
```
```python
import flux
flux.Flux()
```
## Development
We provide a [.devcontainer](https://github.com/flux-framework/flux-python/tree/main/.devcontainer) environment you can open in VSCode
to have an environment ready to go with Flux (and Flux Security). You can follow
the instructions in the DevContainer to build the Flux Python bindings.
By default, this environment installs the latest flux-core.
If you want to build a custom version with Flux core you can do:
```bash
rm -rf ~/flux-core
export FLUX_VERSION=0.49.0
wget https://github.com/flux-framework/flux-core/releases/download/v${FLUX_VERSION}/flux-core-${FLUX_VERSION}.tar.gz
tar -xzvf flux-core-${FLUX_VERSION}.tar.gz
sudo mv flux-core-${FLUX_VERSION} ~/flux-core
rm flux-core-${FLUX_VERSION}.tar.gz
cd ~/flux-core
./configure --prefix=/usr
make
sudo make install
```
And then copy over the Python source to "flux" and build your custom archives and/or wheels:
```bash
$ rm -rf /workspaces/flux-python/flux
$ cp -R src/bindings/python/flux /workspaces/flux-python/flux
$ cd /workspaces/flux-python
# Archive
$ python3 setup.py sdist
# Wheel (will build for the python version you've targeted)
# This isn't currently being done but is a future TODO!
$ python3 setup.py sdist bdist_wheel
```
You can build versions of the Python wheels across 3.6 to 3.10 like:
```bash
# only need to run this once in the container
/bin/bash ./docker/install-mamba.sh
# And this to install the current Flux + version in setup.py as a wheel
# The number is the build number
/bin/bash ./docker/build-wheels.sh 2
```
And if you want to upload:
```bash
$ twine upload dist/*.tar.gz
```
### Automation
In order to (eventually) support a workflow to check for a release, get the release's
flux assets and build, we have a [workflow script](.github/scripts/check_releases.py) that will:
- Get the latest flux releases
- Compare to the latest pypi releases
- If there is a new release (from Flux on GitHub) not in pypi, build and release
And the script also can take an optional version string that will find and build
a version explicitly. You might (to test) do:
```bash
$ export GITHUB_TOKEN=xxxxxxxxxxxxx
$ python .github/scripts/check_releases.py flux-framework --version 0.46.0
```
That basically checks if there should be a build. It will use the container provided
here to install a custom version of flux core that builds the release.
#### Release
SPDX-License-Identifier: LGPL-3.0
LLNL-CODE-764420
Raw data
{
"_id": null,
"home_page": "https://github.com/flux-framework/flux-python",
"name": "flux-python",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "flux, job manager, workload manager, orchestration, hpc",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/57/ca/ad1388a37b9f2defe723a27efa8f3bee71d1a4b40acd940898e9cbac3e92/flux_python-0.66.0.tar.gz",
"platform": null,
"description": "# Flux Python Bindings\n\n> \ud83d\udc0d\ufe0f You called me?\n\nHello! You've found the flux Python bindings, an experiment to build and deploy\nFlux to Pypi without needing to store code alongside Flux. The goal of\nthis experiment is to test them separately. This was originally developed\nat [vsoch/flux-python](https://github.com/vsoch/flux-python) and has \nbeen ported here to automate more officially. The following guides might be useful\nto you:\n\n - \u2b50\ufe0f [Flux Framework Documentation](https://flux-framework.readthedocs.io)\n - \u2b50\ufe0f [Flux Projects](https://flux-framework.org)\n - \u2b50\ufe0f [Tutorials](https://flux-framework.readthedocs.io/en/latest/tutorials/index.html)\n - \u2b50\ufe0f [Installing Flux Python on an LLNL System](https://github.com/flux-framework/flux-python/blob/main/LLNL.md) \n \n\n## Helper Install\n\nIf you don't want to figure out your flux version and install with pip directory (per instructions in [manual install](#manual-install))\nwe provide a helper script [install-flux-python.sh](script/install-flux-python.sh) that will:\n\n1. Determine that Flux and Python are installed with pip\n2. Parse the version of Flux on your PATH\n3. Derive the version and run the command for you!\n\nYou'll first want to make sure that flux is on your path, as is a `python3` executable with pip installed\nas a module alongside it. And you can run it as follows:\n\n```bash\nwget https://raw.githubusercontent.com/flux-framework/flux-python/main/script/install-flux-python.sh\nchmod +x ./install-flux-python.sh\n./install-flux-python.sh\n```\n\nIf a version is missing on pypi or you otherwise have a bug, please open an issue here.\n\n## Manual Install\n\nSince we need to link to Flux libraries, you are advised to install flux and flux-security in the same location\nthat will be discovered via the executable \"flux,\" so typically `/usr` or `/usr/local`. You will need `wheel` installed:\n\n```bash\n$ pip install wheel\n```\n\nYou can do either:\n\n```bash\n# Find the flux version on your system\nflux --version\n\n# Install the bindings that match that version\npip install flux-python==0.48.0\n```\n\nAnd if you are having trouble with the WCI cache, you can also wget the [file directly from pypi](https://pypi.org/project/flux-python/#history)\nand install the .tar.gz directly.\n\n```bash\nwget https://files.pythonhosted.org/packages/25/fb/02951d80e44a19db291f0e7370d4e7d82c0c1b17709a37913881f958dff7/flux-python-0.48.0rc0.tar.gz\npip install flux-python-0.48.0rc0.tar.gz\n```\n\nIf you install to a local (personal) location (e.g., `$HOME/.local`) you'll need to add this to your `PYTHONPATH`\n\n```bash\n# The directory \"flux\" is under the site-packages here\nexport PYTHONPATH=$HOME/.local/lib/python3.7/site-packages\n```\n\nYou can then do some basic testing.\n\n```bash\n$ flux start --test-size=4\n```\n\n**Ensure your PYTHONPATH is correct here** And import flux.\n\n```bash\n$ ipython\n```\n```python\nimport flux\nflux.Flux()\n```\n \n \n## Development\n\nWe provide a [.devcontainer](https://github.com/flux-framework/flux-python/tree/main/.devcontainer) environment you can open in VSCode\nto have an environment ready to go with Flux (and Flux Security). You can follow\nthe instructions in the DevContainer to build the Flux Python bindings.\nBy default, this environment installs the latest flux-core.\nIf you want to build a custom version with Flux core you can do:\n\n```bash\nrm -rf ~/flux-core\nexport FLUX_VERSION=0.49.0\nwget https://github.com/flux-framework/flux-core/releases/download/v${FLUX_VERSION}/flux-core-${FLUX_VERSION}.tar.gz\ntar -xzvf flux-core-${FLUX_VERSION}.tar.gz\nsudo mv flux-core-${FLUX_VERSION} ~/flux-core\nrm flux-core-${FLUX_VERSION}.tar.gz\ncd ~/flux-core\n./configure --prefix=/usr\nmake\nsudo make install\n```\n\nAnd then copy over the Python source to \"flux\" and build your custom archives and/or wheels:\n\n```bash\n$ rm -rf /workspaces/flux-python/flux\n$ cp -R src/bindings/python/flux /workspaces/flux-python/flux\n$ cd /workspaces/flux-python\n# Archive\n$ python3 setup.py sdist\n\n# Wheel (will build for the python version you've targeted)\n# This isn't currently being done but is a future TODO!\n$ python3 setup.py sdist bdist_wheel\n```\n\nYou can build versions of the Python wheels across 3.6 to 3.10 like:\n\n```bash\n# only need to run this once in the container\n/bin/bash ./docker/install-mamba.sh\n\n# And this to install the current Flux + version in setup.py as a wheel\n# The number is the build number\n/bin/bash ./docker/build-wheels.sh 2\n```\n\nAnd if you want to upload:\n\n```bash\n$ twine upload dist/*.tar.gz\n```\n\n\n### Automation\n\nIn order to (eventually) support a workflow to check for a release, get the release's\nflux assets and build, we have a [workflow script](.github/scripts/check_releases.py) that will:\n\n- Get the latest flux releases\n- Compare to the latest pypi releases\n- If there is a new release (from Flux on GitHub) not in pypi, build and release\n\nAnd the script also can take an optional version string that will find and build\na version explicitly. You might (to test) do:\n\n```bash\n$ export GITHUB_TOKEN=xxxxxxxxxxxxx\n$ python .github/scripts/check_releases.py flux-framework --version 0.46.0\n```\n\nThat basically checks if there should be a build. It will use the container provided\nhere to install a custom version of flux core that builds the release.\n\n#### Release\n\nSPDX-License-Identifier: LGPL-3.0\n\nLLNL-CODE-764420\n",
"bugtrack_url": null,
"license": null,
"summary": "Python bindings for the flux resource manager API",
"version": "0.66.0",
"project_urls": {
"Homepage": "https://github.com/flux-framework/flux-python"
},
"split_keywords": [
"flux",
" job manager",
" workload manager",
" orchestration",
" hpc"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "78cbd306bd09825bd23d68b1d41dbf701c716892f9e8e2d19577a918ce937b67",
"md5": "066510a3414b3ea4f578eb1a3e3aa269",
"sha256": "8e9fcbdfe917d442a7e9d948c0e41bcb45d75c14cb923252c96a51a8f7b5201e"
},
"downloads": -1,
"filename": "flux_python-0.66.0-0-cp310-cp310-any.whl",
"has_sig": false,
"md5_digest": "066510a3414b3ea4f578eb1a3e3aa269",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 676785,
"upload_time": "2024-10-07T18:15:58",
"upload_time_iso_8601": "2024-10-07T18:15:58.252150Z",
"url": "https://files.pythonhosted.org/packages/78/cb/d306bd09825bd23d68b1d41dbf701c716892f9e8e2d19577a918ce937b67/flux_python-0.66.0-0-cp310-cp310-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4239c7d2d8005b987517c849f5ffec400a0d3c2d0867be57879fa44a363ab738",
"md5": "a1119990097838342b6cc757eb209121",
"sha256": "d9772ce07805847bdb12022523a8b7557f2b40ec972f2dbebc3bb83639992df1"
},
"downloads": -1,
"filename": "flux_python-0.66.0-0-cp36-cp36m-any.whl",
"has_sig": false,
"md5_digest": "a1119990097838342b6cc757eb209121",
"packagetype": "bdist_wheel",
"python_version": "cp36",
"requires_python": null,
"size": 941177,
"upload_time": "2024-10-07T18:15:59",
"upload_time_iso_8601": "2024-10-07T18:15:59.747957Z",
"url": "https://files.pythonhosted.org/packages/42/39/c7d2d8005b987517c849f5ffec400a0d3c2d0867be57879fa44a363ab738/flux_python-0.66.0-0-cp36-cp36m-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3a3dc6f3b4eb639c9b4a40b8f9fb90762e0a39fa0f2e0694ce5075cd6cf3fe7e",
"md5": "8a4d467eccac7baf9ac510dcb1f33256",
"sha256": "eb6b911ed2a6c16b9db03453176726eb3b8fb5354083486e7aa6e3b0ff727334"
},
"downloads": -1,
"filename": "flux_python-0.66.0-0-cp37-cp37m-any.whl",
"has_sig": false,
"md5_digest": "8a4d467eccac7baf9ac510dcb1f33256",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": null,
"size": 941171,
"upload_time": "2024-10-07T18:16:01",
"upload_time_iso_8601": "2024-10-07T18:16:01.718405Z",
"url": "https://files.pythonhosted.org/packages/3a/3d/c6f3b4eb639c9b4a40b8f9fb90762e0a39fa0f2e0694ce5075cd6cf3fe7e/flux_python-0.66.0-0-cp37-cp37m-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2432a9a531f9a3b60001505af41d6b0ef4c6e470be76ccf669d1d15a697c0cb8",
"md5": "9886be2600f997086f7993369d30a669",
"sha256": "274fba7e44332a9b7b5300de349349eef8551e9d641eff826ea0d6a408d84bf1"
},
"downloads": -1,
"filename": "flux_python-0.66.0-0-cp38-cp38-any.whl",
"has_sig": false,
"md5_digest": "9886be2600f997086f7993369d30a669",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": null,
"size": 416505,
"upload_time": "2024-10-07T18:16:03",
"upload_time_iso_8601": "2024-10-07T18:16:03.026197Z",
"url": "https://files.pythonhosted.org/packages/24/32/a9a531f9a3b60001505af41d6b0ef4c6e470be76ccf669d1d15a697c0cb8/flux_python-0.66.0-0-cp38-cp38-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "405cdf1c27c4198537ee51d29eb1f32e270e8985d0e1c6ca466145703de580c8",
"md5": "7da1deabc7c64c8d1d2e3e86ee618645",
"sha256": "eb50bbade3a2ca016fb84a0ebba20fd04987b3bf5bbb9dea2dd6fd6efd321172"
},
"downloads": -1,
"filename": "flux_python-0.66.0-0-cp39-cp39-any.whl",
"has_sig": false,
"md5_digest": "7da1deabc7c64c8d1d2e3e86ee618645",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 416508,
"upload_time": "2024-10-07T18:16:04",
"upload_time_iso_8601": "2024-10-07T18:16:04.608743Z",
"url": "https://files.pythonhosted.org/packages/40/5c/df1c27c4198537ee51d29eb1f32e270e8985d0e1c6ca466145703de580c8/flux_python-0.66.0-0-cp39-cp39-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "57caad1388a37b9f2defe723a27efa8f3bee71d1a4b40acd940898e9cbac3e92",
"md5": "8576db5f7f6100ec041872696fbe4b7e",
"sha256": "56b6f0356e8bb143629332c1fb0ddaa16b7e6afdf1fa459bb9b3b35d1366c8e3"
},
"downloads": -1,
"filename": "flux_python-0.66.0.tar.gz",
"has_sig": false,
"md5_digest": "8576db5f7f6100ec041872696fbe4b7e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 241768,
"upload_time": "2024-10-07T18:16:06",
"upload_time_iso_8601": "2024-10-07T18:16:06.588595Z",
"url": "https://files.pythonhosted.org/packages/57/ca/ad1388a37b9f2defe723a27efa8f3bee71d1a4b40acd940898e9cbac3e92/flux_python-0.66.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-07 18:16:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "flux-framework",
"github_project": "flux-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "cffi",
"specs": [
[
">=",
"1.1"
]
]
},
{
"name": "pyyaml",
"specs": []
}
],
"lcname": "flux-python"
}
JSON
