# isolated-environment
[](https://github.com/zackees/isolated-environment/actions/workflows/lint.yml)
[](https://github.com/zackees/isolated-environment/actions/workflows/push_macos.yml)
[](https://github.com/zackees/isolated-environment/actions/workflows/push_ubuntu.yml)
[](https://github.com/zackees/isolated-environment/actions/workflows/push_win.yml)
# Summary
Got pinned dependencies in your python package that make it hard to install? Use isolated-environment to package those up in a runtime `venv` that only your package has access to.
This is a package isolation library designed originally for AI developers to solve the problems
of AI dependency conflicts introduced by the various `pytorch`/`tensorflow`/etc incompatibilities within and between AI apps.
*Install*
```bash
pip install isolated-environment
```
*Runtime*
```python
# Example of running "whisper --help" in an isolated-environment
from pathlib import Path
import subprocess
from isolated_environment import isolated_environment_run
TENSOR_VERSION = "2.1.2"
CUDA_VERSION = "cu121"
EXTRA_INDEX_URL = f"https://download.pytorch.org/whl/{CUDA_VERSION}"
HERE = Path(os.path.abspath(os.path.dirname(__file__)))
venv_path = Path(HERE) / "whisper-venv"
requirements = [
"whisper-whisper",
f"torch=={TENSOR_VERSION}+{CUDA_VERSION} --extra-index-url {EXTRA_INDEX_URL}"
]
cmd_list = ["whisper", "--help"]
# Note that shell=False, universal_newlines=True, capture_output=True
cp: subprocess.CompletedProcess = isolated_environment_run(
env_path=venv_path,
requirements=requirements,
cmd_list=cmd_list)
print(cp.stdout)
```
Install cuda pytorch when nvidia-smi is found:
```python
# This generates an environment that should be passed to subprocess.run(...)
def get_environment() -> dict[str, Any]:
"""Returns the environment suitable for subprocess.run(..., env=env,...)"""
venv_dir = HERE / "venv" / "whisper"
deps = [
"openai-whisper",
]
if has_nvidia_smi():
deps.append( # This computer has nvidia cuda installed so install cuda torch.
f"torch=={TENSOR_VERSION}+{CUDA_VERSION} --extra-index-url {EXTRA_INDEX_URL}"
)
else:
# Install CPU version.
deps.append(f"torch=={TENSOR_VERSION}")
env = isolated_environment(venv_dir, deps)
return env
```
Any changes to the pip requirements list between runs will invoke a call to `pip install`.
It moves the install of your chosen dependencies from **install time** to **runtime**. The benefit of this is that you can query the system
and make choices on what needs to be installed. For example in `pip` you can't conditionally install packages based on whether `nvidia-smi` has
been installed (indicating `cuda` acceleration), but with `isolated-environment` this is straightfoward.
# Development
## Install
* First time setup
* clone the repo
* run `./install`
* To develop software, run `. ./activate.sh`
# Windows
This environment requires you to use `git-bash`.
# Linting
Run `./lint.sh` to find linting errors using `ruff`, `flake8` and `mypy`.
# License
This software is free to use for personal and commercial products. However, if you make changes to `isolated-environment` code you must agree to the
following "good-samaritan" stipulations:
* All changes to `isolated-environment` **MUST** be put into a github fork, linked to this github project (https://github.com/zackees/isolated-environment).
* That means clicking on the fork button on this repo, and then putting your changes into that fork.
This agreement means that `isolated-environment` can receive additional features from those that benefit from this package, so that others can benefit as well.
This supplemental licensing supersedes any language in the generic license attached. If you merely use `isolated-environment` as is, without modification,
none of this supplemental license applies to you.
# Releases
* 2.0.3 - Fixed a win32 bug related to finding site packages.
* 2.0.2 - Fixed a deep bug with how macos/linux handles subprocess handles a command list with `shell=True`.
* 2.0.0 - Requirements internally is now just a text file. Sequantially installing requirements is now no longer
possible. Any change to the requirements will cause a full rebuild. This fixes a number of problems
with how requirements are handled. This should now be much more robust. However, the old api is slightly
incompatible with the new one so a full api breaking version has been issued.
* 1.3.4 - Isolation for pip too so that it doesn't bind to the parent pip.
* 1.3.1 - New `full_isolation` mode to allow packages installed on other parts of the system from binding.
* 1.3.1 - Update readme.
* 1.3.0 - Marks a new interface.
* 1.2.7 - Please use `isolated_environment_run()` instead of `isolated_environment`. The latter has
footguns when using Linux when invoking `python` and `shell=True`
* 1.2.6 - Update readme
* 1.2.4 - Now support more build options, instead of just --extra-index-url.
* 1.2.3 - All builds green with complex dependencies!
* 1.2.2 - Tested and fixed complex semversion + build number for isolated_environment
* 1.2.1 - Fixes `isolated_environment()` not installing deps correctly on first go
* 1.2.0 - Now just use `isolated_environment()`, more simple.
* 1.0.6 - `exists` -> `installed()`, adds `pip_list()`, adds `clean()`
* 1.0.5 - Added `exists()`
* 1.0.4 - Added `lock()`
* 1.0.0 - Initial release
Raw data
{
"_id": null,
"home_page": "https://github.com/zackees/isolated-environment",
"name": "isolated-environment",
"maintainer": "Zachary Vorhies",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "template-python-cmd",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/e1/40/58a1830f6d7b1e72d7c0dde0de124d4714952da78002862d141f2700e663/isolated-environment-2.0.3.tar.gz",
"platform": null,
"description": "# isolated-environment\r\n\r\n[](https://github.com/zackees/isolated-environment/actions/workflows/lint.yml)\r\n[](https://github.com/zackees/isolated-environment/actions/workflows/push_macos.yml)\r\n[](https://github.com/zackees/isolated-environment/actions/workflows/push_ubuntu.yml)\r\n[](https://github.com/zackees/isolated-environment/actions/workflows/push_win.yml)\r\n\r\n\r\n\r\n# Summary\r\n\r\nGot pinned dependencies in your python package that make it hard to install? Use isolated-environment to package those up in a runtime `venv` that only your package has access to.\r\n\r\nThis is a package isolation library designed originally for AI developers to solve the problems\r\nof AI dependency conflicts introduced by the various `pytorch`/`tensorflow`/etc incompatibilities within and between AI apps.\r\n\r\n*Install*\r\n```bash\r\npip install isolated-environment\r\n```\r\n\r\n*Runtime*\r\n```python\r\n# Example of running \"whisper --help\" in an isolated-environment\r\nfrom pathlib import Path\r\nimport subprocess\r\nfrom isolated_environment import isolated_environment_run\r\n\r\nTENSOR_VERSION = \"2.1.2\"\r\nCUDA_VERSION = \"cu121\"\r\nEXTRA_INDEX_URL = f\"https://download.pytorch.org/whl/{CUDA_VERSION}\"\r\nHERE = Path(os.path.abspath(os.path.dirname(__file__)))\r\n\r\nvenv_path = Path(HERE) / \"whisper-venv\"\r\nrequirements = [\r\n \"whisper-whisper\",\r\n f\"torch=={TENSOR_VERSION}+{CUDA_VERSION} --extra-index-url {EXTRA_INDEX_URL}\"\r\n]\r\ncmd_list = [\"whisper\", \"--help\"]\r\n# Note that shell=False, universal_newlines=True, capture_output=True\r\ncp: subprocess.CompletedProcess = isolated_environment_run(\r\n env_path=venv_path,\r\n requirements=requirements,\r\n cmd_list=cmd_list)\r\nprint(cp.stdout)\r\n```\r\n\r\nInstall cuda pytorch when nvidia-smi is found:\r\n\r\n```python\r\n# This generates an environment that should be passed to subprocess.run(...)\r\ndef get_environment() -> dict[str, Any]:\r\n \"\"\"Returns the environment suitable for subprocess.run(..., env=env,...)\"\"\"\r\n venv_dir = HERE / \"venv\" / \"whisper\"\r\n deps = [\r\n \"openai-whisper\",\r\n ]\r\n if has_nvidia_smi():\r\n deps.append( # This computer has nvidia cuda installed so install cuda torch.\r\n f\"torch=={TENSOR_VERSION}+{CUDA_VERSION} --extra-index-url {EXTRA_INDEX_URL}\"\r\n )\r\n else:\r\n # Install CPU version.\r\n deps.append(f\"torch=={TENSOR_VERSION}\")\r\n env = isolated_environment(venv_dir, deps)\r\n return env\r\n```\r\n\r\nAny changes to the pip requirements list between runs will invoke a call to `pip install`.\r\n\r\nIt moves the install of your chosen dependencies from **install time** to **runtime**. The benefit of this is that you can query the system\r\nand make choices on what needs to be installed. For example in `pip` you can't conditionally install packages based on whether `nvidia-smi` has\r\nbeen installed (indicating `cuda` acceleration), but with `isolated-environment` this is straightfoward.\r\n\r\n# Development\r\n\r\n## Install\r\n\r\n * First time setup\r\n * clone the repo\r\n * run `./install`\r\n * To develop software, run `. ./activate.sh`\r\n\r\n# Windows\r\n\r\nThis environment requires you to use `git-bash`.\r\n\r\n# Linting\r\n\r\nRun `./lint.sh` to find linting errors using `ruff`, `flake8` and `mypy`.\r\n\r\n# License\r\n\r\nThis software is free to use for personal and commercial products. However, if you make changes to `isolated-environment` code you must agree to the\r\nfollowing \"good-samaritan\" stipulations:\r\n\r\n * All changes to `isolated-environment` **MUST** be put into a github fork, linked to this github project (https://github.com/zackees/isolated-environment).\r\n * That means clicking on the fork button on this repo, and then putting your changes into that fork.\r\n\r\nThis agreement means that `isolated-environment` can receive additional features from those that benefit from this package, so that others can benefit as well.\r\n\r\nThis supplemental licensing supersedes any language in the generic license attached. If you merely use `isolated-environment` as is, without modification,\r\nnone of this supplemental license applies to you.\r\n\r\n# Releases\r\n * 2.0.3 - Fixed a win32 bug related to finding site packages.\r\n * 2.0.2 - Fixed a deep bug with how macos/linux handles subprocess handles a command list with `shell=True`.\r\n * 2.0.0 - Requirements internally is now just a text file. Sequantially installing requirements is now no longer\r\n possible. Any change to the requirements will cause a full rebuild. This fixes a number of problems\r\n with how requirements are handled. This should now be much more robust. However, the old api is slightly\r\n incompatible with the new one so a full api breaking version has been issued.\r\n * 1.3.4 - Isolation for pip too so that it doesn't bind to the parent pip.\r\n * 1.3.1 - New `full_isolation` mode to allow packages installed on other parts of the system from binding.\r\n * 1.3.1 - Update readme.\r\n * 1.3.0 - Marks a new interface.\r\n * 1.2.7 - Please use `isolated_environment_run()` instead of `isolated_environment`. The latter has\r\n footguns when using Linux when invoking `python` and `shell=True`\r\n * 1.2.6 - Update readme\r\n * 1.2.4 - Now support more build options, instead of just --extra-index-url.\r\n * 1.2.3 - All builds green with complex dependencies!\r\n * 1.2.2 - Tested and fixed complex semversion + build number for isolated_environment\r\n * 1.2.1 - Fixes `isolated_environment()` not installing deps correctly on first go\r\n * 1.2.0 - Now just use `isolated_environment()`, more simple.\r\n * 1.0.6 - `exists` -> `installed()`, adds `pip_list()`, adds `clean()`\r\n * 1.0.5 - Added `exists()`\r\n * 1.0.4 - Added `lock()`\r\n * 1.0.0 - Initial release\r\n",
"bugtrack_url": null,
"license": "BSD 3-Clause License",
"summary": "Like Pipx, but allows creation of a virtual environment then populating it.",
"version": "2.0.3",
"project_urls": {
"Homepage": "https://github.com/zackees/isolated-environment"
},
"split_keywords": [
"template-python-cmd"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f9a06fa3de83ae5468f23db259c4ce76c4969bddce53324dbfc2717acd0a6ede",
"md5": "090ee058b8c4d2693cad553a1c5b03ab",
"sha256": "6c3f4555d01ded51c6b4381fbfc18b0deee546e095ebad783df1548a6b68ad93"
},
"downloads": -1,
"filename": "isolated_environment-2.0.3-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "090ee058b8c4d2693cad553a1c5b03ab",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.10",
"size": 8832,
"upload_time": "2024-08-20T23:48:17",
"upload_time_iso_8601": "2024-08-20T23:48:17.244996Z",
"url": "https://files.pythonhosted.org/packages/f9/a0/6fa3de83ae5468f23db259c4ce76c4969bddce53324dbfc2717acd0a6ede/isolated_environment-2.0.3-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e14058a1830f6d7b1e72d7c0dde0de124d4714952da78002862d141f2700e663",
"md5": "2aa638b0e00172934a16683d1c6411fc",
"sha256": "a1bb60ab66e5c859ea98f38dc41177503d83d922b0e9ef1cbae8fba9a378f49d"
},
"downloads": -1,
"filename": "isolated-environment-2.0.3.tar.gz",
"has_sig": false,
"md5_digest": "2aa638b0e00172934a16683d1c6411fc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 19696,
"upload_time": "2024-08-20T23:48:18",
"upload_time_iso_8601": "2024-08-20T23:48:18.738946Z",
"url": "https://files.pythonhosted.org/packages/e1/40/58a1830f6d7b1e72d7c0dde0de124d4714952da78002862d141f2700e663/isolated-environment-2.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-20 23:48:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "zackees",
"github_project": "isolated-environment",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "isolated-environment"
}
JSON
