# Python manual package manager—a package manager written in Python for manually installing a compiled stack
```{toctree}
:maxdepth: 2
:numbered:
:hidden:
api/modules
CHANGELOG
```
<!-- [](https://souk-data-centre.readthedocs.io/en/latest/?badge=latest&style=plastic)
[](https://ickc.github.io/souk-data-centre)
[](https://pypi.org/project/souk-data-centre)
[](https://pypi.org/project/souk-data-centre)
[](https://pypi.org/project/souk-data-centre)
[](https://pypi.org/project/souk-data-centre)
[](https://github.com/ickc/python-pmpm/releases)
[](https://pypi.python.org/pypi/souk-data-centre/)
[](https://pypi.python.org/pypi/souk-data-centre/)
 -->
## Introduction
Python manual package manager is a package manager written in Python for manually installing a compiled stack.
Before you proceed, you should know that for typical usage, `conda` (and its friends `mamba`/`micromamba`) should be enough for the purpose.
`pmpm` built on top of `conda` which also compile some Python packages locally.
Goal:
- Custom compilation of some packages, possibly for optimization such as `-march` and `-mtune` which is beyond what `conda` offers.
- Provide fast re-compile after some small local changes, suitable for development.
Approaches:
- `conda_install`: Both the conda provided stack and the compiled stack from `pmpm` are in the same prefix, this makes activating an environment seamless. It is achieved by cleanly compile an environment using the conda provided stack only, including compilers.
- `system_install`: The conda provided stack and the compiled stack from `pmpm` has a different prefix. This makes the 2 completely separate, where the compiled stack can uses the host compilers. This is useful for example when the vendor provided MPI compilers are needed. This also has more points of failure, as we can't completely control what is in the host environment. `pmpm` only serves as automation for reproducibility. But you probably need to modify how `pmpm` behaves on different host, and you probably need to install packages from the OS package manager.
Alternative approach:
- You can create a conda recipe and use `conda-build` and tweak from there for customization. The only downside probably is the time it takes to re-compile after modifications.
## Installation
```sh
pip install pmpm
```
## Development
```sh
git clone https://github.com/ickc/python-pmpm.git
cd python-pmpm
conda activate
mamba env create -f environment.yml
conda activate pmpm
pip install -e .
```
## Usage
Use one of the example config in this repository:
```sh
pmpm conda_install "$HOME/pmpm-test" --file examples/....yml
```
## Design
When installing from a YAML file such as those given in the `examples/` directory,
`pmpm` behaves as a superset of conda/mamba with an extra `_pmpm` key in the YAML configuration.
`pmpm` will compile packages available in `pmpm.packages` after the conda environment is created,
as defined in the YAML file.
Raw data
{
"_id": null,
"home_page": "https://github.com/ickc/python-pmpm",
"name": "python-pmpm",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "package manager",
"author": "Kolen Cheung",
"author_email": "christian.kolen@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/16/1a/c4b130bfc47a930292a0a0372bb040c5d5c88e14ad452bb541539d0685a7/python_pmpm-0.2.0.tar.gz",
"platform": null,
"description": "# Python manual package manager\u2014a package manager written in Python for manually installing a compiled stack\n\n```{toctree}\n:maxdepth: 2\n:numbered:\n:hidden:\n\napi/modules\nCHANGELOG\n```\n\n<!-- [](https://souk-data-centre.readthedocs.io/en/latest/?badge=latest&style=plastic)\n[](https://ickc.github.io/souk-data-centre)\n\n\n\n[](https://pypi.org/project/souk-data-centre)\n[](https://pypi.org/project/souk-data-centre)\n[](https://pypi.org/project/souk-data-centre)\n[](https://pypi.org/project/souk-data-centre)\n[](https://github.com/ickc/python-pmpm/releases)\n[](https://pypi.python.org/pypi/souk-data-centre/)\n[](https://pypi.python.org/pypi/souk-data-centre/)\n -->\n\n## Introduction\n\nPython manual package manager is a package manager written in Python for manually installing a compiled stack.\nBefore you proceed, you should know that for typical usage, `conda` (and its friends `mamba`/`micromamba`) should be enough for the purpose.\n`pmpm` built on top of `conda` which also compile some Python packages locally.\n\nGoal:\n\n- Custom compilation of some packages, possibly for optimization such as `-march` and `-mtune` which is beyond what `conda` offers.\n- Provide fast re-compile after some small local changes, suitable for development.\n\nApproaches:\n\n- `conda_install`: Both the conda provided stack and the compiled stack from `pmpm` are in the same prefix, this makes activating an environment seamless. It is achieved by cleanly compile an environment using the conda provided stack only, including compilers.\n- `system_install`: The conda provided stack and the compiled stack from `pmpm` has a different prefix. This makes the 2 completely separate, where the compiled stack can uses the host compilers. This is useful for example when the vendor provided MPI compilers are needed. This also has more points of failure, as we can't completely control what is in the host environment. `pmpm` only serves as automation for reproducibility. But you probably need to modify how `pmpm` behaves on different host, and you probably need to install packages from the OS package manager.\n\nAlternative approach:\n\n- You can create a conda recipe and use `conda-build` and tweak from there for customization. The only downside probably is the time it takes to re-compile after modifications.\n\n## Installation\n\n```sh\npip install pmpm\n```\n\n## Development\n\n```sh\ngit clone https://github.com/ickc/python-pmpm.git\ncd python-pmpm\nconda activate\nmamba env create -f environment.yml\nconda activate pmpm\npip install -e .\n```\n\n## Usage\n\nUse one of the example config in this repository:\n\n```sh\npmpm conda_install \"$HOME/pmpm-test\" --file examples/....yml\n```\n\n## Design\n\nWhen installing from a YAML file such as those given in the `examples/` directory,\n`pmpm` behaves as a superset of conda/mamba with an extra `_pmpm` key in the YAML configuration.\n`pmpm` will compile packages available in `pmpm.packages` after the conda environment is created,\nas defined in the YAML file.\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "a package manager written in Python for manually installing a compiled stack",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://ickc.github.io/python-pmpm",
"Homepage": "https://github.com/ickc/python-pmpm",
"Repository": "https://github.com/ickc/python-pmpm"
},
"split_keywords": [
"package",
"manager"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "05343391946770677b5f0bb34362f0aa1767b60bc1743b6e66473c6b10927c6a",
"md5": "18fc800d9d64ddf6378c58fdfd5bd655",
"sha256": "c53c0949e40a5d84502b7dcd38aa5a9ed1fec4366a83ba753c449c7f0408d2a8"
},
"downloads": -1,
"filename": "python_pmpm-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "18fc800d9d64ddf6378c58fdfd5bd655",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 18155,
"upload_time": "2023-12-11T18:38:53",
"upload_time_iso_8601": "2023-12-11T18:38:53.178194Z",
"url": "https://files.pythonhosted.org/packages/05/34/3391946770677b5f0bb34362f0aa1767b60bc1743b6e66473c6b10927c6a/python_pmpm-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "161ac4b130bfc47a930292a0a0372bb040c5d5c88e14ad452bb541539d0685a7",
"md5": "33f48c83823ba0041ac703c02d590662",
"sha256": "a03b903b2a9dbd3736f53160f618f750b4a284359aafb6f1cd88ed17334909bd"
},
"downloads": -1,
"filename": "python_pmpm-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "33f48c83823ba0041ac703c02d590662",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 14379,
"upload_time": "2023-12-11T18:38:54",
"upload_time_iso_8601": "2023-12-11T18:38:54.903998Z",
"url": "https://files.pythonhosted.org/packages/16/1a/c4b130bfc47a930292a0a0372bb040c5d5c88e14ad452bb541539d0685a7/python_pmpm-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-11 18:38:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ickc",
"github_project": "python-pmpm",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "python-pmpm"
}
JSON
