# Axle-Runtime - Python Wheel enhancement library
[](https://app.gitter.im/#/room/#karellen_Lobby:gitter.im)
[](https://github.com/karellen/wheel-axle-runtime/actions/workflows/build.yml)
[](https://coveralls.io/r/karellen/wheel-axle-runtime?branch=master)
[](https://pypi.org/project/wheel-axle-runtime/)
[](https://pypi.org/project/wheel-axle-runtime/)
[](https://pypistats.org/packages/wheel-axle-runtime)
[](https://pypistats.org/packages/wheel-axle-runtime)
[](https://pypistats.org/packages/wheel-axle-runtime)
# This is a companion project to [Wheel Axle/bdist_axle](https://github.com/karellen/wheel-axle)
## Problem
1. Python wheels do not support symlinks.
2. PIP installation procedure is not locally extensible and does not allow adding post-install hooks.
## Solution
**WARNING: THIS IS EXPERIMENTAL BETA SOFTWARE. THERE ARE NO WARRANTIES OF ANY KIND. USE AT YOUR OWN RISK. ADDITIONAL
INCLUDED DISCLAIMERS ALSO APPLY.**
Wheel Axle Runtime library utilizes a little-known trick used in `site.py`'s `.pth` files that allows executing
arbitrary code while the site packages are being added. Thus, specially-crafted wheels can silently execute installed
code on Python interpreter startup, facilitating the "post-install hook" functionality.
### Python Invariants
The core functionality relies on the following Python behaviors:
* `site.py` [processes .pth files](https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/site.py#L171)
* `site.py` [executes .pth import lines](https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/site.py#L186)
* `.pth` file's line is executed with a local
variable [`fullname` denoting the `.pth` file path](https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/site.py#L170)
These invariants have not changed for *18 years*.
### Implementation
Once the distribution-specific `.pth` is executed by the Python interpreter, the Wheel Axle Runtime behaves as follows:
1. The library checks whether a file `.dist-info/axle.done` exists. If it does it is the indication that the
post-install hook has executed successfully and nothing more is to be done, terminating all further processing.
2. A process-wide *inter-thread* lock is acquired.
3. An OS-wide *inter-process file* lock is acquired on a file `.dist-info/axle.lck`.
4. Once the locks are acquired the `.dist-info/axle.done` existence is rechecked (double-checked locking optimization).
5. Now that in-process and inter-process race conditions are excluded the post-install work can begin.
6. Registered `installers` are run in sequence. Installers *should be* idempotent. The following installers are
currently implemented:
1. *LibPython installer* checks for the presence of `.dist-info/require-libpython`.
1. The installer determines the location of the installation: venv, user or other.
2. The list of all libpython library files is located from the `sys.base_exec_prefix`.
3. If the installation is either venv or user and the link to the libpython library doesn't exist the symlink
is created.
2. *Symlinks installer* processes `.dist-info/symlinks.txt`, if any.
1. Based on the location of the `.pth` file being executed the current installation `schema` and its paths are
determined. Currently, installation into a virtual environment or user location is supported and tested.
2. For each symlink the target path is resolved and `realpath` is used to determine the final target path.
3. If the symlink path and symlink target path are within one of the permitted schema locations the symlink is
created. Otherwise, an exception is raised and the processing is aborted.
4. After all symlinks are created, the `.dist-info/RECORD` file is updated to reflect the created symlinks.
3. *Axle installer* finalizes the installation. This installer is always executed last.
1. The `.dist-info/RECORD` is updated with `.dist-info/axle.done` file record.
2. `.dist-info/axle.done` is created.
3. `<distribution name and version>.pth` is then removed. If the file cannot be removed it is left in place.
This can happen on Windows, since the `.pth` file in question is likely opened for exclusive reading on
Windows.
7. Any failure anywhere in the above process will result in an abort, an error message, and a retry the next time
the `.pth` will be activated.
### Security
There are several security requirements and implications of having post-install hooks implemented this way.
1. The installation requires write permissions to the distribution. This will be a problem if the package is installed
as `root` in locations such as `/usr` or `/usr/local`, or is otherwise not write-permitted, unless the post-install
hook is also ran with the sufficient privileges. This is generally acceptable as the primary use is considered to be
installation into virtual envs and user locations. That said, simply running `python -c pass` or any other python
invocation that does activate `site.py` under the required privileges will finalize post-install procedures.
2. There is *an attempt* to ensure that that axle wheels symlinks and targets don't extend beyond the allowed `schema`
locations. *Those attempts are **superficial** and **have not been formally verified**.* For example, it may be
possible to escape the path validation/confinement by:
* hacking symlink creation order
* hacking symlink directory targets
* exploiting OS-specific `realpath` implementation idiosyncrasies (i.e. `strict` vs not, and what is considered
strict)
### TODOs
* Support schema detection for `prefix` installations.
* Validate and verify Windows support.
Raw data
{
"_id": null,
"home_page": "https://github.com/karellen/wheel-axle-runtime",
"name": "wheel-axle-runtime",
"maintainer": "Arcadiy Ivanov",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "arcadiy@karellen.co",
"keywords": "wheel packaging setuptools bdist_wheel symlink postinstall",
"author": "Karellen, Inc.",
"author_email": "supervisor@karellen.co",
"download_url": "https://files.pythonhosted.org/packages/3b/49/d465ef4093fa40112b5da00e6d36c7b36de1579d2e82e60369c543be7083/wheel_axle_runtime-0.0.6.tar.gz",
"platform": null,
"description": "# Axle-Runtime - Python Wheel enhancement library\n\n[](https://app.gitter.im/#/room/#karellen_Lobby:gitter.im)\n[](https://github.com/karellen/wheel-axle-runtime/actions/workflows/build.yml)\n[](https://coveralls.io/r/karellen/wheel-axle-runtime?branch=master)\n\n[](https://pypi.org/project/wheel-axle-runtime/)\n[](https://pypi.org/project/wheel-axle-runtime/)\n\n[](https://pypistats.org/packages/wheel-axle-runtime)\n[](https://pypistats.org/packages/wheel-axle-runtime)\n[](https://pypistats.org/packages/wheel-axle-runtime)\n\n# This is a companion project to [Wheel Axle/bdist_axle](https://github.com/karellen/wheel-axle)\n\n## Problem\n\n1. Python wheels do not support symlinks.\n2. PIP installation procedure is not locally extensible and does not allow adding post-install hooks.\n\n## Solution\n\n**WARNING: THIS IS EXPERIMENTAL BETA SOFTWARE. THERE ARE NO WARRANTIES OF ANY KIND. USE AT YOUR OWN RISK. ADDITIONAL\nINCLUDED DISCLAIMERS ALSO APPLY.**\n\nWheel Axle Runtime library utilizes a little-known trick used in `site.py`'s `.pth` files that allows executing\narbitrary code while the site packages are being added. Thus, specially-crafted wheels can silently execute installed\ncode on Python interpreter startup, facilitating the \"post-install hook\" functionality.\n\n### Python Invariants\n\nThe core functionality relies on the following Python behaviors:\n\n* `site.py` [processes .pth files](https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/site.py#L171)\n* `site.py` [executes .pth import lines](https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/site.py#L186)\n* `.pth` file's line is executed with a local\n variable [`fullname` denoting the `.pth` file path](https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/site.py#L170)\n\nThese invariants have not changed for *18 years*.\n\n### Implementation\n\nOnce the distribution-specific `.pth` is executed by the Python interpreter, the Wheel Axle Runtime behaves as follows:\n\n1. The library checks whether a file `.dist-info/axle.done` exists. If it does it is the indication that the\n post-install hook has executed successfully and nothing more is to be done, terminating all further processing.\n2. A process-wide *inter-thread* lock is acquired.\n3. An OS-wide *inter-process file* lock is acquired on a file `.dist-info/axle.lck`.\n4. Once the locks are acquired the `.dist-info/axle.done` existence is rechecked (double-checked locking optimization).\n5. Now that in-process and inter-process race conditions are excluded the post-install work can begin.\n6. Registered `installers` are run in sequence. Installers *should be* idempotent. The following installers are\n currently implemented:\n 1. *LibPython installer* checks for the presence of `.dist-info/require-libpython`.\n 1. The installer determines the location of the installation: venv, user or other.\n 2. The list of all libpython library files is located from the `sys.base_exec_prefix`.\n 3. If the installation is either venv or user and the link to the libpython library doesn't exist the symlink \n is created.\n 2. *Symlinks installer* processes `.dist-info/symlinks.txt`, if any.\n 1. Based on the location of the `.pth` file being executed the current installation `schema` and its paths are\n determined. Currently, installation into a virtual environment or user location is supported and tested.\n 2. For each symlink the target path is resolved and `realpath` is used to determine the final target path.\n 3. If the symlink path and symlink target path are within one of the permitted schema locations the symlink is\n created. Otherwise, an exception is raised and the processing is aborted.\n 4. After all symlinks are created, the `.dist-info/RECORD` file is updated to reflect the created symlinks.\n 3. *Axle installer* finalizes the installation. This installer is always executed last.\n 1. The `.dist-info/RECORD` is updated with `.dist-info/axle.done` file record.\n 2. `.dist-info/axle.done` is created.\n 3. `<distribution name and version>.pth` is then removed. If the file cannot be removed it is left in place.\n This can happen on Windows, since the `.pth` file in question is likely opened for exclusive reading on\n Windows.\n7. Any failure anywhere in the above process will result in an abort, an error message, and a retry the next time\n the `.pth` will be activated.\n\n### Security\n\nThere are several security requirements and implications of having post-install hooks implemented this way.\n\n1. The installation requires write permissions to the distribution. This will be a problem if the package is installed\n as `root` in locations such as `/usr` or `/usr/local`, or is otherwise not write-permitted, unless the post-install\n hook is also ran with the sufficient privileges. This is generally acceptable as the primary use is considered to be\n installation into virtual envs and user locations. That said, simply running `python -c pass` or any other python\n invocation that does activate `site.py` under the required privileges will finalize post-install procedures.\n2. There is *an attempt* to ensure that that axle wheels symlinks and targets don't extend beyond the allowed `schema`\n locations. *Those attempts are **superficial** and **have not been formally verified**.* For example, it may be\n possible to escape the path validation/confinement by:\n * hacking symlink creation order\n * hacking symlink directory targets\n * exploiting OS-specific `realpath` implementation idiosyncrasies (i.e. `strict` vs not, and what is considered\n strict)\n\n### TODOs\n\n* Support schema detection for `prefix` installations.\n* Validate and verify Windows support.\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "Axle Runtime is the runtime part of the Python Wheel enhancement library",
"version": "0.0.6",
"project_urls": {
"Bug Tracker": "https://github.com/karellen/wheel-axle-runtime/issues",
"Documentation": "https://github.com/karellen/wheel-axle-runtime/",
"Homepage": "https://github.com/karellen/wheel-axle-runtime",
"Source Code": "https://github.com/karellen/wheel-axle-runtime/"
},
"split_keywords": [
"wheel",
"packaging",
"setuptools",
"bdist_wheel",
"symlink",
"postinstall"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6c3451b4ce7fe6189f1d8de84750aa3f52faa815dcb455473fc1f19dbc2c1905",
"md5": "275bdeec1482b08a7a431044fcf3b1e3",
"sha256": "8236c427e36851643983b2dc0a4e5bb7147f41817e3bb415186ef04f33b5b966"
},
"downloads": -1,
"filename": "wheel_axle_runtime-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "275bdeec1482b08a7a431044fcf3b1e3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 14361,
"upload_time": "2024-09-20T02:28:38",
"upload_time_iso_8601": "2024-09-20T02:28:38.256787Z",
"url": "https://files.pythonhosted.org/packages/6c/34/51b4ce7fe6189f1d8de84750aa3f52faa815dcb455473fc1f19dbc2c1905/wheel_axle_runtime-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3b49d465ef4093fa40112b5da00e6d36c7b36de1579d2e82e60369c543be7083",
"md5": "4a114f415b4dde3345e2441662fd888d",
"sha256": "468135e627ac8e102d096d58dba33d0504092e25db3f3087293988a0edb89eef"
},
"downloads": -1,
"filename": "wheel_axle_runtime-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "4a114f415b4dde3345e2441662fd888d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 16822,
"upload_time": "2024-09-20T02:28:39",
"upload_time_iso_8601": "2024-09-20T02:28:39.545507Z",
"url": "https://files.pythonhosted.org/packages/3b/49/d465ef4093fa40112b5da00e6d36c7b36de1579d2e82e60369c543be7083/wheel_axle_runtime-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-20 02:28:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "karellen",
"github_project": "wheel-axle-runtime",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "wheel-axle-runtime"
}
JSON
