# dowsing
Short version:
```
python -m dowsing.pep517 /path/to/repo | jq .
```
or
```
from dowsing.pep517 import get_metadata
dist = get_metadata(Path("/path/to/repo"))
```
## Basic reasoning
I don't want to execute arbitrary `setup.py` in order to find out their basic
metadata. I don't want to use the pep517 module in a sandbox, because commonly
packages forget to list their build-time dependencies.
This project is one step better than grepping source files, but also understands
`build-system` in `pyproject.toml` (from PEP 517/518). It does pretty well run
on a sampling of pypi projects, but does fail on some notable ones (including
setuptools).
When it fails, a key will be `"??"` and due to some quirks in list context, this
can be `["?", "?"]`.
## A rant
The reality of python packaging, even with recent PEPs, is that most nontrivial
python packages do moderately interesting stuff in their `setup.py`:
* Imports (either from local code, or `setup_requires`)
* Fetching things from the Internet
* Running commands
* Making sure native libs are installed, or there's a working C compiler
* Choosing deps based on platform
From the perspective of basically running a distro, they produce messages
intended for humans, rather than actually using the mechanisms that we have in
PEP 508 (environment markers) and 518 (pyproject.toml requires). There is also
no well-specified way to request native libs, and many projects choose to fail
to run `setup.py` when libs are missing.
## Goals
This project is a bridge to find several things out, about primarily setup.py
but also understanding some popular PEP 517/518 builders as a one-stop-shop, about:
* doesn't actually execute, so fetches or execs can't cause it to fail [done]
* cases where we could find out the version string, but it fails to import [done]
* lets you simulate the `pep517` module's output on different platforms [done]
* a lower-level api suitable for making edits to the place where the setup args
are defined [done]
* to list potential imports, and guess at missing build-time deps (something
like `numpy.distutils` is pretty clear) [todo]
## Doing this "right"
A bunch of this is papering over problems with the current reality. If you have
an existing sandbox and are ok with ~30% of projects just failing to build, you
can rely on the `pep517` module's API to actually execute the code on the
current version of python.
If you're willing to run the code and have it take longer, take a look at the
pep517 api `get_requires_for_*` or have it generate the metadata (assuming
what you want is in there). An example is in `dowsing/_demo_pep517.py`
This project's `dowsing.pep517` api is designed to do something similar, but not
fail on missing build-time requirements.
# Further Reading
* PEP 241, Metadata 1.0
* PEP 314, Metadata 1.1
* PEP 345, Metadata 1.2
* PEP 566, Metadata 2.1
* https://packaging.python.org/specifications/core-metadata/
* https://setuptools.readthedocs.io/en/latest/setuptools.html#metadata
# License
dowsing is copyright [Tim Hatch](http://timhatch.com/), and licensed under
the MIT license. I am providing code in this repository to you under an open
source license. This is my personal repository; the license you receive to
my code is from me and not from my employer. See the `LICENSE` file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/python-packaging/dowsing/",
"name": "dowsing",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Tim Hatch",
"author_email": "tim@timhatch.com",
"download_url": "https://files.pythonhosted.org/packages/11/c4/d69c6e1f5db8de84b0a447e75e2e1494b5ff431e4f29c54263ea3d7efb09/dowsing-0.8.0.tar.gz",
"platform": "",
"description": "# dowsing\n\nShort version:\n\n```\npython -m dowsing.pep517 /path/to/repo | jq .\n```\n\nor\n\n```\nfrom dowsing.pep517 import get_metadata\ndist = get_metadata(Path(\"/path/to/repo\"))\n```\n\n## Basic reasoning\n\nI don't want to execute arbitrary `setup.py` in order to find out their basic\nmetadata. I don't want to use the pep517 module in a sandbox, because commonly\npackages forget to list their build-time dependencies.\n\nThis project is one step better than grepping source files, but also understands\n`build-system` in `pyproject.toml` (from PEP 517/518). It does pretty well run\non a sampling of pypi projects, but does fail on some notable ones (including\nsetuptools).\n\nWhen it fails, a key will be `\"??\"` and due to some quirks in list context, this\ncan be `[\"?\", \"?\"]`.\n\n## A rant\n\nThe reality of python packaging, even with recent PEPs, is that most nontrivial\npython packages do moderately interesting stuff in their `setup.py`:\n\n* Imports (either from local code, or `setup_requires`)\n* Fetching things from the Internet\n* Running commands\n* Making sure native libs are installed, or there's a working C compiler\n* Choosing deps based on platform\n\nFrom the perspective of basically running a distro, they produce messages\nintended for humans, rather than actually using the mechanisms that we have in\nPEP 508 (environment markers) and 518 (pyproject.toml requires). There is also\nno well-specified way to request native libs, and many projects choose to fail\nto run `setup.py` when libs are missing.\n\n## Goals\n\nThis project is a bridge to find several things out, about primarily setup.py\nbut also understanding some popular PEP 517/518 builders as a one-stop-shop, about:\n\n* doesn't actually execute, so fetches or execs can't cause it to fail [done]\n* cases where we could find out the version string, but it fails to import [done]\n* lets you simulate the `pep517` module's output on different platforms [done]\n* a lower-level api suitable for making edits to the place where the setup args\n are defined [done]\n* to list potential imports, and guess at missing build-time deps (something\n like `numpy.distutils` is pretty clear) [todo]\n\n## Doing this \"right\"\n\nA bunch of this is papering over problems with the current reality. If you have\nan existing sandbox and are ok with ~30% of projects just failing to build, you\ncan rely on the `pep517` module's API to actually execute the code on the\ncurrent version of python.\n\nIf you're willing to run the code and have it take longer, take a look at the\npep517 api `get_requires_for_*` or have it generate the metadata (assuming\nwhat you want is in there). An example is in `dowsing/_demo_pep517.py`\n\nThis project's `dowsing.pep517` api is designed to do something similar, but not\nfail on missing build-time requirements.\n\n\n# Further Reading\n\n* PEP 241, Metadata 1.0\n* PEP 314, Metadata 1.1\n* PEP 345, Metadata 1.2\n* PEP 566, Metadata 2.1\n* https://packaging.python.org/specifications/core-metadata/\n* https://setuptools.readthedocs.io/en/latest/setuptools.html#metadata\n\n# License\n\ndowsing is copyright [Tim Hatch](http://timhatch.com/), and licensed under\nthe MIT license. I am providing code in this repository to you under an open\nsource license. This is my personal repository; the license you receive to\nmy code is from me and not from my employer. See the `LICENSE` file for details.\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Guess metadata from sdist without running",
"version": "0.8.0",
"project_urls": {
"Homepage": "https://github.com/python-packaging/dowsing/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ec986c421234e829cdd81c6b3ac5664c566de8ea4bd4873a6e5fd28329af7521",
"md5": "ebede6e1f9246f0227cf24c9d08fde15",
"sha256": "f12a4aedcf23ba6f762bd4c64c4b0c90807139959078ed0edea9ac67e8a4a19a"
},
"downloads": -1,
"filename": "dowsing-0.8.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ebede6e1f9246f0227cf24c9d08fde15",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 27449,
"upload_time": "2020-11-10T04:50:06",
"upload_time_iso_8601": "2020-11-10T04:50:06.765249Z",
"url": "https://files.pythonhosted.org/packages/ec/98/6c421234e829cdd81c6b3ac5664c566de8ea4bd4873a6e5fd28329af7521/dowsing-0.8.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "11c4d69c6e1f5db8de84b0a447e75e2e1494b5ff431e4f29c54263ea3d7efb09",
"md5": "acdf8dce30b9261053bd485478503c06",
"sha256": "076ac8c007cda87099a5290aca5f9fb2ca8a312436eca1581126a2528b4e6534"
},
"downloads": -1,
"filename": "dowsing-0.8.0.tar.gz",
"has_sig": false,
"md5_digest": "acdf8dce30b9261053bd485478503c06",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 24763,
"upload_time": "2020-11-10T04:50:08",
"upload_time_iso_8601": "2020-11-10T04:50:08.046746Z",
"url": "https://files.pythonhosted.org/packages/11/c4/d69c6e1f5db8de84b0a447e75e2e1494b5ff431e4f29c54263ea3d7efb09/dowsing-0.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2020-11-10 04:50:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "python-packaging",
"github_project": "dowsing",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "dowsing"
}
JSON
