pyrollup


Namepyrollup JSON
Version 0.1.0 PyPI version JSON
download
home_pagehttps://github.com/mm21/pyrollup
SummarySimple mechanism to rollup API symbols to a Python module from its submodules
upload_time2023-11-21 02:02:26
maintainer
docs_urlNone
authormm21
requires_python>=3.10,<4.0
license
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # pyrollup
Simple mechanism to rollup API symbols to a Python module from its submodules

## Background

Conventionally, a Python package's public API symbols (or at least a subset) are exported from the top-level package. These symbols are typically imported from various submodules in the module hierarchy. This leads to a rather tedious listing of symbols exported (via a module's `__init__.py`) at each level.

For example:

```python
from .submodule_a import ClassA, ClassA2
from .submodule_b import ClassB
from .submodule_c import ClassC

__all__ = [
    ClassA,
    ClassA2
    ClassB,
    ClassC,
]
```

The definition of `__all__` can be omitted, although some schools of thought encourage its use.

## Problem statement

With the above approach, the burden of which symbols are considered public for a given submodule falls on the containing module importing them. In addition, `__all__` must be maintained alongside symbol imports, potentially resulting in duplication.

This can become problematic for complex projects with many nested symbols imported at the top-level package. For example:

```python
from .submodule_a import ClassA, ClassA2
from .submodule_a.submodule_a1 import ClassA1_1, ClassA1_2
from .submodule_a.submodule_a2 import ClassA2_1, ClassA2_2
from .submodule_a.submodule_a3 import ClassA3_1, ClassA3_2
from .submodule_b import ClassB
from .submodule_c import ClassC

__all__ = [
    ClassA,
    ClassA2,
    ClassA1_1,
    ClassA1_2,
    ClassA2_1,
    ClassA2_2,
    ClassA3_1,
    ClassA3_2,
    ClassB,
    ClassC,
]
```

In addition, submodules must similarly maintain their own imports and/or definition of `__all__` if they export symbols from their submodules.

## Proposed solution

Ideally, a module should "rollup" public symbols from its submodules, letting each submodule decide which symbols those are. Submodules should therefore optionally provide an allow-list and block-list to describe which public symbols should be propagated to the parent module.

In other words, a given module should have ownership of:

- Its own public symbols (Python convention)
    - Defined by `__all__`
- Which of those public symbols should be propagated to parent modules (functionality provided by `pyrollup`)
    - Defined by
        - `__rollup__`: allow-list, defaulting to `__all__`
        - `__nrollup__`: block-list, defaulting to `[]`

Then, the example above can be modified as:

```python
from pyrollup import rollup

# import submodules
from . import submodule_a, submodule_b, submodule_c

# import public symbols from each submodule
from .submodule_a import *
from .submodule_b import *
from .submodule_c import *

# export public symbols from each submodule, filtered by allow-list/block-list
__all__ = rollup(submodule_a, submodule_b, submodule_c)
```

This allows a project with a complex module hierarchy to flexibly propagate public symbols from wherever they are defined to the top-level package.

## Downsides

Static analysis tools are unable to evaluate the value of `__all__` since with this approach it is computed dynamically upon import. Therefore, code documentation generator tools using static analysis (e.g. [autoapi](https://github.com/readthedocs/sphinx-autoapi) or [autodoc2](https://github.com/sphinx-extensions2/sphinx-autodoc2)) will fail to detect a module's public symbols.

Tools using the traditional approach of importing the package for which documentation is being generated (e.g. Sphinx's built-in [autodoc](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html)) should have no problem with such dynamic imports.

It is possible to use a "hybrid" approach; the author has had some success with extending [autodoc2](https://github.com/sphinx-extensions2/sphinx-autodoc2) to work with dynamically-evaluated `__all__`. 

TODO: document and generalize this solution

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/mm21/pyrollup",
    "name": "pyrollup",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.10,<4.0",
    "maintainer_email": "",
    "keywords": "",
    "author": "mm21",
    "author_email": "mm21.apps@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/d7/27/79fd7b38ba488482ed0813db78b6e0e49fde9417c967c755ca3800655f6e/pyrollup-0.1.0.tar.gz",
    "platform": null,
    "description": "# pyrollup\nSimple mechanism to rollup API symbols to a Python module from its submodules\n\n## Background\n\nConventionally, a Python package's public API symbols (or at least a subset) are exported from the top-level package. These symbols are typically imported from various submodules in the module hierarchy. This leads to a rather tedious listing of symbols exported (via a module's `__init__.py`) at each level.\n\nFor example:\n\n```python\nfrom .submodule_a import ClassA, ClassA2\nfrom .submodule_b import ClassB\nfrom .submodule_c import ClassC\n\n__all__ = [\n    ClassA,\n    ClassA2\n    ClassB,\n    ClassC,\n]\n```\n\nThe definition of `__all__` can be omitted, although some schools of thought encourage its use.\n\n## Problem statement\n\nWith the above approach, the burden of which symbols are considered public for a given submodule falls on the containing module importing them. In addition, `__all__` must be maintained alongside symbol imports, potentially resulting in duplication.\n\nThis can become problematic for complex projects with many nested symbols imported at the top-level package. For example:\n\n```python\nfrom .submodule_a import ClassA, ClassA2\nfrom .submodule_a.submodule_a1 import ClassA1_1, ClassA1_2\nfrom .submodule_a.submodule_a2 import ClassA2_1, ClassA2_2\nfrom .submodule_a.submodule_a3 import ClassA3_1, ClassA3_2\nfrom .submodule_b import ClassB\nfrom .submodule_c import ClassC\n\n__all__ = [\n    ClassA,\n    ClassA2,\n    ClassA1_1,\n    ClassA1_2,\n    ClassA2_1,\n    ClassA2_2,\n    ClassA3_1,\n    ClassA3_2,\n    ClassB,\n    ClassC,\n]\n```\n\nIn addition, submodules must similarly maintain their own imports and/or definition of `__all__` if they export symbols from their submodules.\n\n## Proposed solution\n\nIdeally, a module should \"rollup\" public symbols from its submodules, letting each submodule decide which symbols those are. Submodules should therefore optionally provide an allow-list and block-list to describe which public symbols should be propagated to the parent module.\n\nIn other words, a given module should have ownership of:\n\n- Its own public symbols (Python convention)\n    - Defined by `__all__`\n- Which of those public symbols should be propagated to parent modules (functionality provided by `pyrollup`)\n    - Defined by\n        - `__rollup__`: allow-list, defaulting to `__all__`\n        - `__nrollup__`: block-list, defaulting to `[]`\n\nThen, the example above can be modified as:\n\n```python\nfrom pyrollup import rollup\n\n# import submodules\nfrom . import submodule_a, submodule_b, submodule_c\n\n# import public symbols from each submodule\nfrom .submodule_a import *\nfrom .submodule_b import *\nfrom .submodule_c import *\n\n# export public symbols from each submodule, filtered by allow-list/block-list\n__all__ = rollup(submodule_a, submodule_b, submodule_c)\n```\n\nThis allows a project with a complex module hierarchy to flexibly propagate public symbols from wherever they are defined to the top-level package.\n\n## Downsides\n\nStatic analysis tools are unable to evaluate the value of `__all__` since with this approach it is computed dynamically upon import. Therefore, code documentation generator tools using static analysis (e.g. [autoapi](https://github.com/readthedocs/sphinx-autoapi) or [autodoc2](https://github.com/sphinx-extensions2/sphinx-autodoc2)) will fail to detect a module's public symbols.\n\nTools using the traditional approach of importing the package for which documentation is being generated (e.g. Sphinx's built-in [autodoc](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html)) should have no problem with such dynamic imports.\n\nIt is possible to use a \"hybrid\" approach; the author has had some success with extending [autodoc2](https://github.com/sphinx-extensions2/sphinx-autodoc2) to work with dynamically-evaluated `__all__`. \n\nTODO: document and generalize this solution\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "Simple mechanism to rollup API symbols to a Python module from its submodules",
    "version": "0.1.0",
    "project_urls": {
        "Homepage": "https://github.com/mm21/pyrollup"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c38f4375e0fbf96962aca89243ee0661956972bf8350c283acc13ec4a6dd7918",
                "md5": "7c997e695f8b9f1f8150577c3ce5b6e0",
                "sha256": "a2f73b823e251f6e3c9e14dfe7ce43323824a62fe5fc66d5e6221db6d9fae6ba"
            },
            "downloads": -1,
            "filename": "pyrollup-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "7c997e695f8b9f1f8150577c3ce5b6e0",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10,<4.0",
            "size": 3807,
            "upload_time": "2023-11-21T02:02:24",
            "upload_time_iso_8601": "2023-11-21T02:02:24.091415Z",
            "url": "https://files.pythonhosted.org/packages/c3/8f/4375e0fbf96962aca89243ee0661956972bf8350c283acc13ec4a6dd7918/pyrollup-0.1.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d72779fd7b38ba488482ed0813db78b6e0e49fde9417c967c755ca3800655f6e",
                "md5": "4987b1a23b3d31ebdfa47ef7d21a4dea",
                "sha256": "388c152b20b5467d2378462dcc6bac121db3bd98ef1902874caa0e30b24a0f21"
            },
            "downloads": -1,
            "filename": "pyrollup-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "4987b1a23b3d31ebdfa47ef7d21a4dea",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10,<4.0",
            "size": 3420,
            "upload_time": "2023-11-21T02:02:26",
            "upload_time_iso_8601": "2023-11-21T02:02:26.087337Z",
            "url": "https://files.pythonhosted.org/packages/d7/27/79fd7b38ba488482ed0813db78b6e0e49fde9417c967c755ca3800655f6e/pyrollup-0.1.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-11-21 02:02:26",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "mm21",
    "github_project": "pyrollup",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "pyrollup"
}
        
Elapsed time: 0.13892s