[](https://bestpractices.coreinfrastructure.org/projects/6907)
# Import Tracker
`Import Tracker` is a Python package offering a number of capabilities related to tracking and managing optional dependencies in Python projects. Specifically, this project enables developers to:
- Track the dependencies of a python project to map each module within the project to the set of dependencies it relies on. This can be useful for debugging of dependencies in large projects.
- Enable lazy import errors in a python projects to prevent code from crashing when uninstalled imports are imported, but not utilized. This can be helpful in large projects, especially those which incorporate lots of hierarchical wild imports, as importing the top level package of such projects can often bring a lot of heavy dependencies into `sys.modules`.
- Programmatically determine the [`install_requires`](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency) and [`extras_require`](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies) arguments to `setuptools.setup` where the extras sets are determined by a set of modules that should be optional.
## Table of contents
<!-- @import "[TOC]" {cmd="toc" depthFrom=2 depthTo=6 orderedList=false} -->
<!-- code_chunk_output -->
- [Table of contents](#table-of-contents)
- [Running Import Tracker](#running-import-tracker)
- [Integrating `import_tracker` into a project](#integrating-import_tracker-into-a-project)
- [Enabling `lazy_import_errors`](#enabling-lazy_import_errors)
- [Using `setup_tools.parse_requirements`](#using-setup_toolsparse_requirements)
- [Gotchas](#gotchas)
- [Minor issue with zsh](#minor-issue-with-zsh)
<!-- /code_chunk_output -->
## Running Import Tracker
To run `import_tracker` against a project, simply invoke the module's main:
```
python3 -m import_tracker --name <my_module>
```
The main supports the following additional arguments:
- `--package`: Allows `--name` to be a relative import (see [`importlib.import_module`](https://docs.python.org/3/library/importlib.html#importlib.import_module))
- `--indent`: Indent the output json for pretty printing
- `--log_level`: Set the level of logging (up to `debug4`) to debug unexpected behavior
- `--submodules`: List of sub-modules to recurse on (or full recursion when no args given)
- `--track_import_stack`: Store the stack trace of imports belonging to the tracked module
- `--detect_transitive`: Mark each dependency as either "direct" (imported directly) or "transitive" (inherited from a direct import)
- `--full_depth`: Track all dependencies, including transitive dependencies of direct third-party deps
- `--show_optional`: Show whether each dependency is optional or required
## Integrating `import_tracker` into a project
When using `import_tracker` to implement optional dependencies in a project, there are two steps to take:
1. Enable `lazy_import_errors` for the set of modules that should be managed as optional
2. Use `setup_tools.parse_requirements` in `setup.py` to determine the `install_requires` and `extras_require` arguments
In the following examples, we'll use a fictitious project with the following structure:
```
my_module/
├── __init__.py
├── utils.py
└── widgets
├── __init__.py
├── widget1.py
└── widget2.py
```
### Enabling `lazy_import_errors`
The `import_tracker.lazy_import_errors` function can be invoked directly to enable lazy import errors globally, or used as a context manager to enable them only for a selcted set of modules.
To globally enable lazy import errors, `my_module/__init__.py` would look like the following:
```py
# Globally enable lazy import errors
from import_tracker import lazy_import_errors
lazy_import_errors()
from . import utils, widgets
```
Alternately, applying lazy import error semantics only to the `widgets` would look like the following:
```py
from import_tracker import lazy_import_errors
# Require all downstream imports from utils to exist
from . import utils
# Enable lazy import errors for widgets
with lazy_import_errors():
from . import widgets
```
When using lazy import errors, there are two ways to customize the error message that is raised when a failed import is used:
1. 1. The `get_extras_modules` argument takes a function which returns a `Set[str]` of the module names that are tracked as extras. If the import error is triggered within a module that is managed as an extras set, the error message is updated to include instructions on which extras set needs to be installed.
2. The `make_error_message` argument allows the caller to specify a fully custom error message generation function.
### Using `setup_tools.parse_requirements`
To take advantage of the automatic dependency parsing when building a package, the `setup.py` would look like the following:
```py
import import_tracker
import os
import setuptools
# Determine the path to the requirements.txt for the project
requirements_file = os.path.join(os.path.dirname(__file__), "requirements.txt")
# Parse the requirement sets
install_requires, extras_require = import_tracker.setup_tools.parse_requirements(
requirements_file=requirements_file,
library_name="my_module",
extras_modules=[
"my_module.widgets.widget1",
"my_module.widgets.widget2",
],
)
# Perform the standard setup call
setuptools.setup(
name="my_module",
author="me",
version="1.2.3",
license="MIT",
install_requires=install_requires,
extras_require=extras_require,
packages=setuptools.find_packages(),
)
```
## Gotchas
### Minor issue with zsh
As mentioned before, when using lazy import errors in `import_tracker`, if the import error is triggered within a module that is managed as an extras set, the error message is updated to include instructions on which extras set needs to be installed. The error message might look something like this:
```bash
ModuleNotFoundError: No module named 'example_module'.
To install the missing dependencies, run `pip install my_module[my_module.example_module]`
```
There might be an issue when running `pip install my_module[my_module.example_module]` within a `zsh` environment, since square brackets in `zsh` have special meanings. We have to escape them by putting \ (`backslash`) before them. So for `zsh`, something like this will work:
```
pip install my_module\[my_module.example_module\]
```
Raw data
{
"_id": null,
"home_page": "https://github.com/IBM/import-tracker",
"name": "import-tracker",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "import,importlib,dependencies",
"author": "Gabe Goodhart",
"author_email": "gabe.l.hart@gmail.com",
"download_url": "",
"platform": null,
"description": "[](https://bestpractices.coreinfrastructure.org/projects/6907)\n\n# Import Tracker\n\n`Import Tracker` is a Python package offering a number of capabilities related to tracking and managing optional dependencies in Python projects. Specifically, this project enables developers to:\n\n- Track the dependencies of a python project to map each module within the project to the set of dependencies it relies on. This can be useful for debugging of dependencies in large projects.\n\n- Enable lazy import errors in a python projects to prevent code from crashing when uninstalled imports are imported, but not utilized. This can be helpful in large projects, especially those which incorporate lots of hierarchical wild imports, as importing the top level package of such projects can often bring a lot of heavy dependencies into `sys.modules`.\n\n- Programmatically determine the [`install_requires`](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency) and [`extras_require`](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies) arguments to `setuptools.setup` where the extras sets are determined by a set of modules that should be optional.\n\n## Table of contents\n\n<!-- @import \"[TOC]\" {cmd=\"toc\" depthFrom=2 depthTo=6 orderedList=false} -->\n\n<!-- code_chunk_output -->\n\n- [Table of contents](#table-of-contents)\n- [Running Import Tracker](#running-import-tracker)\n- [Integrating `import_tracker` into a project](#integrating-import_tracker-into-a-project)\n - [Enabling `lazy_import_errors`](#enabling-lazy_import_errors)\n - [Using `setup_tools.parse_requirements`](#using-setup_toolsparse_requirements)\n- [Gotchas](#gotchas)\n - [Minor issue with zsh](#minor-issue-with-zsh)\n\n<!-- /code_chunk_output -->\n\n## Running Import Tracker\n\nTo run `import_tracker` against a project, simply invoke the module's main:\n\n```\npython3 -m import_tracker --name <my_module>\n```\n\nThe main supports the following additional arguments:\n\n- `--package`: Allows `--name` to be a relative import (see [`importlib.import_module`](https://docs.python.org/3/library/importlib.html#importlib.import_module))\n- `--indent`: Indent the output json for pretty printing\n- `--log_level`: Set the level of logging (up to `debug4`) to debug unexpected behavior\n- `--submodules`: List of sub-modules to recurse on (or full recursion when no args given)\n- `--track_import_stack`: Store the stack trace of imports belonging to the tracked module\n- `--detect_transitive`: Mark each dependency as either \"direct\" (imported directly) or \"transitive\" (inherited from a direct import)\n- `--full_depth`: Track all dependencies, including transitive dependencies of direct third-party deps\n- `--show_optional`: Show whether each dependency is optional or required\n\n## Integrating `import_tracker` into a project\n\nWhen using `import_tracker` to implement optional dependencies in a project, there are two steps to take:\n\n1. Enable `lazy_import_errors` for the set of modules that should be managed as optional\n2. Use `setup_tools.parse_requirements` in `setup.py` to determine the `install_requires` and `extras_require` arguments\n\nIn the following examples, we'll use a fictitious project with the following structure:\n\n```\nmy_module/\n\u251c\u2500\u2500 __init__.py\n\u251c\u2500\u2500 utils.py\n\u2514\u2500\u2500 widgets\n \u251c\u2500\u2500 __init__.py\n \u251c\u2500\u2500 widget1.py\n \u2514\u2500\u2500 widget2.py\n```\n\n### Enabling `lazy_import_errors`\n\nThe `import_tracker.lazy_import_errors` function can be invoked directly to enable lazy import errors globally, or used as a context manager to enable them only for a selcted set of modules.\n\nTo globally enable lazy import errors, `my_module/__init__.py` would look like the following:\n\n```py\n# Globally enable lazy import errors\nfrom import_tracker import lazy_import_errors\nlazy_import_errors()\n\nfrom . import utils, widgets\n```\n\nAlternately, applying lazy import error semantics only to the `widgets` would look like the following:\n\n```py\nfrom import_tracker import lazy_import_errors\n\n# Require all downstream imports from utils to exist\nfrom . import utils\n\n# Enable lazy import errors for widgets\nwith lazy_import_errors():\n from . import widgets\n```\n\nWhen using lazy import errors, there are two ways to customize the error message that is raised when a failed import is used:\n\n1. 1. The `get_extras_modules` argument takes a function which returns a `Set[str]` of the module names that are tracked as extras. If the import error is triggered within a module that is managed as an extras set, the error message is updated to include instructions on which extras set needs to be installed.\n\n2. The `make_error_message` argument allows the caller to specify a fully custom error message generation function.\n\n### Using `setup_tools.parse_requirements`\n\nTo take advantage of the automatic dependency parsing when building a package, the `setup.py` would look like the following:\n\n```py\nimport import_tracker\nimport os\nimport setuptools\n\n# Determine the path to the requirements.txt for the project\nrequirements_file = os.path.join(os.path.dirname(__file__), \"requirements.txt\")\n\n# Parse the requirement sets\ninstall_requires, extras_require = import_tracker.setup_tools.parse_requirements(\n requirements_file=requirements_file,\n library_name=\"my_module\",\n extras_modules=[\n \"my_module.widgets.widget1\",\n \"my_module.widgets.widget2\",\n ],\n)\n\n# Perform the standard setup call\nsetuptools.setup(\n name=\"my_module\",\n author=\"me\",\n version=\"1.2.3\",\n license=\"MIT\",\n install_requires=install_requires,\n extras_require=extras_require,\n packages=setuptools.find_packages(),\n)\n```\n\n## Gotchas\n\n### Minor issue with zsh\n\nAs mentioned before, when using lazy import errors in `import_tracker`, if the import error is triggered within a module that is managed as an extras set, the error message is updated to include instructions on which extras set needs to be installed. The error message might look something like this:\n\n```bash\nModuleNotFoundError: No module named 'example_module'.\n\nTo install the missing dependencies, run `pip install my_module[my_module.example_module]`\n\n```\n\nThere might be an issue when running `pip install my_module[my_module.example_module]` within a `zsh` environment, since square brackets in `zsh` have special meanings. We have to escape them by putting \\ (`backslash`) before them. So for `zsh`, something like this will work:\n\n```\npip install my_module\\[my_module.example_module\\]\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A tool for managing dependencies in a modular python project by tracking which dependencies are needed by which sub-modules",
"version": "3.2.1",
"project_urls": {
"Homepage": "https://github.com/IBM/import-tracker"
},
"split_keywords": [
"import",
"importlib",
"dependencies"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "37bbb43bb7f702e616db5ca5beb8f098a4d01145caaaad4c54c856e7263ced95",
"md5": "b0882ccd98c24d12190cb2dbe6870427",
"sha256": "3b408c85f3e2da4f77d4c5f4b9d79c1cdc2dd270ecbb213fbd83aef168047b1a"
},
"downloads": -1,
"filename": "import_tracker-3.2.1-py310-none-any.whl",
"has_sig": false,
"md5_digest": "b0882ccd98c24d12190cb2dbe6870427",
"packagetype": "bdist_wheel",
"python_version": "py310",
"requires_python": null,
"size": 23171,
"upload_time": "2024-02-20T18:17:05",
"upload_time_iso_8601": "2024-02-20T18:17:05.671693Z",
"url": "https://files.pythonhosted.org/packages/37/bb/b43bb7f702e616db5ca5beb8f098a4d01145caaaad4c54c856e7263ced95/import_tracker-3.2.1-py310-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7eb46b1a775882193edc6dc2f6f7f0c87e2e8ccf9c9b98af12e085c91a8a4956",
"md5": "cae59ba2fd09246adbd7843125974cc0",
"sha256": "3746167218bebf9c595167c7e91e67dcda9c158f7fdff9557c429a89e582bed8"
},
"downloads": -1,
"filename": "import_tracker-3.2.1-py311-none-any.whl",
"has_sig": false,
"md5_digest": "cae59ba2fd09246adbd7843125974cc0",
"packagetype": "bdist_wheel",
"python_version": "py311",
"requires_python": null,
"size": 23171,
"upload_time": "2024-02-20T18:17:05",
"upload_time_iso_8601": "2024-02-20T18:17:05.744761Z",
"url": "https://files.pythonhosted.org/packages/7e/b4/6b1a775882193edc6dc2f6f7f0c87e2e8ccf9c9b98af12e085c91a8a4956/import_tracker-3.2.1-py311-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7b5039394835c556bab01d9bbaf1dee86b4dc9f8170ef5426fbdee0f8668caa9",
"md5": "d7c0df20eaf32493664485dc705a36cc",
"sha256": "daccbf23fc36265eec0c2d42c94ec1bb074a9bd50fc3143c51d9ef77501e08fb"
},
"downloads": -1,
"filename": "import_tracker-3.2.1-py312-none-any.whl",
"has_sig": false,
"md5_digest": "d7c0df20eaf32493664485dc705a36cc",
"packagetype": "bdist_wheel",
"python_version": "py312",
"requires_python": null,
"size": 23173,
"upload_time": "2024-02-20T18:17:09",
"upload_time_iso_8601": "2024-02-20T18:17:09.511515Z",
"url": "https://files.pythonhosted.org/packages/7b/50/39394835c556bab01d9bbaf1dee86b4dc9f8170ef5426fbdee0f8668caa9/import_tracker-3.2.1-py312-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b2214cc19c331cfac18a64a106c819f401930d3353e0db42d55c83fbeda2979c",
"md5": "3fa17fed01f8b9f9823cf76808e4578a",
"sha256": "f353204e93d720ca030cd91cac022f7af57a1674f26b95b01d42ab9f2174b232"
},
"downloads": -1,
"filename": "import_tracker-3.2.1-py37-none-any.whl",
"has_sig": false,
"md5_digest": "3fa17fed01f8b9f9823cf76808e4578a",
"packagetype": "bdist_wheel",
"python_version": "py37",
"requires_python": null,
"size": 23185,
"upload_time": "2024-02-20T18:17:09",
"upload_time_iso_8601": "2024-02-20T18:17:09.619470Z",
"url": "https://files.pythonhosted.org/packages/b2/21/4cc19c331cfac18a64a106c819f401930d3353e0db42d55c83fbeda2979c/import_tracker-3.2.1-py37-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "37f9351a92c4e9c229eb96aa7b165a18cfa3dcf6644dff8566c664ea1e7bc7f8",
"md5": "6c4ee27ae79f4aa5662184eb548c6358",
"sha256": "8a09784c7575448159ba684f8f2a139bb0430f723ad66b2bc29f53ed62985b3d"
},
"downloads": -1,
"filename": "import_tracker-3.2.1-py38-none-any.whl",
"has_sig": false,
"md5_digest": "6c4ee27ae79f4aa5662184eb548c6358",
"packagetype": "bdist_wheel",
"python_version": "py38",
"requires_python": null,
"size": 23186,
"upload_time": "2024-02-20T18:17:10",
"upload_time_iso_8601": "2024-02-20T18:17:10.682114Z",
"url": "https://files.pythonhosted.org/packages/37/f9/351a92c4e9c229eb96aa7b165a18cfa3dcf6644dff8566c664ea1e7bc7f8/import_tracker-3.2.1-py38-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3914f6a25b274aeac8bb43a52ca0817bfb368f050af2e574e1bf16b63d2c6723",
"md5": "faaf88c70634344cd956f99be25cd679",
"sha256": "785a8effc9ba4a66ea924233e4f228749b60ca7f8f33776c553a59e51baf3c48"
},
"downloads": -1,
"filename": "import_tracker-3.2.1-py39-none-any.whl",
"has_sig": false,
"md5_digest": "faaf88c70634344cd956f99be25cd679",
"packagetype": "bdist_wheel",
"python_version": "py39",
"requires_python": null,
"size": 23186,
"upload_time": "2024-02-20T18:17:10",
"upload_time_iso_8601": "2024-02-20T18:17:10.828878Z",
"url": "https://files.pythonhosted.org/packages/39/14/f6a25b274aeac8bb43a52ca0817bfb368f050af2e574e1bf16b63d2c6723/import_tracker-3.2.1-py39-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-20 18:17:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "IBM",
"github_project": "import-tracker",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "import-tracker"
}
JSON
