# typed-lazyimport
Provides type hinted lazy import for slow to load libraries
# Usage
## Simple example and principal behavior
```python
from lazyimport import Libs
torch = Libs.torch # does not trigger torch to be imported, but torch is still a typed variable
print(torch.__version__) # causes torch to be imported and its version is then returned
print(torch.__version__) # returns the same version from the already loaded library
```
## Intended use in a real codebase
```python
from lazyimport import Libs as L
torch = L.torch
def do_something_returning_a_tensor() -> torch.tensor:
return torch.tensor(1)
# do various stuff that don't require torch and doesn't warrant loading it.
if condition:
print(do_something_returning_a_tensor()) # only now do we pay the price for importing torch, since it adds value.
...
```
## Included libraries
The following set of libraries are currently included in `Libs` out of the box, ready for lazy importing (provided you have any of them installed of course).
* `cv2` ([Python OpenCV binding](https://pypi.org/project/opencv-python/))
* `matplotlib`
* `numpy`
* `pandas`
* `pytorch_lightning`
* `sklearn`
* `torch`
* `torchvision`
Check the [source](https://github.com/jojje/typed-lazyimport/blob/0.1.3/lazyimport/lazy_import.py#L82) for the latest set of libraries bundled, since the above list may not always be updated in the readme when new packages are added to the `Libs` _convenience_ library set.
## Extending or lazily loading your own libraries
The provided `Libs` set is offered simply as a convenience, to offer close to zero-friction in order to gain lazy
importing of the most popular libraries. However, if your specific set of libraries isn't included, you can easily
create your own lazy-loaded library set.
Example:
```python
# mylibs.py (for example)
from __future__ import annotations # only needed if you want type hints
from typing import TYPE_CHECKING # -"-
from lazyimport import Lib
if TYPE_CHECKING: # -"-. Skip this block if you don't need type hints.
from PIL import Image
from myown import libs
class Libraries:
Image:Image = Lib('PIL.Image') # the string the same format as "import" requires.
mylibs:libs = Lib('myown.libs')
```
Then you can happily use it with the same ease as the bundled library set:
```python
from mylibs import Libraries as L
flork = L.mylibs.flork
Image = L.Image
flork(Image.open('cat.png'))
```
All type hinted of course.
One point of note is that the type hinting comes from the explicit type annotations used when defining each class variable.
Unfortunately those types are _only_ present when the type-checker runs, and not at program runtime.
This means that the nicer syntax option `Image = lib(Image)` isn't possible, because when the type-checker isn't running,
the type `Image` is never imported from `PIL`, and the program would crash as a result. That's why we have to specify the
import names using a _little_ bit of (horrible) type-unsafe "string programming". No way around it, but it's minimized.
To ensure there's no spelling mistake, my recommendation is to first enter `import <statement>` in the code editor or
interactive python, to verify the string is correct. Then copy the statement verbatim and quote it. That's the
least risky option I found (absent a better python type system).
If you don't want/use any type hints, then creating your own lazy-loaded library set is about as
[DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) and concise as can be.
```python
from lazyimport import Lib
class Libraries:
Image = Lib('PIL.Image')
mylibs = Lib('myown.libs')
```
## Lazy import without library sets
If you want to do some one-off lazy imports, you can use the same proxy primitive the examples up to now have been using
under the hood.
```python
from lazyimport import Import
torch = Import('torch') # no import is performed yet
...
print(torch.tensor(2) ** 2) # now the import happens
```
The downside of this option is that if you want typing, you have to do the typing dance that PEP 484 forces you into,
meaning importing the special annotation related primitives, making use of the type guard with the explicit library imports
within it, then the type declarations etc..
That's the reason this example was provided last and not first, since this package is primarily aimed at offering _typed_
lazy loading. That said, if all you need is to speed up a duck-typed CLI program or similar, there's nothing wrong with
using the lazy loader (proxy) directly, as shown in this last example.
## FAQ
### Q1: Why?
Because many popular libraries are _incredibly_ expensive (slow) to load. Especially ML related libraries.
### Q2: Why another library?
Because none of the ones available was trivial to use, absent of magic string-programming or error-prone dict mapping.
This library provides:
1. Transparent type hinting for a set of popular libraries that are known to be very slow to import.
2. Minimal additional complexity. A single additional import, but otherwise the syntax is pretty much identical to using
normal imports. In particular it allows the programmer to have module wide visibility of the imported types, just like
with a normal import. It chucks away all the PEP 484 boilerplate that otherwise has to be added all across the code
base, just to get some type hints. None of that needed here.
### Q3: Why isn't library X included?
Most likely since I don't use it myself.
The library is first and foremost a convenience library aimed at addressing the slow import of the most popular libraries.
That said, I'm open to adding additional ones. Open an issue, let's discuss and then a PR (since I probably don't have
that library myself).
The cost of adding additional libraries is close to zero, so there's no practical reason for not expanding the set of
libraries provided out of the box in-finitum, other than it becoming harder for users to get an overview of which libraries
are made available.
If you have a special library or libraries you want to lazily load but that's not available in the `Libs` class, you can
easily create your own library set.
See [Extending or lazily loading your own libraries](#extending-or-lazily-loading-your-own-libraries)
as well as the unit-test in this project for an example that does just that. Take a look at the
[lazy_import.py](./lazyimport/lazy_import.py) implementation itself for how to get typing support. It's really trivial
as can be seen by the minimal implementation of this package itself.
### Q4: How can I see when the import happens?
Enable debug logging. `lazy import: <module name>` will be logged whenever a module is imported.
## Development
To run the unit tests
```python
python -m unittest
```
To simplify development, common actions are provided via [Makefile](Makefile) targets:
* test - default targets, runs pytest on the project
* build - create a wheel package distribution, ready to be uploaded to pypi or given to someone else.
* clean - removes temporary files generated as part of the package creation.
Raw data
{
"_id": null,
"home_page": "https://github.com/jojje/typed-lazyimport",
"name": "typed-lazyimport",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.8.1",
"maintainer_email": null,
"keywords": "lazy, import",
"author": "jojje",
"author_email": "tinjon+pypi@gmail.com",
"download_url": null,
"platform": null,
"description": "\n# typed-lazyimport\n\nProvides type hinted lazy import for slow to load libraries\n\n# Usage\n\n## Simple example and principal behavior\n\n```python\nfrom lazyimport import Libs\ntorch = Libs.torch # does not trigger torch to be imported, but torch is still a typed variable\n\nprint(torch.__version__) # causes torch to be imported and its version is then returned\nprint(torch.__version__) # returns the same version from the already loaded library\n```\n\n## Intended use in a real codebase\n\n```python\nfrom lazyimport import Libs as L\ntorch = L.torch\n\ndef do_something_returning_a_tensor() -> torch.tensor:\n return torch.tensor(1)\n\n# do various stuff that don't require torch and doesn't warrant loading it.\nif condition:\n print(do_something_returning_a_tensor()) # only now do we pay the price for importing torch, since it adds value.\n...\n```\n\n## Included libraries\n\nThe following set of libraries are currently included in `Libs` out of the box, ready for lazy importing (provided you have any of them installed of course).\n\n* `cv2` ([Python OpenCV binding](https://pypi.org/project/opencv-python/))\n* `matplotlib`\n* `numpy`\n* `pandas`\n* `pytorch_lightning`\n* `sklearn`\n* `torch`\n* `torchvision`\n\nCheck the [source](https://github.com/jojje/typed-lazyimport/blob/0.1.3/lazyimport/lazy_import.py#L82) for the latest set of libraries bundled, since the above list may not always be updated in the readme when new packages are added to the `Libs` _convenience_ library set.\n\n## Extending or lazily loading your own libraries\n\nThe provided `Libs` set is offered simply as a convenience, to offer close to zero-friction in order to gain lazy\nimporting of the most popular libraries. However, if your specific set of libraries isn't included, you can easily\ncreate your own lazy-loaded library set.\n\nExample:\n\n```python\n# mylibs.py (for example)\nfrom __future__ import annotations # only needed if you want type hints\nfrom typing import TYPE_CHECKING # -\"-\nfrom lazyimport import Lib\n\nif TYPE_CHECKING: # -\"-. Skip this block if you don't need type hints.\n from PIL import Image\n from myown import libs\n\nclass Libraries:\n Image:Image = Lib('PIL.Image') # the string the same format as \"import\" requires.\n mylibs:libs = Lib('myown.libs')\n```\n\nThen you can happily use it with the same ease as the bundled library set:\n\n```python\nfrom mylibs import Libraries as L\nflork = L.mylibs.flork\nImage = L.Image\n\nflork(Image.open('cat.png'))\n```\n\nAll type hinted of course.\n\nOne point of note is that the type hinting comes from the explicit type annotations used when defining each class variable.\nUnfortunately those types are _only_ present when the type-checker runs, and not at program runtime.\nThis means that the nicer syntax option `Image = lib(Image)` isn't possible, because when the type-checker isn't running,\nthe type `Image` is never imported from `PIL`, and the program would crash as a result. That's why we have to specify the\nimport names using a _little_ bit of (horrible) type-unsafe \"string programming\". No way around it, but it's minimized.\n\nTo ensure there's no spelling mistake, my recommendation is to first enter `import <statement>` in the code editor or\ninteractive python, to verify the string is correct. Then copy the statement verbatim and quote it. That's the\nleast risky option I found (absent a better python type system).\n\nIf you don't want/use any type hints, then creating your own lazy-loaded library set is about as\n[DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) and concise as can be.\n\n```python\nfrom lazyimport import Lib\n\nclass Libraries:\n Image = Lib('PIL.Image')\n mylibs = Lib('myown.libs')\n```\n\n## Lazy import without library sets\n\nIf you want to do some one-off lazy imports, you can use the same proxy primitive the examples up to now have been using\nunder the hood.\n\n```python\nfrom lazyimport import Import\n\ntorch = Import('torch') # no import is performed yet\n...\nprint(torch.tensor(2) ** 2) # now the import happens\n```\n\nThe downside of this option is that if you want typing, you have to do the typing dance that PEP 484 forces you into,\nmeaning importing the special annotation related primitives, making use of the type guard with the explicit library imports\nwithin it, then the type declarations etc..\n\nThat's the reason this example was provided last and not first, since this package is primarily aimed at offering _typed_\nlazy loading. That said, if all you need is to speed up a duck-typed CLI program or similar, there's nothing wrong with\nusing the lazy loader (proxy) directly, as shown in this last example.\n\n## FAQ\n\n### Q1: Why?\n\nBecause many popular libraries are _incredibly_ expensive (slow) to load. Especially ML related libraries.\n\n### Q2: Why another library?\n\nBecause none of the ones available was trivial to use, absent of magic string-programming or error-prone dict mapping.\n\nThis library provides:\n\n1. Transparent type hinting for a set of popular libraries that are known to be very slow to import.\n2. Minimal additional complexity. A single additional import, but otherwise the syntax is pretty much identical to using \n normal imports. In particular it allows the programmer to have module wide visibility of the imported types, just like\n with a normal import. It chucks away all the PEP 484 boilerplate that otherwise has to be added all across the code\n base, just to get some type hints. None of that needed here.\n\n### Q3: Why isn't library X included?\n\nMost likely since I don't use it myself.\n\nThe library is first and foremost a convenience library aimed at addressing the slow import of the most popular libraries. \nThat said, I'm open to adding additional ones. Open an issue, let's discuss and then a PR (since I probably don't have\nthat library myself).\n\nThe cost of adding additional libraries is close to zero, so there's no practical reason for not expanding the set of\nlibraries provided out of the box in-finitum, other than it becoming harder for users to get an overview of which libraries\nare made available.\n\nIf you have a special library or libraries you want to lazily load but that's not available in the `Libs` class, you can\neasily create your own library set. \nSee [Extending or lazily loading your own libraries](#extending-or-lazily-loading-your-own-libraries) \nas well as the unit-test in this project for an example that does just that. Take a look at the \n[lazy_import.py](./lazyimport/lazy_import.py) implementation itself for how to get typing support. It's really trivial\nas can be seen by the minimal implementation of this package itself.\n\n### Q4: How can I see when the import happens?\n\nEnable debug logging. `lazy import: <module name>` will be logged whenever a module is imported.\n\n\n## Development\n\nTo run the unit tests\n\n```python\npython -m unittest\n```\n\nTo simplify development, common actions are provided via [Makefile](Makefile) targets:\n\n* test - default targets, runs pytest on the project\n* build - create a wheel package distribution, ready to be uploaded to pypi or given to someone else.\n* clean - removes temporary files generated as part of the package creation.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Provides type hinted lazy import for slow to load libraries",
"version": "0.3.1",
"project_urls": {
"Bug Tracker": "https://github.com/jojje/typed-lazyimport/issues",
"Homepage": "https://github.com/jojje/typed-lazyimport"
},
"split_keywords": [
"lazy",
" import"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "04dd6c98237e8f4f98653f913ee3c4393c9fe94423c257726073e66165d66b7c",
"md5": "f80c6e6a192d0ed5ce5c916a912a3144",
"sha256": "cb24ea4baa925ee32f36105088cf74c30959ee748a3845a0f4c2d9cbfc6d4f72"
},
"downloads": -1,
"filename": "typed_lazyimport-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f80c6e6a192d0ed5ce5c916a912a3144",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.8.1",
"size": 6760,
"upload_time": "2024-07-01T23:11:03",
"upload_time_iso_8601": "2024-07-01T23:11:03.623263Z",
"url": "https://files.pythonhosted.org/packages/04/dd/6c98237e8f4f98653f913ee3c4393c9fe94423c257726073e66165d66b7c/typed_lazyimport-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-01 23:11:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jojje",
"github_project": "typed-lazyimport",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "typed-lazyimport"
}
JSON
