mkinit
======
|CircleCI| |Appveyor| |Codecov| |Pypi| |Downloads| |ReadTheDocs|
+------------------+--------------------------------------------+
| Read the docs | https://mkinit.readthedocs.io |
+------------------+--------------------------------------------+
| Github | https://github.com/Erotemic/mkinit |
+------------------+--------------------------------------------+
| Pypi | https://pypi.org/project/mkinit |
+------------------+--------------------------------------------+
The ``mkinit`` module helps you write ``__init__`` files that expose all submodule
attributes without ``from ? import *``.
``mkinit`` automatically imports all submodules in a package and their members.
It can do this dynamically, or it can statically autogenerate the ``__init__``
for faster import times. Its kinda like using the ``fromimport *`` syntax, but
its easy to replace with text that wont make other developers lose their hair.
This module supports Scientific Python `SPEC1 <https://scientific-python.org/specs/spec-0001/>`_.
Also note that the docs in this readme are somewhat old, and need to be updated
to make best practices more clear. There are a lot of ways you can use the
module, but the current recommended way is to use:
.. code:: bash
mkinit --lazy_loader <path-to-init.py>
Installation
============
.. code:: bash
pip install mkinit
The Pitch
---------
Say you have a python module structured like so:
.. code::
└── mkinit_demo_pkg
├── __init__.py
├── submod.py
└── subpkg
├── __init__.py
└── nested.py
And you would like to make all functions inside of ``submod.py`` and
``nested.py`` available at the top-level of the package.
Imagine the contents of submod.py and nested.py are:
.. code:: python
# --- submod.py ---
def submod_func():
print('This is a submod func in {}'.format(__file__))
# --- nested.py ---
def nested_func():
print('This is a nested func in {}'.format(__file__))
You could manually write:
.. code:: python
from mkinit_demo_pkg.submod import *
from mkinit_demo_pkg.subpkg.nested import *
But that has a few problems. Using ``import *`` makes it hard for people
reading the code to know what is coming from where. Furthermore, if there were
many submodules you wanted to expose attributes of, writing this would become
tedious and hard to maintain.
Enter the mkinit package. It has the ability to autogenerate explicit ``__init__.py``
files using static analysis. Normally, the mkinit CLI only works on one file at
a time, but if we specify the ``--recursive`` flag, then mkinit will
recursively generate ``__init__.py`` files for all subpackages in the package.
Thus running ``mkinit mkinit_demo_pkg --recursive`` will result in a root
``__init__.py`` file that looks like this:
.. code:: python
from mkinit_demo_pkg import submod
from mkinit_demo_pkg import subpkg
from mkinit_demo_pkg.submod import (submod_func,)
from mkinit_demo_pkg.subpkg import (nested, nested_func,)
__all__ = ['nested', 'nested_func', 'submod', 'submod_func', 'subpkg']
That's pretty cool. The mkinit package was able to recursively parse our
package, find all of the defined names, and then generate ``__init__.py`` files
such that all attributes are exposed at the top level of the package.
Furthermore, this file is **readable**. It is perfectly clear exactly what
names are exposed in this module without having to execute anything.
Of course, this isn't a perfect solution. Perhaps only some submodules should
be exposed, perhaps you would rather use relative import statements, maybe you
only want to expose submodule but not their attributes, or vis-versa. Well good
news, because mkinit has command line flags that allow for all of these modes.
See ``mkinit --help`` for more details.
Lastly, while exposing all attributes can be helpful for larger projects,
import time can start to become a consideration. Thankfully,
`PEP 0562 <https://peps.python.org/pep-0562/>`_ outlines
a lazy import specification for Python >= 3.7. As of 2020-12-26 mkinit
supports autogenerating these lazy init files.
Unfortunately, there is no syntax support for lazy imports, so mkinit must
define a ``lazy_import`` boilerplate function in each ``__init__.py`` file.
.. code:: python
def lazy_import(module_name, submodules, submod_attrs):
"""
Boilerplate to define PEP 562 __getattr__ for lazy import
https://www.python.org/dev/peps/pep-0562/
"""
import importlib
import os
name_to_submod = {
func: mod for mod, funcs in submod_attrs.items()
for func in funcs
}
def __getattr__(name):
if name in submodules:
attr = importlib.import_module(
'{module_name}.{name}'.format(
module_name=module_name, name=name)
)
elif name in name_to_submod:
submodname = name_to_submod[name]
module = importlib.import_module(
'{module_name}.{submodname}'.format(
module_name=module_name, submodname=submodname)
)
attr = getattr(module, name)
else:
raise AttributeError(
'No {module_name} attribute {name}'.format(
module_name=module_name, name=name))
globals()[name] = attr
return attr
if os.environ.get('EAGER_IMPORT', ''):
for name in submodules:
__getattr__(name)
for attrs in submod_attrs.values():
for attr in attrs:
__getattr__(attr)
return __getattr__
__getattr__ = lazy_import(
__name__,
submodules={
'submod',
'subpkg',
},
submod_attrs={
'submod': [
'submod_func',
],
'subpkg': [
'nested',
'nested_func',
],
},
)
def __dir__():
return __all__
__all__ = ['nested', 'nested_func', 'submod', 'submod_func', 'subpkg']
Although if you are willing to depend on the
`lazy_loader <https://pypi.org/project/lazy_loader/>`_
package and the ``--lazy_loader`` option (new as of 1.0.0), then this
boilerplate is no longer needed.
By default, lazy imports are not compatibly with statically typed projects (e.g
using mypy or pyright), however, if the
`lazy_loader <https://pypi.org/project/lazy_loader/>`_
package is used, the ``--lazy_loader_typed`` option can be specified to generate
``__init.pyi__`` files in addition to lazily evaulated ``__init.py__`` files.
These interface files are understood by static type checkers and allow the
combination of lazy loading with static type checking.
Command Line Usage
------------------
The following command will statically autogenerate an ``__init__`` file in the
specified path or module name. If one exists, it will only replace text after
the final comment. This means ``mkinit`` wont clobber your custom logic and can
be used to help maintain customized ``__init__.py`` files.
.. code:: bash
mkinit <your_modname_or_modpath> -w
You can also enclose the area allowed to be clobbered in the auto-generation
with special xml-like comments.
Running ``mkint --help`` displays:
.. code::
usage: python -m mkinit [-h] [--dry] [-i] [--diff] [--noattrs] [--nomods] [--noall] [--relative] [--lazy | --lazy_loader] [--black] [--lazy_boilerplate LAZY_BOILERPLATE] [--recursive] [--norespect_all]
[--verbose [VERBOSE]] [--version]
[modname_or_path]
Autogenerate an `__init__.py` that exposes a top-level API.
Behavior is modified depending on the existing content of the
`__init__.py` file (subsequent runs of mkinit are idempotent).
The following `__init__.py` variables modify autogeneration behavior:
`__submodules__` (List[str] | Dict[str, List[str])) -
Indicates the list of submodules to be introspected, if
unspecified all submodules are introspected. Can be a list
of submodule names, or a dictionary mapping each submodule name
to a list of attribute names to expose. If the value is None,
then all attributes are exposed (or __all__) is respected).
`__external__` - Specify external modules to expose the attributes of.
`__explicit__` - Add custom explicitly defined names to this, and
they will be automatically added to the __all__ variable.
`__protected__` - Protected modules are exposed, but their attributes are not.
`__private__` - Private modules and their attributes are not exposed.
`__ignore__` - Tells mkinit to ignore particular attributes
positional arguments:
modname_or_path module or path to generate __init__.py for
options:
-h, --help show this help message and exit
--dry
-i, -w, --write, --inplace
modify / write to the file inplace
--diff show the diff (forces dry mode)
--noattrs Do not generate attribute from imports
--nomods Do not generate modules imports
--noall Do not generate an __all__ variable
--relative Use relative . imports instead of <modname>
--lazy Use lazy imports with more boilerplate but no dependencies (Python >= 3.7 only!)
--lazy_loader Use lazy imports with less boilerplate but requires the lazy_loader module (Python >= 3.7 only!)
--lazy_loader_typed Use lazy imports with the lazy_loader module, additionally generating
``__init__.pyi`` files for static typing (e.g. with mypy or pyright) (Python >= 3.7 only!)
--black Use black formatting
--lazy_boilerplate LAZY_BOILERPLATE
Code that defines a custom lazy_import callable
--recursive If specified, runs mkinit on all subpackages in a package
--norespect_all if False does not respect __all__ attributes of submodules when parsing
--verbose [VERBOSE] Verbosity level
--version print version and exit
Dynamic Usage
-------------
NOTE: Dynamic usage is NOT recommended.
In most cases, we recommend using mkinit command line tool to statically
generate / update the ``__init__.py`` file, but there is an option to to use it
dynamically (although this might be considered worse practice than using
``import *``).
.. code:: python
import mkinit; exec(mkinit.dynamic_init(__name__))
Examples
========
The ``mkinit`` module is used by the `ubelt <https://www.github.com/Erotemic/ubelt>`_ library to explicitly
auto-generate part of the ``__init__.py`` file. This example walks through the
design of this module to illustrate the usage of ``mkinit``.
Step 1 (Optional): Write any custom `__init__` code
----------------------------------------------------
The first section of the ``ubelt`` module consists of manually written code. It
contains coding, ``flake8`` directives, a docstring a few comments, a future
import, and a custom ``__version__`` attribute. Here is an example of this
manually written code in the ``0.2.0.dev0`` version of ``ubelt``.
.. code:: python
# -*- coding: utf-8 -*-
# flake8: noqa
"""
CommandLine:
# Partially regenerate __init__.py
mkinit ubelt
"""
# Todo:
# The following functions and classes are candidates to be ported from utool:
# * reload_class
# * inject_func_as_property
# * accumulate
# * rsync
from __future__ import absolute_import, division, print_function, unicode_literals
__version__ = '0.2.0'
It doesn't particularly matter what the above code is, the point is to
illustrate that ``mkinit`` does not prevent you from customizing your code. By
default auto-generation will only start clobbering existing code after the
final comment, in the file, which is a decent heuristic, but as we will see,
there are other more explicit ways to define exactly where auto-generated code
is allowed.
Step 2 (Optional): Enumerate relevant submodules
------------------------------------------------
After optionally writing any custom code, you may optionally specify exactly
what submodules should be considered when auto-generating imports. This is done
by setting the ``__submodules__`` attribute to a list of submodule names.
In ``ubelt`` this section looks similar to the following:
.. code:: python
__submodules__ = [
'util_arg',
'util_cmd',
'util_dict',
'util_links',
'util_hash',
'util_import',
'orderedset',
'progiter',
]
Note that this step is optional, but recommended. If the ``__submodules__``
package is not specified, then all paths matching the glob expressions ``*.py``
or ``*/__init__.py`` are considered as part of the package.
Step 3: Autogenerate explicitly
-------------------------------
To provide the fastest import times and most readable ``__init__.py`` files, use
the ``mkinit`` command line script to statically parse the submodules and
populate the ``__init__.py`` file with the submodules and their top-level
members.
Before running this script it is good practice to paste the XML-like comment
directives into the ``__init__.py`` file. This restricts where ``mkinit`` is
allowed to autogenerate code, and it also uses the same indentation of the
comments in case you want to run the auto-generated code conditionally. Note,
if the second tag is not specified, then it is assumed that ``mkinit`` can
overwrite everything after the first tag.
.. code:: python
# <AUTOGEN_INIT>
pass
# </AUTOGEN_INIT>
Now that we have inserted the auto-generation tags, we can actually run
``mkinit``. In general this is done by running ``mkinit <path-to-pkg-directory>``.
Assuming the ``ubelt`` repo is checked out in ``~/code/``, the command to
autogenerate its ``__init__.py`` file would be: ``mkinit ~/code/ubelt/ubelt``.
Given the previously specified ``__submodules__``, the resulting auto-generated
portion of the code looks like this:
.. code:: python
# <AUTOGEN_INIT>
from ubelt import util_arg
from ubelt import util_cmd
from ubelt import util_dict
from ubelt import util_links
from ubelt import util_hash
from ubelt import util_import
from ubelt import orderedset
from ubelt import progiter
from ubelt.util_arg import (argflag, argval,)
from ubelt.util_cmd import (cmd,)
from ubelt.util_dict import (AutoDict, AutoOrderedDict, ddict, dict_hist,
dict_subset, dict_take, dict_union, dzip,
find_duplicates, group_items, invert_dict,
map_keys, map_vals, odict,)
from ubelt.util_links import (symlink,)
from ubelt.util_hash import (hash_data, hash_file,)
from ubelt.util_import import (import_module_from_name,
import_module_from_path, modname_to_modpath,
modpath_to_modname, split_modpath,)
from ubelt.orderedset import (OrderedSet, oset,)
from ubelt.progiter import (ProgIter,)
__all__ = ['util_arg', 'util_cmd', 'util_dict', 'util_links', 'util_hash',
'util_import', 'orderedset', 'progiter', 'argflag', 'argval', 'cmd',
'AutoDict', 'AutoOrderedDict', 'ddict', 'dict_hist', 'dict_subset',
'dict_take', 'dict_union', 'dzip', 'find_duplicates', 'group_items',
'invert_dict', 'map_keys', 'map_vals', 'odict', 'symlink',
'hash_data', 'hash_file', 'import_module_from_name',
'import_module_from_path', 'modname_to_modpath',
'modpath_to_modname', 'split_modpath', 'OrderedSet', 'oset',
'ProgIter']
When running the command-line ``mkinit`` tool, the target module is inspected
using static analysis, so no code from the target module is ever run. This
avoids unintended side effects, prevents arbitrary code execution, and ensures
that ``mkinit`` will do something useful even if there would otherwise be a
runtime error.
Step 3 (alternate): Autogenerate dynamically
--------------------------------------------
While running ``mkinit`` from the command line produces the cleanest and most
readable ``__init__.py``, you have to run it every time you make a change to your
library. This is not always desirable especially during rapid development of a
new Python package. In this case it is possible to dynamically execute ``mkinit``
on import of your module. To use dynamic initialization simply paste the
following lines into the ``__init__.py`` file.
.. code:: python
import mkinit
exec(mkinit.dynamic_init(__name__, __submodules__))
This is almost equivalent to running the static command line variant. However,
instead of using static analysis, this will use the Python interpreter to
execute and import all submodules and dynamically inspect the defined members.
This is faster than using static analysis, and in most circumstances there will
be no difference in the resulting imported attributes. To avoid all differences
simply specify the ``__all__`` attribute in each submodule.
Note that inclusion of the ``__submodules__`` attribute is not strictly
necessary. The dynamic version of this function will look in the parent stack
frame for this attribute if it is not specified explicitly as an argument.
It is also possible to achieve a "best of both worlds" trade-off using
conditional logic. Use a conditional block to execute dynamic initialization
and place the static auto-generation tags in the block that is not executed.
This lets you develop without worrying about updating the ``__init__.py`` file,
and lets you statically generate the code for documentation purposes when you
want to. Once the rapid development phase is over, you can remove the dynamic
conditional, keep the auto-generated portion, and forget you ever used ``mkinit``
in the first place!
.. code:: python
__DYNAMIC__ = True
if __DYNAMIC__:
from mkinit import dynamic_mkinit
exec(dynamic_mkinit.dynamic_init(__name__))
else:
# <AUTOGEN_INIT>
from mkinit import dynamic_mkinit
from mkinit import static_mkinit
from mkinit.dynamic_mkinit import (dynamic_init,)
from mkinit.static_mkinit import (autogen_init,)
# </AUTOGEN_INIT>
Behavior Notes
--------------
The ``mkinit`` module is a simple way to execute a complex task. At times it may
seem like magic, although I assure you it is not. To minimize perception of
magic and maximize understanding of its behaviors, please consider the
following:
* When discovering attributes of submodules ``mkinit`` will respect the ``__all__``
attribute by default. In general it is good practice to specify this
property; doing so will also avoid the following caveats.
* Static analysis currently only extracts top-level module attributes. However,
if will also extract attributes defined on all non-error raising paths of
conditional if-else or try-except statements.
* Static analysis currently does not look or account for the usage of the ``del``
operator. Again, these will be accounted for by dynamic analysis.
* In the case where no ``__init__.py`` file exists, the ``mkinit`` command line
tool will create one.
* By default we ignore attributes that are marked as non-public by a leading
underscore
TODO
----
- [ ] Give ``dynamic_init`` an options dict to maintain a compatible API with ``static_init``.
- [ ] If an attribute would be defined twice, then don't define it at all. Currently, it is defined, but its value is not well-defined.
.. |CircleCI| image:: https://circleci.com/gh/Erotemic/mkinit.svg?style=svg
:target: https://circleci.com/gh/Erotemic/mkinit
.. |Travis| image:: https://img.shields.io/travis/Erotemic/mkinit/master.svg?label=Travis%20CI
:target: https://travis-ci.org/Erotemic/mkinit?branch=master
.. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/github/Erotemic/mkinit?branch=master&svg=True
:target: https://ci.appveyor.com/projegt/Erotemic/mkinit/branch/master
.. |Codecov| image:: https://codecov.io/github/Erotemic/mkinit/badge.svg?branch=master&service=github
:target: https://codecov.io/github/Erotemic/mkinit?branch=master
.. |Pypi| image:: https://img.shields.io/pypi/v/mkinit.svg
:target: https://pypi.python.org/pypi/mkinit
.. |Downloads| image:: https://img.shields.io/pypi/dm/mkinit.svg
:target: https://pypistats.org/packages/mkinit
.. |ReadTheDocs| image:: https://readthedocs.org/projects/mkinit/badge/?version=latest
:target: http://mkinit.readthedocs.io/en/latest/
Raw data
{
"_id": null,
"home_page": "https://github.com/Erotemic/mkinit",
"name": "mkinit",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "Jon Crall",
"author_email": "erotemic@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/8b/85/76f750d3b8413390a61208945d5eb30dbfc4dc35697931af03638f40a508/mkinit-1.1.0.tar.gz",
"platform": null,
"description": "mkinit\n======\n\n|CircleCI| |Appveyor| |Codecov| |Pypi| |Downloads| |ReadTheDocs|\n\n\n+------------------+--------------------------------------------+\n| Read the docs | https://mkinit.readthedocs.io |\n+------------------+--------------------------------------------+\n| Github | https://github.com/Erotemic/mkinit |\n+------------------+--------------------------------------------+\n| Pypi | https://pypi.org/project/mkinit |\n+------------------+--------------------------------------------+\n\nThe ``mkinit`` module helps you write ``__init__`` files that expose all submodule\nattributes without ``from ? import *``.\n\n``mkinit`` automatically imports all submodules in a package and their members.\n\nIt can do this dynamically, or it can statically autogenerate the ``__init__``\nfor faster import times. Its kinda like using the ``fromimport *`` syntax, but\nits easy to replace with text that wont make other developers lose their hair.\n\nThis module supports Scientific Python `SPEC1 <https://scientific-python.org/specs/spec-0001/>`_.\n\nAlso note that the docs in this readme are somewhat old, and need to be updated\nto make best practices more clear. There are a lot of ways you can use the\nmodule, but the current recommended way is to use:\n\n\n.. code:: bash\n\n mkinit --lazy_loader <path-to-init.py>\n\nInstallation\n============\n\n.. code:: bash\n\n pip install mkinit\n\n\nThe Pitch\n---------\n\nSay you have a python module structured like so:\n\n.. code::\n\n \u2514\u2500\u2500 mkinit_demo_pkg\n \u251c\u2500\u2500 __init__.py\n \u251c\u2500\u2500 submod.py\n \u2514\u2500\u2500 subpkg\n \u251c\u2500\u2500 __init__.py\n \u2514\u2500\u2500 nested.py\n\n\nAnd you would like to make all functions inside of ``submod.py`` and\n``nested.py`` available at the top-level of the package.\n\nImagine the contents of submod.py and nested.py are:\n\n.. code:: python\n\n # --- submod.py ---\n\n def submod_func():\n print('This is a submod func in {}'.format(__file__))\n\n # --- nested.py ---\n\n def nested_func():\n print('This is a nested func in {}'.format(__file__))\n\nYou could manually write:\n\n\n.. code:: python\n\n from mkinit_demo_pkg.submod import *\n from mkinit_demo_pkg.subpkg.nested import *\n\n\nBut that has a few problems. Using ``import *`` makes it hard for people\nreading the code to know what is coming from where. Furthermore, if there were\nmany submodules you wanted to expose attributes of, writing this would become\ntedious and hard to maintain.\n\nEnter the mkinit package. It has the ability to autogenerate explicit ``__init__.py``\nfiles using static analysis. Normally, the mkinit CLI only works on one file at\na time, but if we specify the ``--recursive`` flag, then mkinit will\nrecursively generate ``__init__.py`` files for all subpackages in the package.\n\nThus running ``mkinit mkinit_demo_pkg --recursive`` will result in a root\n``__init__.py`` file that looks like this:\n\n.. code:: python\n\n from mkinit_demo_pkg import submod\n from mkinit_demo_pkg import subpkg\n\n from mkinit_demo_pkg.submod import (submod_func,)\n from mkinit_demo_pkg.subpkg import (nested, nested_func,)\n\n __all__ = ['nested', 'nested_func', 'submod', 'submod_func', 'subpkg']\n\n\nThat's pretty cool. The mkinit package was able to recursively parse our\npackage, find all of the defined names, and then generate ``__init__.py`` files\nsuch that all attributes are exposed at the top level of the package.\nFurthermore, this file is **readable**. It is perfectly clear exactly what\nnames are exposed in this module without having to execute anything.\n\n\nOf course, this isn't a perfect solution. Perhaps only some submodules should\nbe exposed, perhaps you would rather use relative import statements, maybe you\nonly want to expose submodule but not their attributes, or vis-versa. Well good\nnews, because mkinit has command line flags that allow for all of these modes.\nSee ``mkinit --help`` for more details.\n\n\nLastly, while exposing all attributes can be helpful for larger projects,\nimport time can start to become a consideration. Thankfully,\n`PEP 0562 <https://peps.python.org/pep-0562/>`_ outlines\na lazy import specification for Python >= 3.7. As of 2020-12-26 mkinit\nsupports autogenerating these lazy init files.\n\nUnfortunately, there is no syntax support for lazy imports, so mkinit must\ndefine a ``lazy_import`` boilerplate function in each ``__init__.py`` file.\n\n\n.. code:: python\n\n def lazy_import(module_name, submodules, submod_attrs):\n \"\"\"\n Boilerplate to define PEP 562 __getattr__ for lazy import\n https://www.python.org/dev/peps/pep-0562/\n \"\"\"\n import importlib\n import os\n name_to_submod = {\n func: mod for mod, funcs in submod_attrs.items()\n for func in funcs\n }\n\n def __getattr__(name):\n if name in submodules:\n attr = importlib.import_module(\n '{module_name}.{name}'.format(\n module_name=module_name, name=name)\n )\n elif name in name_to_submod:\n submodname = name_to_submod[name]\n module = importlib.import_module(\n '{module_name}.{submodname}'.format(\n module_name=module_name, submodname=submodname)\n )\n attr = getattr(module, name)\n else:\n raise AttributeError(\n 'No {module_name} attribute {name}'.format(\n module_name=module_name, name=name))\n globals()[name] = attr\n return attr\n\n if os.environ.get('EAGER_IMPORT', ''):\n for name in submodules:\n __getattr__(name)\n\n for attrs in submod_attrs.values():\n for attr in attrs:\n __getattr__(attr)\n return __getattr__\n\n\n __getattr__ = lazy_import(\n __name__,\n submodules={\n 'submod',\n 'subpkg',\n },\n submod_attrs={\n 'submod': [\n 'submod_func',\n ],\n 'subpkg': [\n 'nested',\n 'nested_func',\n ],\n },\n )\n\n def __dir__():\n return __all__\n\n __all__ = ['nested', 'nested_func', 'submod', 'submod_func', 'subpkg']\n\n\nAlthough if you are willing to depend on the\n`lazy_loader <https://pypi.org/project/lazy_loader/>`_\npackage and the ``--lazy_loader`` option (new as of 1.0.0), then this\nboilerplate is no longer needed.\n\nBy default, lazy imports are not compatibly with statically typed projects (e.g\nusing mypy or pyright), however, if the\n`lazy_loader <https://pypi.org/project/lazy_loader/>`_\npackage is used, the ``--lazy_loader_typed`` option can be specified to generate\n``__init.pyi__`` files in addition to lazily evaulated ``__init.py__`` files.\nThese interface files are understood by static type checkers and allow the\ncombination of lazy loading with static type checking.\n\n\nCommand Line Usage\n------------------\n\nThe following command will statically autogenerate an ``__init__`` file in the\nspecified path or module name. If one exists, it will only replace text after\nthe final comment. This means ``mkinit`` wont clobber your custom logic and can\nbe used to help maintain customized ``__init__.py`` files.\n\n.. code:: bash\n\n mkinit <your_modname_or_modpath> -w\n\n\nYou can also enclose the area allowed to be clobbered in the auto-generation\nwith special xml-like comments.\n\nRunning ``mkint --help`` displays:\n\n.. code::\n\n\n usage: python -m mkinit [-h] [--dry] [-i] [--diff] [--noattrs] [--nomods] [--noall] [--relative] [--lazy | --lazy_loader] [--black] [--lazy_boilerplate LAZY_BOILERPLATE] [--recursive] [--norespect_all]\n [--verbose [VERBOSE]] [--version]\n [modname_or_path]\n\n Autogenerate an `__init__.py` that exposes a top-level API.\n\n Behavior is modified depending on the existing content of the\n `__init__.py` file (subsequent runs of mkinit are idempotent).\n\n The following `__init__.py` variables modify autogeneration behavior:\n\n `__submodules__` (List[str] | Dict[str, List[str])) -\n Indicates the list of submodules to be introspected, if\n unspecified all submodules are introspected. Can be a list\n of submodule names, or a dictionary mapping each submodule name\n to a list of attribute names to expose. If the value is None,\n then all attributes are exposed (or __all__) is respected).\n\n `__external__` - Specify external modules to expose the attributes of.\n\n `__explicit__` - Add custom explicitly defined names to this, and\n they will be automatically added to the __all__ variable.\n\n `__protected__` - Protected modules are exposed, but their attributes are not.\n\n `__private__` - Private modules and their attributes are not exposed.\n\n `__ignore__` - Tells mkinit to ignore particular attributes\n\n positional arguments:\n modname_or_path module or path to generate __init__.py for\n\n options:\n -h, --help show this help message and exit\n --dry\n -i, -w, --write, --inplace\n modify / write to the file inplace\n --diff show the diff (forces dry mode)\n --noattrs Do not generate attribute from imports\n --nomods Do not generate modules imports\n --noall Do not generate an __all__ variable\n --relative Use relative . imports instead of <modname>\n --lazy Use lazy imports with more boilerplate but no dependencies (Python >= 3.7 only!)\n --lazy_loader Use lazy imports with less boilerplate but requires the lazy_loader module (Python >= 3.7 only!)\n --lazy_loader_typed Use lazy imports with the lazy_loader module, additionally generating \n ``__init__.pyi`` files for static typing (e.g. with mypy or pyright) (Python >= 3.7 only!)\n --black Use black formatting\n --lazy_boilerplate LAZY_BOILERPLATE\n Code that defines a custom lazy_import callable\n --recursive If specified, runs mkinit on all subpackages in a package\n --norespect_all if False does not respect __all__ attributes of submodules when parsing\n --verbose [VERBOSE] Verbosity level\n --version print version and exit\n\n\nDynamic Usage\n-------------\n\nNOTE: Dynamic usage is NOT recommended.\n\nIn most cases, we recommend using mkinit command line tool to statically\ngenerate / update the ``__init__.py`` file, but there is an option to to use it\ndynamically (although this might be considered worse practice than using\n``import *``).\n\n.. code:: python\n\n import mkinit; exec(mkinit.dynamic_init(__name__))\n\n\nExamples\n========\n\nThe ``mkinit`` module is used by the `ubelt <https://www.github.com/Erotemic/ubelt>`_ library to explicitly\nauto-generate part of the ``__init__.py`` file. This example walks through the\ndesign of this module to illustrate the usage of ``mkinit``.\n\nStep 1 (Optional): Write any custom `__init__` code\n----------------------------------------------------\n\nThe first section of the ``ubelt`` module consists of manually written code. It\ncontains coding, ``flake8`` directives, a docstring a few comments, a future\nimport, and a custom ``__version__`` attribute. Here is an example of this\nmanually written code in the ``0.2.0.dev0`` version of ``ubelt``.\n\n.. code:: python\n\n # -*- coding: utf-8 -*-\n # flake8: noqa\n \"\"\"\n CommandLine:\n # Partially regenerate __init__.py\n mkinit ubelt\n \"\"\"\n # Todo:\n # The following functions and classes are candidates to be ported from utool:\n # * reload_class\n # * inject_func_as_property\n # * accumulate\n # * rsync\n from __future__ import absolute_import, division, print_function, unicode_literals\n\n __version__ = '0.2.0'\n\nIt doesn't particularly matter what the above code is, the point is to\nillustrate that ``mkinit`` does not prevent you from customizing your code. By\ndefault auto-generation will only start clobbering existing code after the\nfinal comment, in the file, which is a decent heuristic, but as we will see,\nthere are other more explicit ways to define exactly where auto-generated code\nis allowed.\n\nStep 2 (Optional): Enumerate relevant submodules\n------------------------------------------------\n\nAfter optionally writing any custom code, you may optionally specify exactly\nwhat submodules should be considered when auto-generating imports. This is done\nby setting the ``__submodules__`` attribute to a list of submodule names.\n\nIn ``ubelt`` this section looks similar to the following:\n\n.. code:: python\n\n __submodules__ = [\n 'util_arg',\n 'util_cmd',\n 'util_dict',\n 'util_links',\n 'util_hash',\n 'util_import',\n 'orderedset',\n 'progiter',\n ]\n\nNote that this step is optional, but recommended. If the ``__submodules__``\npackage is not specified, then all paths matching the glob expressions ``*.py``\nor ``*/__init__.py`` are considered as part of the package.\n\nStep 3: Autogenerate explicitly\n-------------------------------\n\nTo provide the fastest import times and most readable ``__init__.py`` files, use\nthe ``mkinit`` command line script to statically parse the submodules and\npopulate the ``__init__.py`` file with the submodules and their top-level\nmembers.\n\nBefore running this script it is good practice to paste the XML-like comment\ndirectives into the ``__init__.py`` file. This restricts where ``mkinit`` is\nallowed to autogenerate code, and it also uses the same indentation of the\ncomments in case you want to run the auto-generated code conditionally. Note,\nif the second tag is not specified, then it is assumed that ``mkinit`` can\noverwrite everything after the first tag.\n\n.. code:: python\n\n # <AUTOGEN_INIT>\n pass\n # </AUTOGEN_INIT>\n\nNow that we have inserted the auto-generation tags, we can actually run\n``mkinit``. In general this is done by running ``mkinit <path-to-pkg-directory>``.\n\nAssuming the ``ubelt`` repo is checked out in ``~/code/``, the command to\nautogenerate its ``__init__.py`` file would be: ``mkinit ~/code/ubelt/ubelt``.\nGiven the previously specified ``__submodules__``, the resulting auto-generated\nportion of the code looks like this:\n\n\n.. code:: python\n\n # <AUTOGEN_INIT>\n from ubelt import util_arg\n from ubelt import util_cmd\n from ubelt import util_dict\n from ubelt import util_links\n from ubelt import util_hash\n from ubelt import util_import\n from ubelt import orderedset\n from ubelt import progiter\n from ubelt.util_arg import (argflag, argval,)\n from ubelt.util_cmd import (cmd,)\n from ubelt.util_dict import (AutoDict, AutoOrderedDict, ddict, dict_hist,\n dict_subset, dict_take, dict_union, dzip,\n find_duplicates, group_items, invert_dict,\n map_keys, map_vals, odict,)\n from ubelt.util_links import (symlink,)\n from ubelt.util_hash import (hash_data, hash_file,)\n from ubelt.util_import import (import_module_from_name,\n import_module_from_path, modname_to_modpath,\n modpath_to_modname, split_modpath,)\n from ubelt.orderedset import (OrderedSet, oset,)\n from ubelt.progiter import (ProgIter,)\n __all__ = ['util_arg', 'util_cmd', 'util_dict', 'util_links', 'util_hash',\n 'util_import', 'orderedset', 'progiter', 'argflag', 'argval', 'cmd',\n 'AutoDict', 'AutoOrderedDict', 'ddict', 'dict_hist', 'dict_subset',\n 'dict_take', 'dict_union', 'dzip', 'find_duplicates', 'group_items',\n 'invert_dict', 'map_keys', 'map_vals', 'odict', 'symlink',\n 'hash_data', 'hash_file', 'import_module_from_name',\n 'import_module_from_path', 'modname_to_modpath',\n 'modpath_to_modname', 'split_modpath', 'OrderedSet', 'oset',\n 'ProgIter']\n\nWhen running the command-line ``mkinit`` tool, the target module is inspected\nusing static analysis, so no code from the target module is ever run. This\navoids unintended side effects, prevents arbitrary code execution, and ensures\nthat ``mkinit`` will do something useful even if there would otherwise be a\nruntime error.\n\nStep 3 (alternate): Autogenerate dynamically\n--------------------------------------------\n\nWhile running ``mkinit`` from the command line produces the cleanest and most\nreadable ``__init__.py``, you have to run it every time you make a change to your\nlibrary. This is not always desirable especially during rapid development of a\nnew Python package. In this case it is possible to dynamically execute ``mkinit``\non import of your module. To use dynamic initialization simply paste the\nfollowing lines into the ``__init__.py`` file.\n\n.. code:: python\n\n import mkinit\n exec(mkinit.dynamic_init(__name__, __submodules__))\n\nThis is almost equivalent to running the static command line variant. However,\ninstead of using static analysis, this will use the Python interpreter to\nexecute and import all submodules and dynamically inspect the defined members.\nThis is faster than using static analysis, and in most circumstances there will\nbe no difference in the resulting imported attributes. To avoid all differences\nsimply specify the ``__all__`` attribute in each submodule.\n\nNote that inclusion of the ``__submodules__`` attribute is not strictly\nnecessary. The dynamic version of this function will look in the parent stack\nframe for this attribute if it is not specified explicitly as an argument.\n\nIt is also possible to achieve a \"best of both worlds\" trade-off using\nconditional logic. Use a conditional block to execute dynamic initialization\nand place the static auto-generation tags in the block that is not executed.\nThis lets you develop without worrying about updating the ``__init__.py`` file,\nand lets you statically generate the code for documentation purposes when you\nwant to. Once the rapid development phase is over, you can remove the dynamic\nconditional, keep the auto-generated portion, and forget you ever used ``mkinit``\nin the first place!\n\n\n.. code:: python\n\n __DYNAMIC__ = True\n if __DYNAMIC__:\n from mkinit import dynamic_mkinit\n exec(dynamic_mkinit.dynamic_init(__name__))\n else:\n # <AUTOGEN_INIT>\n from mkinit import dynamic_mkinit\n from mkinit import static_mkinit\n from mkinit.dynamic_mkinit import (dynamic_init,)\n from mkinit.static_mkinit import (autogen_init,)\n # </AUTOGEN_INIT>\n\n\nBehavior Notes\n--------------\n\nThe ``mkinit`` module is a simple way to execute a complex task. At times it may\nseem like magic, although I assure you it is not. To minimize perception of\nmagic and maximize understanding of its behaviors, please consider the\nfollowing:\n\n * When discovering attributes of submodules ``mkinit`` will respect the ``__all__``\n attribute by default. In general it is good practice to specify this\n property; doing so will also avoid the following caveats.\n\n * Static analysis currently only extracts top-level module attributes. However,\n if will also extract attributes defined on all non-error raising paths of\n conditional if-else or try-except statements.\n\n * Static analysis currently does not look or account for the usage of the ``del``\n operator. Again, these will be accounted for by dynamic analysis.\n\n * In the case where no ``__init__.py`` file exists, the ``mkinit`` command line\n tool will create one.\n\n * By default we ignore attributes that are marked as non-public by a leading\n underscore\n\nTODO\n----\n\n- [ ] Give ``dynamic_init`` an options dict to maintain a compatible API with ``static_init``.\n\n- [ ] If an attribute would be defined twice, then don't define it at all. Currently, it is defined, but its value is not well-defined.\n\n\n.. |CircleCI| image:: https://circleci.com/gh/Erotemic/mkinit.svg?style=svg\n :target: https://circleci.com/gh/Erotemic/mkinit\n.. |Travis| image:: https://img.shields.io/travis/Erotemic/mkinit/master.svg?label=Travis%20CI\n :target: https://travis-ci.org/Erotemic/mkinit?branch=master\n.. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/github/Erotemic/mkinit?branch=master&svg=True\n :target: https://ci.appveyor.com/projegt/Erotemic/mkinit/branch/master\n.. |Codecov| image:: https://codecov.io/github/Erotemic/mkinit/badge.svg?branch=master&service=github\n :target: https://codecov.io/github/Erotemic/mkinit?branch=master\n.. |Pypi| image:: https://img.shields.io/pypi/v/mkinit.svg\n :target: https://pypi.python.org/pypi/mkinit\n.. |Downloads| image:: https://img.shields.io/pypi/dm/mkinit.svg\n :target: https://pypistats.org/packages/mkinit\n.. |ReadTheDocs| image:: https://readthedocs.org/projects/mkinit/badge/?version=latest\n :target: http://mkinit.readthedocs.io/en/latest/\n\n\n",
"bugtrack_url": null,
"license": "Apache 2",
"summary": "Autogenerate __init__.py files",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/Erotemic/mkinit"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b23b4e5fabaa924292216f23f095009f9e091b49018e18d653eedab2dd25885c",
"md5": "22b0b69365f8c38f34167507c6aaeac8",
"sha256": "b49dca93956c1e3ba2ae84fa31238745cf1d1aaa47ab53aeda5e22d375a3732a"
},
"downloads": -1,
"filename": "mkinit-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "22b0b69365f8c38f34167507c6aaeac8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 63103,
"upload_time": "2024-01-17T17:35:14",
"upload_time_iso_8601": "2024-01-17T17:35:14.697507Z",
"url": "https://files.pythonhosted.org/packages/b2/3b/4e5fabaa924292216f23f095009f9e091b49018e18d653eedab2dd25885c/mkinit-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8b8576f750d3b8413390a61208945d5eb30dbfc4dc35697931af03638f40a508",
"md5": "bd729a2004f1d8b519cc6f8b3fa5bb1e",
"sha256": "052ee0ba314fbacfc2d04314812868053a3a3acbd393e6a7c0aea4b9f697f6da"
},
"downloads": -1,
"filename": "mkinit-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "bd729a2004f1d8b519cc6f8b3fa5bb1e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 63741,
"upload_time": "2024-01-17T17:35:17",
"upload_time_iso_8601": "2024-01-17T17:35:17.217247Z",
"url": "https://files.pythonhosted.org/packages/8b/85/76f750d3b8413390a61208945d5eb30dbfc4dc35697931af03638f40a508/mkinit-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-17 17:35:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Erotemic",
"github_project": "mkinit",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"circle": true,
"appveyor": true,
"requirements": [],
"lcname": "mkinit"
}
JSON
