iterpath


Nameiterpath JSON
Version 0.4.0 PyPI version JSON
download
home_pagehttps://github.com/jwodder/iterpath
SummaryIterate through a file tree
upload_time2022-06-25 16:16:36
maintainer
docs_urlNone
authorJohn Thorvald Wodder II
requires_python~=3.7
licenseMIT
keywords direntry directories directory tree file tree files find path pathlib walk
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            .. image:: http://www.repostatus.org/badges/latest/active.svg
    :target: http://www.repostatus.org/#active
    :alt: Project Status: Active — The project has reached a stable, usable
          state and is being actively developed.

.. image:: https://github.com/jwodder/iterpath/workflows/Test/badge.svg?branch=master
    :target: https://github.com/jwodder/iterpath/actions?workflow=Test
    :alt: CI Status

.. image:: https://codecov.io/gh/jwodder/iterpath/branch/master/graph/badge.svg
    :target: https://codecov.io/gh/jwodder/iterpath

.. image:: https://img.shields.io/pypi/pyversions/iterpath.svg
    :target: https://pypi.org/project/iterpath/

.. image:: https://img.shields.io/github/license/jwodder/iterpath.svg
    :target: https://opensource.org/licenses/MIT
    :alt: MIT License

`GitHub <https://github.com/jwodder/iterpath>`_
| `PyPI <https://pypi.org/project/iterpath/>`_
| `Issues <https://github.com/jwodder/iterpath/issues>`_
| `Changelog <https://github.com/jwodder/iterpath/blob/master/CHANGELOG.md>`_

``iterpath`` lets you iterate over a file tree as a single iterator of
``pathlib.Path`` objects, eliminating the need to combine lists returned by
``os.walk()`` or recursively call ``Path.iterdir()`` or ``os.scandir()``.
Besides the standard ``os.walk()`` options, the library also includes options
for sorting & filtering entries.


Installation
============
``iterpath`` requires Python 3.7 or higher.  Just use `pip
<https://pip.pypa.io>`_ for Python 3 (You have pip, right?) to install it::

    python3 -m pip install iterpath


Example
=======

Iterate over this library's source repository, skipping the ``.git`` and
``test/data`` folders:

>>> import os.path
>>> from iterpath import iterpath
>>> def filterer(dir_entry):
...     if dir_entry.name == ".git":
...         return False
...     elif dir_entry.path == os.path.join(".", "test", "data"):
...         return False
...     else:
...         return True
...
>>> with iterpath(".", sort=True, filter_dirs=filterer) as ip:
...     for p in ip:
...         print(p)
...
.github
.github/workflows
.github/workflows/test.yml
.gitignore
LICENSE
MANIFEST.in
README.rst
TODO.md
pyproject.toml
setup.cfg
src
src/iterpath
src/iterpath/__init__.py
src/iterpath/__pycache__
src/iterpath/__pycache__/__init__.cpython-39.pyc
src/iterpath/py.typed
test
test/test_iterpath.py
tox.ini


API
===

The ``iterpath`` module provides a single function, also named ``iterpath``:

.. code:: python

    iterpath(dirpath: AnyStr | os.PathLike[AnyStr] = os.curdir, **kwargs) -> Iterpath[AnyStr]

Iterate through the file tree rooted at the directory ``dirpath`` (by default,
the current directory) in depth-first order, yielding the files & directories
within as ``pathlib.Path`` instances.

The return value is both an iterator and a context manager.  In order to ensure
that the internal ``os.scandir()`` iterators are closed properly, either call
the ``close()`` method when done or else use it as a context manager like so::

    with iterpath(...) as ip:
        for path in ip:
            ...

If ``return_relative`` is true, the generated ``Path`` objects will be relative
to ``dirpath``.  If ``return_relative`` is false (the default) and ``dirpath``
is an absolute path, the generated ``Path`` objects will be absolute;
otherwise, if ``dirpath`` is a relative path, the ``Path`` objects will be
relative and will have ``dirpath`` as a prefix.

Note that, although ``iterpath()`` yields ``pathlib.Path`` objects, it operates
internally on ``os.DirEntry`` objects, and so any function supplied as the
``sort_key`` parameter or as a filter/exclude parameter must accept
``os.DirEntry`` instances.

Keyword arguments:

``dirs: bool = True``
    Whether to include directories in the output

``topdown: bool = True``
    Whether to yield each directory before (``True``) or after (``False``) its
    contents

``include_root: bool = False``
    Whether to include the ``dirpath`` argument passed to ``iterpath()`` in the
    output

``followlinks: bool = False``
    Whether to treat a symlink to a directory as a directory

``return_relative: bool = False``
    If true, the generated paths will be relative to ``dirpath``

``onerror: Optional[Callable[[OSError], Any]] = None``
    Specify a function to be called whenever an ``OSError`` is encountered
    while iterating over a directory.  If the function reraises the exception,
    ``iterpath()`` aborts; otherwise, it continues with the next directory.  By
    default, ``OSError`` exceptions are ignored.

``sort: bool = False``
    Sort the entries in each directory.  When ``False``, entries are yielded in
    the order returned by ``os.scandir()``.  When ``True``, entries are sorted,
    by default, by filename in ascending order, but this can be changed via the
    ``sort_key`` and ``sort_reverse`` arguments.

``sort_key: Optional[Callable[[os.DirEntry[AnyStr]], _typeshed.SupportsLessThan]] = None``
    Specify a custom key function for sorting directory entries.  Only has an
    effect when ``sort`` is ``True``.

``sort_reverse: bool = False``
    Sort directory entries in reverse order.  Only has an effect when ``sort``
    is ``True``.

``filter: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``
    Specify a predicate to be applied to all files & directories encountered;
    only those for which the predicate returns a true value will be yielded
    (and, for directories, descended into).

    If ``filter`` is specified, it is an error to also specify ``filter_dirs``
    or ``filter_files``.

``filter_dirs: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``
    Specify a predicate to be applied to all directories encountered; only
    those for which the predicate returns a true value will be yielded &
    descended into

``filter_files: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``
    Specify a predicate to be applied to all files encountered; only those for
    which the predicate returns a true value will be yielded

``exclude: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``
    Specify a predicate to be applied to all files & directories encountered;
    only those for which the predicate returns a false value will be yielded
    (and, for directories, descended into).

    If ``exclude`` is specified, it is an error to also specify ``exclude_dirs``
    or ``exclude_files``.

``exclude_dirs: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``
    Specify a predicate to be applied to all directories encountered; only
    those for which the predicate returns a false value will be yielded &
    descended into

``exclude_files: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``
    Specify a predicate to be applied to all files encountered; only those for
    which the predicate returns a false value will be yielded

If both ``filter`` and ``exclude`` are set, a given entry will only be included
if ``filter`` returns true and ``exclude`` returns false (that is, exclusions
take priority over inclusions), and likewise for the directory- and
file-specific arguments.

**Warnings:**

- If ``dirpath`` is a relative path, changing the working directory while
  ``iterpath()`` is in progress will lead to errors, or at least inaccurate
  results.

- Setting ``followlinks`` to ``True`` can result in infinite recursion if a
  symlink points to a parent directory of itself.

Selectors
---------

*New in version 0.3.0*

``iterpath`` also provides a selection of "selector" classes & constants for
easy construction of ``filter`` and ``exclude`` arguments.  Selectors are
callables that return true for ``DirEntry``'s whose (base) names match given
criteria.

Selectors can even be combined using the ``|`` operator:

.. code:: python

    # This only returns entries whose names end in ".txt" or equal "foo.png" or
    # ".hidden":
    iterpath(
        dirpath,
        filter=SelectGlob("*.txt") | SelectNames("foo.png", ".hidden")
    )

    # Exclude all dot-directories and VCS directories:
    iterpath(dirpath, exclude_dirs=SELECT_DOTS | SELECT_VCS_DIRS)

The selectors:

.. code:: python

    class SelectNames(*names: AnyStr, case_sensitive: bool = True)

Selects ``DirEntry``'s whose names are one of ``names``.  If ``case_sensitive``
is ``False``, the check is performed case-insensitively.

.. code:: python

    class SelectGlob(pattern: AnyStr)

Selects ``DirEntry``'s whose names match the given fileglob pattern

.. code:: python

    class SelectRegex(pattern: AnyStr | re.Pattern[AnyStr])

Selects ``DirEntry``'s whose names match (using ``re.search()``) the given
regular expression

.. code:: python

    SELECT_DOTS

Selects ``DirEntry``'s whose names begin with a period

.. code:: python

    SELECT_VCS

Selects ``DirEntry``'s matched by either ``SELECT_VCS_DIRS`` or
``SELECT_VCS_FILES`` (see below)

.. code:: python

    SELECT_VCS_DIRS

Selects the following names of version-control directories: ``.git``, ``.hg``,
``_darcs``, ``.bzr``, ``.svn``, ``_svn``, ``CVS``, ``RCS``

.. code:: python

    SELECT_VCS_FILES

Selects the following names of version-control-specific files:
``.gitattributes``, ``.gitignore``, ``.gitmodules``, ``.mailmap``,
``.hgignore``, ``.hgsigs``, ``.hgtags``, ``.binaries``, ``.boring``,
``.bzrignore``, and all nonempty filenames that end in ``,v``

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/jwodder/iterpath",
    "name": "iterpath",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "~=3.7",
    "maintainer_email": "",
    "keywords": "DirEntry,directories,directory tree,file tree,files,find,path,pathlib,walk",
    "author": "John Thorvald Wodder II",
    "author_email": "iterpath@varonathe.org",
    "download_url": "https://files.pythonhosted.org/packages/46/bb/37ec82f48d9297e64c130bcead60de512d128737ea7dc74aa206c4dd783b/iterpath-0.4.0.tar.gz",
    "platform": null,
    "description": ".. image:: http://www.repostatus.org/badges/latest/active.svg\n    :target: http://www.repostatus.org/#active\n    :alt: Project Status: Active \u2014 The project has reached a stable, usable\n          state and is being actively developed.\n\n.. image:: https://github.com/jwodder/iterpath/workflows/Test/badge.svg?branch=master\n    :target: https://github.com/jwodder/iterpath/actions?workflow=Test\n    :alt: CI Status\n\n.. image:: https://codecov.io/gh/jwodder/iterpath/branch/master/graph/badge.svg\n    :target: https://codecov.io/gh/jwodder/iterpath\n\n.. image:: https://img.shields.io/pypi/pyversions/iterpath.svg\n    :target: https://pypi.org/project/iterpath/\n\n.. image:: https://img.shields.io/github/license/jwodder/iterpath.svg\n    :target: https://opensource.org/licenses/MIT\n    :alt: MIT License\n\n`GitHub <https://github.com/jwodder/iterpath>`_\n| `PyPI <https://pypi.org/project/iterpath/>`_\n| `Issues <https://github.com/jwodder/iterpath/issues>`_\n| `Changelog <https://github.com/jwodder/iterpath/blob/master/CHANGELOG.md>`_\n\n``iterpath`` lets you iterate over a file tree as a single iterator of\n``pathlib.Path`` objects, eliminating the need to combine lists returned by\n``os.walk()`` or recursively call ``Path.iterdir()`` or ``os.scandir()``.\nBesides the standard ``os.walk()`` options, the library also includes options\nfor sorting & filtering entries.\n\n\nInstallation\n============\n``iterpath`` requires Python 3.7 or higher.  Just use `pip\n<https://pip.pypa.io>`_ for Python 3 (You have pip, right?) to install it::\n\n    python3 -m pip install iterpath\n\n\nExample\n=======\n\nIterate over this library's source repository, skipping the ``.git`` and\n``test/data`` folders:\n\n>>> import os.path\n>>> from iterpath import iterpath\n>>> def filterer(dir_entry):\n...     if dir_entry.name == \".git\":\n...         return False\n...     elif dir_entry.path == os.path.join(\".\", \"test\", \"data\"):\n...         return False\n...     else:\n...         return True\n...\n>>> with iterpath(\".\", sort=True, filter_dirs=filterer) as ip:\n...     for p in ip:\n...         print(p)\n...\n.github\n.github/workflows\n.github/workflows/test.yml\n.gitignore\nLICENSE\nMANIFEST.in\nREADME.rst\nTODO.md\npyproject.toml\nsetup.cfg\nsrc\nsrc/iterpath\nsrc/iterpath/__init__.py\nsrc/iterpath/__pycache__\nsrc/iterpath/__pycache__/__init__.cpython-39.pyc\nsrc/iterpath/py.typed\ntest\ntest/test_iterpath.py\ntox.ini\n\n\nAPI\n===\n\nThe ``iterpath`` module provides a single function, also named ``iterpath``:\n\n.. code:: python\n\n    iterpath(dirpath: AnyStr | os.PathLike[AnyStr] = os.curdir, **kwargs) -> Iterpath[AnyStr]\n\nIterate through the file tree rooted at the directory ``dirpath`` (by default,\nthe current directory) in depth-first order, yielding the files & directories\nwithin as ``pathlib.Path`` instances.\n\nThe return value is both an iterator and a context manager.  In order to ensure\nthat the internal ``os.scandir()`` iterators are closed properly, either call\nthe ``close()`` method when done or else use it as a context manager like so::\n\n    with iterpath(...) as ip:\n        for path in ip:\n            ...\n\nIf ``return_relative`` is true, the generated ``Path`` objects will be relative\nto ``dirpath``.  If ``return_relative`` is false (the default) and ``dirpath``\nis an absolute path, the generated ``Path`` objects will be absolute;\notherwise, if ``dirpath`` is a relative path, the ``Path`` objects will be\nrelative and will have ``dirpath`` as a prefix.\n\nNote that, although ``iterpath()`` yields ``pathlib.Path`` objects, it operates\ninternally on ``os.DirEntry`` objects, and so any function supplied as the\n``sort_key`` parameter or as a filter/exclude parameter must accept\n``os.DirEntry`` instances.\n\nKeyword arguments:\n\n``dirs: bool = True``\n    Whether to include directories in the output\n\n``topdown: bool = True``\n    Whether to yield each directory before (``True``) or after (``False``) its\n    contents\n\n``include_root: bool = False``\n    Whether to include the ``dirpath`` argument passed to ``iterpath()`` in the\n    output\n\n``followlinks: bool = False``\n    Whether to treat a symlink to a directory as a directory\n\n``return_relative: bool = False``\n    If true, the generated paths will be relative to ``dirpath``\n\n``onerror: Optional[Callable[[OSError], Any]] = None``\n    Specify a function to be called whenever an ``OSError`` is encountered\n    while iterating over a directory.  If the function reraises the exception,\n    ``iterpath()`` aborts; otherwise, it continues with the next directory.  By\n    default, ``OSError`` exceptions are ignored.\n\n``sort: bool = False``\n    Sort the entries in each directory.  When ``False``, entries are yielded in\n    the order returned by ``os.scandir()``.  When ``True``, entries are sorted,\n    by default, by filename in ascending order, but this can be changed via the\n    ``sort_key`` and ``sort_reverse`` arguments.\n\n``sort_key: Optional[Callable[[os.DirEntry[AnyStr]], _typeshed.SupportsLessThan]] = None``\n    Specify a custom key function for sorting directory entries.  Only has an\n    effect when ``sort`` is ``True``.\n\n``sort_reverse: bool = False``\n    Sort directory entries in reverse order.  Only has an effect when ``sort``\n    is ``True``.\n\n``filter: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``\n    Specify a predicate to be applied to all files & directories encountered;\n    only those for which the predicate returns a true value will be yielded\n    (and, for directories, descended into).\n\n    If ``filter`` is specified, it is an error to also specify ``filter_dirs``\n    or ``filter_files``.\n\n``filter_dirs: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``\n    Specify a predicate to be applied to all directories encountered; only\n    those for which the predicate returns a true value will be yielded &\n    descended into\n\n``filter_files: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``\n    Specify a predicate to be applied to all files encountered; only those for\n    which the predicate returns a true value will be yielded\n\n``exclude: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``\n    Specify a predicate to be applied to all files & directories encountered;\n    only those for which the predicate returns a false value will be yielded\n    (and, for directories, descended into).\n\n    If ``exclude`` is specified, it is an error to also specify ``exclude_dirs``\n    or ``exclude_files``.\n\n``exclude_dirs: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``\n    Specify a predicate to be applied to all directories encountered; only\n    those for which the predicate returns a false value will be yielded &\n    descended into\n\n``exclude_files: Optional[Callable[[os.DirEntry[AnyStr]], Any]] = None``\n    Specify a predicate to be applied to all files encountered; only those for\n    which the predicate returns a false value will be yielded\n\nIf both ``filter`` and ``exclude`` are set, a given entry will only be included\nif ``filter`` returns true and ``exclude`` returns false (that is, exclusions\ntake priority over inclusions), and likewise for the directory- and\nfile-specific arguments.\n\n**Warnings:**\n\n- If ``dirpath`` is a relative path, changing the working directory while\n  ``iterpath()`` is in progress will lead to errors, or at least inaccurate\n  results.\n\n- Setting ``followlinks`` to ``True`` can result in infinite recursion if a\n  symlink points to a parent directory of itself.\n\nSelectors\n---------\n\n*New in version 0.3.0*\n\n``iterpath`` also provides a selection of \"selector\" classes & constants for\neasy construction of ``filter`` and ``exclude`` arguments.  Selectors are\ncallables that return true for ``DirEntry``'s whose (base) names match given\ncriteria.\n\nSelectors can even be combined using the ``|`` operator:\n\n.. code:: python\n\n    # This only returns entries whose names end in \".txt\" or equal \"foo.png\" or\n    # \".hidden\":\n    iterpath(\n        dirpath,\n        filter=SelectGlob(\"*.txt\") | SelectNames(\"foo.png\", \".hidden\")\n    )\n\n    # Exclude all dot-directories and VCS directories:\n    iterpath(dirpath, exclude_dirs=SELECT_DOTS | SELECT_VCS_DIRS)\n\nThe selectors:\n\n.. code:: python\n\n    class SelectNames(*names: AnyStr, case_sensitive: bool = True)\n\nSelects ``DirEntry``'s whose names are one of ``names``.  If ``case_sensitive``\nis ``False``, the check is performed case-insensitively.\n\n.. code:: python\n\n    class SelectGlob(pattern: AnyStr)\n\nSelects ``DirEntry``'s whose names match the given fileglob pattern\n\n.. code:: python\n\n    class SelectRegex(pattern: AnyStr | re.Pattern[AnyStr])\n\nSelects ``DirEntry``'s whose names match (using ``re.search()``) the given\nregular expression\n\n.. code:: python\n\n    SELECT_DOTS\n\nSelects ``DirEntry``'s whose names begin with a period\n\n.. code:: python\n\n    SELECT_VCS\n\nSelects ``DirEntry``'s matched by either ``SELECT_VCS_DIRS`` or\n``SELECT_VCS_FILES`` (see below)\n\n.. code:: python\n\n    SELECT_VCS_DIRS\n\nSelects the following names of version-control directories: ``.git``, ``.hg``,\n``_darcs``, ``.bzr``, ``.svn``, ``_svn``, ``CVS``, ``RCS``\n\n.. code:: python\n\n    SELECT_VCS_FILES\n\nSelects the following names of version-control-specific files:\n``.gitattributes``, ``.gitignore``, ``.gitmodules``, ``.mailmap``,\n``.hgignore``, ``.hgsigs``, ``.hgtags``, ``.binaries``, ``.boring``,\n``.bzrignore``, and all nonempty filenames that end in ``,v``\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Iterate through a file tree",
    "version": "0.4.0",
    "split_keywords": [
        "direntry",
        "directories",
        "directory tree",
        "file tree",
        "files",
        "find",
        "path",
        "pathlib",
        "walk"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "638efe3d1d258da72c39a121c24c456d",
                "sha256": "7a7ac9199999a1ba52f179902d0778cad7cdeaa70247354060edcb2c29753738"
            },
            "downloads": -1,
            "filename": "iterpath-0.4.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "638efe3d1d258da72c39a121c24c456d",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "~=3.7",
            "size": 11212,
            "upload_time": "2022-06-25T16:16:35",
            "upload_time_iso_8601": "2022-06-25T16:16:35.463420Z",
            "url": "https://files.pythonhosted.org/packages/f8/c1/fbe4efc05469d4088b0413952ab7213c9ef9ffe2b2626ce338e8bd37a232/iterpath-0.4.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "e30cfdb41a0986dc80a5a7f56f8b48a5",
                "sha256": "47b899938fa3840774b9df91db2a90f7769bfc84b4e550ab814ab974fc12d772"
            },
            "downloads": -1,
            "filename": "iterpath-0.4.0.tar.gz",
            "has_sig": false,
            "md5_digest": "e30cfdb41a0986dc80a5a7f56f8b48a5",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "~=3.7",
            "size": 16374,
            "upload_time": "2022-06-25T16:16:36",
            "upload_time_iso_8601": "2022-06-25T16:16:36.761403Z",
            "url": "https://files.pythonhosted.org/packages/46/bb/37ec82f48d9297e64c130bcead60de512d128737ea7dc74aa206c4dd783b/iterpath-0.4.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2022-06-25 16:16:36",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "jwodder",
    "github_project": "iterpath",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "tox": true,
    "lcname": "iterpath"
}
        
Elapsed time: 0.38578s