Assorted filesystem related utility functions,
some of which have been bloating cs.fileutils for too long.
*Latest release 20241122*:
* FSPathBasedSingleton: add a .promote method to promote a filesystem path to an instance.
* FSPathBasedSingleton: new fspath_normalised(fspath) class method to produce a normalised form of the fspath for use as the key to the singleton registry.
## <a name="atomic_directory"></a>`atomic_directory(*da, **dkw)`
Decorator for a function which fills in a directory
which calls the function against a temporary directory
then renames the temporary to the target name on completion.
Parameters:
* `infill_func`: the function to fill in the target directory
* `make_placeholder`: optional flag, default `False`:
if true an empty directory will be make at the target name
and after completion it will be removed and the completed
directory renamed to the target name
## <a name="findup"></a>`findup(dirpath: str, criterion: Union[str, Callable[[str], Any]]) -> str`
Walk up the filesystem tree looking for a directory where
`criterion(fspath)` is not `None`, where `fspath` starts at `dirpath`.
Return the result of `criterion(fspath)`.
Return `None` if no such path is found.
Parameters:
* `dirpath`: the starting directory
* `criterion`: a `str` or a callable accepting a `str`
If `criterion` is a `str`, use look for the existence of `os.path.join(fspath,criterion)`
Example:
# find a directory containing a `.envrc` file
envrc_path = findup('.', '.envrc')
# find a Tagger rules file for the Downloads directory
rules_path = findup(expanduser('~/Downloads', '.taggerrc')
## <a name="fnmatchdir"></a>`fnmatchdir(dirpath, fnglob)`
Return a list of the names in `dirpath` matching the glob `fnglob`.
## <a name="FSPathBasedSingleton"></a>Class `FSPathBasedSingleton(cs.obj.SingletonMixin, HasFSPath, cs.deco.Promotable)`
The basis for a `SingletonMixin` based on `realpath(self.fspath)`.
*`FSPathBasedSingleton.__init__(self, fspath: Optional[str] = None, lock=None)`*:
Initialise the singleton:
On the first call:
- set `.fspath` to `self._resolve_fspath(fspath)`
- set `._lock` to `lock` (or `cs.threads.NRLock()` if not specified)
*`FSPathBasedSingleton.fspath_normalised(fspath: str)`*:
Return the normalised form of the filesystem path `fspath`,
used as the key for the singleton registry.
This default returns `realpath(fspath)`.
As a contracting example, the `cs.ebooks.kindle.classic.KindleTree`
class tries to locate the directory containing the book
database, and returns its realpath, allowing some imprecision.
*`FSPathBasedSingleton.promote(obj)`*:
Promote `None` or `str` to a `CalibreTree`.
## <a name="HasFSPath"></a>Class `HasFSPath`
A mixin for an object with a `.fspath` attribute representing a filesystem location.
The `__init__` method just sets the `.fspath` attribute, and
need not be called if the main class takes care of that itself.
*`HasFSPath.fnmatch(self, fnglob)`*:
Return a list of the names in `self.fspath` matching the glob `fnglob`.
*`HasFSPath.listdir(self)`*:
Return `os.listdir(self.fspath)`.
*`HasFSPath.pathto(self, *subpaths)`*:
The full path to `subpaths`, comprising a relative path
below `self.fspath`.
This is a shim for `os.path.join` which requires that all
the `subpaths` be relative paths.
*`HasFSPath.shortpath`*:
The short version of `self.fspath`.
## <a name="is_valid_rpath"></a>`is_valid_rpath(rpath, log=None) -> bool`
Test that `rpath` is a clean relative path with no funny business.
This is a Boolean wrapper for `validate_rpath()`.
## <a name="longpath"></a>`longpath(path, prefixes=None)`
Return `path` with prefixes and environment variables substituted.
The converse of `shortpath()`.
## <a name="needdir"></a>`needdir(dirpath, mode=511, *, use_makedirs=False, log=None) -> bool`
Create the directory `dirpath` if missing.
Return `True` if the directory was made, `False` otherwise.
Parameters:
* `dirpath`: the required directory path
* `mode`: the permissions mode, default `0o777`
* `log`: log `makedirs` or `mkdir` call
* `use_makedirs`: optional creation mode, default `False`;
if true, use `os.makedirs`, otherwise `os.mkdir`
## <a name="rpaths"></a>`rpaths(dirpath='.', **scan_kw)`
A shim for `scandirtree` to yield relative file paths from a directory.
Parameters:
* `dirpath`: optional top directory, default `'.'`
Other keyword arguments are passed to `scandirtree`.
## <a name="scandirpaths"></a>`scandirpaths(dirpath='.', **scan_kw)`
A shim for `scandirtree` to yield filesystem paths from a directory.
Parameters:
* `dirpath`: optional top directory, default `'.'`
Other keyword arguments are passed to `scandirtree`.
## <a name="scandirtree"></a>`scandirtree(dirpath='.', *, include_dirs=False, name_selector=None, only_suffixes=None, skip_suffixes=None, sort_names=False, follow_symlinks=False, recurse=True)`
Generator to recurse over `dirpath`, yielding `(is_dir,subpath)`
for all selected subpaths.
Parameters:
* `dirpath`: the directory to scan, default `'.'`
* `include_dirs`: if true yield directories; default `False`
* `name_selector`: optional callable to select particular names;
the default is to select names not starting with a dot (`'.'`)
* `only_suffixes`: if supplied, skip entries whose extension
is not in `only_suffixes`
* `skip_suffixes`: if supplied, skip entries whose extension
is in `skip_suffixes`
* `sort_names`: option flag, default `False`; yield entires
in lexical order if true
* `follow_symlinks`: optional flag, default `False`; passed to `scandir`
* `recurse`: optional flag, default `True`; if true, recurse into subdrectories
## <a name="shortpath"></a>`shortpath(fspath, prefixes=None, *, collapseuser=False, foldsymlinks=False)`
Return `fspath` with the first matching leading prefix replaced.
Parameters:
* `prefixes`: optional list of `(prefix,subst)` pairs
* `collapseuser`: optional flag to enable detection of user
home directory paths; default `False`
* `foldsymlinks`: optional flag to enable detection of
convenience symlinks which point deeper into the path;
default `False`
The `prefixes` is an optional iterable of `(prefix,subst)`
to consider for replacement. Each `prefix` is subject to
environment variable substitution before consideration.
The default `prefixes` is from `SHORTPATH_PREFIXES_DEFAULT`:
`(('$HOME/', '~/'),)`.
## <a name="validate_rpath"></a>`validate_rpath(rpath: str)`
Test that `rpath` is a clean relative path with no funny business;
raise `ValueError` if the test fails.
Tests:
- not empty or '.' or '..'
- not an absolute path
- normalised
- does not walk up out of its parent directory
Examples:
>>> validate_rpath('')
False
>>> validate_rpath('.')
# Release Log
*Release 20241122*:
* FSPathBasedSingleton: add a .promote method to promote a filesystem path to an instance.
* FSPathBasedSingleton: new fspath_normalised(fspath) class method to produce a normalised form of the fspath for use as the key to the singleton registry.
*Release 20241007*:
FSPathBasedSingleton.__init__: use an NRLock for the default lock, using a late import with fallback to Lock.
*Release 20241005*:
needdir: now returns True if the directory was made, False if it already existed.
*Release 20240630*:
FSPathBasedSingleton: recent Pythons seem to check that __init__ returns None, subclasses must test another way.
*Release 20240623*:
* shortpath(foldsymlinks=True): only examine symlinks which have clean subpaths in their link text - this avoids junk and also avoids stat()ing links which might be symlinks to mount points which might be offline.
* scandirtree: clean up the logic, possibly fix repeated mention of directories.
*Release 20240522*:
shortpath: new collapseuser=False, foldsymlinks=False parameters, rename DEFAULT_SHORTEN_PREFIXES to SHORTPATH_PREFIXES_DEFAULT.
*Release 20240422*:
New scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.
*Release 20240412*:
HasFSPath: explain that the __init__ is optional in the docstring.
*Release 20240316*:
Fixed release upload artifacts.
*Release 20240201*:
* FSPathBasedSingleton: drop the default_factory parameter/attribute, let default_attr specify a callable.
* Singleton._resolve_fspath: fix reference to class name.
*Release 20231129*:
* HasFSPath: new listdir method.
* HasFSPath.pathto: accept multiple relative subpaths.
* FSPathBasedSingleton: accept cls.FSPATH_FACTORY as a factory function for the default fspath, makes it possible to defer the path lookup.
* Replace is_clean_subpath with validate_rpath/is_valid_rpath pair.
*Release 20230806*:
* Reimplement fnmatchdir using fnmatch.filter.
* No longer claim Python 2 compatibility.
*Release 20230401*:
HasFSPath.shortpath: hand call before .fspath set.
*Release 20221221*:
Replace use of cs.env.envsub with os.path.expandvars and drop unused environ parameter.
*Release 20220918*:
* FSPathBasedSingleton.__init__: return True on the first call, False on subsequent calls.
* FSPathBasedSingleton.__init__: probe __dict__ for '_lock' instead of using hasattr (which plays poorly this early on with classes with their own __getattr__).
* needdir: accept optional `log` parameter to log mkdir or makedirs.
* HasFSPath: add a default __str__.
*Release 20220805*:
Doc update.
*Release 20220530*:
FSPathBasedSingleton._resolve_fspath: new `envvar` and `default_attr` parameters.
*Release 20220429*:
* New HasFSPath and FSPathBasedSingleton.
* Add longpath and shortpath from cs.fileutils.
* New is_clean_subpath(subpath).
* New needdir(path).
* New fnmatchdir(dirpath,fnglob) pulled out from HasFSPath.fnmatch(fnglob).
*Release 20220327*:
New module cs.fs to contain more filesystem focussed functions than cs.fileutils, which is feeling a bit bloated.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs-fs",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python2, python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/01/39/66ad5e52447aafd9bf301bcf6dcc0ba5759a94f220e6e2ba0d76dbc97605/cs_fs-20241122.tar.gz",
"platform": null,
"description": "Assorted filesystem related utility functions,\nsome of which have been bloating cs.fileutils for too long.\n\n*Latest release 20241122*:\n* FSPathBasedSingleton: add a .promote method to promote a filesystem path to an instance.\n* FSPathBasedSingleton: new fspath_normalised(fspath) class method to produce a normalised form of the fspath for use as the key to the singleton registry.\n\n## <a name=\"atomic_directory\"></a>`atomic_directory(*da, **dkw)`\n\nDecorator for a function which fills in a directory\nwhich calls the function against a temporary directory\nthen renames the temporary to the target name on completion.\n\nParameters:\n* `infill_func`: the function to fill in the target directory\n* `make_placeholder`: optional flag, default `False`:\n if true an empty directory will be make at the target name\n and after completion it will be removed and the completed\n directory renamed to the target name\n\n## <a name=\"findup\"></a>`findup(dirpath: str, criterion: Union[str, Callable[[str], Any]]) -> str`\n\nWalk up the filesystem tree looking for a directory where\n`criterion(fspath)` is not `None`, where `fspath` starts at `dirpath`.\nReturn the result of `criterion(fspath)`.\nReturn `None` if no such path is found.\n\nParameters:\n* `dirpath`: the starting directory\n* `criterion`: a `str` or a callable accepting a `str`\n\nIf `criterion` is a `str`, use look for the existence of `os.path.join(fspath,criterion)`\n\nExample:\n\n # find a directory containing a `.envrc` file\n envrc_path = findup('.', '.envrc')\n\n # find a Tagger rules file for the Downloads directory\n rules_path = findup(expanduser('~/Downloads', '.taggerrc')\n\n## <a name=\"fnmatchdir\"></a>`fnmatchdir(dirpath, fnglob)`\n\nReturn a list of the names in `dirpath` matching the glob `fnglob`.\n\n## <a name=\"FSPathBasedSingleton\"></a>Class `FSPathBasedSingleton(cs.obj.SingletonMixin, HasFSPath, cs.deco.Promotable)`\n\nThe basis for a `SingletonMixin` based on `realpath(self.fspath)`.\n\n*`FSPathBasedSingleton.__init__(self, fspath: Optional[str] = None, lock=None)`*:\nInitialise the singleton:\n\nOn the first call:\n- set `.fspath` to `self._resolve_fspath(fspath)`\n- set `._lock` to `lock` (or `cs.threads.NRLock()` if not specified)\n\n*`FSPathBasedSingleton.fspath_normalised(fspath: str)`*:\nReturn the normalised form of the filesystem path `fspath`,\nused as the key for the singleton registry.\n\nThis default returns `realpath(fspath)`.\n\nAs a contracting example, the `cs.ebooks.kindle.classic.KindleTree`\nclass tries to locate the directory containing the book\ndatabase, and returns its realpath, allowing some imprecision.\n\n*`FSPathBasedSingleton.promote(obj)`*:\nPromote `None` or `str` to a `CalibreTree`.\n\n## <a name=\"HasFSPath\"></a>Class `HasFSPath`\n\nA mixin for an object with a `.fspath` attribute representing a filesystem location.\n\nThe `__init__` method just sets the `.fspath` attribute, and\nneed not be called if the main class takes care of that itself.\n\n*`HasFSPath.fnmatch(self, fnglob)`*:\nReturn a list of the names in `self.fspath` matching the glob `fnglob`.\n\n*`HasFSPath.listdir(self)`*:\nReturn `os.listdir(self.fspath)`.\n\n*`HasFSPath.pathto(self, *subpaths)`*:\nThe full path to `subpaths`, comprising a relative path\nbelow `self.fspath`.\nThis is a shim for `os.path.join` which requires that all\nthe `subpaths` be relative paths.\n\n*`HasFSPath.shortpath`*:\nThe short version of `self.fspath`.\n\n## <a name=\"is_valid_rpath\"></a>`is_valid_rpath(rpath, log=None) -> bool`\n\nTest that `rpath` is a clean relative path with no funny business.\n\nThis is a Boolean wrapper for `validate_rpath()`.\n\n## <a name=\"longpath\"></a>`longpath(path, prefixes=None)`\n\nReturn `path` with prefixes and environment variables substituted.\nThe converse of `shortpath()`.\n\n## <a name=\"needdir\"></a>`needdir(dirpath, mode=511, *, use_makedirs=False, log=None) -> bool`\n\nCreate the directory `dirpath` if missing.\nReturn `True` if the directory was made, `False` otherwise.\n\nParameters:\n* `dirpath`: the required directory path\n* `mode`: the permissions mode, default `0o777`\n* `log`: log `makedirs` or `mkdir` call\n* `use_makedirs`: optional creation mode, default `False`;\n if true, use `os.makedirs`, otherwise `os.mkdir`\n\n## <a name=\"rpaths\"></a>`rpaths(dirpath='.', **scan_kw)`\n\nA shim for `scandirtree` to yield relative file paths from a directory.\n\nParameters:\n* `dirpath`: optional top directory, default `'.'`\n\nOther keyword arguments are passed to `scandirtree`.\n\n## <a name=\"scandirpaths\"></a>`scandirpaths(dirpath='.', **scan_kw)`\n\nA shim for `scandirtree` to yield filesystem paths from a directory.\n\nParameters:\n* `dirpath`: optional top directory, default `'.'`\n\nOther keyword arguments are passed to `scandirtree`.\n\n## <a name=\"scandirtree\"></a>`scandirtree(dirpath='.', *, include_dirs=False, name_selector=None, only_suffixes=None, skip_suffixes=None, sort_names=False, follow_symlinks=False, recurse=True)`\n\nGenerator to recurse over `dirpath`, yielding `(is_dir,subpath)`\nfor all selected subpaths.\n\nParameters:\n* `dirpath`: the directory to scan, default `'.'`\n* `include_dirs`: if true yield directories; default `False`\n* `name_selector`: optional callable to select particular names;\n the default is to select names not starting with a dot (`'.'`)\n* `only_suffixes`: if supplied, skip entries whose extension\n is not in `only_suffixes`\n* `skip_suffixes`: if supplied, skip entries whose extension\n is in `skip_suffixes`\n* `sort_names`: option flag, default `False`; yield entires\n in lexical order if true\n* `follow_symlinks`: optional flag, default `False`; passed to `scandir`\n* `recurse`: optional flag, default `True`; if true, recurse into subdrectories\n\n## <a name=\"shortpath\"></a>`shortpath(fspath, prefixes=None, *, collapseuser=False, foldsymlinks=False)`\n\nReturn `fspath` with the first matching leading prefix replaced.\n\nParameters:\n* `prefixes`: optional list of `(prefix,subst)` pairs\n* `collapseuser`: optional flag to enable detection of user\n home directory paths; default `False`\n* `foldsymlinks`: optional flag to enable detection of\n convenience symlinks which point deeper into the path;\n default `False`\n\nThe `prefixes` is an optional iterable of `(prefix,subst)`\nto consider for replacement. Each `prefix` is subject to\nenvironment variable substitution before consideration.\nThe default `prefixes` is from `SHORTPATH_PREFIXES_DEFAULT`:\n`(('$HOME/', '~/'),)`.\n\n## <a name=\"validate_rpath\"></a>`validate_rpath(rpath: str)`\n\nTest that `rpath` is a clean relative path with no funny business;\nraise `ValueError` if the test fails.\n\nTests:\n- not empty or '.' or '..'\n- not an absolute path\n- normalised\n- does not walk up out of its parent directory\n\nExamples:\n\n >>> validate_rpath('')\n False\n >>> validate_rpath('.')\n\n# Release Log\n\n\n\n*Release 20241122*:\n* FSPathBasedSingleton: add a .promote method to promote a filesystem path to an instance.\n* FSPathBasedSingleton: new fspath_normalised(fspath) class method to produce a normalised form of the fspath for use as the key to the singleton registry.\n\n*Release 20241007*:\nFSPathBasedSingleton.__init__: use an NRLock for the default lock, using a late import with fallback to Lock.\n\n*Release 20241005*:\nneeddir: now returns True if the directory was made, False if it already existed.\n\n*Release 20240630*:\nFSPathBasedSingleton: recent Pythons seem to check that __init__ returns None, subclasses must test another way.\n\n*Release 20240623*:\n* shortpath(foldsymlinks=True): only examine symlinks which have clean subpaths in their link text - this avoids junk and also avoids stat()ing links which might be symlinks to mount points which might be offline.\n* scandirtree: clean up the logic, possibly fix repeated mention of directories.\n\n*Release 20240522*:\nshortpath: new collapseuser=False, foldsymlinks=False parameters, rename DEFAULT_SHORTEN_PREFIXES to SHORTPATH_PREFIXES_DEFAULT.\n\n*Release 20240422*:\nNew scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.\n\n*Release 20240412*:\nHasFSPath: explain that the __init__ is optional in the docstring.\n\n*Release 20240316*:\nFixed release upload artifacts.\n\n*Release 20240201*:\n* FSPathBasedSingleton: drop the default_factory parameter/attribute, let default_attr specify a callable.\n* Singleton._resolve_fspath: fix reference to class name.\n\n*Release 20231129*:\n* HasFSPath: new listdir method.\n* HasFSPath.pathto: accept multiple relative subpaths.\n* FSPathBasedSingleton: accept cls.FSPATH_FACTORY as a factory function for the default fspath, makes it possible to defer the path lookup.\n* Replace is_clean_subpath with validate_rpath/is_valid_rpath pair.\n\n*Release 20230806*:\n* Reimplement fnmatchdir using fnmatch.filter.\n* No longer claim Python 2 compatibility.\n\n*Release 20230401*:\nHasFSPath.shortpath: hand call before .fspath set.\n\n*Release 20221221*:\nReplace use of cs.env.envsub with os.path.expandvars and drop unused environ parameter.\n\n*Release 20220918*:\n* FSPathBasedSingleton.__init__: return True on the first call, False on subsequent calls.\n* FSPathBasedSingleton.__init__: probe __dict__ for '_lock' instead of using hasattr (which plays poorly this early on with classes with their own __getattr__).\n* needdir: accept optional `log` parameter to log mkdir or makedirs.\n* HasFSPath: add a default __str__.\n\n*Release 20220805*:\nDoc update.\n\n*Release 20220530*:\nFSPathBasedSingleton._resolve_fspath: new `envvar` and `default_attr` parameters.\n\n*Release 20220429*:\n* New HasFSPath and FSPathBasedSingleton.\n* Add longpath and shortpath from cs.fileutils.\n* New is_clean_subpath(subpath).\n* New needdir(path).\n* New fnmatchdir(dirpath,fnglob) pulled out from HasFSPath.fnmatch(fnglob).\n\n*Release 20220327*:\nNew module cs.fs to contain more filesystem focussed functions than cs.fileutils, which is feeling a bit bloated.\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 or later (GPLv3+)",
"summary": "Assorted filesystem related utility functions, some of which have been bloating cs.fileutils for too long.",
"version": "20241122",
"project_urls": {
"MonoRepo Commits": "https://bitbucket.org/cameron_simpson/css/commits/branch/main",
"Monorepo Git Mirror": "https://github.com/cameron-simpson/css",
"Monorepo Hg/Mercurial Mirror": "https://hg.sr.ht/~cameron-simpson/css",
"Source": "https://github.com/cameron-simpson/css/blob/main/lib/python/cs/fs.py"
},
"split_keywords": [
"python2",
" python3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "132ee01b879d10fbce6a18c64b486e68251d953fe13db7a3af292d9b5a1a7e34",
"md5": "ccb9ec06b82f58a3ef8fddf7674795a4",
"sha256": "6f3b8d25bb7b2151afa2eda255c7f99a50d1166d9f575328cba72dda7dd18b74"
},
"downloads": -1,
"filename": "cs_fs-20241122-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ccb9ec06b82f58a3ef8fddf7674795a4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 10935,
"upload_time": "2024-11-22T00:51:03",
"upload_time_iso_8601": "2024-11-22T00:51:03.052259Z",
"url": "https://files.pythonhosted.org/packages/13/2e/e01b879d10fbce6a18c64b486e68251d953fe13db7a3af292d9b5a1a7e34/cs_fs-20241122-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "013966ad5e52447aafd9bf301bcf6dcc0ba5759a94f220e6e2ba0d76dbc97605",
"md5": "77b1d0e768dd260c1a23ccbec9dc3b2d",
"sha256": "d9469a13a13a6f4c322195fb3aaa3e0b6748fdfc8a33c7bba9b709174c70c1a2"
},
"downloads": -1,
"filename": "cs_fs-20241122.tar.gz",
"has_sig": false,
"md5_digest": "77b1d0e768dd260c1a23ccbec9dc3b2d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 12138,
"upload_time": "2024-11-22T00:51:04",
"upload_time_iso_8601": "2024-11-22T00:51:04.553877Z",
"url": "https://files.pythonhosted.org/packages/01/39/66ad5e52447aafd9bf301bcf6dcc0ba5759a94f220e6e2ba0d76dbc97605/cs_fs-20241122.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 00:51:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cameron-simpson",
"github_project": "css",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cs-fs"
}
JSON
