Assorted filesystem related utility functions,
some of which have been bloating cs.fileutils for too long.
*Latest release 20240422*:
New scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.
## Function `atomic_directory(*da, **dkw)`
Decorator for a function which fills in a directory
which calls the function against a temporary directory
then renames the temporary to the target name on completion.
Parameters:
* `infill_func`: the function to fill in the target directory
* `make_placeholder`: optional flag, default `False`:
if true an empty directory will be make at the target name
and after completion it will be removed and the completed
directory renamed to the target name
## Function `findup(dirpath: str, criterion: Union[str, Callable[[str], Any]]) -> str`
Walk up the filesystem tree looking for a directory where
`criterion(fspath)` is not `None`, where `fspath` starts at `dirpath`.
Return the result of `criterion(fspath)`.
Return `None` if no such path is found.
Parameters:
* `dirpath`: the starting directory
* `criterion`: a `str` or a callable accepting a `str`
If `criterion` is a `str`, use look for the existence of `os.path.join(fspath,criterion)`
Example:
# find a directory containing a `.envrc` file
envrc_path = findup('.', '.envrc')
# find a Tagger rules file for the Downloads directory
rules_path = findup(expanduser('~/Downloads', '.taggerrc')
## Function `fnmatchdir(dirpath, fnglob)`
Return a list of the names in `dirpath` matching the glob `fnglob`.
## Class `FSPathBasedSingleton(cs.obj.SingletonMixin, HasFSPath)`
The basis for a `SingletonMixin` based on `realpath(self.fspath)`.
*Method `FSPathBasedSingleton.__init__(self, fspath: Optional[str] = None, lock=None)`*:
Initialise the singleton:
On the first call:
- set `.fspath` to `self._resolve_fspath(fspath)`
- set `._lock` to `lock` (or `threading.Lock()` if not specified)
- return `True`
On subsequent calls return `False`.
## Class `HasFSPath`
A mixin for an object with a `.fspath` attribute representing a filesystem location.
The `__init__` method just sets the `.fspath` attribute, and
need not be called if the main class takes care of that itself.
*Method `HasFSPath.fnmatch(self, fnglob)`*:
Return a list of the names in `self.fspath` matching the glob `fnglob`.
*Method `HasFSPath.listdir(self)`*:
Return `os.listdir(self.fspath)`.
*Method `HasFSPath.pathto(self, *subpaths)`*:
The full path to `subpaths`, comprising a relative path
below `self.fspath`.
This is a shim for `os.path.join` which requires that all
the `subpaths` be relative paths.
*Property `HasFSPath.shortpath`*:
The short version of `self.fspath`.
## Function `is_valid_rpath(rpath, log=None) -> bool`
Test that `rpath` is a clean relative path with no funny business.
This is a Boolean wrapper for `validate_rpath()`.
## Function `longpath(path, prefixes=None)`
Return `path` with prefixes and environment variables substituted.
The converse of `shortpath()`.
## Function `needdir(dirpath, mode=511, *, use_makedirs=False, log=None)`
Create the directory `dirpath` if missing.
Parameters:
* `dirpath`: the required directory path
* `mode`: the permissions mode, default `0o777`
* `log`: log `makedirs` or `mkdir` call
* `use_makedirs`: optional creation mode, default `False`;
if true, use `os.makedirs`, otherwise `os.mkdir`
## Function `rpaths(dirpath='.', **scan_kw)`
A shim for `scandirtree` to yield relative file paths from a directory.
Parameters:
* `dirpath`: optional top directory, default `'.'`
Other keyword arguments are passed to `scandirtree`.
## Function `scandirpaths(dirpath='.', **scan_kw)`
A shim for `scandirtree` to yield filesystem paths from a directory.
Parameters:
* `dirpath`: optional top directory, default `'.'`
Other keyword arguments are passed to `scandirtree`.
## Function `scandirtree(dirpath='.', *, include_dirs=False, name_selector=None, only_suffixes=None, skip_suffixes=None, sort_names=False, follow_symlinks=False, recurse=True)`
Generator to recurse over `dirpath`, yielding `(is_dir,subpath)`
for all selected subpaths.
Parameters:
* `dirpath`: the directory to scan, default `'.'`
* `include_dirs`: if true yield directories; default `False`
* `name_selector`: optional callable to select particular names;
the default is to select names no starting with a dot (`'.'`)
* `only_suffixes`: if supplied, skip entries whose extension
is not in `only_suffixes`
* `skip_suffixes`: if supplied, skip entries whose extension
is in `skip_suffixes`
* `sort_names`: option flag, default `False`; yield entires
in lexical order if true
* `follow_symlinks`: optional flag, default `False`; passed to `scandir`
* `recurse`: optional flag, default `True`; if true, recurse into subdrectories
## Function `shortpath(path, prefixes=None)`
Return `path` with the first matching leading prefix replaced.
Parameters:
* `environ`: environment mapping if not os.environ
* `prefixes`: optional iterable of `(prefix,subst)` to consider for replacement;
each `prefix` is subject to environment variable
substitution before consideration
The default considers "$HOME/" for replacement by "~/".
## Function `validate_rpath(rpath: str)`
Test that `rpath` is a clean relative path with no funny business;
raise `ValueError` if the test fails.
Tests:
- not empty or '.' or '..'
- not an absolute path
- normalised
- does not walk up out of its parent directory
Examples:
>>> validate_rpath('')
False
>>> validate_rpath('.')
# Release Log
*Release 20240422*:
New scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.
*Release 20240412*:
HasFSPath: explain that the __init__ is optional in the docstring.
*Release 20240316*:
Fixed release upload artifacts.
*Release 20240201*:
* FSPathBasedSingleton: drop the default_factory parameter/attribute, let default_attr specify a callable.
* Singleton._resolve_fspath: fix reference to class name.
*Release 20231129*:
* HasFSPath: new listdir method.
* HasFSPath.pathto: accept multiple relative subpaths.
* FSPathBasedSingleton: accept cls.FSPATH_FACTORY as a factory function for the default fspath, makes it possible to defer the path lookup.
* Replace is_clean_subpath with validate_rpath/is_valid_rpath pair.
*Release 20230806*:
* Reimplement fnmatchdir using fnmatch.filter.
* No longer claim Python 2 compatibility.
*Release 20230401*:
HasFSPath.shortpath: hand call before .fspath set.
*Release 20221221*:
Replace use of cs.env.envsub with os.path.expandvars and drop unused environ parameter.
*Release 20220918*:
* FSPathBasedSingleton.__init__: return True on the first call, False on subsequent calls.
* FSPathBasedSingleton.__init__: probe __dict__ for '_lock' instead of using hasattr (which plays poorly this early on with classes with their own __getattr__).
* needdir: accept optional `log` parameter to log mkdir or makedirs.
* HasFSPath: add a default __str__.
*Release 20220805*:
Doc update.
*Release 20220530*:
FSPathBasedSingleton._resolve_fspath: new `envvar` and `default_attr` parameters.
*Release 20220429*:
* New HasFSPath and FSPathBasedSingleton.
* Add longpath and shortpath from cs.fileutils.
* New is_clean_subpath(subpath).
* New needdir(path).
* New fnmatchdir(dirpath,fnglob) pulled out from HasFSPath.fnmatch(fnglob).
*Release 20220327*:
New module cs.fs to contain more filesystem focussed functions than cs.fileutils, which is feeling a bit bloated.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs.fs",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python2, python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/f1/31/ac4ac4d2e63fb11dfa5836e062ec60e30350d1d9af056a1a430259e4d8bf/cs.fs-20240422.tar.gz",
"platform": null,
"description": "Assorted filesystem related utility functions,\nsome of which have been bloating cs.fileutils for too long.\n\n*Latest release 20240422*:\nNew scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.\n\n## Function `atomic_directory(*da, **dkw)`\n\nDecorator for a function which fills in a directory\nwhich calls the function against a temporary directory\nthen renames the temporary to the target name on completion.\n\nParameters:\n* `infill_func`: the function to fill in the target directory\n* `make_placeholder`: optional flag, default `False`:\n if true an empty directory will be make at the target name\n and after completion it will be removed and the completed\n directory renamed to the target name\n\n## Function `findup(dirpath: str, criterion: Union[str, Callable[[str], Any]]) -> str`\n\nWalk up the filesystem tree looking for a directory where\n`criterion(fspath)` is not `None`, where `fspath` starts at `dirpath`.\nReturn the result of `criterion(fspath)`.\nReturn `None` if no such path is found.\n\nParameters:\n* `dirpath`: the starting directory\n* `criterion`: a `str` or a callable accepting a `str`\n\nIf `criterion` is a `str`, use look for the existence of `os.path.join(fspath,criterion)`\n\nExample:\n\n # find a directory containing a `.envrc` file\n envrc_path = findup('.', '.envrc')\n\n # find a Tagger rules file for the Downloads directory\n rules_path = findup(expanduser('~/Downloads', '.taggerrc')\n\n## Function `fnmatchdir(dirpath, fnglob)`\n\nReturn a list of the names in `dirpath` matching the glob `fnglob`.\n\n## Class `FSPathBasedSingleton(cs.obj.SingletonMixin, HasFSPath)`\n\nThe basis for a `SingletonMixin` based on `realpath(self.fspath)`.\n\n*Method `FSPathBasedSingleton.__init__(self, fspath: Optional[str] = None, lock=None)`*:\nInitialise the singleton:\n\nOn the first call:\n- set `.fspath` to `self._resolve_fspath(fspath)`\n- set `._lock` to `lock` (or `threading.Lock()` if not specified)\n- return `True`\nOn subsequent calls return `False`.\n\n## Class `HasFSPath`\n\nA mixin for an object with a `.fspath` attribute representing a filesystem location.\n\nThe `__init__` method just sets the `.fspath` attribute, and\nneed not be called if the main class takes care of that itself.\n\n*Method `HasFSPath.fnmatch(self, fnglob)`*:\nReturn a list of the names in `self.fspath` matching the glob `fnglob`.\n\n*Method `HasFSPath.listdir(self)`*:\nReturn `os.listdir(self.fspath)`.\n\n*Method `HasFSPath.pathto(self, *subpaths)`*:\nThe full path to `subpaths`, comprising a relative path\nbelow `self.fspath`.\nThis is a shim for `os.path.join` which requires that all\nthe `subpaths` be relative paths.\n\n*Property `HasFSPath.shortpath`*:\nThe short version of `self.fspath`.\n\n## Function `is_valid_rpath(rpath, log=None) -> bool`\n\nTest that `rpath` is a clean relative path with no funny business.\n\nThis is a Boolean wrapper for `validate_rpath()`.\n\n## Function `longpath(path, prefixes=None)`\n\nReturn `path` with prefixes and environment variables substituted.\nThe converse of `shortpath()`.\n\n## Function `needdir(dirpath, mode=511, *, use_makedirs=False, log=None)`\n\nCreate the directory `dirpath` if missing.\n\nParameters:\n* `dirpath`: the required directory path\n* `mode`: the permissions mode, default `0o777`\n* `log`: log `makedirs` or `mkdir` call\n* `use_makedirs`: optional creation mode, default `False`;\n if true, use `os.makedirs`, otherwise `os.mkdir`\n\n## Function `rpaths(dirpath='.', **scan_kw)`\n\nA shim for `scandirtree` to yield relative file paths from a directory.\n\nParameters:\n* `dirpath`: optional top directory, default `'.'`\n\nOther keyword arguments are passed to `scandirtree`.\n\n## Function `scandirpaths(dirpath='.', **scan_kw)`\n\nA shim for `scandirtree` to yield filesystem paths from a directory.\n\nParameters:\n* `dirpath`: optional top directory, default `'.'`\n\nOther keyword arguments are passed to `scandirtree`.\n\n## Function `scandirtree(dirpath='.', *, include_dirs=False, name_selector=None, only_suffixes=None, skip_suffixes=None, sort_names=False, follow_symlinks=False, recurse=True)`\n\nGenerator to recurse over `dirpath`, yielding `(is_dir,subpath)`\nfor all selected subpaths.\n\nParameters:\n* `dirpath`: the directory to scan, default `'.'`\n* `include_dirs`: if true yield directories; default `False`\n* `name_selector`: optional callable to select particular names;\n the default is to select names no starting with a dot (`'.'`)\n* `only_suffixes`: if supplied, skip entries whose extension\n is not in `only_suffixes`\n* `skip_suffixes`: if supplied, skip entries whose extension\n is in `skip_suffixes`\n* `sort_names`: option flag, default `False`; yield entires\n in lexical order if true\n* `follow_symlinks`: optional flag, default `False`; passed to `scandir`\n* `recurse`: optional flag, default `True`; if true, recurse into subdrectories\n\n## Function `shortpath(path, prefixes=None)`\n\nReturn `path` with the first matching leading prefix replaced.\n\nParameters:\n* `environ`: environment mapping if not os.environ\n* `prefixes`: optional iterable of `(prefix,subst)` to consider for replacement;\n each `prefix` is subject to environment variable\n substitution before consideration\n The default considers \"$HOME/\" for replacement by \"~/\".\n\n## Function `validate_rpath(rpath: str)`\n\nTest that `rpath` is a clean relative path with no funny business;\nraise `ValueError` if the test fails.\n\nTests:\n- not empty or '.' or '..'\n- not an absolute path\n- normalised\n- does not walk up out of its parent directory\n\nExamples:\n\n >>> validate_rpath('')\n False\n >>> validate_rpath('.')\n\n# Release Log\n\n\n\n*Release 20240422*:\nNew scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.\n\n*Release 20240412*:\nHasFSPath: explain that the __init__ is optional in the docstring.\n\n*Release 20240316*:\nFixed release upload artifacts.\n\n*Release 20240201*:\n* FSPathBasedSingleton: drop the default_factory parameter/attribute, let default_attr specify a callable.\n* Singleton._resolve_fspath: fix reference to class name.\n\n*Release 20231129*:\n* HasFSPath: new listdir method.\n* HasFSPath.pathto: accept multiple relative subpaths.\n* FSPathBasedSingleton: accept cls.FSPATH_FACTORY as a factory function for the default fspath, makes it possible to defer the path lookup.\n* Replace is_clean_subpath with validate_rpath/is_valid_rpath pair.\n\n*Release 20230806*:\n* Reimplement fnmatchdir using fnmatch.filter.\n* No longer claim Python 2 compatibility.\n\n*Release 20230401*:\nHasFSPath.shortpath: hand call before .fspath set.\n\n*Release 20221221*:\nReplace use of cs.env.envsub with os.path.expandvars and drop unused environ parameter.\n\n*Release 20220918*:\n* FSPathBasedSingleton.__init__: return True on the first call, False on subsequent calls.\n* FSPathBasedSingleton.__init__: probe __dict__ for '_lock' instead of using hasattr (which plays poorly this early on with classes with their own __getattr__).\n* needdir: accept optional `log` parameter to log mkdir or makedirs.\n* HasFSPath: add a default __str__.\n\n*Release 20220805*:\nDoc update.\n\n*Release 20220530*:\nFSPathBasedSingleton._resolve_fspath: new `envvar` and `default_attr` parameters.\n\n*Release 20220429*:\n* New HasFSPath and FSPathBasedSingleton.\n* Add longpath and shortpath from cs.fileutils.\n* New is_clean_subpath(subpath).\n* New needdir(path).\n* New fnmatchdir(dirpath,fnglob) pulled out from HasFSPath.fnmatch(fnglob).\n\n*Release 20220327*:\nNew module cs.fs to contain more filesystem focussed functions than cs.fileutils, which is feeling a bit bloated.\n\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 or later (GPLv3+)",
"summary": "Assorted filesystem related utility functions, some of which have been bloating cs.fileutils for too long.",
"version": "20240422",
"project_urls": {
"URL": "https://bitbucket.org/cameron_simpson/css/commits/all"
},
"split_keywords": [
"python2",
" python3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e3e9bf02ebc4487756a52de9941b80fae5e515a1e0b74d5d310f12a0afc0a777",
"md5": "78b48200fb6f15f56ef435d00140ca0c",
"sha256": "7f272c2572b2ae206d99ec726b7f759ef9643fa60fca975b83e5d23af4252d36"
},
"downloads": -1,
"filename": "cs.fs-20240422-py3-none-any.whl",
"has_sig": false,
"md5_digest": "78b48200fb6f15f56ef435d00140ca0c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 8751,
"upload_time": "2024-04-22T02:50:41",
"upload_time_iso_8601": "2024-04-22T02:50:41.805387Z",
"url": "https://files.pythonhosted.org/packages/e3/e9/bf02ebc4487756a52de9941b80fae5e515a1e0b74d5d310f12a0afc0a777/cs.fs-20240422-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f131ac4ac4d2e63fb11dfa5836e062ec60e30350d1d9af056a1a430259e4d8bf",
"md5": "b45d70486f0a49e46c612c0c921d75bb",
"sha256": "a9aeb9d4ef0ea08c088de0ee1b76f9265aa4cdaff5fb0df820d6e1cbd1673e67"
},
"downloads": -1,
"filename": "cs.fs-20240422.tar.gz",
"has_sig": false,
"md5_digest": "b45d70486f0a49e46c612c0c921d75bb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 8203,
"upload_time": "2024-04-22T02:50:43",
"upload_time_iso_8601": "2024-04-22T02:50:43.878622Z",
"url": "https://files.pythonhosted.org/packages/f1/31/ac4ac4d2e63fb11dfa5836e062ec60e30350d1d9af056a1a430259e4d8bf/cs.fs-20240422.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-22 02:50:43",
"github": false,
"gitlab": false,
"bitbucket": true,
"codeberg": false,
"bitbucket_user": "cameron_simpson",
"bitbucket_project": "css",
"lcname": "cs.fs"
}
JSON