### Context
* Created: September 2020
* Author: Renier Kramer, renier.kramer@hdsr.nl
* Python version: >3.5
### Description
A python project that serves as an interface for finding directories and files by
combining best of both worlds: glob/rglob (speed) and regex (flexibility).
### Usage path_finder.FileFinder
```
# pip install path-finder
# pip install pathlib
from pathlib import Path
from path_finder import FileFinder
start_dir1 = Path('start_search_from_this_dir')
start_dir2 = Path('and_start_search_from_this_dir')
limit_depth = True
depth = 2 # 2, so search in start_dir1, subdir and subsubdirs (same for start_dir2)
filename_regex = '^[0-9]{8}_blabla'
extension = '.csv' # choose from ('.jpg', '.png', '.txt', '.xml', '.csv', '.xlsx', '.pdf', '.h5', '.nc', '.zip')
file_finder = FileFinder(
multi_start_dir=[start_dir1, start_dir2],
extension=extension,
limit_depth=True,
depth=depth,
filename_regex=filename_regex
)
paths = file_finder.paths # returns a List[Path]
paths_empty_files = file_finder.paths_empty_file # returns a List[Path]
```
### Usage path_finder.DirFinder
```
# pip install path-finder (or conda install --channel hdsr-mid path-finder)
# pip install pathlib
from pathlib import Path
from path_finder import DirFinder
dir_finder = DirFinder(
single_start_dir=Path('start_search_from_this_dir')
exclude_empty_dirs=True,
limit_depth=True,
depth=0, # so only search in single_start_dir
)
paths = dir_finder.paths # returns a List[Path]
paths_empty_files = dir_finder.paths_empty_file # returns a List[Path]
```
### License
[MIT][mit]
### Releases
[PyPi][pypi]
### Contributions
All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome.
Issues are posted on: https://github.com/hdsr-mid/path_finder/issues
[pypi]: https://pypi.org/project/path-finder/
[mit]: https://github.com/hdsr-mid/path_finder/blob/main/LICENSE.txt
### Test coverage (release v1.6)
```
---------- coverage: platform win32, python 3.7.11-final-0 ---
Name Stmts Miss Cover
------------------------------------------------
path_finder\__init__.py 2 0 100%
path_finder\base.py 45 8 82%
path_finder\dir_finder.py 65 11 83%
path_finder\file_finder.py 58 0 100%
setup.py 10 10 0%
------------------------------------------------
TOTAL 180 29 84%
```
### Conda general tips
#### Build conda environment (on Windows) from any directory using environment.yml:
Note1: prefix is not set in the enviroment.yml as then conda does not handle it very well
Note2: env_directory can be anywhere, it does not have to be in your code project
```
> conda env create --prefix <env_directory><env_name> --file <path_to_project>/environment.yml
# example: conda env create --prefix C:/Users/xxx/.conda/envs/project_xx --file C:/Users/code_projects/xx/environment.yml
> conda info --envs # verify that <env_name> (project_xx) is in this list
```
#### Start the application from any directory:
```
> conda activate <env_name>
At any location:
> (<env_name>) python <path_to_project>/main.py
```
#### Test the application:
```
> conda activate <env_name>
> cd <path_to_project>
> pytest # make sure pytest is installed (conda install pytest)
```
#### List all conda environments on your machine:
```
At any location:
> conda info --envs
```
#### Delete a conda environment:
```
Get directory where environment is located
> conda info --envs
Remove the enviroment
> conda env remove --name <env_name>
Finally, remove the left-over directory by hand
```
#### Write dependencies to environment.yml:
The goal is to keep the .yml as short as possible (not include sub-dependencies), yet make the environment
reproducible. Why? If you do 'conda install matplotlib' you also install sub-dependencies like pyqt, qt
icu, and sip. You should not include these sub-dependencies in your .yml as:
- including sub-dependencies result in an unnecessary strict environment (difficult to solve when conflicting)
- sub-dependencies will be installed when dependencies are being installed
```
> conda activate <conda_env_name>
Recommended:
> conda env export --from-history --no-builds | findstr -v "prefix" > --file <path_to_project>/environment_new.yml
Alternative:
> conda env export --no-builds | findstr -v "prefix" > --file <path_to_project>/environment_new.yml
--from-history:
Only include packages that you have explicitly asked for, as opposed to including every package in the
environment. This flag works regardless how you created the environment (through CMD or Anaconda Navigator).
--no-builds:
By default, the YAML includes platform-specific build constraints. If you transfer across platforms (e.g.
win32 to 64) omit the build info with '--no-builds'.
```
#### Pip and Conda:
If a package is not available on all conda channels, but available as pip package, one can install pip as a dependency.
Note that mixing packages from conda and pip is always a potential problem: conda calls pip, but pip does not know
how to satisfy missing dependencies with packages from Anaconda repositories.
```
> conda activate <env_name>
> conda install pip
> pip install <pip_package>
```
The environment.yml might look like:
```
channels:
- defaults
dependencies:
- <a conda package>=<version>
- pip
- pip:
- <a pip package>==<version>
```
You can also write a requirements.txt file:
```
> pip list --format=freeze > <path_to_project>/requirements.txt
```
Raw data
{
"_id": null,
"home_page": "https://github.com/hdsr-mid/path_finder",
"name": "path-finder",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "interface,path,directory,filename,glob,regex,find",
"author": "Renier Kramer",
"author_email": "renier.kramer@hdsr.nl",
"download_url": "https://files.pythonhosted.org/packages/c6/d8/2de7fd37ec23ad9f862faae90ddadbec22083d8da974e1872ad129f63027/path_finder-1.7.tar.gz",
"platform": null,
"description": "### Context\r\n* Created: September 2020\r\n* Author: Renier Kramer, renier.kramer@hdsr.nl\r\n* Python version: >3.5\r\n\r\n### Description\r\nA python project that serves as an interface for finding directories and files by \r\ncombining best of both worlds: glob/rglob (speed) and regex (flexibility).\r\n\r\n### Usage path_finder.FileFinder\r\n```\r\n# pip install path-finder\r\n# pip install pathlib\r\nfrom pathlib import Path\r\nfrom path_finder import FileFinder\r\n\r\nstart_dir1 = Path('start_search_from_this_dir')\r\nstart_dir2 = Path('and_start_search_from_this_dir')\r\nlimit_depth = True\r\ndepth = 2 # 2, so search in start_dir1, subdir and subsubdirs (same for start_dir2) \r\nfilename_regex = '^[0-9]{8}_blabla'\r\nextension = '.csv' # choose from ('.jpg', '.png', '.txt', '.xml', '.csv', '.xlsx', '.pdf', '.h5', '.nc', '.zip') \r\n\r\nfile_finder = FileFinder(\r\n multi_start_dir=[start_dir1, start_dir2],\r\n extension=extension,\r\n limit_depth=True, \r\n depth=depth,\r\n filename_regex=filename_regex\r\n)\r\n \r\npaths = file_finder.paths # returns a List[Path]\r\npaths_empty_files = file_finder.paths_empty_file # returns a List[Path]\r\n```\r\n\r\n### Usage path_finder.DirFinder\r\n```\r\n# pip install path-finder (or conda install --channel hdsr-mid path-finder)\r\n# pip install pathlib\r\nfrom pathlib import Path\r\nfrom path_finder import DirFinder\r\n\r\ndir_finder = DirFinder(\r\n single_start_dir=Path('start_search_from_this_dir')\r\n exclude_empty_dirs=True,\r\n limit_depth=True,\r\n depth=0, # so only search in single_start_dir\r\n)\r\n\r\npaths = dir_finder.paths # returns a List[Path]\r\npaths_empty_files = dir_finder.paths_empty_file # returns a List[Path]\r\n```\r\n\r\n### License \r\n[MIT][mit]\r\n\r\n### Releases\r\n[PyPi][pypi]\r\n\r\n### Contributions\r\nAll contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome.\r\nIssues are posted on: https://github.com/hdsr-mid/path_finder/issues\r\n\r\n[pypi]: https://pypi.org/project/path-finder/\r\n[mit]: https://github.com/hdsr-mid/path_finder/blob/main/LICENSE.txt\r\n\r\n### Test coverage (release v1.6)\r\n```\r\n---------- coverage: platform win32, python 3.7.11-final-0 ---\r\nName Stmts Miss Cover\r\n------------------------------------------------\r\npath_finder\\__init__.py 2 0 100%\r\npath_finder\\base.py 45 8 82%\r\npath_finder\\dir_finder.py 65 11 83%\r\npath_finder\\file_finder.py 58 0 100%\r\nsetup.py 10 10 0%\r\n------------------------------------------------\r\nTOTAL 180 29 84%\r\n\r\n```\r\n\r\n### Conda general tips\r\n#### Build conda environment (on Windows) from any directory using environment.yml:\r\nNote1: prefix is not set in the enviroment.yml as then conda does not handle it very well\r\nNote2: env_directory can be anywhere, it does not have to be in your code project\r\n```\r\n> conda env create --prefix <env_directory><env_name> --file <path_to_project>/environment.yml\r\n# example: conda env create --prefix C:/Users/xxx/.conda/envs/project_xx --file C:/Users/code_projects/xx/environment.yml\r\n> conda info --envs # verify that <env_name> (project_xx) is in this list \r\n```\r\n#### Start the application from any directory:\r\n```\r\n> conda activate <env_name>\r\nAt any location:\r\n> (<env_name>) python <path_to_project>/main.py\r\n```\r\n#### Test the application:\r\n```\r\n> conda activate <env_name>\r\n> cd <path_to_project>\r\n> pytest # make sure pytest is installed (conda install pytest)\r\n```\r\n#### List all conda environments on your machine:\r\n```\r\nAt any location:\r\n> conda info --envs\r\n```\r\n#### Delete a conda environment:\r\n```\r\nGet directory where environment is located \r\n> conda info --envs\r\nRemove the enviroment\r\n> conda env remove --name <env_name>\r\nFinally, remove the left-over directory by hand\r\n```\r\n#### Write dependencies to environment.yml:\r\nThe goal is to keep the .yml as short as possible (not include sub-dependencies), yet make the environment \r\nreproducible. Why? If you do 'conda install matplotlib' you also install sub-dependencies like pyqt, qt \r\nicu, and sip. You should not include these sub-dependencies in your .yml as:\r\n- including sub-dependencies result in an unnecessary strict environment (difficult to solve when conflicting)\r\n- sub-dependencies will be installed when dependencies are being installed\r\n```\r\n> conda activate <conda_env_name>\r\n\r\nRecommended:\r\n> conda env export --from-history --no-builds | findstr -v \"prefix\" > --file <path_to_project>/environment_new.yml \r\n\r\nAlternative:\r\n> conda env export --no-builds | findstr -v \"prefix\" > --file <path_to_project>/environment_new.yml \r\n\r\n--from-history: \r\n Only include packages that you have explicitly asked for, as opposed to including every package in the \r\n environment. This flag works regardless how you created the environment (through CMD or Anaconda Navigator).\r\n--no-builds:\r\n By default, the YAML includes platform-specific build constraints. If you transfer across platforms (e.g. \r\n win32 to 64) omit the build info with '--no-builds'.\r\n```\r\n#### Pip and Conda:\r\nIf a package is not available on all conda channels, but available as pip package, one can install pip as a dependency.\r\nNote that mixing packages from conda and pip is always a potential problem: conda calls pip, but pip does not know \r\nhow to satisfy missing dependencies with packages from Anaconda repositories. \r\n```\r\n> conda activate <env_name>\r\n> conda install pip\r\n> pip install <pip_package>\r\n```\r\nThe environment.yml might look like:\r\n```\r\nchannels:\r\n - defaults\r\ndependencies:\r\n - <a conda package>=<version>\r\n - pip\r\n - pip:\r\n - <a pip package>==<version>\r\n```\r\nYou can also write a requirements.txt file:\r\n```\r\n> pip list --format=freeze > <path_to_project>/requirements.txt\r\n```\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "An interface for finding directories and files",
"version": "1.7",
"project_urls": {
"Download": "https://github.com/hdsr-mid/path_finder/archive/v1.7.tar.gz",
"Homepage": "https://github.com/hdsr-mid/path_finder"
},
"split_keywords": [
"interface",
"path",
"directory",
"filename",
"glob",
"regex",
"find"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c6d82de7fd37ec23ad9f862faae90ddadbec22083d8da974e1872ad129f63027",
"md5": "cb8723456bc525a73e430009df4c08ea",
"sha256": "02fffccf594372e2ad8f61edbab5de35bc2faf2dc240f08675ef36a0894c0766"
},
"downloads": -1,
"filename": "path_finder-1.7.tar.gz",
"has_sig": false,
"md5_digest": "cb8723456bc525a73e430009df4c08ea",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 8432,
"upload_time": "2023-11-13T15:37:18",
"upload_time_iso_8601": "2023-11-13T15:37:18.807312Z",
"url": "https://files.pythonhosted.org/packages/c6/d8/2de7fd37ec23ad9f862faae90ddadbec22083d8da974e1872ad129f63027/path_finder-1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-13 15:37:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hdsr-mid",
"github_project": "path_finder",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "path-finder"
}
JSON
