# scandir-rs
The Python module is called `scandir_rs` and installable via `pip`. It is an alternative to `os.walk()` and `os.scandir()` with more features and higher speed. On Linux it is **3 - 11 times faster** and on Windows **6 - 70 time faster** (see [benchmarks](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/benchmarks.md)).
It releases the GIL and the scanning is done in a background thread. With different methods intermediate results can be read.
If you are just interested in directory statistics you can use the `Count`.
`scandir_rs` contains following classes:
- `Count` for determining statistics of a directory.
- `Walk` for getting names of directory entries.
- `Scandir` for getting detailed stats of directory entries.
For the API see:
- Class [Count](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/count.md)
- Class [Walk](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/walk.md)
- Class [Scandir](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/scandir.md)
## Installation
For building this wheel from source you need the tool `maturin`.
Install `maturin`:
```sh
cargo install maturin
```
IMPORTANT: In order to build this project at least Rust version 1.61 is needed!
**Build wheel:**
Change to directory `pyscandir`.
Build wheel (on Linux):
```sh
maturin build --release --strip
```
Build wheel on Windows:
```sh
maturin build --release --strip --no-sdist
```
``maturin`` will build the wheels for all Python versions installed on your system.
Alternatively you can use the build script `build_wheels.py`. The precondition to run this script is to have `pyenv` installed.
The script can build the wheel for specific Python versions or for all Python versions installed by `pyenv`.
In addition it runs ``pytest`` after successfull creation of each wheel.
```sh
python build_wheels.py
```
By default the script will build the wheel for the current Python interpreter.
If you want to build the wheel for specific Python version(s) by providing the argument `--versions`.
```sh
python build_wheels.py --versions 3.11.8,3.12.2
```
To build the wheel for all installed Python versions:
```sh
python build_wheels.py --versions *
```
Instruction how to install ``pyenv`` can be found [here](https://github.com/pyenv/pyenv).
## Examples
Get statistics of a directory:
```python
from scandir_rs import Count, ReturnType
print(Count("/usr", return_type=ReturnType.Ext).collect())
```
The `collect` method releases the GIL. So other Python threads can run in parallel.
The same, but asynchronously in background using a class instance:
```python
from scandir_rs import Count, ReturnType
instance = Count("/usr", return_type=ReturnType.Ext)
instance.start() # Start scanning the directory in background
...
values = instance.results() # Returns the current statistics. Can be read at any time
...
if instance.busy(): # Check if the task is still running.
...
instance.stop() # If you want to cancel the task
...
instance.join() # Wait for the instance to finish.
```
and with a context manager:
```python
import time
from scandir_rs import Count, ReturnType
with Count("/usr", return_type=ReturnType.Ext) as instance:
while instance.busy():
statistics = instance.results()
# Do something
time.sleep(0.01)
print(instance.results())
```
``os.walk()`` example:
```python
from scandir_rs import Walk
for root, dirs, files in Walk("/usr"):
# Do something
```
with extended data:
```python
from scandir_rs import Walk, ReturnType
for root, dirs, files, symlinks, other, errors in Walk("/usr", return_type=ReturnType.Ext):
# Do something
```
``os.scandir()`` example:
```python
from scandir_rs import Scandir, ReturnType
for path, entry in Scandir("~/workspace", return_type=ReturnType.Ext):
# entry is a custom DirEntry object
```
Raw data
{
"_id": null,
"home_page": "https://github.com/brmmm3/scandir-rs",
"name": "scandir-rs",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "fast, scandir, walk",
"author": "Martin Bammer <mrbm74@gmail.com>",
"author_email": "Martin Bammer <mrbm74@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/bd/50/69bd3689e927d9e7324171aa8ed3fa1b3a27ed63b4f8bdb064e13536f8a0/scandir_rs-2.7.1.tar.gz",
"platform": null,
"description": "# scandir-rs\n\nThe Python module is called `scandir_rs` and installable via `pip`. It is an alternative to `os.walk()` and `os.scandir()` with more features and higher speed. On Linux it is **3 - 11 times faster** and on Windows **6 - 70 time faster** (see [benchmarks](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/benchmarks.md)). \nIt releases the GIL and the scanning is done in a background thread. With different methods intermediate results can be read.\n\nIf you are just interested in directory statistics you can use the `Count`.\n\n`scandir_rs` contains following classes:\n\n- `Count` for determining statistics of a directory.\n- `Walk` for getting names of directory entries.\n- `Scandir` for getting detailed stats of directory entries.\n\nFor the API see:\n\n- Class [Count](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/count.md)\n- Class [Walk](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/walk.md)\n- Class [Scandir](https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/scandir.md)\n\n## Installation\n\nFor building this wheel from source you need the tool `maturin`.\n\nInstall `maturin`:\n\n```sh\ncargo install maturin\n```\n\nIMPORTANT: In order to build this project at least Rust version 1.61 is needed!\n\n**Build wheel:**\n\nChange to directory `pyscandir`.\n\nBuild wheel (on Linux):\n\n```sh\nmaturin build --release --strip\n```\n\nBuild wheel on Windows:\n\n```sh\nmaturin build --release --strip --no-sdist\n```\n\n``maturin`` will build the wheels for all Python versions installed on your system.\n\nAlternatively you can use the build script `build_wheels.py`. The precondition to run this script is to have `pyenv` installed.\nThe script can build the wheel for specific Python versions or for all Python versions installed by `pyenv`.\nIn addition it runs ``pytest`` after successfull creation of each wheel.\n\n```sh\npython build_wheels.py\n```\n\nBy default the script will build the wheel for the current Python interpreter.\nIf you want to build the wheel for specific Python version(s) by providing the argument `--versions`.\n\n```sh\npython build_wheels.py --versions 3.11.8,3.12.2\n```\n\nTo build the wheel for all installed Python versions:\n\n```sh\npython build_wheels.py --versions *\n```\n\nInstruction how to install ``pyenv`` can be found [here](https://github.com/pyenv/pyenv).\n\n## Examples\n\nGet statistics of a directory:\n\n```python\nfrom scandir_rs import Count, ReturnType\n\nprint(Count(\"/usr\", return_type=ReturnType.Ext).collect())\n```\n\nThe `collect` method releases the GIL. So other Python threads can run in parallel.\n\nThe same, but asynchronously in background using a class instance:\n\n```python\nfrom scandir_rs import Count, ReturnType\n\ninstance = Count(\"/usr\", return_type=ReturnType.Ext)\ninstance.start() # Start scanning the directory in background\n...\nvalues = instance.results() # Returns the current statistics. Can be read at any time\n...\nif instance.busy(): # Check if the task is still running.\n...\ninstance.stop() # If you want to cancel the task\n...\ninstance.join() # Wait for the instance to finish.\n```\n\nand with a context manager:\n\n```python\nimport time\n\nfrom scandir_rs import Count, ReturnType\n\nwith Count(\"/usr\", return_type=ReturnType.Ext) as instance:\n while instance.busy():\n statistics = instance.results()\n # Do something\n time.sleep(0.01)\n print(instance.results())\n```\n\n``os.walk()`` example:\n\n```python\nfrom scandir_rs import Walk\n\nfor root, dirs, files in Walk(\"/usr\"):\n # Do something\n```\n\nwith extended data:\n\n```python\nfrom scandir_rs import Walk, ReturnType\n\nfor root, dirs, files, symlinks, other, errors in Walk(\"/usr\", return_type=ReturnType.Ext):\n # Do something\n```\n\n``os.scandir()`` example:\n\n```python\nfrom scandir_rs import Scandir, ReturnType\n\nfor path, entry in Scandir(\"~/workspace\", return_type=ReturnType.Ext):\n # entry is a custom DirEntry object\n```\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A fast file tree scanner written in Rust",
"version": "2.7.1",
"project_urls": {
"Homepage": "https://github.com/brmmm3/scandir-rs",
"Source Code": "https://github.com/brmmm3/scandir-rs"
},
"split_keywords": [
"fast",
" scandir",
" walk"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b7bdab5ec6cdcf9540309d9edd100712d5130d6d0bb383312c7710c27e35641e",
"md5": "bc91efa63b48bd6fe44ce1cac4ee134a",
"sha256": "918e487d8004e36730d7cf78fc5d6b41aaf76bfd4a539ddf1f3b84a65b493090"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp310-cp310-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "bc91efa63b48bd6fe44ce1cac4ee134a",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.7",
"size": 576769,
"upload_time": "2024-04-17T10:41:58",
"upload_time_iso_8601": "2024-04-17T10:41:58.055935Z",
"url": "https://files.pythonhosted.org/packages/b7/bd/ab5ec6cdcf9540309d9edd100712d5130d6d0bb383312c7710c27e35641e/scandir_rs-2.7.1-cp310-cp310-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "53e3f5325fda4ccf035dc271f0076d665e419f7a544156f6d3c749ffd071aef1",
"md5": "acbedc7483e1fab046e75436957ddf1e",
"sha256": "8b31ac380f4f941fe5b139b7577ac5c16eeb6cd9a8c0a8b40881cf827d4a6966"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp310-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "acbedc7483e1fab046e75436957ddf1e",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.7",
"size": 436731,
"upload_time": "2024-04-17T10:42:00",
"upload_time_iso_8601": "2024-04-17T10:42:00.257424Z",
"url": "https://files.pythonhosted.org/packages/53/e3/f5325fda4ccf035dc271f0076d665e419f7a544156f6d3c749ffd071aef1/scandir_rs-2.7.1-cp310-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9876a42bd14a354d0bbce2ed30dde777dfcb2fd8c880507cb3914a9de3efe5fa",
"md5": "c59944b659d39ac4b646031069109466",
"sha256": "c29606b82c17b7c7dec776e2db8bbfdbce5d0f21cec7d3e3cbd22b9e62fc5401"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp311-cp311-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "c59944b659d39ac4b646031069109466",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.7",
"size": 576774,
"upload_time": "2024-04-17T10:42:02",
"upload_time_iso_8601": "2024-04-17T10:42:02.938584Z",
"url": "https://files.pythonhosted.org/packages/98/76/a42bd14a354d0bbce2ed30dde777dfcb2fd8c880507cb3914a9de3efe5fa/scandir_rs-2.7.1-cp311-cp311-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b71176277c7537dba1753b5e376bf3f74707f6c7c3e277629afee78572cdcb0f",
"md5": "18f6599b11df2a033a58b54f032cfb1b",
"sha256": "f5b290c6a01bfb1b32f75d7a8ee4e1ce94a4eb8a3a527e9df1bac4fd461d0227"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp311-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "18f6599b11df2a033a58b54f032cfb1b",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.7",
"size": 436767,
"upload_time": "2024-04-17T10:42:05",
"upload_time_iso_8601": "2024-04-17T10:42:05.394807Z",
"url": "https://files.pythonhosted.org/packages/b7/11/76277c7537dba1753b5e376bf3f74707f6c7c3e277629afee78572cdcb0f/scandir_rs-2.7.1-cp311-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "787bc93f05e4e76ebb7f265e352748a38261923e018548f46b0a08b1f0888753",
"md5": "f3de7e4bd81f327041a3a2a35d71d342",
"sha256": "5fd55ef0f1de9454e8304abe4999294417dee92b53cef3c83bf2e0beb78aaf6c"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp312-cp312-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "f3de7e4bd81f327041a3a2a35d71d342",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.7",
"size": 578152,
"upload_time": "2024-04-17T10:42:07",
"upload_time_iso_8601": "2024-04-17T10:42:07.407909Z",
"url": "https://files.pythonhosted.org/packages/78/7b/c93f05e4e76ebb7f265e352748a38261923e018548f46b0a08b1f0888753/scandir_rs-2.7.1-cp312-cp312-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2a2b6d2542fe72f1705b40b8bb633c2e230823204bafc35e43fbaba1de8464d7",
"md5": "47ae8f7b0e3bc37e041570b083eb86b3",
"sha256": "15ac0e27759d77031c3eb9ea4812314d983b95647881b8b8612921fe5f55a8e3"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp312-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "47ae8f7b0e3bc37e041570b083eb86b3",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.7",
"size": 441607,
"upload_time": "2024-04-17T10:42:09",
"upload_time_iso_8601": "2024-04-17T10:42:09.979140Z",
"url": "https://files.pythonhosted.org/packages/2a/2b/6d2542fe72f1705b40b8bb633c2e230823204bafc35e43fbaba1de8464d7/scandir_rs-2.7.1-cp312-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b64c54fa058ba7b14c135ea4aea264dae93e19741616de2388bba0e0c7660ebe",
"md5": "bbbc8be7d34b9886117474727ddd3cd1",
"sha256": "9323d4672763108907b7d0d29ab02e7e0b615e8dc357edd2c438a43ac305509f"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp37-cp37m-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "bbbc8be7d34b9886117474727ddd3cd1",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": ">=3.7",
"size": 577606,
"upload_time": "2024-04-17T10:42:13",
"upload_time_iso_8601": "2024-04-17T10:42:13.134220Z",
"url": "https://files.pythonhosted.org/packages/b6/4c/54fa058ba7b14c135ea4aea264dae93e19741616de2388bba0e0c7660ebe/scandir_rs-2.7.1-cp37-cp37m-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bea9b108c7da8c80f927d05b040b60ab6e9ddae147e60a41c068b8e85bbfe16e",
"md5": "34fd7cc1417ac5e040e16400267c9c75",
"sha256": "683bbe911099b867ca495d5c793f602462725950bc211ab757dd97250931cabb"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp37-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "34fd7cc1417ac5e040e16400267c9c75",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": ">=3.7",
"size": 437256,
"upload_time": "2024-04-17T10:42:15",
"upload_time_iso_8601": "2024-04-17T10:42:15.700549Z",
"url": "https://files.pythonhosted.org/packages/be/a9/b108c7da8c80f927d05b040b60ab6e9ddae147e60a41c068b8e85bbfe16e/scandir_rs-2.7.1-cp37-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a0e2a601043272d1a5745273825638e96bcc62cf5817f93f0df822d7ce1ba953",
"md5": "2f2db89920288d22bd83cfecfdb73e4e",
"sha256": "f9f6b8c2b51cc66479b1bc69fa54b2349535f056ccb3ee870a0a3e0e43388f96"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp38-cp38-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "2f2db89920288d22bd83cfecfdb73e4e",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.7",
"size": 577503,
"upload_time": "2024-04-17T10:42:17",
"upload_time_iso_8601": "2024-04-17T10:42:17.771463Z",
"url": "https://files.pythonhosted.org/packages/a0/e2/a601043272d1a5745273825638e96bcc62cf5817f93f0df822d7ce1ba953/scandir_rs-2.7.1-cp38-cp38-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "97267fddb52a184eb5f3ccdf70bef997ef020d8b513a5fde56776d1e393f7b55",
"md5": "5ed94e93e98f86a4e12a0ef5e6b67cfe",
"sha256": "830fcb682d804a30c186f622798996d4fc67f91d601899194a563a6d5fb3bbd8"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp38-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "5ed94e93e98f86a4e12a0ef5e6b67cfe",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.7",
"size": 437302,
"upload_time": "2024-04-17T10:42:19",
"upload_time_iso_8601": "2024-04-17T10:42:19.682934Z",
"url": "https://files.pythonhosted.org/packages/97/26/7fddb52a184eb5f3ccdf70bef997ef020d8b513a5fde56776d1e393f7b55/scandir_rs-2.7.1-cp38-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "02d7c70b934394231592a67c198fde4650b7f03e992917c7b2044c347af090d8",
"md5": "718139c4bb5f73b5bff501114821b471",
"sha256": "2ea8ca7e20d7b10ff59b32042c4cbb9c0472ec47e344f99c437acdbab0e0886c"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp39-cp39-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "718139c4bb5f73b5bff501114821b471",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.7",
"size": 577346,
"upload_time": "2024-04-17T10:42:21",
"upload_time_iso_8601": "2024-04-17T10:42:21.837890Z",
"url": "https://files.pythonhosted.org/packages/02/d7/c70b934394231592a67c198fde4650b7f03e992917c7b2044c347af090d8/scandir_rs-2.7.1-cp39-cp39-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "569a482927b024e50257d70ebf6dc525b55b0ec2d5415683795d29a3d418f727",
"md5": "9869f9d4bfac33f8af90b916a4c943f1",
"sha256": "ee36a5c11150c8e3d05245c6ff38ac4a23122df77ba738a6ecce3740fa65c0fa"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1-cp39-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "9869f9d4bfac33f8af90b916a4c943f1",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.7",
"size": 437031,
"upload_time": "2024-04-17T10:42:23",
"upload_time_iso_8601": "2024-04-17T10:42:23.851330Z",
"url": "https://files.pythonhosted.org/packages/56/9a/482927b024e50257d70ebf6dc525b55b0ec2d5415683795d29a3d418f727/scandir_rs-2.7.1-cp39-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bd5069bd3689e927d9e7324171aa8ed3fa1b3a27ed63b4f8bdb064e13536f8a0",
"md5": "5c6eb93533712187d94646ce2d5d1826",
"sha256": "49909f0e9e535e989176fc53d6ac0db829dfed65781b749c9c3275b7adbbe279"
},
"downloads": -1,
"filename": "scandir_rs-2.7.1.tar.gz",
"has_sig": false,
"md5_digest": "5c6eb93533712187d94646ce2d5d1826",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 39045,
"upload_time": "2024-04-17T10:42:25",
"upload_time_iso_8601": "2024-04-17T10:42:25.551611Z",
"url": "https://files.pythonhosted.org/packages/bd/50/69bd3689e927d9e7324171aa8ed3fa1b3a27ed63b4f8bdb064e13536f8a0/scandir_rs-2.7.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-17 10:42:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "brmmm3",
"github_project": "scandir-rs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "scandir-rs"
}
JSON
