# 📁 Async `pathlib` for Python
`aiopath` is a complete implementation of Python's [`pathlib`](https://docs.python.org/3/library/pathlib.html) that's compatible with [`asyncio`](https://docs.python.org/3/library/asyncio.html), [`trio`](https://github.com/python-trio/trio), and the [`async/await` syntax](https://www.python.org/dev/peps/pep-0492/).
All I/O performed by `aiopath` is asynchronous and [awaitable](https://docs.python.org/3/library/asyncio-task.html#awaitables).
Check out [`📂 app_paths`](https://github.com/alexdelorenzo/app_paths) for an example of library that uses `aiopath`, as well as the [`pyclean` script here](https://alexdelorenzo.dev/notes/pyclean).
## Use case
If you're writing asynchronous Python code and want to take advantage of `pathlib`'s conveniences, but don't want to mix blocking and [non-blocking I/O](https://en.wikipedia.org/wiki/Asynchronous_I/O), then you can reach for `aiopath`.
For example, if you're writing an async [web scraping](https://en.wikipedia.org/wiki/Web_scraping) script, you might want to make several concurrent requests to websites and save the responses to persistent storage:
```python3
from asyncio import run, gather
from aiohttp import ClientSession
from aiopath import AsyncPath
async def save_page(url: str, name: str):
path = AsyncPath(name)
if await path.exists():
return
async with ClientSession() as session, session.get(url) as response:
content: bytes = await response.read()
await path.write_bytes(content)
async def main():
urls = [
'https://example.com',
'https://github.com/alexdelorenzo/aiopath',
'https://alexdelorenzo.dev',
'https://dupebot.firstbyte.dev'
]
tasks = (
save_page(url, f'{index}.html')
for index, url in enumerate(urls)
)
await gather(*tasks)
run(main())
```
If you used `pathlib` instead of `aiopath`, tasks accessing the disk would block the event loop, and async tasks accessing the network would suspend until the event loop was unblocked.
By using `aiopath`, the script can access the network and disk concurrently.
## Implementation
`aiopath` is a direct reimplementation of [CPython's `pathlib.py`](https://github.com/python/cpython/blob/master/Lib/pathlib.py) and shares some of its code. `aiopath`'s class hierarchy [directly matches the one from `pathlib`](https://docs.python.org/3/library/pathlib.html), where `Path` inherits from `PurePath`, `AsyncPath` inherits from `AsyncPurePath`, and so on.
With `aiopath`, methods that perform I/O are asynchronous and awaitable, and methods that perform I/O and return iterators in `pathlib` now return [async generators](https://www.python.org/dev/peps/pep-0525/). `aiopath` goes one step further, and wraps [`os.scandir()`](https://docs.python.org/3/library/os.html#os.scandir) and [`DirEntry`](https://docs.python.org/3/library/os.html#os.DirEntry) to make [`AsyncPath.glob()`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob) completely async.
`aiopath` is typed with Python [type annotations](https://docs.python.org/3/library/typing.html), and if using the `aiofile` back end, it takes advantage of [`libaio`](https://pagure.io/libaio) for async I/O on Linux.
# Usage
`aiopath`'s API directly matches [`pathlib`](https://docs.python.org/3/library/pathlib.html), so check out the standard library documentation for [`PurePath`](https://docs.python.org/3/library/pathlib.html#pure-paths) and [`Path`](https://docs.python.org/3/library/pathlib.html#methods).
### Running examples
To run the following examples with top-level `await` expressions, [launch an asynchronous Python REPL](https://www.integralist.co.uk/posts/python-asyncio/#running-async-code-in-the-repl) using `python3 -m asyncio` or an [IPython shell](https://ipython.org/).
You'll also need to install `asynctempfile` via PyPI, like so `python3 -m pip install asynctempfile`.
## Replacing `pathlib`
All of `pathlib.Path`'s methods that perform synchronous I/O are reimplemented as asynchronous methods. `PurePath` methods are not asynchronous because they don't perform I/O.
```python3
from pathlib import Path
from asynctempfile import NamedTemporaryFile
from aiopath import AsyncPath
async with NamedTemporaryFile() as temp:
path = Path(temp.name)
apath = AsyncPath(temp.name)
# check existence
## sync
assert path.exists()
## async
assert await apath.exists()
# check if file
## sync
assert path.is_file()
## async
assert await apath.is_file()
# touch
path.touch()
await apath.touch()
# PurePath methods are not async
assert path.is_absolute() == apath.is_absolute()
assert path.as_uri() == apath.as_uri()
# read and write text
text: str = 'example'
await apath.write_text(text)
assert await apath.read_text() == text
assert not path.exists()
assert not await apath.exists()
```
You can convert `pathlib.Path` objects to `aiopath.AsyncPath` objects, and vice versa:
```python3
from pathlib import Path
from aiopath import AsyncPath
home: Path = Path.home()
ahome: AsyncPath = AsyncPath(home)
path: Path = Path(ahome)
assert isinstance(home, Path)
assert isinstance(ahome, AsyncPath)
assert isinstance(path, Path)
# AsyncPath and Path objects can point to the same file
assert str(home) == str(ahome) == str(path)
# AsyncPath and Path objects are equivalent
assert home == ahome
```
`AsyncPath` is a subclass of `Path` and `PurePath`, and a subclass of `AsyncPurePath`:
```python3
from pathlib import Path, PurePath
from aiopath import AsyncPath, AsyncPurePath
assert issubclass(AsyncPath, Path)
assert issubclass(AsyncPath, PurePath)
assert issubclass(AsyncPath, AsyncPurePath)
assert issubclass(AsyncPurePath, PurePath)
path: AsyncPath = await AsyncPath.home()
assert isinstance(path, Path)
assert isinstance(path, PurePath)
assert isinstance(path, AsyncPurePath)
```
Check out the test files in the [`tests` directory](https://github.com/alexdelorenzo/aiopath/blob/main/tests) for more examples of how `aiopath` compares to `pathlib`.
## Opening a file
You can get an asynchronous [file-like object handle](https://docs.python.org/3/glossary.html#term-file-object) by using [asynchronous context managers](https://docs.python.org/3/reference/datamodel.html#asynchronous-context-managers).
`AsyncPath.open()`'s async context manager yields an [`anyio.AsyncFile`](https://anyio.readthedocs.io/en/stable/api.html#async-file-i-o) object.
```python3
from asynctempfile import NamedTemporaryFile
from aiopath import AsyncPath
text: str = 'example'
# you can access a file with async context managers
async with NamedTemporaryFile() as temp:
path = AsyncPath(temp.name)
async with path.open(mode='w') as file:
await file.write(text)
async with path.open(mode='r') as file:
result: str = await file.read()
assert result == text
# or you can use the read/write convenience methods
async with NamedTemporaryFile() as temp:
path = AsyncPath(temp.name)
await path.write_text(text)
result: str = await path.read_text()
assert result == text
content: bytes = text.encode()
await path.write_bytes(content)
result: bytes = await path.read_bytes()
assert result == content
```
## [Globbing](https://en.wikipedia.org/wiki/Glob_(programming))
`aiopath` implements [`pathlib` globbing](https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob) using async I/O and async generators.
```python3
from aiopath import AsyncPath
home: AsyncPath = await AsyncPath.home()
async for path in home.glob('*'):
assert isinstance(path, AsyncPath)
print(path)
downloads: AsyncPath = home / 'Downloads'
if await downloads.exists():
# this might take a while
paths: list[AsyncPath] = \
[path async for path in downloads.glob('**/*')]
```
# Installation
## Dependencies
- A POSIX compliant OS, or Windows
- Python 3.7+
- `requirements.txt`
<!--#### Linux dependencies
If you're using a 4.18 or newer kernel and have [`libaio`](https://pagure.io/libaio) installed, `aiopath` will use it via `aiofile`. You can install `libaio` on Debian/Ubuntu like so:
```bash
$ sudo apt install libaio1 libaio-dev
```-->
## PyPI
```bash
$ python3 -m pip install aiopath
```
#### Python 3.9 and older
`aiopath` for Python 3.9 and older is available on PyPI under versions `0.5.x` and lower.
#### Python 3.10 and newer
`aiopath` for Python 3.10 and newer is available on PyPI under versions `0.6.x` and higher.
## GitHub
Download a release archive for your Python version from [the releases page](https://github.com/alexdelorenzo/aiopath/releases).
Then to install, run:
```bash
$ python3 -m pip install -r requirements.txt
$ python3 setup.py install
```
#### Python 3.9 and older
`aiopath` for Python 3.9 and older is developed on the [`Python-3.9` branch](https://github.com/alexdelorenzo/aiopath/tree/Python-3.9).
#### Python 3.10 and newer
`aiopath` for Python 3.10 and newer is developed on the [`Python-3.10` branch](https://github.com/alexdelorenzo/aiopath/tree/Python-3.10).
# Support
Want to support this project and [other open-source projects](https://github.com/alexdelorenzo) like it?
<a href="https://www.buymeacoffee.com/alexdelorenzo" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="60px" style="height: 60px !important;width: 217px !important;max-width:25%" ></a>
# License
See `LICENSE`. If you'd like to use this project with a different license, please get in touch.
# Credit
See [`CREDIT.md`](/CREDIT.md).
Raw data
{
"_id": null,
"home_page": "https://github.com/AlexDeLorenzo",
"name": "aiopath",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": "",
"keywords": "",
"author": "Alex DeLorenzo <alex@alexdelorenzo.dev>",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/a4/6f/b377e3eb293feb57e457b33961004cc3716794ef97428d2ae2a326599b1c/aiopath-0.7.7.tar.gz",
"platform": null,
"description": "# \ud83d\udcc1 Async `pathlib` for Python\n`aiopath` is a complete implementation of Python's [`pathlib`](https://docs.python.org/3/library/pathlib.html) that's compatible with [`asyncio`](https://docs.python.org/3/library/asyncio.html), [`trio`](https://github.com/python-trio/trio), and the [`async/await` syntax](https://www.python.org/dev/peps/pep-0492/). \n\nAll I/O performed by `aiopath` is asynchronous and [awaitable](https://docs.python.org/3/library/asyncio-task.html#awaitables).\n\nCheck out [`\ud83d\udcc2 app_paths`](https://github.com/alexdelorenzo/app_paths) for an example of library that uses `aiopath`, as well as the [`pyclean` script here](https://alexdelorenzo.dev/notes/pyclean).\n\n## Use case\nIf you're writing asynchronous Python code and want to take advantage of `pathlib`'s conveniences, but don't want to mix blocking and [non-blocking I/O](https://en.wikipedia.org/wiki/Asynchronous_I/O), then you can reach for `aiopath`.\n\nFor example, if you're writing an async [web scraping](https://en.wikipedia.org/wiki/Web_scraping) script, you might want to make several concurrent requests to websites and save the responses to persistent storage:\n```python3\nfrom asyncio import run, gather\n\nfrom aiohttp import ClientSession\nfrom aiopath import AsyncPath\n\n\nasync def save_page(url: str, name: str):\n path = AsyncPath(name)\n\n if await path.exists():\n return\n\n async with ClientSession() as session, session.get(url) as response:\n content: bytes = await response.read()\n\n await path.write_bytes(content)\n\n\nasync def main():\n urls = [\n 'https://example.com',\n 'https://github.com/alexdelorenzo/aiopath',\n 'https://alexdelorenzo.dev',\n 'https://dupebot.firstbyte.dev'\n ]\n\n tasks = (\n save_page(url, f'{index}.html')\n for index, url in enumerate(urls)\n )\n\n await gather(*tasks)\n\n\nrun(main())\n```\nIf you used `pathlib` instead of `aiopath`, tasks accessing the disk would block the event loop, and async tasks accessing the network would suspend until the event loop was unblocked.\n\nBy using `aiopath`, the script can access the network and disk concurrently.\n\n## Implementation \n`aiopath` is a direct reimplementation of [CPython's `pathlib.py`](https://github.com/python/cpython/blob/master/Lib/pathlib.py) and shares some of its code. `aiopath`'s class hierarchy [directly matches the one from `pathlib`](https://docs.python.org/3/library/pathlib.html), where `Path` inherits from `PurePath`, `AsyncPath` inherits from `AsyncPurePath`, and so on.\n\nWith `aiopath`, methods that perform I/O are asynchronous and awaitable, and methods that perform I/O and return iterators in `pathlib` now return [async generators](https://www.python.org/dev/peps/pep-0525/). `aiopath` goes one step further, and wraps [`os.scandir()`](https://docs.python.org/3/library/os.html#os.scandir) and [`DirEntry`](https://docs.python.org/3/library/os.html#os.DirEntry) to make [`AsyncPath.glob()`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob) completely async.\n\n`aiopath` is typed with Python [type annotations](https://docs.python.org/3/library/typing.html), and if using the `aiofile` back end, it takes advantage of [`libaio`](https://pagure.io/libaio) for async I/O on Linux.\n\n# Usage\n `aiopath`'s API directly matches [`pathlib`](https://docs.python.org/3/library/pathlib.html), so check out the standard library documentation for [`PurePath`](https://docs.python.org/3/library/pathlib.html#pure-paths) and [`Path`](https://docs.python.org/3/library/pathlib.html#methods).\n \n### Running examples\nTo run the following examples with top-level `await` expressions, [launch an asynchronous Python REPL](https://www.integralist.co.uk/posts/python-asyncio/#running-async-code-in-the-repl) using `python3 -m asyncio` or an [IPython shell](https://ipython.org/).\n\nYou'll also need to install `asynctempfile` via PyPI, like so `python3 -m pip install asynctempfile`.\n\n## Replacing `pathlib`\nAll of `pathlib.Path`'s methods that perform synchronous I/O are reimplemented as asynchronous methods. `PurePath` methods are not asynchronous because they don't perform I/O.\n\n```python3\nfrom pathlib import Path\n\nfrom asynctempfile import NamedTemporaryFile\nfrom aiopath import AsyncPath\n\n\nasync with NamedTemporaryFile() as temp:\n path = Path(temp.name)\n apath = AsyncPath(temp.name)\n\n # check existence\n ## sync\n assert path.exists()\n ## async\n assert await apath.exists()\n\n # check if file\n ## sync\n assert path.is_file()\n ## async\n assert await apath.is_file()\n\n # touch\n path.touch()\n await apath.touch()\n\n # PurePath methods are not async\n assert path.is_absolute() == apath.is_absolute()\n assert path.as_uri() == apath.as_uri()\n\n # read and write text\n text: str = 'example'\n await apath.write_text(text)\n assert await apath.read_text() == text\n\nassert not path.exists()\nassert not await apath.exists()\n```\n\nYou can convert `pathlib.Path` objects to `aiopath.AsyncPath` objects, and vice versa:\n```python3\nfrom pathlib import Path\nfrom aiopath import AsyncPath\n\n\nhome: Path = Path.home()\nahome: AsyncPath = AsyncPath(home)\npath: Path = Path(ahome)\n\nassert isinstance(home, Path)\nassert isinstance(ahome, AsyncPath)\nassert isinstance(path, Path)\n\n# AsyncPath and Path objects can point to the same file\nassert str(home) == str(ahome) == str(path)\n\n# AsyncPath and Path objects are equivalent\nassert home == ahome\n```\n\n`AsyncPath` is a subclass of `Path` and `PurePath`, and a subclass of `AsyncPurePath`:\n```python3\nfrom pathlib import Path, PurePath\nfrom aiopath import AsyncPath, AsyncPurePath\n\n\nassert issubclass(AsyncPath, Path)\nassert issubclass(AsyncPath, PurePath)\nassert issubclass(AsyncPath, AsyncPurePath)\nassert issubclass(AsyncPurePath, PurePath)\n\npath: AsyncPath = await AsyncPath.home()\n\nassert isinstance(path, Path)\nassert isinstance(path, PurePath)\nassert isinstance(path, AsyncPurePath) \n```\n\nCheck out the test files in the [`tests` directory](https://github.com/alexdelorenzo/aiopath/blob/main/tests) for more examples of how `aiopath` compares to `pathlib`.\n\n## Opening a file\nYou can get an asynchronous [file-like object handle](https://docs.python.org/3/glossary.html#term-file-object) by using [asynchronous context managers](https://docs.python.org/3/reference/datamodel.html#asynchronous-context-managers). \n\n`AsyncPath.open()`'s async context manager yields an [`anyio.AsyncFile`](https://anyio.readthedocs.io/en/stable/api.html#async-file-i-o) object.\n\n```python3\nfrom asynctempfile import NamedTemporaryFile\nfrom aiopath import AsyncPath\n\n\ntext: str = 'example'\n\n# you can access a file with async context managers\nasync with NamedTemporaryFile() as temp:\n path = AsyncPath(temp.name)\n\n async with path.open(mode='w') as file:\n await file.write(text)\n\n async with path.open(mode='r') as file:\n result: str = await file.read()\n\n assert result == text\n\n# or you can use the read/write convenience methods\nasync with NamedTemporaryFile() as temp:\n path = AsyncPath(temp.name)\n\n await path.write_text(text)\n result: str = await path.read_text()\n assert result == text\n\n content: bytes = text.encode()\n\n await path.write_bytes(content)\n result: bytes = await path.read_bytes()\n assert result == content\n```\n\n## [Globbing](https://en.wikipedia.org/wiki/Glob_(programming))\n`aiopath` implements [`pathlib` globbing](https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob) using async I/O and async generators.\n\n```python3\nfrom aiopath import AsyncPath\n\n\nhome: AsyncPath = await AsyncPath.home()\n\nasync for path in home.glob('*'):\n assert isinstance(path, AsyncPath)\n print(path)\n\ndownloads: AsyncPath = home / 'Downloads'\n\nif await downloads.exists():\n # this might take a while\n paths: list[AsyncPath] = \\\n [path async for path in downloads.glob('**/*')]\n```\n\n# Installation\n## Dependencies\n - A POSIX compliant OS, or Windows\n - Python 3.7+\n - `requirements.txt`\n\n<!--#### Linux dependencies\nIf you're using a 4.18 or newer kernel and have [`libaio`](https://pagure.io/libaio) installed, `aiopath` will use it via `aiofile`. You can install `libaio` on Debian/Ubuntu like so:\n```bash\n$ sudo apt install libaio1 libaio-dev\n```-->\n\n## PyPI\n```bash\n$ python3 -m pip install aiopath\n```\n\n#### Python 3.9 and older\n`aiopath` for Python 3.9 and older is available on PyPI under versions `0.5.x` and lower.\n\n#### Python 3.10 and newer\n`aiopath` for Python 3.10 and newer is available on PyPI under versions `0.6.x` and higher.\n\n## GitHub\nDownload a release archive for your Python version from [the releases page](https://github.com/alexdelorenzo/aiopath/releases).\n\nThen to install, run:\n```bash\n$ python3 -m pip install -r requirements.txt\n$ python3 setup.py install\n```\n\n#### Python 3.9 and older\n`aiopath` for Python 3.9 and older is developed on the [`Python-3.9` branch](https://github.com/alexdelorenzo/aiopath/tree/Python-3.9).\n\n#### Python 3.10 and newer\n`aiopath` for Python 3.10 and newer is developed on the [`Python-3.10` branch](https://github.com/alexdelorenzo/aiopath/tree/Python-3.10).\n\n# Support\nWant to support this project and [other open-source projects](https://github.com/alexdelorenzo) like it?\n\n<a href=\"https://www.buymeacoffee.com/alexdelorenzo\" target=\"_blank\"><img src=\"https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png\" alt=\"Buy Me A Coffee\" height=\"60px\" style=\"height: 60px !important;width: 217px !important;max-width:25%\" ></a>\n\n# License\nSee `LICENSE`. If you'd like to use this project with a different license, please get in touch.\n\n\n# Credit\nSee [`CREDIT.md`](/CREDIT.md).\n",
"bugtrack_url": null,
"license": "LGPL-3.0",
"summary": "\ud83d\udcc1 Async pathlib for Python",
"version": "0.7.7",
"project_urls": {
"Homepage": "https://github.com/AlexDeLorenzo"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "97df6a3826363e4848ffa8972410ab5234d3802597ae92f4a6397600149112c7",
"md5": "4cdf75f4a5f84533b120159ed1ac7021",
"sha256": "cd5d18de8ede167e1db659f02ee448fe085f923cb8e194407ccc568bffc4fe4e"
},
"downloads": -1,
"filename": "aiopath-0.7.7-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "4cdf75f4a5f84533b120159ed1ac7021",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.12",
"size": 12200,
"upload_time": "2023-10-16T22:37:09",
"upload_time_iso_8601": "2023-10-16T22:37:09.595553Z",
"url": "https://files.pythonhosted.org/packages/97/df/6a3826363e4848ffa8972410ab5234d3802597ae92f4a6397600149112c7/aiopath-0.7.7-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a46fb377e3eb293feb57e457b33961004cc3716794ef97428d2ae2a326599b1c",
"md5": "fe90ca36690f98dfffc44536192aa327",
"sha256": "ad4b9d09ae08ddf6d39dd06e7b0a353939e89528da571c0cd4f3fe071aefad4f"
},
"downloads": -1,
"filename": "aiopath-0.7.7.tar.gz",
"has_sig": false,
"md5_digest": "fe90ca36690f98dfffc44536192aa327",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 16925,
"upload_time": "2023-10-16T22:37:11",
"upload_time_iso_8601": "2023-10-16T22:37:11.381980Z",
"url": "https://files.pythonhosted.org/packages/a4/6f/b377e3eb293feb57e457b33961004cc3716794ef97428d2ae2a326599b1c/aiopath-0.7.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-16 22:37:11",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "aiopath"
}
JSON