AIOFile
=======
.. image:: https://github.com/mosquito/aiofile/workflows/tox/badge.svg
:target: https://github.com/mosquito/aiofile/actions?query=branch%3Amaster
:alt: Github Actions
.. image:: https://img.shields.io/pypi/v/aiofile.svg
:target: https://pypi.python.org/pypi/aiofile/
:alt: Latest Version
.. image:: https://img.shields.io/pypi/wheel/aiofile.svg
:target: https://pypi.python.org/pypi/aiofile/
.. image:: https://img.shields.io/pypi/pyversions/aiofile.svg
:target: https://pypi.python.org/pypi/aiofile/
.. image:: https://img.shields.io/pypi/l/aiofile.svg
:target: https://pypi.python.org/pypi/aiofile/
.. image:: https://coveralls.io/repos/github/mosquito/aiofile/badge.svg?branch=master
:target: https://coveralls.io/github/mosquito/aiofile?branch=master
Real asynchronous file operations with asyncio support.
Status
------
Development - Stable
Features
--------
* Since version 2.0.0 using `caio`_, which contains linux ``libaio`` and two
thread-based implementations (c-based and pure-python).
* AIOFile has no internal pointer. You should pass ``offset`` and
``chunk_size`` for each operation or use helpers (Reader or Writer).
The simples way is to use ``async_open`` for creating object with
file-like interface.
* For Linux using implementation based on `libaio`_.
* For POSIX (MacOS X and optional Linux) using implementation
based on `threadpool`_.
* Otherwise using pure-python thread-based implementation.
* Implementation chooses automatically depending on system compatibility.
.. _caio: https://pypi.org/project/caio
.. _libaio: https://pagure.io/libaio
.. _threadpool: https://github.com/mbrossard/threadpool/
Limitations
-----------
* Linux native AIO implementation is not able to open special files.
Asynchronous operations against special fs like ``/proc/`` ``/sys/`` are not
supported by the kernel. It's not a `aiofile`s or `caio` issue.
In this cases, you might switch to thread-based implementations
(see troubleshooting_ section).
However, when used on supported file systems, the linux implementation has a
smaller overhead and is preferred but it's not a silver bullet.
Code examples
-------------
All code examples requires python 3.6+.
High-level API
++++++++++++++
``async_open`` helper
~~~~~~~~~~~~~~~~~~~~~
Helper mimics python file-like objects, it returns file-like
objects with similar but async methods.
Supported methods:
* ``async def read(length = -1)`` - reading chunk from file, when length is
``-1``, will be reading file to the end.
* ``async def write(data)`` - writing chunk to file
* ``def seek(offset)`` - setting file pointer position
* ``def tell()`` - returns current file pointer position
* ``async def readline(size=-1, newline="\n")`` - read chunks until
newline or EOF. Since version 3.7.0 ``__aiter__`` returns ``LineReader``.
This method is suboptimal for small lines because it doesn't reuse read buffer.
When you want to read file by lines please avoid using ``async_open``
use ``LineReader`` instead.
* ``def __aiter__() -> LineReader`` - iterator over lines.
* ``def iter_chunked(chunk_size: int = 32768) -> Reader`` - iterator over
chunks.
* ``.file`` property contains AIOFile object
Basic example:
.. code-block:: python
:name: test_basic
import asyncio
from pathlib import Path
from tempfile import gettempdir
from aiofile import async_open
tmp_filename = Path(gettempdir()) / "hello.txt"
async def main():
async with async_open(tmp_filename, 'w+') as afp:
await afp.write("Hello ")
await afp.write("world")
afp.seek(0)
print(await afp.read())
await afp.write("Hello from\nasync world")
print(await afp.readline())
print(await afp.readline())
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Example without context manager:
.. code-block:: python
:name: test_no_context_manager
import asyncio
import atexit
import os
from tempfile import mktemp
from aiofile import async_open
TMP_NAME = mktemp()
atexit.register(os.unlink, TMP_NAME)
async def main():
afp = await async_open(TMP_NAME, "w")
await afp.write("Hello")
await afp.close()
asyncio.run(main())
assert open(TMP_NAME, "r").read() == "Hello"
Concatenate example program (``cat``):
.. code-block:: python
import asyncio
import sys
from argparse import ArgumentParser
from pathlib import Path
from aiofile import async_open
parser = ArgumentParser(
description="Read files line by line using asynchronous io API"
)
parser.add_argument("file_name", nargs="+", type=Path)
async def main(arguments):
for src in arguments.file_name:
async with async_open(src, "r") as afp:
async for line in afp:
sys.stdout.write(line)
asyncio.run(main(parser.parse_args()))
Copy file example program (``cp``):
.. code-block:: python
import asyncio
from argparse import ArgumentParser
from pathlib import Path
from aiofile import async_open
parser = ArgumentParser(
description="Copying files using asynchronous io API"
)
parser.add_argument("source", type=Path)
parser.add_argument("dest", type=Path)
parser.add_argument("--chunk-size", type=int, default=65535)
async def main(arguments):
async with async_open(arguments.source, "rb") as src, \
async_open(arguments.dest, "wb") as dest:
async for chunk in src.iter_chunked(arguments.chunk_size):
await dest.write(chunk)
asyncio.run(main(parser.parse_args()))
Example with opening already open file pointer:
.. code-block:: python
:name: test_opened
import asyncio
from typing import IO, Any
from aiofile import async_open
async def main(fp: IO[Any]):
async with async_open(fp) as afp:
await afp.write("Hello from\nasync world")
print(await afp.readline())
with open("test.txt", "w+") as fp:
asyncio.run(main(fp))
Linux native aio doesn't support reading and writing special files
(e.g. procfs/sysfs/unix pipes/etc.), so you can perform operations with
these files using compatible context objects.
.. code-block:: python
import asyncio
from aiofile import async_open
from caio import thread_aio_asyncio
from contextlib import AsyncExitStack
async def main():
async with AsyncExitStack() as stack:
# Custom context should be reused
ctx = await stack.enter_async_context(
thread_aio_asyncio.AsyncioContext()
)
# Open special file with custom context
src = await stack.enter_async_context(
async_open("/proc/cpuinfo", "r", context=ctx)
)
# Open regular file with default context
dest = await stack.enter_async_context(
async_open("/tmp/cpuinfo", "w")
)
# Copying file content line by line
async for line in src:
await dest.write(line)
asyncio.run(main())
Low-level API
++++++++++++++
The `AIOFile` class is a low-level interface for asynchronous file operations, and the read and write methods accept
an `offset=0` in bytes at which the operation will be performed.
This allows you to do many independent IO operations on an once open file without moving the virtual carriage.
For example, you may make 10 concurrent HTTP requests by specifying the `Range` header, and asynchronously write
one opened file, while the offsets must either be calculated manually, or use 10 instances of `Writer` with
specified initial offsets.
In order to provide sequential reading and writing, there is `Writer`, `Reader` and `LineReader`. Keep in mind
`async_open` is not the same as AIOFile, it provides a similar interface for file operations, it simulates methods
like read or write as it is implemented in the built-in open.
.. code-block:: python
:name: test_low_level_api
import asyncio
from aiofile import AIOFile
async def main():
async with AIOFile("hello.txt", 'w+') as afp:
payload = "Hello world\n"
await asyncio.gather(
*[afp.write(payload, offset=i * len(payload)) for i in range(10)]
)
await afp.fsync()
assert await afp.read(len(payload) * 10) == payload * 10
asyncio.run(main())
The Low-level API in fact is just little bit sugared ``caio`` API.
.. code-block:: python
import asyncio
from aiofile import AIOFile
async def main():
async with AIOFile("/tmp/hello.txt", 'w+') as afp:
await afp.write("Hello ")
await afp.write("world", offset=7)
await afp.fsync()
print(await afp.read())
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
``Reader`` and ``Writer``
~~~~~~~~~~~~~~~~~~~~~~~~~
When you want to read or write file linearly following example
might be helpful.
.. code-block:: python
import asyncio
from aiofile import AIOFile, Reader, Writer
async def main():
async with AIOFile("/tmp/hello.txt", 'w+') as afp:
writer = Writer(afp)
reader = Reader(afp, chunk_size=8)
await writer("Hello")
await writer(" ")
await writer("World")
await afp.fsync()
async for chunk in reader:
print(chunk)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
``LineReader`` - read file line by line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LineReader is a helper that is very effective when you want to read a file
linearly and line by line.
It contains a buffer and will read the fragments of the file chunk by
chunk into the buffer, where it will try to find lines.
The default chunk size is 4KB.
.. code-block:: python
import asyncio
from aiofile import AIOFile, LineReader, Writer
async def main():
async with AIOFile("/tmp/hello.txt", 'w+') as afp:
writer = Writer(afp)
await writer("Hello")
await writer(" ")
await writer("World")
await writer("\n")
await writer("\n")
await writer("From async world")
await afp.fsync()
async for line in LineReader(afp):
print(line)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
When you want to read file by lines please avoid to use ``async_open``
use ``LineReader`` instead.
More examples
-------------
Useful examples with ``aiofile``
Async CSV Dict Reader
+++++++++++++++++++++
.. code-block:: python
import asyncio
import io
from csv import DictReader
from aiofile import AIOFile, LineReader
class AsyncDictReader:
def __init__(self, afp, **kwargs):
self.buffer = io.BytesIO()
self.file_reader = LineReader(
afp, line_sep=kwargs.pop('line_sep', '\n'),
chunk_size=kwargs.pop('chunk_size', 4096),
offset=kwargs.pop('offset', 0),
)
self.reader = DictReader(
io.TextIOWrapper(
self.buffer,
encoding=kwargs.pop('encoding', 'utf-8'),
errors=kwargs.pop('errors', 'replace'),
), **kwargs,
)
self.line_num = 0
def __aiter__(self):
return self
async def __anext__(self):
if self.line_num == 0:
header = await self.file_reader.readline()
self.buffer.write(header)
line = await self.file_reader.readline()
if not line:
raise StopAsyncIteration
self.buffer.write(line)
self.buffer.seek(0)
try:
result = next(self.reader)
except StopIteration as e:
raise StopAsyncIteration from e
self.buffer.seek(0)
self.buffer.truncate(0)
self.line_num = self.reader.line_num
return result
async def main():
async with AIOFile('sample.csv', 'rb') as afp:
async for item in AsyncDictReader(afp):
print(item)
asyncio.run(main())
.. _troubleshooting:
Troubleshooting
---------------
The caio ``linux`` implementation works normal for modern linux kernel versions
and file systems. So you may have problems specific for your environment.
It's not a bug and might be resolved some ways:
1. Upgrade the kernel
2. Use compatible file systems
3. Use threads based or pure python implementation.
The caio since version 0.7.0 contains some ways to do this.
1. In runtime use the environment variable ``CAIO_IMPL`` with
possible values:
* ``linux`` - use native linux kernels aio mechanism
* ``thread`` - use thread based implementation written in C
* ``python`` - use pure python implementation
2. File ``default_implementation`` located near ``__init__.py`` in caio
installation path. It's useful for distros package maintainers. This file
might contains comments (lines starts with ``#`` symbol) and the first line
should be one of ``linux`` ``thread`` or ``python``.
3. You might manually manage contexts:
.. code-block:: python
import asyncio
from aiofile import async_open
from caio import linux_aio_asyncio, thread_aio_asyncio
async def main():
linux_ctx = linux_aio_asyncio.AsyncioContext()
threads_ctx = thread_aio_asyncio.AsyncioContext()
async with async_open("/tmp/test.txt", "w", context=linux_ctx) as afp:
await afp.write("Hello")
async with async_open("/tmp/test.txt", "r", context=threads_ctx) as afp:
print(await afp.read())
asyncio.run(main())
Raw data
{
"_id": null,
"home_page": "http://github.com/mosquito/aiofile",
"name": "aiofile",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7, <4",
"maintainer_email": "",
"keywords": "aio,python,asyncio,fileio,io",
"author": "Dmitry Orlov <me@mosquito.su>",
"author_email": "me@mosquito.su",
"download_url": "https://files.pythonhosted.org/packages/41/6d/e03ef9694346a55da965538f3e28461bd9fc71e21f997269d03b3cfc8755/aiofile-3.8.8.tar.gz",
"platform": "POSIX",
"description": "AIOFile\n=======\n\n.. image:: https://github.com/mosquito/aiofile/workflows/tox/badge.svg\n :target: https://github.com/mosquito/aiofile/actions?query=branch%3Amaster\n :alt: Github Actions\n\n.. image:: https://img.shields.io/pypi/v/aiofile.svg\n :target: https://pypi.python.org/pypi/aiofile/\n :alt: Latest Version\n\n.. image:: https://img.shields.io/pypi/wheel/aiofile.svg\n :target: https://pypi.python.org/pypi/aiofile/\n\n.. image:: https://img.shields.io/pypi/pyversions/aiofile.svg\n :target: https://pypi.python.org/pypi/aiofile/\n\n.. image:: https://img.shields.io/pypi/l/aiofile.svg\n :target: https://pypi.python.org/pypi/aiofile/\n\n.. image:: https://coveralls.io/repos/github/mosquito/aiofile/badge.svg?branch=master\n :target: https://coveralls.io/github/mosquito/aiofile?branch=master\n\n\n\nReal asynchronous file operations with asyncio support.\n\n\nStatus\n------\n\nDevelopment - Stable\n\n\nFeatures\n--------\n\n* Since version 2.0.0 using `caio`_, which contains linux ``libaio`` and two\n thread-based implementations (c-based and pure-python).\n* AIOFile has no internal pointer. You should pass ``offset`` and\n ``chunk_size`` for each operation or use helpers (Reader or Writer).\n The simples way is to use ``async_open`` for creating object with\n file-like interface.\n* For Linux using implementation based on `libaio`_.\n* For POSIX (MacOS X and optional Linux) using implementation\n based on `threadpool`_.\n* Otherwise using pure-python thread-based implementation.\n* Implementation chooses automatically depending on system compatibility.\n\n.. _caio: https://pypi.org/project/caio\n.. _libaio: https://pagure.io/libaio\n.. _threadpool: https://github.com/mbrossard/threadpool/\n\n\nLimitations\n-----------\n\n* Linux native AIO implementation is not able to open special files.\n Asynchronous operations against special fs like ``/proc/`` ``/sys/`` are not\n supported by the kernel. It's not a `aiofile`s or `caio` issue.\n In this cases, you might switch to thread-based implementations\n (see troubleshooting_ section).\n However, when used on supported file systems, the linux implementation has a\n smaller overhead and is preferred but it's not a silver bullet.\n\nCode examples\n-------------\n\nAll code examples requires python 3.6+.\n\nHigh-level API\n++++++++++++++\n\n``async_open`` helper\n~~~~~~~~~~~~~~~~~~~~~\n\nHelper mimics python file-like objects, it returns file-like\nobjects with similar but async methods.\n\nSupported methods:\n\n* ``async def read(length = -1)`` - reading chunk from file, when length is\n ``-1``, will be reading file to the end.\n* ``async def write(data)`` - writing chunk to file\n* ``def seek(offset)`` - setting file pointer position\n* ``def tell()`` - returns current file pointer position\n* ``async def readline(size=-1, newline=\"\\n\")`` - read chunks until\n newline or EOF. Since version 3.7.0 ``__aiter__`` returns ``LineReader``.\n\n This method is suboptimal for small lines because it doesn't reuse read buffer.\n When you want to read file by lines please avoid using ``async_open``\n use ``LineReader`` instead.\n* ``def __aiter__() -> LineReader`` - iterator over lines.\n* ``def iter_chunked(chunk_size: int = 32768) -> Reader`` - iterator over\n chunks.\n* ``.file`` property contains AIOFile object\n\n\nBasic example:\n\n.. code-block:: python\n :name: test_basic\n\n import asyncio\n from pathlib import Path\n from tempfile import gettempdir\n\n from aiofile import async_open\n\n tmp_filename = Path(gettempdir()) / \"hello.txt\"\n\n async def main():\n async with async_open(tmp_filename, 'w+') as afp:\n await afp.write(\"Hello \")\n await afp.write(\"world\")\n afp.seek(0)\n\n print(await afp.read())\n\n await afp.write(\"Hello from\\nasync world\")\n print(await afp.readline())\n print(await afp.readline())\n\n loop = asyncio.get_event_loop()\n loop.run_until_complete(main())\n\nExample without context manager:\n\n.. code-block:: python\n :name: test_no_context_manager\n\n import asyncio\n import atexit\n import os\n from tempfile import mktemp\n\n from aiofile import async_open\n\n\n TMP_NAME = mktemp()\n atexit.register(os.unlink, TMP_NAME)\n\n\n async def main():\n afp = await async_open(TMP_NAME, \"w\")\n await afp.write(\"Hello\")\n await afp.close()\n\n\n asyncio.run(main())\n assert open(TMP_NAME, \"r\").read() == \"Hello\"\n\n\nConcatenate example program (``cat``):\n\n.. code-block:: python\n\n import asyncio\n import sys\n from argparse import ArgumentParser\n from pathlib import Path\n\n from aiofile import async_open\n\n parser = ArgumentParser(\n description=\"Read files line by line using asynchronous io API\"\n )\n parser.add_argument(\"file_name\", nargs=\"+\", type=Path)\n\n async def main(arguments):\n for src in arguments.file_name:\n async with async_open(src, \"r\") as afp:\n async for line in afp:\n sys.stdout.write(line)\n\n\n asyncio.run(main(parser.parse_args()))\n\n\nCopy file example program (``cp``):\n\n.. code-block:: python\n\n import asyncio\n from argparse import ArgumentParser\n from pathlib import Path\n\n from aiofile import async_open\n\n parser = ArgumentParser(\n description=\"Copying files using asynchronous io API\"\n )\n parser.add_argument(\"source\", type=Path)\n parser.add_argument(\"dest\", type=Path)\n parser.add_argument(\"--chunk-size\", type=int, default=65535)\n\n\n async def main(arguments):\n async with async_open(arguments.source, \"rb\") as src, \\\n async_open(arguments.dest, \"wb\") as dest:\n async for chunk in src.iter_chunked(arguments.chunk_size):\n await dest.write(chunk)\n\n\n asyncio.run(main(parser.parse_args()))\n\n\nExample with opening already open file pointer:\n\n.. code-block:: python\n :name: test_opened\n\n import asyncio\n from typing import IO, Any\n from aiofile import async_open\n\n\n async def main(fp: IO[Any]):\n async with async_open(fp) as afp:\n await afp.write(\"Hello from\\nasync world\")\n print(await afp.readline())\n\n\n with open(\"test.txt\", \"w+\") as fp:\n asyncio.run(main(fp))\n\n\nLinux native aio doesn't support reading and writing special files\n(e.g. procfs/sysfs/unix pipes/etc.), so you can perform operations with\nthese files using compatible context objects.\n\n.. code-block:: python\n\n import asyncio\n from aiofile import async_open\n from caio import thread_aio_asyncio\n from contextlib import AsyncExitStack\n\n\n async def main():\n async with AsyncExitStack() as stack:\n\n # Custom context should be reused\n ctx = await stack.enter_async_context(\n thread_aio_asyncio.AsyncioContext()\n )\n\n # Open special file with custom context\n src = await stack.enter_async_context(\n async_open(\"/proc/cpuinfo\", \"r\", context=ctx)\n )\n\n # Open regular file with default context\n dest = await stack.enter_async_context(\n async_open(\"/tmp/cpuinfo\", \"w\")\n )\n\n # Copying file content line by line\n async for line in src:\n await dest.write(line)\n\n\n asyncio.run(main())\n\nLow-level API\n++++++++++++++\n\nThe `AIOFile` class is a low-level interface for asynchronous file operations, and the read and write methods accept\nan `offset=0` in bytes at which the operation will be performed.\n\nThis allows you to do many independent IO operations on an once open file without moving the virtual carriage.\n\nFor example, you may make 10 concurrent HTTP requests by specifying the `Range` header, and asynchronously write\none opened file, while the offsets must either be calculated manually, or use 10 instances of `Writer` with\nspecified initial offsets.\n\nIn order to provide sequential reading and writing, there is `Writer`, `Reader` and `LineReader`. Keep in mind\n`async_open` is not the same as AIOFile, it provides a similar interface for file operations, it simulates methods\nlike read or write as it is implemented in the built-in open.\n\n.. code-block:: python\n :name: test_low_level_api\n\n import asyncio\n from aiofile import AIOFile\n\n\n async def main():\n async with AIOFile(\"hello.txt\", 'w+') as afp:\n payload = \"Hello world\\n\"\n\n await asyncio.gather(\n *[afp.write(payload, offset=i * len(payload)) for i in range(10)]\n )\n\n await afp.fsync()\n\n assert await afp.read(len(payload) * 10) == payload * 10\n\n asyncio.run(main())\n\nThe Low-level API in fact is just little bit sugared ``caio`` API.\n\n.. code-block:: python\n\n import asyncio\n from aiofile import AIOFile\n\n\n async def main():\n async with AIOFile(\"/tmp/hello.txt\", 'w+') as afp:\n await afp.write(\"Hello \")\n await afp.write(\"world\", offset=7)\n await afp.fsync()\n\n print(await afp.read())\n\n\n loop = asyncio.get_event_loop()\n loop.run_until_complete(main())\n\n\n``Reader`` and ``Writer``\n~~~~~~~~~~~~~~~~~~~~~~~~~\n\nWhen you want to read or write file linearly following example\nmight be helpful.\n\n.. code-block:: python\n\n import asyncio\n from aiofile import AIOFile, Reader, Writer\n\n\n async def main():\n async with AIOFile(\"/tmp/hello.txt\", 'w+') as afp:\n writer = Writer(afp)\n reader = Reader(afp, chunk_size=8)\n\n await writer(\"Hello\")\n await writer(\" \")\n await writer(\"World\")\n await afp.fsync()\n\n async for chunk in reader:\n print(chunk)\n\n\n loop = asyncio.get_event_loop()\n loop.run_until_complete(main())\n\n\n\n``LineReader`` - read file line by line\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nLineReader is a helper that is very effective when you want to read a file\nlinearly and line by line.\n\nIt contains a buffer and will read the fragments of the file chunk by\nchunk into the buffer, where it will try to find lines.\n\nThe default chunk size is 4KB.\n\n.. code-block:: python\n\n import asyncio\n from aiofile import AIOFile, LineReader, Writer\n\n\n async def main():\n async with AIOFile(\"/tmp/hello.txt\", 'w+') as afp:\n writer = Writer(afp)\n\n await writer(\"Hello\")\n await writer(\" \")\n await writer(\"World\")\n await writer(\"\\n\")\n await writer(\"\\n\")\n await writer(\"From async world\")\n await afp.fsync()\n\n async for line in LineReader(afp):\n print(line)\n\n\n loop = asyncio.get_event_loop()\n loop.run_until_complete(main())\n\n\nWhen you want to read file by lines please avoid to use ``async_open``\nuse ``LineReader`` instead.\n\n\nMore examples\n-------------\n\nUseful examples with ``aiofile``\n\nAsync CSV Dict Reader\n+++++++++++++++++++++\n\n.. code-block:: python\n\n import asyncio\n import io\n from csv import DictReader\n\n from aiofile import AIOFile, LineReader\n\n\n class AsyncDictReader:\n def __init__(self, afp, **kwargs):\n self.buffer = io.BytesIO()\n self.file_reader = LineReader(\n afp, line_sep=kwargs.pop('line_sep', '\\n'),\n chunk_size=kwargs.pop('chunk_size', 4096),\n offset=kwargs.pop('offset', 0),\n )\n self.reader = DictReader(\n io.TextIOWrapper(\n self.buffer,\n encoding=kwargs.pop('encoding', 'utf-8'),\n errors=kwargs.pop('errors', 'replace'),\n ), **kwargs,\n )\n self.line_num = 0\n\n def __aiter__(self):\n return self\n\n async def __anext__(self):\n if self.line_num == 0:\n header = await self.file_reader.readline()\n self.buffer.write(header)\n\n line = await self.file_reader.readline()\n\n if not line:\n raise StopAsyncIteration\n\n self.buffer.write(line)\n self.buffer.seek(0)\n\n try:\n result = next(self.reader)\n except StopIteration as e:\n raise StopAsyncIteration from e\n\n self.buffer.seek(0)\n self.buffer.truncate(0)\n self.line_num = self.reader.line_num\n\n return result\n\n\n async def main():\n async with AIOFile('sample.csv', 'rb') as afp:\n async for item in AsyncDictReader(afp):\n print(item)\n\n\n asyncio.run(main())\n\n\n.. _troubleshooting:\n\nTroubleshooting\n---------------\n\nThe caio ``linux`` implementation works normal for modern linux kernel versions\nand file systems. So you may have problems specific for your environment.\nIt's not a bug and might be resolved some ways:\n\n1. Upgrade the kernel\n2. Use compatible file systems\n3. Use threads based or pure python implementation.\n\nThe caio since version 0.7.0 contains some ways to do this.\n\n1. In runtime use the environment variable ``CAIO_IMPL`` with\npossible values:\n\n* ``linux`` - use native linux kernels aio mechanism\n* ``thread`` - use thread based implementation written in C\n* ``python`` - use pure python implementation\n\n2. File ``default_implementation`` located near ``__init__.py`` in caio\ninstallation path. It's useful for distros package maintainers. This file\nmight contains comments (lines starts with ``#`` symbol) and the first line\nshould be one of ``linux`` ``thread`` or ``python``.\n\n3. You might manually manage contexts:\n\n.. code-block:: python\n\n import asyncio\n\n from aiofile import async_open\n from caio import linux_aio_asyncio, thread_aio_asyncio\n\n\n async def main():\n linux_ctx = linux_aio_asyncio.AsyncioContext()\n threads_ctx = thread_aio_asyncio.AsyncioContext()\n\n async with async_open(\"/tmp/test.txt\", \"w\", context=linux_ctx) as afp:\n await afp.write(\"Hello\")\n\n async with async_open(\"/tmp/test.txt\", \"r\", context=threads_ctx) as afp:\n print(await afp.read())\n\n\n asyncio.run(main())\n\n\n",
"bugtrack_url": null,
"license": "Apache 2",
"summary": "Asynchronous file operations.",
"version": "3.8.8",
"project_urls": {
"Homepage": "http://github.com/mosquito/aiofile"
},
"split_keywords": [
"aio",
"python",
"asyncio",
"fileio",
"io"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "76eab6aae3275bb0991c43fff401e7b8db198047b6f692e8954520ec7aed3652",
"md5": "93c0f7bae4c498d920d3fa62bb97b527",
"sha256": "41e8845cce055779cd77713d949a339deb012eab605b857765e8f8e52a5ed811"
},
"downloads": -1,
"filename": "aiofile-3.8.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "93c0f7bae4c498d920d3fa62bb97b527",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7, <4",
"size": 19866,
"upload_time": "2023-08-22T12:48:37",
"upload_time_iso_8601": "2023-08-22T12:48:37.144548Z",
"url": "https://files.pythonhosted.org/packages/76/ea/b6aae3275bb0991c43fff401e7b8db198047b6f692e8954520ec7aed3652/aiofile-3.8.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "416de03ef9694346a55da965538f3e28461bd9fc71e21f997269d03b3cfc8755",
"md5": "1c2dd8c0890c6345e35c844d81a056e6",
"sha256": "41f3dc40bd730459d58610476e82e5efb2f84ae6e9fa088a9545385d838b8a43"
},
"downloads": -1,
"filename": "aiofile-3.8.8.tar.gz",
"has_sig": false,
"md5_digest": "1c2dd8c0890c6345e35c844d81a056e6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7, <4",
"size": 19456,
"upload_time": "2023-08-22T12:48:38",
"upload_time_iso_8601": "2023-08-22T12:48:38.880505Z",
"url": "https://files.pythonhosted.org/packages/41/6d/e03ef9694346a55da965538f3e28461bd9fc71e21f997269d03b3cfc8755/aiofile-3.8.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-22 12:48:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mosquito",
"github_project": "aiofile",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "aiofile"
}
JSON
