| Name | kvikio-cu12 JSON |
| Version |
25.8.0
JSON |
| download |
| home_page | None |
| Summary | KvikIO - GPUDirect Storage |
| upload_time | 2025-08-07 13:50:58 |
| maintainer | None |
| docs_url | None |
| author | NVIDIA Corporation |
| requires_python | >=3.10 |
| license | Apache-2.0 |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# KvikIO: High Performance File IO
## Summary
KvikIO (pronounced "kuh-VICK-eye-oh", see [here](https://ordnet.dk/ddo_en/dict?query=kvik) for pronunciation of kvik) is a Python and C++ library for high performance file IO. It provides C++ and Python
bindings to [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html),
which enables [GPUDirect Storage (GDS)](https://developer.nvidia.com/blog/gpudirect-storage/).
KvikIO also works efficiently when GDS isn't available and can read/write both host and device data seamlessly.
### Features
* Object oriented API of [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html) with C++/Python exception handling.
* A Python [Zarr](https://zarr.readthedocs.io/en/stable/) backend for reading and writing GPU data to file seamlessly.
* Concurrent reads and writes using an internal thread pool.
* Non-blocking API.
* Transparently handles reads and writes to/from memory on both host and device.
* (Deprecated) Provides Python bindings to [nvCOMP](https://docs.nvidia.com/cuda/nvcomp/py_api.html).
### Documentation
* Python: <https://docs.rapids.ai/api/kvikio/nightly/>
* C++: <https://docs.rapids.ai/api/libkvikio/nightly/>
### Examples
#### Python
```python
import cupy
import kvikio
def main(path):
a = cupy.arange(100)
f = kvikio.CuFile(path, "w")
# Write whole array to file
f.write(a)
f.close()
b = cupy.empty_like(a)
f = kvikio.CuFile(path, "r")
# Read whole array from file
f.read(b)
assert all(a == b)
f.close()
# Use contexmanager
c = cupy.empty_like(a)
with kvikio.CuFile(path, "r") as f:
f.read(c)
assert all(a == c)
# Non-blocking read
d = cupy.empty_like(a)
with kvikio.CuFile(path, "r") as f:
future1 = f.pread(d[:50])
future2 = f.pread(d[50:], file_offset=d[:50].nbytes)
# Note: must wait for futures before exiting block
# at which point the file is closed.
future1.get() # Wait for first read
future2.get() # Wait for second read
assert all(a == d)
if __name__ == "__main__":
main("/tmp/kvikio-hello-world-file")
```
#### C++
```c++
#include <cstddef>
#include <future>
#include <cuda_runtime.h>
#include <kvikio/file_handle.hpp>
int main()
{
// Create two arrays `a` and `b`
constexpr std::size_t size = 100;
void *a = nullptr;
void *b = nullptr;
cudaMalloc(&a, size);
cudaMalloc(&b, size);
// Write `a` to file
kvikio::FileHandle fw("test-file", "w");
std::size_t written = fw.write(a, size);
fw.close();
// Read file into `b`
kvikio::FileHandle fr("test-file", "r");
std::size_t read = fr.read(b, size);
fr.close();
// Read file into `b` in parallel using 16 threads
kvikio::default_thread_pool::reset(16);
{
// FileHandles have RAII semantics
kvikio::FileHandle f("test-file", "r");
std::future<std::size_t> future = f.pread(b_dev, sizeof(a), 0); // Non-blocking
std::size_t read = future.get(); // Blocking
// Notice, `f` closes automatically on destruction.
}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "kvikio-cu12",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": null,
"author": "NVIDIA Corporation",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# KvikIO: High Performance File IO\n\n## Summary\n\nKvikIO (pronounced \"kuh-VICK-eye-oh\", see [here](https://ordnet.dk/ddo_en/dict?query=kvik) for pronunciation of kvik) is a Python and C++ library for high performance file IO. It provides C++ and Python\nbindings to [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html),\nwhich enables [GPUDirect Storage (GDS)](https://developer.nvidia.com/blog/gpudirect-storage/).\nKvikIO also works efficiently when GDS isn't available and can read/write both host and device data seamlessly.\n\n\n### Features\n\n* Object oriented API of [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html) with C++/Python exception handling.\n* A Python [Zarr](https://zarr.readthedocs.io/en/stable/) backend for reading and writing GPU data to file seamlessly.\n* Concurrent reads and writes using an internal thread pool.\n* Non-blocking API.\n* Transparently handles reads and writes to/from memory on both host and device.\n* (Deprecated) Provides Python bindings to [nvCOMP](https://docs.nvidia.com/cuda/nvcomp/py_api.html).\n\n\n### Documentation\n * Python: <https://docs.rapids.ai/api/kvikio/nightly/>\n * C++: <https://docs.rapids.ai/api/libkvikio/nightly/>\n\n\n### Examples\n\n#### Python\n```python\nimport cupy\nimport kvikio\n\ndef main(path):\n a = cupy.arange(100)\n f = kvikio.CuFile(path, \"w\")\n # Write whole array to file\n f.write(a)\n f.close()\n\n b = cupy.empty_like(a)\n f = kvikio.CuFile(path, \"r\")\n # Read whole array from file\n f.read(b)\n assert all(a == b)\n f.close()\n\n # Use contexmanager\n c = cupy.empty_like(a)\n with kvikio.CuFile(path, \"r\") as f:\n f.read(c)\n assert all(a == c)\n\n # Non-blocking read\n d = cupy.empty_like(a)\n with kvikio.CuFile(path, \"r\") as f:\n future1 = f.pread(d[:50])\n future2 = f.pread(d[50:], file_offset=d[:50].nbytes)\n # Note: must wait for futures before exiting block\n # at which point the file is closed.\n future1.get() # Wait for first read\n future2.get() # Wait for second read\n assert all(a == d)\n\n\nif __name__ == \"__main__\":\n main(\"/tmp/kvikio-hello-world-file\")\n```\n\n#### C++\n```c++\n#include <cstddef>\n#include <future>\n#include <cuda_runtime.h>\n#include <kvikio/file_handle.hpp>\n\nint main()\n{\n // Create two arrays `a` and `b`\n constexpr std::size_t size = 100;\n void *a = nullptr;\n void *b = nullptr;\n cudaMalloc(&a, size);\n cudaMalloc(&b, size);\n\n // Write `a` to file\n kvikio::FileHandle fw(\"test-file\", \"w\");\n std::size_t written = fw.write(a, size);\n fw.close();\n\n // Read file into `b`\n kvikio::FileHandle fr(\"test-file\", \"r\");\n std::size_t read = fr.read(b, size);\n fr.close();\n\n // Read file into `b` in parallel using 16 threads\n kvikio::default_thread_pool::reset(16);\n {\n // FileHandles have RAII semantics\n kvikio::FileHandle f(\"test-file\", \"r\");\n std::future<std::size_t> future = f.pread(b_dev, sizeof(a), 0); // Non-blocking\n std::size_t read = future.get(); // Blocking\n // Notice, `f` closes automatically on destruction.\n }\n}\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "KvikIO - GPUDirect Storage",
"version": "25.8.0",
"project_urls": {
"Homepage": "https://github.com/rapidsai/kvikio"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e91eaa4b7ab4f0a5faa94445bd1135f172da6a29211e97298cf99cf2f398755a",
"md5": "6bc33843b16c2639a3103d1cc2431b41",
"sha256": "dc799e15466e24851c882730e399661443e14720950ccc357ac5b5ee8308b135"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"has_sig": false,
"md5_digest": "6bc33843b16c2639a3103d1cc2431b41",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 668566,
"upload_time": "2025-08-07T13:50:58",
"upload_time_iso_8601": "2025-08-07T13:50:58.104363Z",
"url": "https://files.pythonhosted.org/packages/e9/1e/aa4b7ab4f0a5faa94445bd1135f172da6a29211e97298cf99cf2f398755a/kvikio_cu12-25.8.0-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f6758fe15826ab759c5dd144fef6779379431a6d1d8595344ee902c83cb7b306",
"md5": "0a67d7b7211b3af25050a7869ad0d45b",
"sha256": "8a7d445304df781d5144fec25d733a34a0935d8e383262ed21cd5b9f70d02d89"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "0a67d7b7211b3af25050a7869ad0d45b",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 714568,
"upload_time": "2025-08-07T13:43:56",
"upload_time_iso_8601": "2025-08-07T13:43:56.504470Z",
"url": "https://files.pythonhosted.org/packages/f6/75/8fe15826ab759c5dd144fef6779379431a6d1d8595344ee902c83cb7b306/kvikio_cu12-25.8.0-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "58b7fbc1968592c92396efd3dc68f330d0e3da98a1150bfae414029bea9f9fe1",
"md5": "95c7896e4fd4a1b70ee2f60e8c8b821c",
"sha256": "51094df8e0793485ae5021174b65bcd2bc584853cd832c142e6b463380bf4109"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"has_sig": false,
"md5_digest": "95c7896e4fd4a1b70ee2f60e8c8b821c",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 672337,
"upload_time": "2025-08-07T13:50:13",
"upload_time_iso_8601": "2025-08-07T13:50:13.422881Z",
"url": "https://files.pythonhosted.org/packages/58/b7/fbc1968592c92396efd3dc68f330d0e3da98a1150bfae414029bea9f9fe1/kvikio_cu12-25.8.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c36f62a56dff50260a5bfc40f19bf0101f1e859b284c01762985626f226fd626",
"md5": "4d8f1f4cd42235e75c7d3d7d9647252c",
"sha256": "5cf3c8c6fff7167f342f0a48f84c7e9dbc6bc2af5cae5d9f01261ee0b09066b6"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "4d8f1f4cd42235e75c7d3d7d9647252c",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 715741,
"upload_time": "2025-08-07T13:43:32",
"upload_time_iso_8601": "2025-08-07T13:43:32.911601Z",
"url": "https://files.pythonhosted.org/packages/c3/6f/62a56dff50260a5bfc40f19bf0101f1e859b284c01762985626f226fd626/kvikio_cu12-25.8.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3f7d6ca65b1c4109a1a65c6568c006da3bc6e224c4c41c48433a40c06094cf66",
"md5": "cbdfdb6ba1c197dc6dbdc4aa5f3c67bd",
"sha256": "b09ffb4dd71b4800ae14593aa4112bee1c2fc5ce929329078b326a396b9d6cff"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"has_sig": false,
"md5_digest": "cbdfdb6ba1c197dc6dbdc4aa5f3c67bd",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 647986,
"upload_time": "2025-08-07T13:49:27",
"upload_time_iso_8601": "2025-08-07T13:49:27.185831Z",
"url": "https://files.pythonhosted.org/packages/3f/7d/6ca65b1c4109a1a65c6568c006da3bc6e224c4c41c48433a40c06094cf66/kvikio_cu12-25.8.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7a7a1b83ec71526db41c03bb4876e9176ce457e5d3bee330a0b58f167aa96d6b",
"md5": "51887432598216e83de6d25e7e1b3d1b",
"sha256": "dbf35716e2f78c5154b8193ab46832660147269e553f9222bf6408e4220c1171"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "51887432598216e83de6d25e7e1b3d1b",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 694195,
"upload_time": "2025-08-07T13:43:09",
"upload_time_iso_8601": "2025-08-07T13:43:09.925906Z",
"url": "https://files.pythonhosted.org/packages/7a/7a/1b83ec71526db41c03bb4876e9176ce457e5d3bee330a0b58f167aa96d6b/kvikio_cu12-25.8.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f5eff5383ef3719bdc4c87518b14adccb54374a0b4e47c64f7d88c5a16279774",
"md5": "af66b90715c07dadc325836b8d659a57",
"sha256": "7639b81c7d3a3637d238ecf098408e7986bf261c28e32cb8c7238a7c56c21dcb"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"has_sig": false,
"md5_digest": "af66b90715c07dadc325836b8d659a57",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": ">=3.10",
"size": 643826,
"upload_time": "2025-08-07T13:48:39",
"upload_time_iso_8601": "2025-08-07T13:48:39.382950Z",
"url": "https://files.pythonhosted.org/packages/f5/ef/f5383ef3719bdc4c87518b14adccb54374a0b4e47c64f7d88c5a16279774/kvikio_cu12-25.8.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3a1283da80837a6619fcca7e73adbac074462018a25fa7a15aa0c55f7a4028f7",
"md5": "1849cdcc6596ee8cd6814f05ab99a7d9",
"sha256": "8758676d47e9ea195e8f1130576186a586f95521909b5eba0e118469aa923dc2"
},
"downloads": -1,
"filename": "kvikio_cu12-25.8.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "1849cdcc6596ee8cd6814f05ab99a7d9",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": ">=3.10",
"size": 690722,
"upload_time": "2025-08-07T13:42:45",
"upload_time_iso_8601": "2025-08-07T13:42:45.489292Z",
"url": "https://files.pythonhosted.org/packages/3a/12/83da80837a6619fcca7e73adbac074462018a25fa7a15aa0c55f7a4028f7/kvikio_cu12-25.8.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-07 13:50:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rapidsai",
"github_project": "kvikio",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "kvikio-cu12"
}
