# v3iofs
<!--
[](https://github.com/v3io/v3io-fs/actions?query=workflow%3ACI)
-->
[](https://opensource.org/licenses/Apache-2.0)
An [fsspec][fsspec] driver for [v3io][v3io].
## Examples
### Python
```python
>>> from v3iofs import V3ioFS # Will register 'v3io' protocol in ffspec
>>> fs = V3ioFS('api.app.yh48.iguazio-cd2.com', v3io_access_key='s3cr3t')
>>> fs.ls('/container/path')
```
### Pandas
```python
# Use V3IO_ACCESS_KEY & V3IO_API from environment
>> df = pd.read_csv('v3io://container/path/to/file.csv')
```
### Dask
```python
>>> from v3iofs import V3ioFS
>>> from dask import bag
# Use V3IO_ACCESS_KEY & V3IO_API from environment
>>> url = 'v3io://container/path'
>>> file = bag.read_text(url)
>>> data, _ = file.compute()
# Pass key in storage_options
>>> storage_options={
... 'v3io_api': 'webapi.app.iguazio.com',
... 'v3io_access_key': 's3cr3t',
... }
>>> file = bag.read_text(url, storage_options=storage_options)
>>> data, _ = file.compute()
```
## Development
### Testing
You need to set `V3IO_ACCESS_KEY` and `V3IO_API` environment variables.
Then run `make test` to run the tests.
### Environment
Deployment requirements are in `requirements.txt` and development requirements
are in `dev-requirements.txt`.
```
$ python -m venv venv
$ ./venv/bin/python -m pip install -r dev-requirements.txt
```
[fsspec]: https://filesystem-spec.readthedocs.io
[v3io]: https://www.iguazio.com/docs/tutorials/latest-release/getting-started/containers/
Raw data
{
"_id": null,
"home_page": "https://github.com/v3io/v3io-fs",
"name": "v3iofs",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Iguazio",
"author_email": "yaronh@iguazio.com",
"download_url": "https://files.pythonhosted.org/packages/f6/13/56e1bc41c19b4d6cd06efd812fbebf9d076f2310ef669f22238eb7394d5d/v3iofs-0.1.16.tar.gz",
"platform": null,
"description": "# v3iofs\n\n<!--\n[](https://github.com/v3io/v3io-fs/actions?query=workflow%3ACI)\n-->\n[](https://opensource.org/licenses/Apache-2.0)\n\n\nAn [fsspec][fsspec] driver for [v3io][v3io].\n\n\n## Examples\n\n\n### Python\n\n```python\n>>> from v3iofs import V3ioFS # Will register 'v3io' protocol in ffspec\n>>> fs = V3ioFS('api.app.yh48.iguazio-cd2.com', v3io_access_key='s3cr3t')\n>>> fs.ls('/container/path')\n```\n\n### Pandas\n\n```python\n# Use V3IO_ACCESS_KEY & V3IO_API from environment\n>> df = pd.read_csv('v3io://container/path/to/file.csv')\n```\n\n### Dask\n\n```python\n>>> from v3iofs import V3ioFS\n>>> from dask import bag\n\n# Use V3IO_ACCESS_KEY & V3IO_API from environment\n>>> url = 'v3io://container/path'\n>>> file = bag.read_text(url)\n>>> data, _ = file.compute()\n\n# Pass key in storage_options\n>>> storage_options={\n... 'v3io_api': 'webapi.app.iguazio.com',\n... 'v3io_access_key': 's3cr3t',\n... }\n>>> file = bag.read_text(url, storage_options=storage_options)\n>>> data, _ = file.compute()\n```\n\n## Development\n\n\n### Testing\n\nYou need to set `V3IO_ACCESS_KEY` and `V3IO_API` environment variables.\nThen run `make test` to run the tests.\n\n\n### Environment\n\nDeployment requirements are in `requirements.txt` and development requirements\nare in `dev-requirements.txt`.\n\n```\n$ python -m venv venv\n$ ./venv/bin/python -m pip install -r dev-requirements.txt\n```\n\n\n[fsspec]: https://filesystem-spec.readthedocs.io\n[v3io]: https://www.iguazio.com/docs/tutorials/latest-release/getting-started/containers/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "fsspec driver for v3io",
"version": "0.1.16",
"project_urls": {
"Homepage": "https://github.com/v3io/v3io-fs"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3a706d2da0bc78a3491185e24d1873c77d17dd2fde48d47804e66bc576154b97",
"md5": "bee94394b045bccc747c1790dbfa9932",
"sha256": "c8a9f0bd0e6c8061940bd05ad5f2281e8d61375323101624b40bf3b8f3b36c29"
},
"downloads": -1,
"filename": "v3iofs-0.1.16-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bee94394b045bccc747c1790dbfa9932",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 13925,
"upload_time": "2023-06-01T04:57:32",
"upload_time_iso_8601": "2023-06-01T04:57:32.241901Z",
"url": "https://files.pythonhosted.org/packages/3a/70/6d2da0bc78a3491185e24d1873c77d17dd2fde48d47804e66bc576154b97/v3iofs-0.1.16-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f61356e1bc41c19b4d6cd06efd812fbebf9d076f2310ef669f22238eb7394d5d",
"md5": "59d5758dcaf8a37dd2f2082f8d5414e8",
"sha256": "61b9894171e24f38c9db2f9d2ddb4f87a3b7e14bc4b641cb1ed56d7ad754ae87"
},
"downloads": -1,
"filename": "v3iofs-0.1.16.tar.gz",
"has_sig": false,
"md5_digest": "59d5758dcaf8a37dd2f2082f8d5414e8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 17747,
"upload_time": "2023-06-01T04:57:33",
"upload_time_iso_8601": "2023-06-01T04:57:33.850772Z",
"url": "https://files.pythonhosted.org/packages/f6/13/56e1bc41c19b4d6cd06efd812fbebf9d076f2310ef669f22238eb7394d5d/v3iofs-0.1.16.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-01 04:57:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "v3io",
"github_project": "v3io-fs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "v3iofs"
}
JSON
