<!-- PROJECT BADGES -->
[![Python Version][python-shield]][python-url]
[![MIT License][license-shield]][license-url]
[![Coverage][coverage-shield]][coverage-url]
<!-- What is Fluke? -->
## What is Fluke?
Fluke is a Python package that acts as a higher-level API to
cloud services that primarily relate to object storage and
messaging. Fluke manages to hide away much of the complexity
that derives from working with said services, aiding you in
completing your tasks fast and hassle-free! Fluke achieves this by:
* Wrapping object storage services within a *File/Dir* API
inspired by the more familiar file storage, through which
you are able to interact with your data no matter where
they reside, be it the local file system, a remote server,
or a bucket in the cloud.
* Greatly reducing the intricacies of working with message
queues by viewing them as mere data structures that support
three elementary operations, that is, *push*, *peek*, and *poll*.
<!-- Installation -->
## Installation
You can start using Fluke by installing it via pip.
Note that *fluke* requires Python >= 3.9.
```sh
pip install fluke-api
```
<!-- Usage example -->
## Usage Example
In this example, we will be using Fluke in order to:
1. Poll an Amazon SQS queue every minute for new messages. Each of these messages
contains the path of a newly uploaded file to an Amazon S3 bucket.
2. Use said messages in order to locate the corresponding files and transfer
them to a remote server.
First things first, we need to be able to authenticate with both AWS
and the remote server. In order to achieve this, we will be importing
from ``fluke.auth``:
```python
from fluke.auth import AWSAuth, RemoteAuth
# This object will be used to authenticate
# with AWS.
aws_auth = AWSAuth(
aws_access_key_id="aws_access_key",
aws_secret_access_key="aws_secret_key")
# This object will be used to authenticate
# with the remote machine.
rmt_auth = RemoteAuth.from_password(
hostname="host",
username="user",
password="password")
```
Next, we just need to import from ``fluke.queues`` and ``fluke.storage``
so that we gain access to any necessary resources in order to perform
the data transfer:
```python
from fluke.queues import AmazonSQSQueue
from fluke.storage import AmazonS3Dir, RemoteDir
with (
AmazonSQSQueue(auth=aws_auth, queue='queue') as queue,
AmazonS3Dir(auth=aws_auth, bucket='bucket') as bucket,
RemoteDir(auth=rmt_auth, path='/home/user/dir', create_if_missing=True) as rmt_dir
):
for batch in queue.poll(polling_frequency=60):
for msg in batch:
bucket.get_file(path=msg).transfer_to(dst=rmt_dir)
```
And that's basically it!
You can learn more about Fluke by visiting the [Fluke Documentation Page][docs-url].
<!-- MARKDOWN LINKS & IMAGES -->
[python-shield]: https://img.shields.io/badge/python-3.9+-blue
[python-url]: https://www.python.org/downloads/release/python-390/
[license-shield]: https://img.shields.io/badge/license-MIT-red
[license-url]: https://github.com/manoss96/fluke/blob/main/LICENSE
[coverage-shield]: https://coveralls.io/repos/github/manoss96/fluke/badge.svg?branch=main&service=github
[coverage-url]: https://coveralls.io/github/manoss96/fluke?branch=main
[docs-url]: https://fluke.readthedocs.io/en/latest/
Raw data
{
"_id": null,
"home_page": "",
"name": "fluke-api",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "cloud,storage,message-queues,aws,azure,gcp",
"author": "Manos Stoumpos",
"author_email": "manosstoumpos@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f5/98/4da9d96a313e6d19ec9187115f7a91737bcba3e33f34620d58ef7c687794/fluke-api-0.5.0.tar.gz",
"platform": null,
"description": "<!-- PROJECT BADGES -->\n[![Python Version][python-shield]][python-url]\n[![MIT License][license-shield]][license-url]\n[![Coverage][coverage-shield]][coverage-url]\n\n\n\n<!-- What is Fluke? -->\n## What is Fluke?\n\nFluke is a Python package that acts as a higher-level API to\ncloud services that primarily relate to object storage and\nmessaging. Fluke manages to hide away much of the complexity\nthat derives from working with said services, aiding you in\ncompleting your tasks fast and hassle-free! Fluke achieves this by:\n\n* Wrapping object storage services within a *File/Dir* API\n inspired by the more familiar file storage, through which\n you are able to interact with your data no matter where\n they reside, be it the local file system, a remote server,\n or a bucket in the cloud.\n\n* Greatly reducing the intricacies of working with message\n queues by viewing them as mere data structures that support\n three elementary operations, that is, *push*, *peek*, and *poll*.\n\n\n<!-- Installation -->\n## Installation\n\nYou can start using Fluke by installing it via pip.\nNote that *fluke* requires Python >= 3.9.\n\n```sh\npip install fluke-api\n```\n\n\n<!-- Usage example -->\n## Usage Example\n\nIn this example, we will be using Fluke in order to:\n\n1. Poll an Amazon SQS queue every minute for new messages. Each of these messages\n contains the path of a newly uploaded file to an Amazon S3 bucket.\n2. Use said messages in order to locate the corresponding files and transfer\n them to a remote server.\n\nFirst things first, we need to be able to authenticate with both AWS\nand the remote server. In order to achieve this, we will be importing\nfrom ``fluke.auth``:\n\n```python\nfrom fluke.auth import AWSAuth, RemoteAuth\n\n# This object will be used to authenticate\n# with AWS.\naws_auth = AWSAuth(\n aws_access_key_id=\"aws_access_key\",\n aws_secret_access_key=\"aws_secret_key\")\n\n# This object will be used to authenticate\n# with the remote machine.\nrmt_auth = RemoteAuth.from_password(\n hostname=\"host\",\n username=\"user\",\n password=\"password\")\n```\n\nNext, we just need to import from ``fluke.queues`` and ``fluke.storage``\nso that we gain access to any necessary resources in order to perform\nthe data transfer:\n\n```python\nfrom fluke.queues import AmazonSQSQueue\nfrom fluke.storage import AmazonS3Dir, RemoteDir\n\nwith (\n AmazonSQSQueue(auth=aws_auth, queue='queue') as queue,\n AmazonS3Dir(auth=aws_auth, bucket='bucket') as bucket,\n RemoteDir(auth=rmt_auth, path='/home/user/dir', create_if_missing=True) as rmt_dir\n):\n for batch in queue.poll(polling_frequency=60):\n for msg in batch:\n bucket.get_file(path=msg).transfer_to(dst=rmt_dir)\n```\n\nAnd that's basically it!\n\nYou can learn more about Fluke by visiting the [Fluke Documentation Page][docs-url].\n\n\n<!-- MARKDOWN LINKS & IMAGES -->\n[python-shield]: https://img.shields.io/badge/python-3.9+-blue\n[python-url]: https://www.python.org/downloads/release/python-390/\n[license-shield]: https://img.shields.io/badge/license-MIT-red\n[license-url]: https://github.com/manoss96/fluke/blob/main/LICENSE\n[coverage-shield]: https://coveralls.io/repos/github/manoss96/fluke/badge.svg?branch=main&service=github\n[coverage-url]: https://coveralls.io/github/manoss96/fluke?branch=main\n[docs-url]: https://fluke.readthedocs.io/en/latest/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Cloud-agnostic Python API",
"version": "0.5.0",
"project_urls": {
"Bug Tracker": "https://github.com/manoss96/fluke/issues",
"Changelog": "https://github.com/manoss96/fluke/blob/main/CHANGELOG.md",
"Documentation": "https://fluke.rtfd.io",
"Homepage": "https://github.com/manoss96/fluke"
},
"split_keywords": [
"cloud",
"storage",
"message-queues",
"aws",
"azure",
"gcp"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "926e92cd8309ae5ff69f090ab88abae6e162747b2dd2f2e9fc51fb37cfe9d522",
"md5": "19972e9d45e68ec8fb7ca3c5354d9896",
"sha256": "99df382f73f5d7908f2838717a9dce554997b39d1978b2f4a6c9f608aa0f83f4"
},
"downloads": -1,
"filename": "fluke_api-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "19972e9d45e68ec8fb7ca3c5354d9896",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 40717,
"upload_time": "2023-08-20T14:50:47",
"upload_time_iso_8601": "2023-08-20T14:50:47.951179Z",
"url": "https://files.pythonhosted.org/packages/92/6e/92cd8309ae5ff69f090ab88abae6e162747b2dd2f2e9fc51fb37cfe9d522/fluke_api-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f5984da9d96a313e6d19ec9187115f7a91737bcba3e33f34620d58ef7c687794",
"md5": "226aae065f76e497c2079f626530f6ab",
"sha256": "a5fdccdbf4c8d9545103d3957fe4fcf1643a842480c88a4333cfe6915b7fa69d"
},
"downloads": -1,
"filename": "fluke-api-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "226aae065f76e497c2079f626530f6ab",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 62451,
"upload_time": "2023-08-20T14:50:49",
"upload_time_iso_8601": "2023-08-20T14:50:49.687864Z",
"url": "https://files.pythonhosted.org/packages/f5/98/4da9d96a313e6d19ec9187115f7a91737bcba3e33f34620d58ef7c687794/fluke-api-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-20 14:50:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "manoss96",
"github_project": "fluke",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "fluke-api"
}
JSON
