# AWS S3 Tools
[](https://github.com/dlite-tools/aws-s3-tools/actions/workflows/test.yml)
[](https://codecov.io/gh/dlite-tools/aws-s3-tools)
AWS S3 Tools is a Python package to make it easier to interact with S3 objects, where you can:
- List S3 bucket content
- Check if an S3 object exists
- Download/upload S3 objects to/from local files
- Read/write S3 objects into/from Python variables
- Delete/move/copy S3 objects
The AWS S3 authentication is done via boto3 package, via environment variables, aws config file, or parameters.
All S3 objects functions, in this package, have the option to set AWS Session authentication by passing the following dictionary on the `aws_auth` parameter, with the schema below (not all field are required).
To understand more about AWS authentication mechanism, [read boto3 documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html).
```python
aws_auth = {
'region_name': 'REGION',
'aws_access_key_id': 'ACCESS_KEY',
'aws_secret_access_key': 'SECRET_KEY',
'aws_session_token': 'SESSION_TOKEN',
'profile_name': 'PROFILE_NAME'
}
```
---
## Installation
You can install AWS S3 Tools from PyPi with `pip` or your favorite package manager:
```shell
pip install aws-s3-tools
```
Add the ``-U`` switch to update to the current version, if AWS S3 Tools is already installed.
If you want to use the **progress bar** feature when downloading or uploading,
you need to install an extra dependency.
```shell
pip install aws-s3-tools[progress]
```
---
## Usage
[The full documentation can be found here](https://aws-s3-tools.readthedocs.io/en/latest/index.html).
```python
from s3_tools import object_exists
if object_exists("my-bucket", "s3-prefix/object.data"):
# Your code goes here
else:
print("Object not found")
```
Example to use the progress bar:
```python
from s3_tools import upload_folder_to_prefix
result = upload_folder_to_prefix(
bucket='dlite-tools',
prefix='aws-s3-tools',
search_str='*.py',
threads=2,
folder='s3_tools',
show_progress=True
)
```
Progress bar when running the code above:
---
## Contributions
All contributions, bug reports, bug fixes, documentation improvements,
enhancements and ideas are welcome.
A detailed overview on how to contribute can be found in the
[contributing guide](CONTRIBUTING.md)
on GitHub.
---
## Issues
Go [here](https://github.com/dlite-tools/aws-s3-tools/issues) to submit feature
requests or bugfixes.
---
## License and Credits
`AWS S3 Tools` is licensed under the [MIT license](LICENSE) and is written and
maintained by:
- Daniel Ferrari ([@FerrariDG](https://github.com/FerrariDG))
- Carlos Alves ([@cmalves](https://github.com/cmalves))
- Tomás Osório ([@tomassosorio](https://github.com/tomassosorio/))
---
## Acknowledgement
The idea from these functions come from an amazing team that I worked with. This repo is a refactor and documentation to make this public to everyone.
Many thanks to:
- [Anabela Nogueira](https://www.linkedin.com/in/abnogueira/)
- [Carlos Alves](https://www.linkedin.com/in/carlosmalves/)
- [João Machado](https://www.linkedin.com/in/machadojpf/)
- [Renato Dantas](https://www.linkedin.com/in/renatomoura/)
- [Ricardo Garcia](https://www.linkedin.com/in/ricardo-g-oliveira/)
- [Tomás Osório](https://www.linkedin.com/in/tomas-osorio/)
Raw data
{
"_id": null,
"home_page": "https://github.com/dlite-tools/aws-s3-tools",
"name": "aws-s3-tools",
"maintainer": "Daniel Ferrari, Carlos Alves, Tom\u00e1s Os\u00f3rio",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "aws,s3,tools,package",
"author": "Daniel Ferrari",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/79/65/2cd30bf9141721af0458ac7e61ce36a14627f80b9378f00d5f377bd3c338/aws_s3_tools-0.3.0.tar.gz",
"platform": null,
"description": "# AWS S3 Tools\n\n\n[](https://github.com/dlite-tools/aws-s3-tools/actions/workflows/test.yml)\n[](https://codecov.io/gh/dlite-tools/aws-s3-tools)\n\n\n\n\nAWS S3 Tools is a Python package to make it easier to interact with S3 objects, where you can:\n\n- List S3 bucket content\n- Check if an S3 object exists\n- Download/upload S3 objects to/from local files\n- Read/write S3 objects into/from Python variables\n- Delete/move/copy S3 objects\n\nThe AWS S3 authentication is done via boto3 package, via environment variables, aws config file, or parameters.\nAll S3 objects functions, in this package, have the option to set AWS Session authentication by passing the following dictionary on the `aws_auth` parameter, with the schema below (not all field are required).\nTo understand more about AWS authentication mechanism, [read boto3 documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html).\n\n```python\naws_auth = {\n 'region_name': 'REGION',\n 'aws_access_key_id': 'ACCESS_KEY',\n 'aws_secret_access_key': 'SECRET_KEY',\n 'aws_session_token': 'SESSION_TOKEN',\n 'profile_name': 'PROFILE_NAME'\n}\n```\n\n---\n\n## Installation\n\nYou can install AWS S3 Tools from PyPi with `pip` or your favorite package manager:\n\n```shell\npip install aws-s3-tools\n```\n\nAdd the ``-U`` switch to update to the current version, if AWS S3 Tools is already installed.\n\nIf you want to use the **progress bar** feature when downloading or uploading,\nyou need to install an extra dependency.\n\n```shell\npip install aws-s3-tools[progress]\n```\n\n---\n\n## Usage\n\n[The full documentation can be found here](https://aws-s3-tools.readthedocs.io/en/latest/index.html).\n\n```python\nfrom s3_tools import object_exists\n\nif object_exists(\"my-bucket\", \"s3-prefix/object.data\"):\n # Your code goes here\nelse:\n print(\"Object not found\")\n```\n\nExample to use the progress bar:\n\n```python\nfrom s3_tools import upload_folder_to_prefix\n\nresult = upload_folder_to_prefix(\n bucket='dlite-tools',\n prefix='aws-s3-tools',\n search_str='*.py',\n threads=2,\n folder='s3_tools',\n show_progress=True\n)\n```\n\nProgress bar when running the code above:\n\n\n\n---\n\n## Contributions\n\nAll contributions, bug reports, bug fixes, documentation improvements,\nenhancements and ideas are welcome.\n\nA detailed overview on how to contribute can be found in the\n[contributing guide](CONTRIBUTING.md)\non GitHub.\n\n---\n\n## Issues\n\nGo [here](https://github.com/dlite-tools/aws-s3-tools/issues) to submit feature\nrequests or bugfixes.\n\n---\n\n## License and Credits\n\n`AWS S3 Tools` is licensed under the [MIT license](LICENSE) and is written and\nmaintained by:\n\n- Daniel Ferrari ([@FerrariDG](https://github.com/FerrariDG))\n- Carlos Alves ([@cmalves](https://github.com/cmalves))\n- Tom\u00e1s Os\u00f3rio ([@tomassosorio](https://github.com/tomassosorio/))\n\n---\n\n## Acknowledgement\n\nThe idea from these functions come from an amazing team that I worked with. This repo is a refactor and documentation to make this public to everyone.\n\nMany thanks to:\n\n- [Anabela Nogueira](https://www.linkedin.com/in/abnogueira/)\n- [Carlos Alves](https://www.linkedin.com/in/carlosmalves/)\n- [Jo\u00e3o Machado](https://www.linkedin.com/in/machadojpf/)\n- [Renato Dantas](https://www.linkedin.com/in/renatomoura/)\n- [Ricardo Garcia](https://www.linkedin.com/in/ricardo-g-oliveira/)\n- [Tom\u00e1s Os\u00f3rio](https://www.linkedin.com/in/tomas-osorio/)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "AWS S3 tools package",
"version": "0.3.0",
"project_urls": {
"Documentation": "https://aws-s3-tools.readthedocs.io/en/latest/index.html",
"Homepage": "https://github.com/dlite-tools/aws-s3-tools",
"Repository": "https://github.com/dlite-tools/aws-s3-tools"
},
"split_keywords": [
"aws",
"s3",
"tools",
"package"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3bd761117bece8b45f2e11474c4b11f869948a89be22c72a70eaa2589da4a3e2",
"md5": "f8481ddb93f3f5e2c7b72456de8463b1",
"sha256": "e77ddf08e258e2acabfbbfe47b017ba69e76e91c60788edf47d269c725418872"
},
"downloads": -1,
"filename": "aws_s3_tools-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f8481ddb93f3f5e2c7b72456de8463b1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 22179,
"upload_time": "2023-06-13T10:11:53",
"upload_time_iso_8601": "2023-06-13T10:11:53.962094Z",
"url": "https://files.pythonhosted.org/packages/3b/d7/61117bece8b45f2e11474c4b11f869948a89be22c72a70eaa2589da4a3e2/aws_s3_tools-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "79652cd30bf9141721af0458ac7e61ce36a14627f80b9378f00d5f377bd3c338",
"md5": "2841104d6bd213a56d909482741ea77c",
"sha256": "c8ec8f1505e3373df863c1f2a587e76161c0b7b789c920bd66ed582570f3e2bd"
},
"downloads": -1,
"filename": "aws_s3_tools-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "2841104d6bd213a56d909482741ea77c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 14173,
"upload_time": "2023-06-13T10:11:56",
"upload_time_iso_8601": "2023-06-13T10:11:56.031298Z",
"url": "https://files.pythonhosted.org/packages/79/65/2cd30bf9141721af0458ac7e61ce36a14627f80b9378f00d5f377bd3c338/aws_s3_tools-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-13 10:11:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dlite-tools",
"github_project": "aws-s3-tools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "aws-s3-tools"
}
JSON
