| Name | s5commander JSON |
| Version |
0.0.1.dev2
JSON |
| download |
| home_page | |
| Summary | |
| upload_time | 2023-09-08 09:53:52 |
| maintainer | |
| docs_url | None |
| author | Clément POIRET |
| requires_python | >=3.7,<4.0 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<div align="center">
<h1 align="center">
<img src="https://raw.githubusercontent.com/PKief/vscode-material-icon-theme/ec559a9f6bfd399b82bb44393651661b08aaf7ba/icons/folder-markdown-open.svg" width="100" />
<br>s5commander
</h1>
<h3>◦ S5commander: Unleash the Power of <a href="https://github.com/peak/s5cmd" target="_blank">s5cmd</a> directly in Python</h3>
<p align="center">
<img src="https://img.shields.io/badge/Python-3776AB.svg?style&logo=Python&logoColor=white" alt="Python" />
</p>
<img src="https://img.shields.io/github/languages/top/clementpoiret/s5commander?style&color=5D6D7E" alt="GitHub top language" />
<img src="https://img.shields.io/github/languages/code-size/clementpoiret/s5commander?style&color=5D6D7E" alt="GitHub code size in bytes" />
<img src="https://img.shields.io/github/commit-activity/m/clementpoiret/s5commander?style&color=5D6D7E" alt="GitHub commit activity" />
<img src="https://img.shields.io/github/license/clementpoiret/s5commander?style&color=5D6D7E" alt="GitHub license" />
</div>
---
## 📒 Table of Contents
- [📒 Table of Contents](#-table-of-contents)
- [📍 Overview](#-overview)
- [⚙️ Features](#-features)
- [📂 Project Structure](#project-structure)
- [🧩 Modules](#modules)
- [🚀 Getting Started](#-getting-started)
- [🗺 Roadmap](#-roadmap)
- [🤝 Contributing](#-contributing)
- [📄 License](#-license)
- [👏 Acknowledgments](#-acknowledgments)
---
## 📍 Overview
The s5commander project is a Python library that allows users to interact with S3 buckets using the awesome s5cmd command-line tool.
To quote its description, [`s5cmd`](https://github.com/peak/s5cmd) is a very fast S3 and local filesystem execution tool.
It comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files.
> In short, s5cmd offers a very fast speed. For uploads, s5cmd is 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively.
The goal of this library is to offer a wrapper around this software.
It offers functionality for performing various operations like listing, copying, and removing files, creating and removing buckets, checking disk usage, reading files, and utilizing data pipes.
This project simplifies management and manipulation of S3 data using `s5cmd` inside a Python project, making it easier for users to work with S3 buckets and improve their productivity.
---
## ⚙️ Features
| Feature | Description |
| ---------------------- | ------------------------------------- |
| **⚙️ Architecture** | The codebase follows a modular and object-oriented design. It provides a wrapper for the s5cmd command-line tool and uses classes and functions to handle different actions related to S3 bucket management and file operations. |
| **📖 Documentation** | The documentation will come later... |
| **🔗 Dependencies** | The codebase depends on the s5cmd command-line tool, which is an external dependency for interacting with the AWS S3 service. It relies on necessary AWS credentials and configuration provided by the user. |
| **🧩 Modularity** | The codebase is organized into separate modules: `commander.py` contains classes and functions related to S3 bucket operations and file management. It allows components to be reused or extended for different S3 operations. |
| **✔️ Testing** | Tests will also come later :) |
| **⚡️ Performance** | The performance of the codebase is directly dependent on the underlying s5cmd tool and AWS S3 service. As the codebase primarily acts as a wrapper, its own performance impact is limited. |
| **🔐 Security** | The codebase relies on AWS credentials for accessing and interacting with S3 buckets, ensuring proper authentication and authorization. Proper handling and protection of these credentials is crucial for maintaining security.|
| **🔀 Version Control** | The codebase is hosted on GitHub and follows a standard Git version control workflow. Commits are tracked, and the repository can be cloned, branched, and versions can be audited using Git tools and techniques. |
| **🔌 Integrations** | The codebase is designed to interact with cloud providers using an S3 protocol, including other providers than AWS services. |
---
## 📂 Project Structure
---
## 🧩 Modules
<details closed><summary>S5commander</summary>
| File | Summary |
| --- | --- |
| [commander.py](https://github.com/clementpoiret/s5commander/blob/main/s5commander/commander.py) | The code provides a wrapper for the s5cmd command-line tool, allowing users to interact with an S3 bucket. It supports functionalities like listing files, copying files, removing files, moving files, creating/removing buckets, selecting buckets, checking disk usage, reading files, and using data pipe. |
</details>
---
## 🚀 Getting Started
### ✔️ Prerequisites
Before you begin, ensure that you have the following prerequisites installed:
> - `s5cmd`
### 📦 Installation
1. Clone the s5commander repository:
```sh
git clone https://github.com/clementpoiret/s5commander
```
2. Change to the project directory:
```sh
cd s5commander
```
3. Install the dependencies:
```sh
pip install -r requirements.txt
```
### 🎮 Using s5commander
```python
```
### 🧪 Running Tests
```sh
pytest
```
---
## 🗺 Roadmap
> - [X] `list buckets and objects`
> - [X] `download method`
> - [X] `upload method`
> - [ ] `copy objects`
> - [ ] `remove objects`
> - [ ] `move/rename objects`
> - [ ] `make bucket`
> - [ ] `remove bucket`
> - [ ] `run SQL queries on objects`
> - [ ] `show object size usage`
> - [ ] `print remote object content`
> - [ ] `stream to remote from stdin`
> - [ ] `run commands in batch`
> - [ ] `sync objects`
---
## 🤝 Contributing
Contributions are always welcome! Please follow these steps:
1. Fork the project repository. This creates a copy of the project on your account that you can modify without affecting the original project.
2. Clone the forked repository to your local machine using a Git client like Git or GitHub Desktop.
3. Create a new branch with a descriptive name (e.g., `new-feature-branch` or `bugfix-issue-123`).
```sh
git checkout -b new-feature-branch
```
4. Make changes to the project's codebase.
5. Commit your changes to your local branch with a clear commit message that explains the changes you've made.
```sh
git commit -m 'Implemented new feature.'
```
6. Push your changes to your forked repository on GitHub using the following command
```sh
git push origin new-feature-branch
```
7. Create a new pull request to the original project repository. In the pull request, describe the changes you've made and why they're necessary.
The project maintainers will review your changes and provide feedback or merge them into the main branch.
---
## 📄 License
This project is licensed under the MIT License. See the [LICENSE](LICENSE.md) file for additional info.
---
Raw data
{
"_id": null,
"home_page": "",
"name": "s5commander",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Cl\u00e9ment POIRET",
"author_email": "poiret.clement@outlook.fr",
"download_url": "https://files.pythonhosted.org/packages/f7/29/ce15fa38199a100786a2290f8def878c42c5946b1ae6be6b51e99e594875/s5commander-0.0.1.dev2.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n<h1 align=\"center\">\n<img src=\"https://raw.githubusercontent.com/PKief/vscode-material-icon-theme/ec559a9f6bfd399b82bb44393651661b08aaf7ba/icons/folder-markdown-open.svg\" width=\"100\" />\n<br>s5commander\n</h1>\n<h3>\u25e6 S5commander: Unleash the Power of <a href=\"https://github.com/peak/s5cmd\" target=\"_blank\">s5cmd</a> directly in Python</h3>\n\n<p align=\"center\">\n<img src=\"https://img.shields.io/badge/Python-3776AB.svg?style&logo=Python&logoColor=white\" alt=\"Python\" />\n</p>\n<img src=\"https://img.shields.io/github/languages/top/clementpoiret/s5commander?style&color=5D6D7E\" alt=\"GitHub top language\" />\n<img src=\"https://img.shields.io/github/languages/code-size/clementpoiret/s5commander?style&color=5D6D7E\" alt=\"GitHub code size in bytes\" />\n<img src=\"https://img.shields.io/github/commit-activity/m/clementpoiret/s5commander?style&color=5D6D7E\" alt=\"GitHub commit activity\" />\n<img src=\"https://img.shields.io/github/license/clementpoiret/s5commander?style&color=5D6D7E\" alt=\"GitHub license\" />\n</div>\n\n---\n\n## \ud83d\udcd2 Table of Contents\n- [\ud83d\udcd2 Table of Contents](#-table-of-contents)\n- [\ud83d\udccd Overview](#-overview)\n- [\u2699\ufe0f Features](#-features)\n- [\ud83d\udcc2 Project Structure](#project-structure)\n- [\ud83e\udde9 Modules](#modules)\n- [\ud83d\ude80 Getting Started](#-getting-started)\n- [\ud83d\uddfa Roadmap](#-roadmap)\n- [\ud83e\udd1d Contributing](#-contributing)\n- [\ud83d\udcc4 License](#-license)\n- [\ud83d\udc4f Acknowledgments](#-acknowledgments)\n\n---\n\n\n## \ud83d\udccd Overview\n\nThe s5commander project is a Python library that allows users to interact with S3 buckets using the awesome s5cmd command-line tool.\nTo quote its description, [`s5cmd`](https://github.com/peak/s5cmd) is a very fast S3 and local filesystem execution tool.\nIt comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files.\n\n> In short, s5cmd offers a very fast speed. For uploads, s5cmd is 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively.\n\nThe goal of this library is to offer a wrapper around this software.\nIt offers functionality for performing various operations like listing, copying, and removing files, creating and removing buckets, checking disk usage, reading files, and utilizing data pipes.\n\nThis project simplifies management and manipulation of S3 data using `s5cmd` inside a Python project, making it easier for users to work with S3 buckets and improve their productivity.\n\n---\n\n## \u2699\ufe0f Features\n\n| Feature | Description |\n| ---------------------- | ------------------------------------- |\n| **\u2699\ufe0f Architecture** | The codebase follows a modular and object-oriented design. It provides a wrapper for the s5cmd command-line tool and uses classes and functions to handle different actions related to S3 bucket management and file operations. |\n| **\ud83d\udcd6 Documentation** | The documentation will come later... |\n| **\ud83d\udd17 Dependencies** | The codebase depends on the s5cmd command-line tool, which is an external dependency for interacting with the AWS S3 service. It relies on necessary AWS credentials and configuration provided by the user. |\n| **\ud83e\udde9 Modularity** | The codebase is organized into separate modules: `commander.py` contains classes and functions related to S3 bucket operations and file management. It allows components to be reused or extended for different S3 operations. |\n| **\u2714\ufe0f Testing** | Tests will also come later :) |\n| **\u26a1\ufe0f Performance** | The performance of the codebase is directly dependent on the underlying s5cmd tool and AWS S3 service. As the codebase primarily acts as a wrapper, its own performance impact is limited. |\n| **\ud83d\udd10 Security** | The codebase relies on AWS credentials for accessing and interacting with S3 buckets, ensuring proper authentication and authorization. Proper handling and protection of these credentials is crucial for maintaining security.|\n| **\ud83d\udd00 Version Control** | The codebase is hosted on GitHub and follows a standard Git version control workflow. Commits are tracked, and the repository can be cloned, branched, and versions can be audited using Git tools and techniques. |\n| **\ud83d\udd0c Integrations** | The codebase is designed to interact with cloud providers using an S3 protocol, including other providers than AWS services. |\n\n---\n\n\n## \ud83d\udcc2 Project Structure\n\n\n\n\n---\n\n## \ud83e\udde9 Modules\n\n<details closed><summary>S5commander</summary>\n\n| File | Summary |\n| --- | --- |\n| [commander.py](https://github.com/clementpoiret/s5commander/blob/main/s5commander/commander.py) | The code provides a wrapper for the s5cmd command-line tool, allowing users to interact with an S3 bucket. It supports functionalities like listing files, copying files, removing files, moving files, creating/removing buckets, selecting buckets, checking disk usage, reading files, and using data pipe. |\n\n</details>\n\n---\n\n## \ud83d\ude80 Getting Started\n\n### \u2714\ufe0f Prerequisites\n\nBefore you begin, ensure that you have the following prerequisites installed:\n> - `s5cmd`\n\n### \ud83d\udce6 Installation\n\n1. Clone the s5commander repository:\n```sh\ngit clone https://github.com/clementpoiret/s5commander\n```\n\n2. Change to the project directory:\n```sh\ncd s5commander\n```\n\n3. Install the dependencies:\n```sh\npip install -r requirements.txt\n```\n\n### \ud83c\udfae Using s5commander\n\n```python\n```\n\n### \ud83e\uddea Running Tests\n```sh\npytest\n```\n\n---\n\n\n## \ud83d\uddfa Roadmap\n\n> - [X] `list buckets and objects`\n> - [X] `download method`\n> - [X] `upload method`\n> - [ ] `copy objects`\n> - [ ] `remove objects`\n> - [ ] `move/rename objects`\n> - [ ] `make bucket`\n> - [ ] `remove bucket`\n> - [ ] `run SQL queries on objects`\n> - [ ] `show object size usage`\n> - [ ] `print remote object content`\n> - [ ] `stream to remote from stdin`\n> - [ ] `run commands in batch`\n> - [ ] `sync objects`\n\n---\n\n## \ud83e\udd1d Contributing\n\nContributions are always welcome! Please follow these steps:\n1. Fork the project repository. This creates a copy of the project on your account that you can modify without affecting the original project.\n2. Clone the forked repository to your local machine using a Git client like Git or GitHub Desktop.\n3. Create a new branch with a descriptive name (e.g., `new-feature-branch` or `bugfix-issue-123`).\n```sh\ngit checkout -b new-feature-branch\n```\n4. Make changes to the project's codebase.\n5. Commit your changes to your local branch with a clear commit message that explains the changes you've made.\n```sh\ngit commit -m 'Implemented new feature.'\n```\n6. Push your changes to your forked repository on GitHub using the following command\n```sh\ngit push origin new-feature-branch\n```\n7. Create a new pull request to the original project repository. In the pull request, describe the changes you've made and why they're necessary.\nThe project maintainers will review your changes and provide feedback or merge them into the main branch.\n\n---\n\n## \ud83d\udcc4 License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE.md) file for additional info.\n\n---\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "",
"version": "0.0.1.dev2",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3ae9b13cee73fc90f9909b9168b0e59dcb2592cab2b07280478a2673a0ee05cc",
"md5": "226647880e5b1da18ea4138de6d1fa4c",
"sha256": "303da43882981e0c495d4a9a5da41c213ca9f15496b3ab8ccf33f924dc1026cb"
},
"downloads": -1,
"filename": "s5commander-0.0.1.dev2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "226647880e5b1da18ea4138de6d1fa4c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7,<4.0",
"size": 6709,
"upload_time": "2023-09-08T09:53:50",
"upload_time_iso_8601": "2023-09-08T09:53:50.328921Z",
"url": "https://files.pythonhosted.org/packages/3a/e9/b13cee73fc90f9909b9168b0e59dcb2592cab2b07280478a2673a0ee05cc/s5commander-0.0.1.dev2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f729ce15fa38199a100786a2290f8def878c42c5946b1ae6be6b51e99e594875",
"md5": "a3e99ae7fe8e12690fc6f90a5873f158",
"sha256": "4413c3e7f4f7dab8f0d8dab9854010c5a763af8dfbb4bf3b2576215e24c899d8"
},
"downloads": -1,
"filename": "s5commander-0.0.1.dev2.tar.gz",
"has_sig": false,
"md5_digest": "a3e99ae7fe8e12690fc6f90a5873f158",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7,<4.0",
"size": 6035,
"upload_time": "2023-09-08T09:53:52",
"upload_time_iso_8601": "2023-09-08T09:53:52.135459Z",
"url": "https://files.pythonhosted.org/packages/f7/29/ce15fa38199a100786a2290f8def878c42c5946b1ae6be6b51e99e594875/s5commander-0.0.1.dev2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-08 09:53:52",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "s5commander"
}
