# wdg-file-storage
`wdg-file-storage` is a Python package that simplifies interactions with S3-compatible storage systems. It provides a reusable and extendable client for operations such as file upload, download, and presigned URL generation. Designed with scalability and testability in mind, this package is ideal for multi-account and multi-region setups.
---
## Features
- **Upload and Download Files**: Seamlessly manage file transfers to and from S3 buckets.
- **Generate Presigned URLs**: Create time-limited URLs for secure file access.
- **Lazy Initialization**: Efficient resource management by initializing the S3 client only when needed.
- **Custom Session Management**: Support for multi-account and multi-region setups with custom `boto3.Session` instances.
- **Thread-Safe Singleton Design**: Ensures a single instance for consistent client usage.
---
## Utils and Helper Function
- **save_files_meta_data** Save files meta data with dynamic model
## Requirements
- Python 3.8+
- `boto3`
- Django (to access `settings` module)
---
## Installation
Install the package using `pip`:
```bash
pip install wdg-file-storage
```
---
## Configuration
Add the following required settings in your Django project's `settings.py`:
```python
S3_ENDPOINT_URL = "your-s3-endpoint-url"
S3_ACCESS_KEY_ID = "your-access-key-id"
S3_SECRET_ACCESS_KEY = "your-secret-access-key"
```
---
## Usage
### Default Client
#### Upload a File
```python
from wdg_file_storage import S3Client
s3_client = S3Client()
s3_client.upload_file("my_bucket", "example.txt", b"File content here")
```
#### Generate a Presigned URL
```python
url = s3_client.generate_presigned_url(
bucket_name="my_bucket",
key="example.txt",
method="get_object",
expiry=3600
)
print(f"Presigned URL: {url}")
```
#### Download a File
```python
file_data = s3_client.download_file("my_bucket", "example.txt")
print(file_data.decode())
```
### Custom Session
#### Use a Custom boto3 Session
```python
from boto3.session import Session
from wdg_file_storage import S3Client
custom_session = Session(
aws_access_key_id="custom_access_key",
aws_secret_access_key="custom_secret_key",
region_name="custom_region"
)
s3_client = S3Client(session=custom_session)
s3_client.upload_file("custom_bucket", "example.txt", b"File content here")
```
### List Files in a Bucket
```python
files = s3_client.list_files("my_bucket")
print(files)
```
---
## Development
### Clone the Repository
```bash
git clone https://github.com/yourusername/wdg-file-storage.git
cd wdg-file-storage
```
### Install Dependencies
```bash
pip install -r requirements.txt
```
### Run Tests
To ensure everything is working as expected:
```bash
pytest
```
---
## Publishing to PyPI
1. Update `setup.py` with the correct package information.
2. Build the distribution:
```bash
python setup.py sdist bdist_wheel
```
3. Upload to PyPI using `twine`:
```bash
twine upload dist/*
```
---
## License
This project is licensed under the MIT License. See the `LICENSE` file for more details.
---
## Contributing
Contributions are welcome! Please open an issue or submit a pull request to contribute to this project.
---
## Contact
For issues or inquiries, please contact [your-email@example.com].
Raw data
{
"_id": null,
"home_page": "https://github.com/devit-chea/drf-file-storage",
"name": "wdg-file-storage",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "storage service s3",
"author": "DC CD",
"author_email": "you@dccd.com",
"download_url": "https://files.pythonhosted.org/packages/08/5b/1f4dc84fa2ca216866b2ecb5198ab6b6114955d4c0fe17bc76661999c229/wdg-file-storage-0.1.4.tar.gz",
"platform": null,
"description": "# wdg-file-storage\r\n\r\n`wdg-file-storage` is a Python package that simplifies interactions with S3-compatible storage systems. It provides a reusable and extendable client for operations such as file upload, download, and presigned URL generation. Designed with scalability and testability in mind, this package is ideal for multi-account and multi-region setups.\r\n\r\n---\r\n\r\n## Features\r\n\r\n- **Upload and Download Files**: Seamlessly manage file transfers to and from S3 buckets.\r\n- **Generate Presigned URLs**: Create time-limited URLs for secure file access.\r\n- **Lazy Initialization**: Efficient resource management by initializing the S3 client only when needed.\r\n- **Custom Session Management**: Support for multi-account and multi-region setups with custom `boto3.Session` instances.\r\n- **Thread-Safe Singleton Design**: Ensures a single instance for consistent client usage.\r\n\r\n---\r\n\r\n## Utils and Helper Function\r\n\r\n- **save_files_meta_data** Save files meta data with dynamic model\r\n\r\n## Requirements\r\n\r\n- Python 3.8+\r\n- `boto3`\r\n- Django (to access `settings` module)\r\n\r\n---\r\n\r\n## Installation\r\n\r\nInstall the package using `pip`:\r\n\r\n```bash\r\npip install wdg-file-storage\r\n```\r\n\r\n---\r\n\r\n## Configuration\r\n\r\nAdd the following required settings in your Django project's `settings.py`:\r\n\r\n```python\r\nS3_ENDPOINT_URL = \"your-s3-endpoint-url\"\r\nS3_ACCESS_KEY_ID = \"your-access-key-id\"\r\nS3_SECRET_ACCESS_KEY = \"your-secret-access-key\"\r\n```\r\n\r\n---\r\n\r\n## Usage\r\n\r\n### Default Client\r\n\r\n#### Upload a File\r\n```python\r\nfrom wdg_file_storage import S3Client\r\n\r\ns3_client = S3Client()\r\ns3_client.upload_file(\"my_bucket\", \"example.txt\", b\"File content here\")\r\n```\r\n\r\n#### Generate a Presigned URL\r\n```python\r\nurl = s3_client.generate_presigned_url(\r\n bucket_name=\"my_bucket\",\r\n key=\"example.txt\",\r\n method=\"get_object\",\r\n expiry=3600\r\n)\r\nprint(f\"Presigned URL: {url}\")\r\n```\r\n\r\n#### Download a File\r\n```python\r\nfile_data = s3_client.download_file(\"my_bucket\", \"example.txt\")\r\nprint(file_data.decode())\r\n```\r\n\r\n### Custom Session\r\n\r\n#### Use a Custom boto3 Session\r\n```python\r\nfrom boto3.session import Session\r\nfrom wdg_file_storage import S3Client\r\n\r\ncustom_session = Session(\r\n aws_access_key_id=\"custom_access_key\",\r\n aws_secret_access_key=\"custom_secret_key\",\r\n region_name=\"custom_region\"\r\n)\r\n\r\ns3_client = S3Client(session=custom_session)\r\ns3_client.upload_file(\"custom_bucket\", \"example.txt\", b\"File content here\")\r\n```\r\n\r\n### List Files in a Bucket\r\n```python\r\nfiles = s3_client.list_files(\"my_bucket\")\r\nprint(files)\r\n```\r\n\r\n---\r\n\r\n## Development\r\n\r\n### Clone the Repository\r\n\r\n```bash\r\ngit clone https://github.com/yourusername/wdg-file-storage.git\r\ncd wdg-file-storage\r\n```\r\n\r\n### Install Dependencies\r\n\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\n### Run Tests\r\n\r\nTo ensure everything is working as expected:\r\n\r\n```bash\r\npytest\r\n```\r\n\r\n---\r\n\r\n## Publishing to PyPI\r\n\r\n1. Update `setup.py` with the correct package information.\r\n2. Build the distribution:\r\n\r\n```bash\r\npython setup.py sdist bdist_wheel\r\n```\r\n\r\n3. Upload to PyPI using `twine`:\r\n\r\n```bash\r\ntwine upload dist/*\r\n```\r\n\r\n---\r\n\r\n## License\r\n\r\nThis project is licensed under the MIT License. See the `LICENSE` file for more details.\r\n\r\n---\r\n\r\n## Contributing\r\n\r\nContributions are welcome! Please open an issue or submit a pull request to contribute to this project.\r\n\r\n---\r\n\r\n## Contact\r\n\r\nFor issues or inquiries, please contact [your-email@example.com].\r\n\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Support for s3 storage backends in Django",
"version": "0.1.4",
"project_urls": {
"Homepage": "https://github.com/devit-chea/drf-file-storage"
},
"split_keywords": [
"storage",
"service",
"s3"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "8b4ea3fd45c7b607f96d9a0569f39f42377ccb2a526580e6b0b4fc2d162690ac",
"md5": "62ee0ba3360dd192a0532136842734a9",
"sha256": "171fb59fd9dab6acf00ddd5c8ed5e1ff3161fb9225296e264814cdab644834c6"
},
"downloads": -1,
"filename": "wdg_file_storage-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "62ee0ba3360dd192a0532136842734a9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 17813,
"upload_time": "2025-02-20T08:34:43",
"upload_time_iso_8601": "2025-02-20T08:34:43.142915Z",
"url": "https://files.pythonhosted.org/packages/8b/4e/a3fd45c7b607f96d9a0569f39f42377ccb2a526580e6b0b4fc2d162690ac/wdg_file_storage-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "085b1f4dc84fa2ca216866b2ecb5198ab6b6114955d4c0fe17bc76661999c229",
"md5": "8dd7a2108bbcea44fd4ed1da4bae8790",
"sha256": "2355192b70eda7b4ef24442b9c95a8f5d73cc155d6534a511d8ac3db76996e39"
},
"downloads": -1,
"filename": "wdg-file-storage-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "8dd7a2108bbcea44fd4ed1da4bae8790",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 13625,
"upload_time": "2025-02-20T08:34:45",
"upload_time_iso_8601": "2025-02-20T08:34:45.301600Z",
"url": "https://files.pythonhosted.org/packages/08/5b/1f4dc84fa2ca216866b2ecb5198ab6b6114955d4c0fe17bc76661999c229/wdg-file-storage-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-20 08:34:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "devit-chea",
"github_project": "drf-file-storage",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "wdg-file-storage"
}
JSON
