# wdg-file-storage
`wdg-file-storage` is a Python package that simplifies interactions with S3-compatible storage systems. It provides a reusable and extendable client for operations such as file upload, download, and presigned URL generation. Designed with scalability and testability in mind, this package is ideal for multi-account and multi-region setups.
---
## Features
- **Upload and Download Files**: Seamlessly manage file transfers to and from S3 buckets.
- **Generate Presigned URLs**: Create time-limited URLs for secure file access.
- **Lazy Initialization**: Efficient resource management by initializing the S3 client only when needed.
- **Custom Session Management**: Support for multi-account and multi-region setups with custom `boto3.Session` instances.
- **Thread-Safe Singleton Design**: Ensures a single instance for consistent client usage.
---
## Utils and Helper Function
- **save_files_meta_data** Save files meta data with dynamic model
## Requirements
- Python 3.8+
- `boto3`
- Django (to access `settings` module)
---
## Installation
Install the package using `pip`:
```bash
pip install wdg-file-storage
```
---
## Configuration
Add the following required settings in your Django project's `settings.py`:
```python
S3_ENDPOINT_URL = "your-s3-endpoint-url"
S3_ACCESS_KEY_ID = "your-access-key-id"
S3_SECRET_ACCESS_KEY = "your-secret-access-key"
```
---
## Usage
### Default Client
#### Upload a File
```python
from wdg_file_storage import S3Client
s3_client = S3Client()
s3_client.upload_file("my_bucket", "example.txt", b"File content here")
```
#### Generate a Presigned URL
```python
url = s3_client.generate_presigned_url(
bucket_name="my_bucket",
key="example.txt",
method="get_object",
expiry=3600
)
print(f"Presigned URL: {url}")
```
#### Download a File
```python
file_data = s3_client.download_file("my_bucket", "example.txt")
print(file_data.decode())
```
### Custom Session
#### Use a Custom boto3 Session
```python
from boto3.session import Session
from wdg_file_storage import S3Client
custom_session = Session(
aws_access_key_id="custom_access_key",
aws_secret_access_key="custom_secret_key",
region_name="custom_region"
)
s3_client = S3Client(session=custom_session)
s3_client.upload_file("custom_bucket", "example.txt", b"File content here")
```
### List Files in a Bucket
```python
files = s3_client.list_files("my_bucket")
print(files)
```
---
## Development
### Clone the Repository
```bash
git clone https://github.com/yourusername/wdg-file-storage.git
cd wdg-file-storage
```
### Install Dependencies
```bash
pip install -r requirements.txt
```
### Run Tests
To ensure everything is working as expected:
```bash
pytest
```
---
## Publishing to PyPI
1. Update `setup.py` with the correct package information.
2. Build the distribution:
```bash
python setup.py sdist bdist_wheel
```
3. Upload to PyPI using `twine`:
```bash
twine upload dist/*
```
---
## License
This project is licensed under the MIT License. See the `LICENSE` file for more details.
---
## Contributing
Contributions are welcome! Please open an issue or submit a pull request to contribute to this project.
---
## Contact
For issues or inquiries, please contact [your-email@example.com].
Raw data
{
"_id": null,
"home_page": "https://github.com/devit-chea/wdg_core_file_storage",
"name": "wdg-core-file-storage",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "storage service s3",
"author": "Devit Chea",
"author_email": "devit.chea1998@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/b1/ab/2e80d4ce0cf49c646c58a9e9879b85c6859c62c5411f26de69573144bec4/wdg_core_file_storage-0.12.0.tar.gz",
"platform": null,
"description": "# wdg-file-storage\n\n`wdg-file-storage` is a Python package that simplifies interactions with S3-compatible storage systems. It provides a reusable and extendable client for operations such as file upload, download, and presigned URL generation. Designed with scalability and testability in mind, this package is ideal for multi-account and multi-region setups.\n\n---\n\n## Features\n\n- **Upload and Download Files**: Seamlessly manage file transfers to and from S3 buckets.\n- **Generate Presigned URLs**: Create time-limited URLs for secure file access.\n- **Lazy Initialization**: Efficient resource management by initializing the S3 client only when needed.\n- **Custom Session Management**: Support for multi-account and multi-region setups with custom `boto3.Session` instances.\n- **Thread-Safe Singleton Design**: Ensures a single instance for consistent client usage.\n\n---\n\n## Utils and Helper Function\n\n- **save_files_meta_data** Save files meta data with dynamic model\n\n## Requirements\n\n- Python 3.8+\n- `boto3`\n- Django (to access `settings` module)\n\n---\n\n## Installation\n\nInstall the package using `pip`:\n\n```bash\npip install wdg-file-storage\n```\n\n---\n\n## Configuration\n\nAdd the following required settings in your Django project's `settings.py`:\n\n```python\nS3_ENDPOINT_URL = \"your-s3-endpoint-url\"\nS3_ACCESS_KEY_ID = \"your-access-key-id\"\nS3_SECRET_ACCESS_KEY = \"your-secret-access-key\"\n```\n\n---\n\n## Usage\n\n### Default Client\n\n#### Upload a File\n```python\nfrom wdg_file_storage import S3Client\n\ns3_client = S3Client()\ns3_client.upload_file(\"my_bucket\", \"example.txt\", b\"File content here\")\n```\n\n#### Generate a Presigned URL\n```python\nurl = s3_client.generate_presigned_url(\n bucket_name=\"my_bucket\",\n key=\"example.txt\",\n method=\"get_object\",\n expiry=3600\n)\nprint(f\"Presigned URL: {url}\")\n```\n\n#### Download a File\n```python\nfile_data = s3_client.download_file(\"my_bucket\", \"example.txt\")\nprint(file_data.decode())\n```\n\n### Custom Session\n\n#### Use a Custom boto3 Session\n```python\nfrom boto3.session import Session\nfrom wdg_file_storage import S3Client\n\ncustom_session = Session(\n aws_access_key_id=\"custom_access_key\",\n aws_secret_access_key=\"custom_secret_key\",\n region_name=\"custom_region\"\n)\n\ns3_client = S3Client(session=custom_session)\ns3_client.upload_file(\"custom_bucket\", \"example.txt\", b\"File content here\")\n```\n\n### List Files in a Bucket\n```python\nfiles = s3_client.list_files(\"my_bucket\")\nprint(files)\n```\n\n---\n\n## Development\n\n### Clone the Repository\n\n```bash\ngit clone https://github.com/yourusername/wdg-file-storage.git\ncd wdg-file-storage\n```\n\n### Install Dependencies\n\n```bash\npip install -r requirements.txt\n```\n\n### Run Tests\n\nTo ensure everything is working as expected:\n\n```bash\npytest\n```\n\n---\n\n## Publishing to PyPI\n\n1. Update `setup.py` with the correct package information.\n2. Build the distribution:\n\n```bash\npython setup.py sdist bdist_wheel\n```\n\n3. Upload to PyPI using `twine`:\n\n```bash\ntwine upload dist/*\n```\n\n---\n\n## License\n\nThis project is licensed under the MIT License. See the `LICENSE` file for more details.\n\n---\n\n## Contributing\n\nContributions are welcome! Please open an issue or submit a pull request to contribute to this project.\n\n---\n\n## Contact\n\nFor issues or inquiries, please contact [your-email@example.com].\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Support for s3 storage backends in Django",
"version": "0.12.0",
"project_urls": {
"Homepage": "https://github.com/devit-chea/wdg_core_file_storage"
},
"split_keywords": [
"storage",
"service",
"s3"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "182ae7def70a42ec4305d7d015ea3e3e78c64a4701dd07d1fee57b2b507f0963",
"md5": "9ed8fec89ff98afa9317eb8cc513f4f9",
"sha256": "b9370b061491bff6d5f44f417339c4c6c86fff387325a0cd3092f2e519f42984"
},
"downloads": -1,
"filename": "wdg_core_file_storage-0.12.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9ed8fec89ff98afa9317eb8cc513f4f9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 29246,
"upload_time": "2025-02-22T06:05:05",
"upload_time_iso_8601": "2025-02-22T06:05:05.007963Z",
"url": "https://files.pythonhosted.org/packages/18/2a/e7def70a42ec4305d7d015ea3e3e78c64a4701dd07d1fee57b2b507f0963/wdg_core_file_storage-0.12.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b1ab2e80d4ce0cf49c646c58a9e9879b85c6859c62c5411f26de69573144bec4",
"md5": "ca796ee6e3fa5aa289a9d5ffef5f4a88",
"sha256": "1f1d51a3635521641d0e2e995861d44550ce3e6f8de7e2d4330bdafb87439976"
},
"downloads": -1,
"filename": "wdg_core_file_storage-0.12.0.tar.gz",
"has_sig": false,
"md5_digest": "ca796ee6e3fa5aa289a9d5ffef5f4a88",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 19673,
"upload_time": "2025-02-22T06:05:06",
"upload_time_iso_8601": "2025-02-22T06:05:06.959662Z",
"url": "https://files.pythonhosted.org/packages/b1/ab/2e80d4ce0cf49c646c58a9e9879b85c6859c62c5411f26de69573144bec4/wdg_core_file_storage-0.12.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-22 06:05:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "devit-chea",
"github_project": "wdg_core_file_storage",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "Django",
"specs": []
},
{
"name": "djangorestframework",
"specs": []
},
{
"name": "boto3",
"specs": []
},
{
"name": "django-storages",
"specs": []
},
{
"name": "setuptools",
"specs": []
},
{
"name": "twine",
"specs": []
}
],
"lcname": "wdg-core-file-storage"
}
JSON
