# S3 Log Handler
A Python logging handler that asynchronously uploads logs to Amazon S3 in batches.
## Features
- Asynchronous upload to S3 using aioboto3
- Batching of log messages for efficient uploading
- Automatic compression of logs using gzip
- Configurable batch size and flush intervals
- Thread-safe operation
- Graceful shutdown handling with proper cleanup
## Installation
```bash
pip install s3-log-handler
```
## Usage
```python
import logging
from s3_log_handler import S3LogHandler
# Configure AWS credentials
client_params = {
"aws_access_key_id": "your_access_key",
"aws_secret_access_key": "your_secret_key",
"region_name": "your_region"
}
# Create and configure the handler
handler = S3LogHandler(
client_params=client_params,
bucket_name="your-bucket-name",
log_prefix="application-logs",
batch_size=100,
flush_interval=300
)
# Add handler to logger
logger = logging.getLogger()
logger.addHandler(handler)
# Log messages will now be uploaded to S3
logger.info("Hello, S3!")
```
## Configuration
- `client_params`: Dictionary of AWS client parameters
- `bucket_name`: Name of the S3 bucket
- `log_prefix`: Prefix for S3 keys (default: "application-logs")
- `batch_size`: Number of logs to batch before upload (default: 100)
- `flush_interval`: Maximum seconds between uploads (default: 300)
## License
MIT License - see LICENSE file for details
Raw data
{
"_id": null,
"home_page": "https://github.com/yourusername/s3-log-handler",
"name": "s3-log-handler",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "logging, s3, aws, handler, async",
"author": "Your Name",
"author_email": "your.email@example.com",
"download_url": "https://files.pythonhosted.org/packages/e5/4c/42a81b2900cac1fc3cf2cf5bfabd7fd5bc6eef46087d7bb0416c53e4ee0f/s3_log_handler-0.1.0.tar.gz",
"platform": null,
"description": "# S3 Log Handler\n\nA Python logging handler that asynchronously uploads logs to Amazon S3 in batches.\n\n## Features\n\n- Asynchronous upload to S3 using aioboto3\n- Batching of log messages for efficient uploading\n- Automatic compression of logs using gzip\n- Configurable batch size and flush intervals\n- Thread-safe operation\n- Graceful shutdown handling with proper cleanup\n\n## Installation\n\n```bash\npip install s3-log-handler\n```\n\n## Usage\n\n```python\nimport logging\nfrom s3_log_handler import S3LogHandler\n\n# Configure AWS credentials\nclient_params = {\n \"aws_access_key_id\": \"your_access_key\",\n \"aws_secret_access_key\": \"your_secret_key\",\n \"region_name\": \"your_region\"\n}\n\n# Create and configure the handler\nhandler = S3LogHandler(\n client_params=client_params,\n bucket_name=\"your-bucket-name\",\n log_prefix=\"application-logs\",\n batch_size=100,\n flush_interval=300\n)\n\n# Add handler to logger\nlogger = logging.getLogger()\nlogger.addHandler(handler)\n\n# Log messages will now be uploaded to S3\nlogger.info(\"Hello, S3!\")\n```\n\n## Configuration\n\n- `client_params`: Dictionary of AWS client parameters\n- `bucket_name`: Name of the S3 bucket\n- `log_prefix`: Prefix for S3 keys (default: \"application-logs\")\n- `batch_size`: Number of logs to batch before upload (default: 100)\n- `flush_interval`: Maximum seconds between uploads (default: 300)\n\n## License\n\nMIT License - see LICENSE file for details\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python logging handler that uploads logs to Amazon S3",
"version": "0.1.0",
"project_urls": {
"Documentation": "https://github.com/yourusername/s3-log-handler#readme",
"Homepage": "https://github.com/yourusername/s3-log-handler",
"Repository": "https://github.com/yourusername/s3-log-handler"
},
"split_keywords": [
"logging",
" s3",
" aws",
" handler",
" async"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c72c4ce330c0e012b6945803924e55b846a860d402679148568f02f98f5c7fdd",
"md5": "630555b21b4b072b35df034a71d3050c",
"sha256": "6b05aff724b9edf70e2e11fd46619f094c8ed5ef27c307bc71481359cd51bfb1"
},
"downloads": -1,
"filename": "s3_log_handler-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "630555b21b4b072b35df034a71d3050c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 5430,
"upload_time": "2024-10-26T09:47:34",
"upload_time_iso_8601": "2024-10-26T09:47:34.443288Z",
"url": "https://files.pythonhosted.org/packages/c7/2c/4ce330c0e012b6945803924e55b846a860d402679148568f02f98f5c7fdd/s3_log_handler-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e54c42a81b2900cac1fc3cf2cf5bfabd7fd5bc6eef46087d7bb0416c53e4ee0f",
"md5": "8fbaa83a7b97e5f0d1eabd8d5fda9861",
"sha256": "b8b69e671981f218b22638a9eb4bd5ad227ba0f994fd567ea17aac09aac9b5fc"
},
"downloads": -1,
"filename": "s3_log_handler-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "8fbaa83a7b97e5f0d1eabd8d5fda9861",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 5178,
"upload_time": "2024-10-26T09:47:36",
"upload_time_iso_8601": "2024-10-26T09:47:36.385381Z",
"url": "https://files.pythonhosted.org/packages/e5/4c/42a81b2900cac1fc3cf2cf5bfabd7fd5bc6eef46087d7bb0416c53e4ee0f/s3_log_handler-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-26 09:47:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "s3-log-handler",
"github_not_found": true,
"lcname": "s3-log-handler"
}
JSON
