# LambdaDynamoLock
`LambdaDynamoLock` is a Python library designed to ensure atomic executions within AWS Lambda functions by leveraging DynamoDB. It provides a mechanism to prevent concurrent executions of Lambda functions that could lead to race conditions or duplicate processing. This library is particularly useful for distributed applications where Lambda functions are triggered in response to events and require coordination or state management.
## Features
- **Atomic Execution**: Ensures that only one instance of a Lambda function processes a specific task at a time.
- **Status Management**: Tracks the execution status of tasks in DynamoDB, marking them as in progress or finished.
- **Timeout Handling**: Supports time-based expiry for task locks, making it resilient to failures or stalls in execution.
- **Easy Integration**: Designed to be easily integrated into existing AWS Lambda functions with minimal configuration.
## Installation
Install `LambdaDynamoLock` using pip:
```bash
pip install lambda_dynamo_lock
```
## Prerequisites
An AWS account and AWS CLI configured with DynamoDB access.
A DynamoDB table set up for tracking execution statuses.
# Usage
Below is a quick example to help you get started:
```python
from lambda_dynamo_lock import LambdaDynamoLock
import os
# DynamoDB table and primary key
TABLE_NAME = 'YourDynamoDBTableName'
PRIMARY_KEY = 'YourPrimaryKey'
# Initialize LambdaDynamoLock
ldl = LambdaDynamoLock(
table_name=TABLE_NAME,
primary_key=PRIMARY_KEY,
region_name="eu-west-1",
endpoint_url="http://docker.for.mac.localhost:8000/" if os.environ.get("AWS_SAM_LOCAL") else None
)
# Attempt to write a lock key for today's date
date_to_run = '2023-01-01'
# delete items in Dynamo for old executions
ldl.delete_items_finished_or_old(keys=[date_to_run], item_execution_valid_for=20)
# write in Dynamo the date_to_run and block other executions to the same key
result = ldl.write_atomically_a_key(key=date_to_run)
if result:
# If lock is acquired, perform your task
print("Lock acquired, proceeding with task.")
## run_your_etl(date_to_run)
# Remember to update the status to 'finished' after completing your task
ldl.update_status(key=date_to_run)
else:
# If lock couldn't be acquired, another instance is already processing the task
print("Task already in progress by another instance.")
# wait until other instances with the same key finish
ldl.wait_other_instances_finish(keys=[date_to_run])
```
## Configuration
LambdaDynamoLock can be configured with several parameters at initialization to fit your needs:
**table_name:** Name of the DynamoDB table used for tracking execution.
**primary_key:** The primary key attribute name in your DynamoDB table.
**region_name:** AWS region where your DynamoDB table is located.
**endpoint_url:** Custom endpoint URL, useful for local testing with DynamoDB Local.
### Additional method parameters:
- `delete_items_finished_or_old` method:
- **item_execution_valid_for** (optional, default=20): Time in minutes to consider an execution old enough to be deleted.
- `wait_other_instances_finish` method:
- **timeout** (optional): Maximum time in seconds to wait for the 'finished' status.
- **time_to_retry** (optional): Time in seconds to wait before trying to validate the status again.
## Support and Contact
Having trouble with `LambdaDynamoLock`? Check out our [GitHub issues](https://github.com/jdaarevalo/lambda_dynamo_lock/issues) or contact support and we’ll help you sort it out. Feel free to wirte a message to jdaarevalo@gmail.com
## Contributing
Contributions are welcome! Feel free to open an issue or submit a pull request on GitHub.
## License
This project is licensed under the MIT License - see the LICENSE file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/jdaarevalo/lambda_dynamo_lock",
"name": "lambda-dynamo-lock",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Jose David Arevalo",
"author_email": "jdaarevalo@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/4d/e2/e7be6aec6e7096291e6ca9fb8f8943a2b88a2c460a7ecf21973cf6f2a0c2/lambda_dynamo_lock-0.1.3.tar.gz",
"platform": null,
"description": "# LambdaDynamoLock\n\n`LambdaDynamoLock` is a Python library designed to ensure atomic executions within AWS Lambda functions by leveraging DynamoDB. It provides a mechanism to prevent concurrent executions of Lambda functions that could lead to race conditions or duplicate processing. This library is particularly useful for distributed applications where Lambda functions are triggered in response to events and require coordination or state management.\n\n## Features\n\n- **Atomic Execution**: Ensures that only one instance of a Lambda function processes a specific task at a time.\n- **Status Management**: Tracks the execution status of tasks in DynamoDB, marking them as in progress or finished.\n- **Timeout Handling**: Supports time-based expiry for task locks, making it resilient to failures or stalls in execution.\n- **Easy Integration**: Designed to be easily integrated into existing AWS Lambda functions with minimal configuration.\n\n## Installation\n\nInstall `LambdaDynamoLock` using pip:\n\n```bash\npip install lambda_dynamo_lock\n```\n\n## Prerequisites\nAn AWS account and AWS CLI configured with DynamoDB access.\nA DynamoDB table set up for tracking execution statuses.\n\n# Usage\nBelow is a quick example to help you get started:\n\n```python\nfrom lambda_dynamo_lock import LambdaDynamoLock\nimport os\n\n# DynamoDB table and primary key\nTABLE_NAME = 'YourDynamoDBTableName'\nPRIMARY_KEY = 'YourPrimaryKey'\n\n# Initialize LambdaDynamoLock\nldl = LambdaDynamoLock(\n table_name=TABLE_NAME,\n primary_key=PRIMARY_KEY,\n region_name=\"eu-west-1\",\n endpoint_url=\"http://docker.for.mac.localhost:8000/\" if os.environ.get(\"AWS_SAM_LOCAL\") else None\n)\n\n# Attempt to write a lock key for today's date\ndate_to_run = '2023-01-01'\n\n# delete items in Dynamo for old executions\nldl.delete_items_finished_or_old(keys=[date_to_run], item_execution_valid_for=20)\n\n# write in Dynamo the date_to_run and block other executions to the same key\nresult = ldl.write_atomically_a_key(key=date_to_run)\n\nif result:\n # If lock is acquired, perform your task\n print(\"Lock acquired, proceeding with task.\")\n ## run_your_etl(date_to_run)\n # Remember to update the status to 'finished' after completing your task\n ldl.update_status(key=date_to_run)\nelse:\n # If lock couldn't be acquired, another instance is already processing the task\n print(\"Task already in progress by another instance.\")\n # wait until other instances with the same key finish\n ldl.wait_other_instances_finish(keys=[date_to_run])\n\n```\n\n## Configuration\n\nLambdaDynamoLock can be configured with several parameters at initialization to fit your needs:\n\n**table_name:** Name of the DynamoDB table used for tracking execution.\n**primary_key:** The primary key attribute name in your DynamoDB table.\n**region_name:** AWS region where your DynamoDB table is located.\n**endpoint_url:** Custom endpoint URL, useful for local testing with DynamoDB Local.\n\n\n### Additional method parameters:\n\n\n- `delete_items_finished_or_old` method:\n - **item_execution_valid_for** (optional, default=20): Time in minutes to consider an execution old enough to be deleted.\n- `wait_other_instances_finish` method:\n - **timeout** (optional): Maximum time in seconds to wait for the 'finished' status.\n - **time_to_retry** (optional): Time in seconds to wait before trying to validate the status again.\n\n\n## Support and Contact\n\nHaving trouble with `LambdaDynamoLock`? Check out our [GitHub issues](https://github.com/jdaarevalo/lambda_dynamo_lock/issues) or contact support and we\u2019ll help you sort it out. Feel free to wirte a message to jdaarevalo@gmail.com\n\n\n## Contributing\n\nContributions are welcome! Feel free to open an issue or submit a pull request on GitHub.\n\n## License\nThis project is licensed under the MIT License - see the LICENSE file for details.\n",
"bugtrack_url": null,
"license": "",
"summary": "A utility for atomic DynamoDB operations in AWS Lambda functions",
"version": "0.1.3",
"project_urls": {
"Homepage": "https://github.com/jdaarevalo/lambda_dynamo_lock"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ae127725d46736e0101b632ab6d667209e21c4fa311f6460071eb92229847f5c",
"md5": "a11836a1458086678598f609fab2d5ac",
"sha256": "f249ecbf6716a0c704c7415827f289f361fefa1d54074a9f3e9e2d56b506c97f"
},
"downloads": -1,
"filename": "lambda_dynamo_lock-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a11836a1458086678598f609fab2d5ac",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 6658,
"upload_time": "2024-03-07T07:45:51",
"upload_time_iso_8601": "2024-03-07T07:45:51.633698Z",
"url": "https://files.pythonhosted.org/packages/ae/12/7725d46736e0101b632ab6d667209e21c4fa311f6460071eb92229847f5c/lambda_dynamo_lock-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4de2e7be6aec6e7096291e6ca9fb8f8943a2b88a2c460a7ecf21973cf6f2a0c2",
"md5": "27ccded131750f2d99cfea5cf29c2e74",
"sha256": "8dd032f992f79ce4f1f0f6df73343c7351b3e5adee55daf97460aac7a39e3e1e"
},
"downloads": -1,
"filename": "lambda_dynamo_lock-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "27ccded131750f2d99cfea5cf29c2e74",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 6152,
"upload_time": "2024-03-07T07:45:53",
"upload_time_iso_8601": "2024-03-07T07:45:53.275099Z",
"url": "https://files.pythonhosted.org/packages/4d/e2/e7be6aec6e7096291e6ca9fb8f8943a2b88a2c460a7ecf21973cf6f2a0c2/lambda_dynamo_lock-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-07 07:45:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jdaarevalo",
"github_project": "lambda_dynamo_lock",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "boto3",
"specs": []
},
{
"name": "botocore",
"specs": []
}
],
"lcname": "lambda-dynamo-lock"
}
JSON
