# AtomicExecutionControl
[](https://pypi.org/project/atomic-execution-control/)
[](https://pepy.tech/badge/atomic-execution-control)
[](https://github.com/gouline/dbt-metabase/blob/master/LICENSE)
`AtomicExecutionControl` is a Python library crafted to address the complexities of ensuring atomic operations across distributed applications, particularly those deployed on AWS services like Lambda, Fargate, and EC2. By leveraging DynamoDB, this library offers a robust mechanism to prevent concurrent executions, thus mitigating race conditions and duplicate processing risks. Whether you're handling event-driven workflows, orchestrating microservices, or ensuring data integrity across distributed systems, `AtomicExecutionControl` simplifies state management and execution coordination, making your applications more reliable, efficient and most important 'atomic'.
## Features
- **Atomic Execution**: Ensures that only one instance of a the function processes a specific task at a time.
- **Status Management**: Tracks the execution status of tasks in DynamoDB, marking them as in progress or finished.
- **Timeout Handling**: Supports time-based expiry for task locks, making it resilient to failures or stalls in execution.
- **Easy Integration**: Designed to be easily integrated into existing AWS Lambda, Fargate or EC2 functions with minimal configuration.
## Quick Start
Get up and running with `AtomicExecutionControl` in just a few steps:
1. Install the package:
```bash
pip install atomic_execution_control
```
2. Set up a DynamoDB table in AWS with a primary key.
3. Use the following snippet to ensure atomic execution in your project:
```python
from atomic_execution_control import AtomicExecutionControl
# Initialize with your DynamoDB table details
aec = AtomicExecutionControl(table_name="YourTable", primary_key="YourKey")
# Start controlling execution atomically
```
## Prerequisites
A DynamoDB table set up for tracking execution statuses.
## Configuration
`AtomicExecutionControl` can be configured with the following parameters during initialization:
- **`table_name`** (required): The name of the DynamoDB table used for tracking execution.
- **`primary_key`** (required): The primary key attribute name in your DynamoDB table.
- **`region_name`** (optional, default=`"eu-west-1"`): The AWS region where your DynamoDB table is located.
- **`endpoint_url`** (optional): Custom endpoint URL, useful for local testing with DynamoDB Local.
### Additional method parameters:
- `delete_items_finished_or_old` method:
- **item_execution_valid_for** (optional, default=20): Time in minutes to consider an execution old enough to be deleted.
- `wait_other_instances_finish` method:
- **timeout** (optional): Maximum time in seconds to wait for the 'finished' status.
- **time_to_retry** (optional): Time in seconds to wait before trying to validate the status again.
## Examples
### Preventing Duplicate Processing
Below is a quick example to help you get started:
```python
from atomic_execution_control import AtomicExecutionControl
import os
# DynamoDB table and primary key
TABLE_NAME = 'YourDynamoDBTableName'
PRIMARY_KEY = 'YourPrimaryKey'
# Test also in your local environment
# verify the endpoint_url, this url "http://localhost:8000" is also valid.
endpoint_url = "http://docker.for.mac.localhost:8000/" if os.environ.get("AWS_SAM_LOCAL") else None
# Initialize AtomicExecutionControl
aec = AtomicExecutionControl(
table_name=TABLE_NAME,
primary_key=PRIMARY_KEY,
region_name="eu-west-1",
endpoint_url=endpoint_url
)
# In case you already log into an specific AWS account via AWS Single Sign-On (SSO)
# you can send the profile_name parameter as follow.
profile_name = 'AwsProfileName'
aec = AtomicExecutionControl(
table_name=TABLE_NAME,
primary_key=PRIMARY_KEY,
region_name="eu-west-1",
endpoint_url=endpoint_url,
profile_name=profile_name
)
# Assume you have an event-driven architecture where multiple events
# could trigger the same task
date_to_run = '2024-01-01'
# delete items in Dynamo for old executions
aec.delete_items_finished_or_old(keys=[date_to_run], item_execution_valid_for=20)
# write in Dynamo the date_to_run and block other executions to the same key
result = aec.write_atomically_a_key(key=date_to_run)
if result:
# Process the event
print("Lock acquired, proceeding with task.")
## run_your_etl(date_to_run)
# Remember to update the status to 'finished' after completing your task
aec.update_status(key=date_to_run)
else:
# If lock couldn't be acquired, another instance is already processing the task
print("Task already in progress by another instance.")
# wait until other instances with the same key finish
aec.wait_other_instances_finish(keys=[date_to_run])
```
## Support and Contact
Encountering issues or have questions about integrating `AtomicExecutionControl`? We're here to help!
- **Ask a Question or Report an Issue**: For technical issues or questions, please [open an issue](https://github.com/jdaarevalo/atomic_execution_control/issues) on our GitHub repository.
- **Email Us**: For direct support or inquiries, email us at jdaarevalo@gmail.com - we aim to respond as quickly as possible.
- **Suggestions**: Your feedback and suggestions are invaluable in making `AtomicExecutionControl` better. Don't hesitate to reach out with ideas for improvements or new features!
## Contributing
Contributions are welcome! Feel free to open an issue or submit a pull request on GitHub.
## License
This project is licensed under the MIT License - see the LICENSE file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/jdaarevalo/atomic_execution_control",
"name": "atomic-execution-control",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "atomic execution, DynamoDB, AWS Lambda, AWS Fargate, EC2, distributed systems, concurrency control, race condition prevention, cloud computing, AWS services, state management",
"author": "Jose David Arevalo",
"author_email": "jdaarevalo@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/35/96/05539f8ebb34aed350a10d7959f76d24fabd05009632c67a472789ae3b87/atomic_execution_control-0.1.5.tar.gz",
"platform": null,
"description": "# AtomicExecutionControl\n\n[](https://pypi.org/project/atomic-execution-control/)\n[](https://pepy.tech/badge/atomic-execution-control)\n[](https://github.com/gouline/dbt-metabase/blob/master/LICENSE)\n\n\n`AtomicExecutionControl` is a Python library crafted to address the complexities of ensuring atomic operations across distributed applications, particularly those deployed on AWS services like Lambda, Fargate, and EC2. By leveraging DynamoDB, this library offers a robust mechanism to prevent concurrent executions, thus mitigating race conditions and duplicate processing risks. Whether you're handling event-driven workflows, orchestrating microservices, or ensuring data integrity across distributed systems, `AtomicExecutionControl` simplifies state management and execution coordination, making your applications more reliable, efficient and most important 'atomic'.\n\n\n## Features\n\n- **Atomic Execution**: Ensures that only one instance of a the function processes a specific task at a time.\n- **Status Management**: Tracks the execution status of tasks in DynamoDB, marking them as in progress or finished.\n- **Timeout Handling**: Supports time-based expiry for task locks, making it resilient to failures or stalls in execution.\n- **Easy Integration**: Designed to be easily integrated into existing AWS Lambda, Fargate or EC2 functions with minimal configuration.\n\n\n## Quick Start\n\nGet up and running with `AtomicExecutionControl` in just a few steps:\n\n1. Install the package:\n\n```bash\npip install atomic_execution_control\n```\n\n2. Set up a DynamoDB table in AWS with a primary key.\n\n\n3. Use the following snippet to ensure atomic execution in your project:\n\n```python\n\nfrom atomic_execution_control import AtomicExecutionControl\n\n# Initialize with your DynamoDB table details\naec = AtomicExecutionControl(table_name=\"YourTable\", primary_key=\"YourKey\")\n\n# Start controlling execution atomically\n```\n\n\n## Prerequisites\n\nA DynamoDB table set up for tracking execution statuses.\n\n\n## Configuration\n\n`AtomicExecutionControl` can be configured with the following parameters during initialization:\n\n- **`table_name`** (required): The name of the DynamoDB table used for tracking execution.\n- **`primary_key`** (required): The primary key attribute name in your DynamoDB table.\n- **`region_name`** (optional, default=`\"eu-west-1\"`): The AWS region where your DynamoDB table is located.\n- **`endpoint_url`** (optional): Custom endpoint URL, useful for local testing with DynamoDB Local.\n\n\n### Additional method parameters:\n\n\n- `delete_items_finished_or_old` method:\n - **item_execution_valid_for** (optional, default=20): Time in minutes to consider an execution old enough to be deleted.\n- `wait_other_instances_finish` method:\n - **timeout** (optional): Maximum time in seconds to wait for the 'finished' status.\n - **time_to_retry** (optional): Time in seconds to wait before trying to validate the status again.\n\n\n## Examples\n\n### Preventing Duplicate Processing\n\nBelow is a quick example to help you get started:\n\n```python\nfrom atomic_execution_control import AtomicExecutionControl\nimport os\n\n# DynamoDB table and primary key\nTABLE_NAME = 'YourDynamoDBTableName'\nPRIMARY_KEY = 'YourPrimaryKey'\n\n# Test also in your local environment \n# verify the endpoint_url, this url \"http://localhost:8000\" is also valid.\nendpoint_url = \"http://docker.for.mac.localhost:8000/\" if os.environ.get(\"AWS_SAM_LOCAL\") else None\n \n# Initialize AtomicExecutionControl\naec = AtomicExecutionControl(\n table_name=TABLE_NAME,\n primary_key=PRIMARY_KEY,\n region_name=\"eu-west-1\",\n endpoint_url=endpoint_url\n)\n\n# In case you already log into an specific AWS account via AWS Single Sign-On (SSO)\n# you can send the profile_name parameter as follow.\nprofile_name = 'AwsProfileName'\naec = AtomicExecutionControl(\n table_name=TABLE_NAME,\n primary_key=PRIMARY_KEY,\n region_name=\"eu-west-1\",\n endpoint_url=endpoint_url,\n profile_name=profile_name\n)\n\n# Assume you have an event-driven architecture where multiple events \n# could trigger the same task\ndate_to_run = '2024-01-01'\n\n\n# delete items in Dynamo for old executions\naec.delete_items_finished_or_old(keys=[date_to_run], item_execution_valid_for=20)\n\n# write in Dynamo the date_to_run and block other executions to the same key\nresult = aec.write_atomically_a_key(key=date_to_run)\n\nif result:\n # Process the event\n \n print(\"Lock acquired, proceeding with task.\")\n \n ## run_your_etl(date_to_run)\n \n # Remember to update the status to 'finished' after completing your task\n aec.update_status(key=date_to_run)\nelse:\n # If lock couldn't be acquired, another instance is already processing the task\n print(\"Task already in progress by another instance.\")\n # wait until other instances with the same key finish\n aec.wait_other_instances_finish(keys=[date_to_run])\n\n```\n\n## Support and Contact\n\nEncountering issues or have questions about integrating `AtomicExecutionControl`? We're here to help!\n\n- **Ask a Question or Report an Issue**: For technical issues or questions, please [open an issue](https://github.com/jdaarevalo/atomic_execution_control/issues) on our GitHub repository.\n- **Email Us**: For direct support or inquiries, email us at jdaarevalo@gmail.com - we aim to respond as quickly as possible.\n- **Suggestions**: Your feedback and suggestions are invaluable in making `AtomicExecutionControl` better. Don't hesitate to reach out with ideas for improvements or new features!\n\n\n## Contributing\n\nContributions are welcome! Feel free to open an issue or submit a pull request on GitHub.\n\n## License\nThis project is licensed under the MIT License - see the LICENSE file for details.\n",
"bugtrack_url": null,
"license": null,
"summary": "Ensures atomic executions across AWS services using DynamoDB to prevent race conditions in distributed applications.",
"version": "0.1.5",
"project_urls": {
"Homepage": "https://github.com/jdaarevalo/atomic_execution_control"
},
"split_keywords": [
"atomic execution",
" dynamodb",
" aws lambda",
" aws fargate",
" ec2",
" distributed systems",
" concurrency control",
" race condition prevention",
" cloud computing",
" aws services",
" state management"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3987742e0742c3bcff9d32dbed7d72165a12b767631921b2bfe099354a91f1d0",
"md5": "6fc209d2820a77be2ee488829b9d2968",
"sha256": "fa7db146e4a5584bf7dbcbd320f02a50dd969ac5e37c82322c41fa61bb2b4368"
},
"downloads": -1,
"filename": "atomic_execution_control-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6fc209d2820a77be2ee488829b9d2968",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 7886,
"upload_time": "2024-04-02T13:11:06",
"upload_time_iso_8601": "2024-04-02T13:11:06.724890Z",
"url": "https://files.pythonhosted.org/packages/39/87/742e0742c3bcff9d32dbed7d72165a12b767631921b2bfe099354a91f1d0/atomic_execution_control-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "359605539f8ebb34aed350a10d7959f76d24fabd05009632c67a472789ae3b87",
"md5": "28f32c7b8d03e861f156bbdcc0d63266",
"sha256": "3536369082bd136da89e44773eee6cc8aa392369c39d6cf1867761b3b7b96b81"
},
"downloads": -1,
"filename": "atomic_execution_control-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "28f32c7b8d03e861f156bbdcc0d63266",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7641,
"upload_time": "2024-04-02T13:11:13",
"upload_time_iso_8601": "2024-04-02T13:11:13.003714Z",
"url": "https://files.pythonhosted.org/packages/35/96/05539f8ebb34aed350a10d7959f76d24fabd05009632c67a472789ae3b87/atomic_execution_control-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-02 13:11:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jdaarevalo",
"github_project": "atomic_execution_control",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "boto3",
"specs": []
},
{
"name": "botocore",
"specs": []
}
],
"lcname": "atomic-execution-control"
}
JSON
