# NUI Lambda Shared Utilities
[](https://badge.fury.io/py/nui-lambda-shared-utils)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
Enterprise-grade utilities for AWS Lambda functions with Slack, Elasticsearch, and monitoring integrations. This package provides standardized, production-ready patterns for common serverless operations while maintaining flexibility and configurability.
## Key Features
- **AWS Secrets Manager Integration** - Secure credential management with caching
- **Slack Messaging** - Rich formatting, threading, file uploads, and channel management
- **Elasticsearch Operations** - Query builders, index management, and health monitoring
- **Database Connections** - Connection pooling, automatic retries, and transaction management
- **CloudWatch Metrics** - Batched publishing with custom dimensions
- **Error Handling** - Intelligent retry patterns with exponential backoff
- **Timezone Utilities** - Timezone handling and formatting
- **Configurable Defaults** - Environment-aware configuration system
## Quick Start
### Installation
```bash
pip install nui-lambda-shared-utils
# With specific extras for optional dependencies
pip install nui-lambda-shared-utils[all] # All integrations
pip install nui-lambda-shared-utils[slack] # Slack only
pip install nui-lambda-shared-utils[elasticsearch] # Elasticsearch only
pip install nui-lambda-shared-utils[database] # Database only
```
### Basic Configuration
```python
import nui_lambda_shared_utils as nui
# Configure for your environment (optional - uses sensible defaults)
nui.configure(
es_host="your-elasticsearch-host:9200",
es_credentials_secret="your-es-secret-name",
slack_credentials_secret="your-slack-secret-name",
db_credentials_secret="your-database-secret-name"
)
# Or use environment variables:
# ES_HOST, ES_CREDENTIALS_SECRET, SLACK_CREDENTIALS_SECRET, etc.
```
## Usage Examples
### Secrets Management
```python
from nui_lambda_shared_utils import get_secret, get_slack_credentials, get_database_credentials
# Generic secret retrieval
api_keys = get_secret("my-service/api-keys")
# Specialized getters with normalized field names
slack_creds = get_slack_credentials() # Uses configured secret name
db_creds = get_database_credentials() # Standardized: host, port, username, password, database
```
### Slack Integration
```python
from nui_lambda_shared_utils import SlackClient, SlackBlockBuilder
# Initialize client
slack = SlackClient()
# Simple message
response = slack.send_message(
channel='#alerts',
text='Service deployment complete'
)
# Rich formatted message with blocks
builder = SlackBlockBuilder()
blocks = (builder
.add_header("Production Alert", emoji="warning")
.add_section("Error Rate", "15.2% (above 10% threshold)")
.add_section("Time Range", "Last 10 minutes")
.add_divider()
.add_context("Alert ID: PROD-2025-001")
.build()
)
slack.send_message(channel='#incidents', blocks=blocks)
# File upload
slack.send_file(
channel='#reports',
content='Daily metrics...',
filename='metrics-2025-01-05.csv',
title='Daily Metrics Report'
)
# Thread reply
slack.reply_to_thread(
channel='#incidents',
thread_ts='1735689600.123',
text='Issue resolved!'
)
```
### Elasticsearch Operations
```python
from nui_lambda_shared_utils import ElasticsearchClient, ESQueryBuilder
# Initialize client
es = ElasticsearchClient()
# Query builder for complex searches
query_builder = ESQueryBuilder()
query = (query_builder
.date_range("@timestamp", "now-1h", "now")
.term("level", "ERROR")
.wildcard("message", "*timeout*")
.build()
)
results = es.search(index="logs-*", body={"query": query})
# Built-in query helpers
error_stats = es.get_error_stats(
index="application-logs",
time_range="1h",
service_filter="payment-service"
)
```
### Database Connections
```python
from nui_lambda_shared_utils import DatabaseClient
# Initialize with connection pooling
db = DatabaseClient()
# Execute queries with automatic retry
async with db.get_connection() as conn:
results = await conn.execute(
"SELECT * FROM orders WHERE created_at > %s",
[datetime.now() - timedelta(hours=1)]
)
# Bulk operations
records = [{"id": 1, "name": "Item 1"}, {"id": 2, "name": "Item 2"}]
db.bulk_insert("items", records)
```
### CloudWatch Metrics
```python
from nui_lambda_shared_utils import MetricsPublisher, StandardMetrics
# Initialize publisher
metrics = MetricsPublisher(namespace="MyApplication")
# Individual metrics
metrics.put_metric("ProcessedItems", 150, "Count")
metrics.put_metric("ResponseTime", 245.5, "Milliseconds")
# Batch publishing (more efficient)
metrics.batch_publish([
StandardMetrics.count("Errors", 3),
StandardMetrics.duration("ProcessingTime", 1.2),
StandardMetrics.gauge("QueueSize", 45)
])
# Decorator for Lambda performance tracking
@track_lambda_performance(metrics)
def lambda_handler(event, context):
# Your Lambda logic here
return {"statusCode": 200}
```
### Error Handling
```python
from nui_lambda_shared_utils import with_retry, retry_on_network_error, handle_lambda_error
# Automatic retry with exponential backoff
@retry_on_network_error
def call_external_api():
response = requests.get("https://api.external-service.com/data")
return response.json()
# Custom retry configuration
@with_retry(max_attempts=5, initial_delay=2.0)
def critical_operation():
# Operation that might fail
return perform_critical_task()
# Lambda error handling wrapper
@handle_lambda_error
def lambda_handler(event, context):
# Your Lambda logic here
return {"statusCode": 200, "body": "Success"}
```
## Configuration
The package supports multiple configuration methods:
### Environment Variables
```bash
# Core settings
ES_HOST=localhost:9200 # Elasticsearch host
ES_CREDENTIALS_SECRET=elasticsearch-creds # AWS secret name for ES credentials
DB_CREDENTIALS_SECRET=database-creds # AWS secret name for DB credentials
SLACK_CREDENTIALS_SECRET=slack-bot-token # AWS secret name for Slack token
# AWS settings
AWS_REGION=us-east-1 # AWS region for services
```
### Programmatic Configuration
```python
import nui_lambda_shared_utils as nui
# Configure all at once
config = nui.Config(
es_host="prod-elastic:9200",
es_credentials_secret="prod/elasticsearch",
db_credentials_secret="prod/database",
slack_credentials_secret="prod/slack",
aws_region="us-west-2"
)
nui.set_config(config)
# Or configure specific settings
nui.configure(
es_host="localhost:9200",
slack_credentials_secret="dev/slack-token"
)
```
## AWS Infrastructure Requirements
This package expects certain AWS resources to be available:
### Secrets Manager
Create secrets with these structures:
```json
// Elasticsearch credentials
{
"host": "your-elasticsearch-host:9200",
"username": "elastic",
"password": "your-password"
}
// Database credentials
{
"host": "your-db-host",
"port": 3306,
"username": "dbuser",
"password": "dbpassword",
"database": "your_database"
}
// Slack credentials
{
"bot_token": "xoxb-your-slack-bot-token",
"webhook_url": "https://hooks.slack.com/..." // optional
}
```
### IAM Permissions
Your Lambda execution role needs:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": "arn:aws:secretsmanager:*:*:secret:your-secret-*"
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricData"
],
"Resource": "*"
}
]
}
```
## Testing
```bash
# Install with dev dependencies
pip install nui-lambda-shared-utils[dev]
# Run tests
pytest
# Run tests with coverage
pytest --cov=nui_lambda_shared_utils --cov-report=html
# Run specific test categories
pytest -m unit # Unit tests only
pytest -m integration # Integration tests (requires AWS access)
```
## Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## Documentation & Support
- **GitHub Repository**: https://github.com/nuimarkets/nui-lambda-shared-utils
- **Issue Tracker**: https://github.com/nuimarkets/nui-lambda-shared-utils/issues
- **PyPI Package**: https://pypi.org/project/nui-lambda-shared-utils/
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## About NUI Markets
NUI Markets is a technology company focused on building innovative trading and marketplace platforms. This package represents our commitment to open-source tooling and enterprise-grade infrastructure patterns.
Raw data
{
"_id": null,
"home_page": "https://github.com/nuimarkets/nui-lambda-shared-utils",
"name": "nui-lambda-shared-utils",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "aws, lambda, utilities, slack, elasticsearch, monitoring, serverless",
"author": "NUI Markets",
"author_email": "NUI Markets <develop@nuimarkets.com>",
"download_url": "https://files.pythonhosted.org/packages/34/c3/68ac84509ab40fc76efc496eddd0ad0543d9b749c656638acb9d50b0a23a/nui_lambda_shared_utils-1.0.0.tar.gz",
"platform": null,
"description": "# NUI Lambda Shared Utilities\n\n[](https://badge.fury.io/py/nui-lambda-shared-utils)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n\nEnterprise-grade utilities for AWS Lambda functions with Slack, Elasticsearch, and monitoring integrations. This package provides standardized, production-ready patterns for common serverless operations while maintaining flexibility and configurability.\n\n## Key Features\n\n- **AWS Secrets Manager Integration** - Secure credential management with caching\n- **Slack Messaging** - Rich formatting, threading, file uploads, and channel management\n- **Elasticsearch Operations** - Query builders, index management, and health monitoring\n- **Database Connections** - Connection pooling, automatic retries, and transaction management\n- **CloudWatch Metrics** - Batched publishing with custom dimensions\n- **Error Handling** - Intelligent retry patterns with exponential backoff\n- **Timezone Utilities** - Timezone handling and formatting\n- **Configurable Defaults** - Environment-aware configuration system\n\n## Quick Start\n\n### Installation\n\n```bash\npip install nui-lambda-shared-utils\n\n# With specific extras for optional dependencies\npip install nui-lambda-shared-utils[all] # All integrations\npip install nui-lambda-shared-utils[slack] # Slack only\npip install nui-lambda-shared-utils[elasticsearch] # Elasticsearch only\npip install nui-lambda-shared-utils[database] # Database only\n```\n\n### Basic Configuration\n\n```python\nimport nui_lambda_shared_utils as nui\n\n# Configure for your environment (optional - uses sensible defaults)\nnui.configure(\n es_host=\"your-elasticsearch-host:9200\",\n es_credentials_secret=\"your-es-secret-name\",\n slack_credentials_secret=\"your-slack-secret-name\",\n db_credentials_secret=\"your-database-secret-name\"\n)\n\n# Or use environment variables:\n# ES_HOST, ES_CREDENTIALS_SECRET, SLACK_CREDENTIALS_SECRET, etc.\n```\n\n## Usage Examples\n\n### Secrets Management\n\n```python\nfrom nui_lambda_shared_utils import get_secret, get_slack_credentials, get_database_credentials\n\n# Generic secret retrieval\napi_keys = get_secret(\"my-service/api-keys\")\n\n# Specialized getters with normalized field names\nslack_creds = get_slack_credentials() # Uses configured secret name\ndb_creds = get_database_credentials() # Standardized: host, port, username, password, database\n```\n\n### Slack Integration\n\n```python\nfrom nui_lambda_shared_utils import SlackClient, SlackBlockBuilder\n\n# Initialize client\nslack = SlackClient()\n\n# Simple message\nresponse = slack.send_message(\n channel='#alerts',\n text='Service deployment complete'\n)\n\n# Rich formatted message with blocks\nbuilder = SlackBlockBuilder()\nblocks = (builder\n .add_header(\"Production Alert\", emoji=\"warning\")\n .add_section(\"Error Rate\", \"15.2% (above 10% threshold)\")\n .add_section(\"Time Range\", \"Last 10 minutes\")\n .add_divider()\n .add_context(\"Alert ID: PROD-2025-001\")\n .build()\n)\n\nslack.send_message(channel='#incidents', blocks=blocks)\n\n# File upload\nslack.send_file(\n channel='#reports',\n content='Daily metrics...',\n filename='metrics-2025-01-05.csv',\n title='Daily Metrics Report'\n)\n\n# Thread reply\nslack.reply_to_thread(\n channel='#incidents',\n thread_ts='1735689600.123',\n text='Issue resolved!'\n)\n```\n\n### Elasticsearch Operations\n\n```python\nfrom nui_lambda_shared_utils import ElasticsearchClient, ESQueryBuilder\n\n# Initialize client\nes = ElasticsearchClient()\n\n# Query builder for complex searches\nquery_builder = ESQueryBuilder()\nquery = (query_builder\n .date_range(\"@timestamp\", \"now-1h\", \"now\")\n .term(\"level\", \"ERROR\")\n .wildcard(\"message\", \"*timeout*\")\n .build()\n)\n\nresults = es.search(index=\"logs-*\", body={\"query\": query})\n\n# Built-in query helpers\nerror_stats = es.get_error_stats(\n index=\"application-logs\", \n time_range=\"1h\",\n service_filter=\"payment-service\"\n)\n```\n\n### Database Connections\n\n```python\nfrom nui_lambda_shared_utils import DatabaseClient\n\n# Initialize with connection pooling\ndb = DatabaseClient()\n\n# Execute queries with automatic retry\nasync with db.get_connection() as conn:\n results = await conn.execute(\n \"SELECT * FROM orders WHERE created_at > %s\",\n [datetime.now() - timedelta(hours=1)]\n )\n\n# Bulk operations\nrecords = [{\"id\": 1, \"name\": \"Item 1\"}, {\"id\": 2, \"name\": \"Item 2\"}]\ndb.bulk_insert(\"items\", records)\n```\n\n### CloudWatch Metrics\n\n```python\nfrom nui_lambda_shared_utils import MetricsPublisher, StandardMetrics\n\n# Initialize publisher\nmetrics = MetricsPublisher(namespace=\"MyApplication\")\n\n# Individual metrics\nmetrics.put_metric(\"ProcessedItems\", 150, \"Count\")\nmetrics.put_metric(\"ResponseTime\", 245.5, \"Milliseconds\")\n\n# Batch publishing (more efficient)\nmetrics.batch_publish([\n StandardMetrics.count(\"Errors\", 3),\n StandardMetrics.duration(\"ProcessingTime\", 1.2),\n StandardMetrics.gauge(\"QueueSize\", 45)\n])\n\n# Decorator for Lambda performance tracking\n@track_lambda_performance(metrics)\ndef lambda_handler(event, context):\n # Your Lambda logic here\n return {\"statusCode\": 200}\n```\n\n### Error Handling\n\n```python\nfrom nui_lambda_shared_utils import with_retry, retry_on_network_error, handle_lambda_error\n\n# Automatic retry with exponential backoff\n@retry_on_network_error\ndef call_external_api():\n response = requests.get(\"https://api.external-service.com/data\")\n return response.json()\n\n# Custom retry configuration\n@with_retry(max_attempts=5, initial_delay=2.0)\ndef critical_operation():\n # Operation that might fail\n return perform_critical_task()\n\n# Lambda error handling wrapper\n@handle_lambda_error\ndef lambda_handler(event, context):\n # Your Lambda logic here\n return {\"statusCode\": 200, \"body\": \"Success\"}\n```\n\n## Configuration\n\nThe package supports multiple configuration methods:\n\n### Environment Variables\n\n```bash\n# Core settings\nES_HOST=localhost:9200 # Elasticsearch host\nES_CREDENTIALS_SECRET=elasticsearch-creds # AWS secret name for ES credentials\nDB_CREDENTIALS_SECRET=database-creds # AWS secret name for DB credentials\nSLACK_CREDENTIALS_SECRET=slack-bot-token # AWS secret name for Slack token\n\n# AWS settings\nAWS_REGION=us-east-1 # AWS region for services\n```\n\n### Programmatic Configuration\n\n```python\nimport nui_lambda_shared_utils as nui\n\n# Configure all at once\nconfig = nui.Config(\n es_host=\"prod-elastic:9200\",\n es_credentials_secret=\"prod/elasticsearch\",\n db_credentials_secret=\"prod/database\",\n slack_credentials_secret=\"prod/slack\",\n aws_region=\"us-west-2\"\n)\nnui.set_config(config)\n\n# Or configure specific settings\nnui.configure(\n es_host=\"localhost:9200\",\n slack_credentials_secret=\"dev/slack-token\"\n)\n```\n\n## AWS Infrastructure Requirements\n\nThis package expects certain AWS resources to be available:\n\n### Secrets Manager\n\nCreate secrets with these structures:\n\n```json\n// Elasticsearch credentials\n{\n \"host\": \"your-elasticsearch-host:9200\",\n \"username\": \"elastic\", \n \"password\": \"your-password\"\n}\n\n// Database credentials\n{\n \"host\": \"your-db-host\",\n \"port\": 3306,\n \"username\": \"dbuser\",\n \"password\": \"dbpassword\", \n \"database\": \"your_database\"\n}\n\n// Slack credentials\n{\n \"bot_token\": \"xoxb-your-slack-bot-token\",\n \"webhook_url\": \"https://hooks.slack.com/...\" // optional\n}\n```\n\n### IAM Permissions\n\nYour Lambda execution role needs:\n\n```json\n{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Action\": [\n \"secretsmanager:GetSecretValue\"\n ],\n \"Resource\": \"arn:aws:secretsmanager:*:*:secret:your-secret-*\"\n },\n {\n \"Effect\": \"Allow\", \n \"Action\": [\n \"cloudwatch:PutMetricData\"\n ],\n \"Resource\": \"*\"\n }\n ]\n}\n```\n\n## Testing\n\n```bash\n# Install with dev dependencies\npip install nui-lambda-shared-utils[dev]\n\n# Run tests\npytest\n\n# Run tests with coverage\npytest --cov=nui_lambda_shared_utils --cov-report=html\n\n# Run specific test categories\npytest -m unit # Unit tests only\npytest -m integration # Integration tests (requires AWS access)\n```\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## Documentation & Support\n\n- **GitHub Repository**: https://github.com/nuimarkets/nui-lambda-shared-utils\n- **Issue Tracker**: https://github.com/nuimarkets/nui-lambda-shared-utils/issues\n- **PyPI Package**: https://pypi.org/project/nui-lambda-shared-utils/\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## About NUI Markets\n\nNUI Markets is a technology company focused on building innovative trading and marketplace platforms. This package represents our commitment to open-source tooling and enterprise-grade infrastructure patterns.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Enterprise-grade utilities for AWS Lambda functions with Slack, Elasticsearch, and monitoring integrations",
"version": "1.0.0",
"project_urls": {
"Bug Tracker": "https://github.com/nuimarkets/nui-lambda-shared-utils/issues",
"Documentation": "https://github.com/nuimarkets/nui-lambda-shared-utils/blob/main/README.md",
"Homepage": "https://github.com/nuimarkets/nui-lambda-shared-utils",
"Source": "https://github.com/nuimarkets/nui-lambda-shared-utils"
},
"split_keywords": [
"aws",
" lambda",
" utilities",
" slack",
" elasticsearch",
" monitoring",
" serverless"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "73a419ca24ecb745964d73c6fe1efc29869273750102f9920a586c58a0e2b1ae",
"md5": "d72e5b7d5d34f2d69e84cb0c7532e7f4",
"sha256": "448181e6d00727982588436c927cd7ab3c5a3cefb9c7e5dee8323c7ce4bfb37c"
},
"downloads": -1,
"filename": "nui_lambda_shared_utils-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d72e5b7d5d34f2d69e84cb0c7532e7f4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 46025,
"upload_time": "2025-09-05T03:42:33",
"upload_time_iso_8601": "2025-09-05T03:42:33.033566Z",
"url": "https://files.pythonhosted.org/packages/73/a4/19ca24ecb745964d73c6fe1efc29869273750102f9920a586c58a0e2b1ae/nui_lambda_shared_utils-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "34c368ac84509ab40fc76efc496eddd0ad0543d9b749c656638acb9d50b0a23a",
"md5": "91716690ae9164b85e9928d78bd68530",
"sha256": "ac099bdc7250495a1a973a139384e94491cba3fe66fe861f405ba8d94095bf3a"
},
"downloads": -1,
"filename": "nui_lambda_shared_utils-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "91716690ae9164b85e9928d78bd68530",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 71913,
"upload_time": "2025-09-05T03:42:34",
"upload_time_iso_8601": "2025-09-05T03:42:34.463668Z",
"url": "https://files.pythonhosted.org/packages/34/c3/68ac84509ab40fc76efc496eddd0ad0543d9b749c656638acb9d50b0a23a/nui_lambda_shared_utils-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-05 03:42:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nuimarkets",
"github_project": "nui-lambda-shared-utils",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "nui-lambda-shared-utils"
}
JSON
