# Generic DynamoDB Repository
A powerful, production-ready Python package for DynamoDB operations with repository pattern supporting both **synchronous** and **asynchronous** operations.
## Features
- **Dual Interface**: Both sync and async implementations with identical APIs
- **Repository Pattern**: Clean, standardized interface for DynamoDB operations
- **Comprehensive Operations**: CRUD, batch operations, queries, and index-based searches
- **Advanced Filtering**: Powerful client-side filtering with multiple operators and conditions
- **Auto-Serialization**: Automatic data type conversion for DynamoDB compatibility
- **Expiration Support**: Built-in TTL handling for automatic data expiration
- **Composite Key Support**: Full support for partition + sort key tables
- **Debug Mode**: Safe testing without actual database operations
- **Extensive Logging**: Comprehensive logging support for debugging
- **Type Hints**: Full type annotations for better IDE support
## Installation
```bash
pip install generic-repo
```
The package includes both synchronous and asynchronous functionality out of the box.
### Development Installation
```bash
pip install generic-repo[dev]
```
## Quick Start
### Synchronous Usage
```python
from generic_repo import GenericRepository
# Create repository - no need for boto3 setup!
repo = GenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1', # Optional: defaults to AWS SDK default
data_expiration_days=30 # Optional: TTL support
)
# Basic operations
item = repo.save('user-123', {'name': 'John Doe', 'email': 'john@example.com'})
loaded_item = repo.load('user-123')
repo.delete('user-123')
```
### Asynchronous Usage
```python
import asyncio
from generic_repo import AsyncGenericRepository
async def main():
# Create async repository - no need for aioboto3 setup!
async with AsyncGenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1', # Optional: defaults to AWS SDK default
data_expiration_days=30
) as repo:
# Basic async operations
item = await repo.save('user-123', {'name': 'John Doe', 'email': 'john@example.com'})
loaded_item = await repo.load('user-123')
# Async generator for scanning
async for item in repo.load_all():
print(item)
# Async scanning with filters
async for item in repo.load_all(filters={'status': 'active'}):
print(f"Active item: {item}")
asyncio.run(main())
```
## API Reference
Both `GenericRepository` and `AsyncGenericRepository` provide identical APIs:
### Basic Operations
- `load(key)` / `await load(key)` - Load item by primary key
- `save(key, data)` / `await save(key, data)` - Save item
- `delete(key)` / `await delete(key)` - Delete item
- `load_or_throw(key)` / `await load_or_throw(key)` - Load item or raise error
### Batch Operations
- `save_batch(items)` / `await save_batch(items)` - Save multiple items
- `delete_batch_by_keys(keys)` / `await delete_batch_by_keys(keys)` - Delete multiple items
### Query Operations
- `find_all(partition_key, filters=None)` / `await find_all(partition_key, filters=None)` - Find all items with partition key
- `find_all_with_index(index, key, value, filters=None)` / `await find_all_with_index(index, key, value, filters=None)` - Query using GSI/LSI
- `find_one_with_index(index, key, value, filters=None)` / `await find_one_with_index(index, key, value, filters=None)` - Find first item using GSI/LSI
- `load_all(filters=None)` / `async for item in load_all(filters=None)` - Scan entire table
### Composite Key Support
- `load_by_composite_key(key_dict)` / `await load_by_composite_key(key_dict)`
- `save_with_composite_key(item_data)` / `await save_with_composite_key(item_data)`
- `delete_by_composite_key(key_dict)` / `await delete_by_composite_key(key_dict)`
## Best Practices
### For PyPI Package Users
```python
from generic_repo import GenericRepository, AsyncGenericRepository
# Both sync and async functionality included out of the box
```
### Error Handling
```python
try:
repo = GenericRepository(table_name='your-table-name', primary_key_name='id', region_name='us-east-1')
item = repo.load_or_throw('nonexistent-key')
except ValueError as e:
print(f"Item not found: {e}")
```
### Debug Mode
```python
# Safe for testing - won't make actual database calls
repo = GenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1',
debug_mode=True
)
```
## Requirements
- Python 3.9+
**Note**: boto3, aioboto3, and related dependencies are automatically installed and managed by the package. You don't need to install them manually!
## License
MIT License - See LICENSE file for details.
## Contributing
See CONTRIBUTING.md for development setup and contribution guidelines.
## Changelog
See CHANGELOG.md for version history and changes.
## ๐ Features
- **Simple & Composite Key Support**: Works with both simple primary key tables and composite key (partition + sort key) tables
- **Comprehensive CRUD Operations**: Create, Read, Update, Delete operations with error handling
- **Batch Operations**: Efficient batch save and delete operations that automatically handle DynamoDB's 25-item limit
- **Advanced Querying**: Query operations with automatic pagination support
- **Powerful Filtering**: Client-side filtering with 12+ operators (eq, ne, gt, lt, contains, between, etc.)
- **Index Support**: Query operations on Global Secondary Indexes (GSI) and Local Secondary Indexes (LSI)
- **Automatic Data Serialization**: Handles Python to DynamoDB data type conversion seamlessly
- **Built-in Expiration**: Optional automatic item expiration using TTL
- **Debug Mode**: Testing-friendly debug mode that skips actual database operations
- **Comprehensive Logging**: Built-in logging support for monitoring and debugging
- **Type Hints**: Full type annotations for better IDE support and code quality
## ๐ฆ Installation
### From PyPI (Recommended)
```bash
pip install generic-repo
```
### From GitHub
```bash
pip install git+https://github.com/subratamal/generic-repo.git
```
### For Development
```bash
git clone https://github.com/subratamal/generic-repo.git
cd generic-repo
pip install -e .
```
## ๐ง Requirements
- Python 3.9+
**Note**: All AWS dependencies (boto3, aioboto3, botocore, etc.) are automatically managed by the package - no manual installation required!
## ๐ Quick Start
### Basic Setup
```python
from generic_repo import GenericRepository
# Create repository instance - no boto3 setup needed!
repo = GenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1', # Optional: defaults to AWS SDK default
data_expiration_days=30, # Optional: items expire after 30 days
debug_mode=False
)
```
### Basic Operations
```python
# Save an item
item_data = {'name': 'John Doe', 'email': 'john@example.com', 'age': 30}
saved_item = repo.save('user-123', item_data)
# Load an item
user = repo.load('user-123')
if user:
print(f"User: {user['name']}")
# Load with exception if not found
try:
user = repo.load_or_throw('user-123')
print(f"User: {user['name']}")
except ValueError as e:
print(f"User not found: {e}")
# Delete an item
repo.delete('user-123')
```
### Composite Key Operations
```python
# For tables with partition key + sort key
composite_data = {
'partition_key': 'USER',
'sort_key': 'profile#123',
'name': 'John Doe',
'email': 'john@example.com'
}
# Save with composite key
repo.save_with_composite_key(composite_data)
# Load with composite key
key_dict = {'partition_key': 'USER', 'sort_key': 'profile#123'}
user = repo.load_by_composite_key(key_dict)
# Delete with composite key
repo.delete_by_composite_key(key_dict)
```
### Batch Operations
```python
# Batch save multiple items
users = [
{'id': 'user-1', 'name': 'Alice', 'email': 'alice@example.com'},
{'id': 'user-2', 'name': 'Bob', 'email': 'bob@example.com'},
{'id': 'user-3', 'name': 'Charlie', 'email': 'charlie@example.com'}
]
repo.save_batch(users)
# Batch delete by keys
keys_to_delete = [
{'id': 'user-1'},
{'id': 'user-2'},
{'id': 'user-3'}
]
repo.delete_batch_by_keys(keys_to_delete)
```
### Query Operations
```python
# Find all items with a specific partition key
items = repo.find_all('USER')
# Find items with filtering
active_users = repo.find_all('USER', filters={'status': 'active'})
# Scan all items in the table (use carefully!)
for item in repo.load_all():
print(f"Item: {item}")
# Scan with filtering
for item in repo.load_all(filters={'age': {'gt': 18}}):
print(f"Adult: {item}")
# Count items in table
total_items = repo.count()
print(f"Total items: {total_items}")
```
### Index-Based Queries
```python
# Query using Global Secondary Index (GSI)
items = repo.find_all_with_index(
index_name='email-index',
key_name='email',
key_value='john@example.com'
)
# Query with additional filtering
active_admins = repo.find_all_with_index(
index_name='role-index',
key_name='role',
key_value='admin',
filters={'status': 'active', 'last_login': {'exists': True}}
)
# Find first matching item from index
item = repo.find_one_with_index(
index_name='status-index',
key_name='status',
key_value='active'
)
# Find first item with filtering
recent_active = repo.find_one_with_index(
index_name='status-index',
key_name='status',
key_value='active',
filters={'last_activity': {'gt': '2024-01-01'}}
)
```
## ๐ Advanced Filtering
The repository supports powerful filtering capabilities for refining query results. Filters can be applied to `load_all()`, `find_all()`, `find_all_with_index()`, and `find_one_with_index()` methods.
### Filter Formats
#### 1. Simple Equality
```python
# Find all active users
active_users = repo.find_all('USER', filters={'status': 'active'})
# Scan for items with specific category
async for item in repo.load_all(filters={'category': 'electronics'}):
print(item)
```
#### 2. Comparison Operators
```python
# Users older than 25
filters = {'age': {'gt': 25}}
older_users = repo.find_all('USER', filters=filters)
# Products with price between $10 and $50
filters = {'price': {'between': [10, 50]}}
products = repo.find_all('PRODUCT', filters=filters)
# Items with score >= 90
filters = {'score': {'ge': 90}}
high_scores = repo.find_all('SCORE', filters=filters)
```
#### 3. String Operations
```python
# Names containing "John"
filters = {'name': {'contains': 'John'}}
users = repo.find_all('USER', filters=filters)
# Emails starting with "admin"
filters = {'email': {'begins_with': 'admin'}}
admins = repo.find_all('USER', filters=filters)
```
#### 4. List and Set Operations
```python
# Users in specific cities
filters = {'city': {'in': ['New York', 'Los Angeles', 'Chicago']}}
city_users = repo.find_all('USER', filters=filters)
# Items with tags containing "python"
filters = {'tags': {'contains': 'python'}}
items = repo.find_all('ITEM', filters=filters)
```
#### 5. Existence Checks
```python
# Items that have an optional field
filters = {'optional_field': {'exists': True}}
items_with_field = repo.find_all('ITEM', filters=filters)
# Items without deleted_at field (active items)
filters = {'deleted_at': {'not_exists': True}}
active_items = repo.find_all('ITEM', filters=filters)
```
#### 6. Multiple Conditions (AND Logic)
```python
# Active users older than 18 in New York
filters = {
'status': 'active',
'age': {'gt': 18},
'city': 'New York'
}
users = repo.find_all('USER', filters=filters)
```
#### 7. Type-Explicit Filters
```python
# For precise numeric comparisons
filters = {
'price': {
'value': 19.99,
'type': 'N', # Numeric type
'operator': 'ge'
}
}
products = repo.find_all('PRODUCT', filters=filters)
```
### Supported Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `eq` | Equals (default) | `{'status': 'active'}` |
| `ne` | Not equals | `{'status': {'ne': 'deleted'}}` |
| `lt` | Less than | `{'age': {'lt': 30}}` |
| `le` | Less than or equal | `{'age': {'le': 30}}` |
| `gt` | Greater than | `{'score': {'gt': 85}}` |
| `ge` | Greater than or equal | `{'score': {'ge': 85}}` |
| `between` | Between two values | `{'age': {'between': [18, 65]}}` |
| `in` | In list of values | `{'status': {'in': ['active', 'pending']}}` |
| `contains` | Contains substring/value | `{'name': {'contains': 'John'}}` |
| `begins_with` | String begins with | `{'email': {'begins_with': 'admin'}}` |
| `exists` | Attribute exists | `{'phone': {'exists': True}}` |
| `not_exists` | Attribute doesn't exist | `{'deleted_at': {'not_exists': True}}` |
### Filtering with Index Queries
```python
# Find active users in a specific index with additional filters
active_admins = repo.find_all_with_index(
index_name='role-index',
key_name='role',
key_value='admin',
filters={'status': 'active', 'last_login': {'exists': True}}
)
# Async version
async for user in repo.find_all_with_index(
index_name='status-index',
key_name='status',
key_value='active',
filters={'age': {'gt': 21}}
):
print(f"Adult active user: {user['name']}")
```
### Performance Notes
- Filters are applied **after** the initial query/scan operation
- For better performance, use proper indexing strategies rather than relying solely on filters
- Filters work on the client side after data retrieval, so they don't reduce DynamoDB read costs
- Consider using GSI/LSI for frequently filtered attributes
## ๐๏ธ Advanced Configuration
### Custom Logger
```python
import logging
# Setup custom logger
logger = logging.getLogger('my-app')
logger.setLevel(logging.INFO)
repo = GenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1',
logger=logger
)
```
### Debug Mode for Testing
```python
# Enable debug mode to skip actual database operations
repo = GenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1',
debug_mode=True # Perfect for unit testing
)
```
### Automatic Item Expiration
```python
# Items will automatically expire after 7 days
repo = GenericRepository(
table_name='your-table-name',
primary_key_name='id',
region_name='us-east-1',
data_expiration_days=7
)
```
## ๐งช Testing
The package includes comprehensive test coverage. Run tests with:
```bash
# Install development dependencies
pip install -e .[dev]
# Run tests
python -m pytest tests/
# Run with coverage
python -m pytest tests/ --cov=generic_repo --cov-report=html
```
## ๐ค Contributing
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
### Development Setup
```bash
git clone https://github.com/subratamal/generic-repo.git
cd generic-repo
pip install -e .[dev]
```
### Code Quality
This project uses:
- **Ruff** for linting and formatting
- **Type hints** for better code quality
- **Comprehensive docstrings** for documentation
```bash
# Format code
ruff check --fix .
ruff format .
```
## ๐ License
This project is licensed under the MIT License. See the LICENSE file for details.
## ๐ Links
- **GitHub Repository**: https://github.com/subratamal/generic-repo
- **PyPI Package**: https://pypi.org/project/generic-repo/
- **Documentation**: https://github.com/subratamal/generic-repo/wiki
- **Issue Tracker**: https://github.com/subratamal/generic-repo/issues
## ๐ Support
- **Email**: 06.subrat@gmail.com
- **GitHub Issues**: https://github.com/subratamal/generic-repo/issues
## ๐ฏ Roadmap
- [x] Async/await support for better performance
- [x] Advanced filtering with multiple operators and conditions
- [ ] More advanced query builders
- [ ] OR logic support for filters
- [ ] Built-in caching layer
- [ ] CloudFormation templates for common DynamoDB setups
- [ ] Integration with AWS CDK
---
**Made with โค๏ธ by Subrat**
Raw data
{
"_id": null,
"home_page": null,
"name": "generic-repo",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Subrat <06.subrata@gmail.com>",
"keywords": "async, asyncio, aws, batch-operations, boto3, crud, data-access, database, dynamodb, nosql, orm, repository",
"author": null,
"author_email": "Subrat <06.subrata@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/52/43/3b6d71e103e9fe6bef645ecdda2b8f5616b3451ccd8ebcaff8baa8f58bab/generic_repo-2.0.2.tar.gz",
"platform": null,
"description": "# Generic DynamoDB Repository\n\nA powerful, production-ready Python package for DynamoDB operations with repository pattern supporting both **synchronous** and **asynchronous** operations.\n\n## Features\n\n- **Dual Interface**: Both sync and async implementations with identical APIs\n- **Repository Pattern**: Clean, standardized interface for DynamoDB operations\n- **Comprehensive Operations**: CRUD, batch operations, queries, and index-based searches\n- **Advanced Filtering**: Powerful client-side filtering with multiple operators and conditions\n- **Auto-Serialization**: Automatic data type conversion for DynamoDB compatibility\n- **Expiration Support**: Built-in TTL handling for automatic data expiration\n- **Composite Key Support**: Full support for partition + sort key tables\n- **Debug Mode**: Safe testing without actual database operations\n- **Extensive Logging**: Comprehensive logging support for debugging\n- **Type Hints**: Full type annotations for better IDE support\n\n## Installation\n\n```bash\npip install generic-repo\n```\n\nThe package includes both synchronous and asynchronous functionality out of the box.\n\n### Development Installation\n```bash\npip install generic-repo[dev]\n```\n\n## Quick Start\n\n### Synchronous Usage\n\n```python\nfrom generic_repo import GenericRepository\n\n# Create repository - no need for boto3 setup!\nrepo = GenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1', # Optional: defaults to AWS SDK default\n data_expiration_days=30 # Optional: TTL support\n)\n\n# Basic operations\nitem = repo.save('user-123', {'name': 'John Doe', 'email': 'john@example.com'})\nloaded_item = repo.load('user-123')\nrepo.delete('user-123')\n```\n\n### Asynchronous Usage\n\n```python\nimport asyncio\nfrom generic_repo import AsyncGenericRepository\n\nasync def main():\n # Create async repository - no need for aioboto3 setup!\n async with AsyncGenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1', # Optional: defaults to AWS SDK default\n data_expiration_days=30\n ) as repo:\n # Basic async operations\n item = await repo.save('user-123', {'name': 'John Doe', 'email': 'john@example.com'})\n loaded_item = await repo.load('user-123')\n \n # Async generator for scanning\n async for item in repo.load_all():\n print(item)\n \n # Async scanning with filters\n async for item in repo.load_all(filters={'status': 'active'}):\n print(f\"Active item: {item}\")\n\nasyncio.run(main())\n```\n\n## API Reference\n\nBoth `GenericRepository` and `AsyncGenericRepository` provide identical APIs:\n\n### Basic Operations\n- `load(key)` / `await load(key)` - Load item by primary key\n- `save(key, data)` / `await save(key, data)` - Save item\n- `delete(key)` / `await delete(key)` - Delete item\n- `load_or_throw(key)` / `await load_or_throw(key)` - Load item or raise error\n\n### Batch Operations\n- `save_batch(items)` / `await save_batch(items)` - Save multiple items\n- `delete_batch_by_keys(keys)` / `await delete_batch_by_keys(keys)` - Delete multiple items\n\n### Query Operations\n- `find_all(partition_key, filters=None)` / `await find_all(partition_key, filters=None)` - Find all items with partition key\n- `find_all_with_index(index, key, value, filters=None)` / `await find_all_with_index(index, key, value, filters=None)` - Query using GSI/LSI\n- `find_one_with_index(index, key, value, filters=None)` / `await find_one_with_index(index, key, value, filters=None)` - Find first item using GSI/LSI\n- `load_all(filters=None)` / `async for item in load_all(filters=None)` - Scan entire table\n\n### Composite Key Support\n- `load_by_composite_key(key_dict)` / `await load_by_composite_key(key_dict)`\n- `save_with_composite_key(item_data)` / `await save_with_composite_key(item_data)`\n- `delete_by_composite_key(key_dict)` / `await delete_by_composite_key(key_dict)`\n\n## Best Practices\n\n### For PyPI Package Users\n\n```python\nfrom generic_repo import GenericRepository, AsyncGenericRepository\n\n# Both sync and async functionality included out of the box\n```\n\n### Error Handling\n\n```python\ntry:\n repo = GenericRepository(table_name='your-table-name', primary_key_name='id', region_name='us-east-1')\n item = repo.load_or_throw('nonexistent-key')\nexcept ValueError as e:\n print(f\"Item not found: {e}\")\n```\n\n### Debug Mode\n\n```python\n# Safe for testing - won't make actual database calls\nrepo = GenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1',\n debug_mode=True\n)\n```\n\n## Requirements\n\n- Python 3.9+\n\n**Note**: boto3, aioboto3, and related dependencies are automatically installed and managed by the package. You don't need to install them manually!\n\n## License\n\nMIT License - See LICENSE file for details.\n\n## Contributing\n\nSee CONTRIBUTING.md for development setup and contribution guidelines.\n\n## Changelog\n\nSee CHANGELOG.md for version history and changes.\n\n## \ud83d\ude80 Features\n\n- **Simple & Composite Key Support**: Works with both simple primary key tables and composite key (partition + sort key) tables\n- **Comprehensive CRUD Operations**: Create, Read, Update, Delete operations with error handling\n- **Batch Operations**: Efficient batch save and delete operations that automatically handle DynamoDB's 25-item limit\n- **Advanced Querying**: Query operations with automatic pagination support\n- **Powerful Filtering**: Client-side filtering with 12+ operators (eq, ne, gt, lt, contains, between, etc.)\n- **Index Support**: Query operations on Global Secondary Indexes (GSI) and Local Secondary Indexes (LSI)\n- **Automatic Data Serialization**: Handles Python to DynamoDB data type conversion seamlessly\n- **Built-in Expiration**: Optional automatic item expiration using TTL\n- **Debug Mode**: Testing-friendly debug mode that skips actual database operations\n- **Comprehensive Logging**: Built-in logging support for monitoring and debugging\n- **Type Hints**: Full type annotations for better IDE support and code quality\n\n## \ud83d\udce6 Installation\n\n### From PyPI (Recommended)\n```bash\npip install generic-repo\n```\n\n### From GitHub\n```bash\npip install git+https://github.com/subratamal/generic-repo.git\n```\n\n### For Development\n```bash\ngit clone https://github.com/subratamal/generic-repo.git\ncd generic-repo\npip install -e .\n```\n\n## \ud83d\udd27 Requirements\n\n- Python 3.9+\n\n**Note**: All AWS dependencies (boto3, aioboto3, botocore, etc.) are automatically managed by the package - no manual installation required!\n\n## \ud83d\udcd6 Quick Start\n\n### Basic Setup\n\n```python\nfrom generic_repo import GenericRepository\n\n# Create repository instance - no boto3 setup needed!\nrepo = GenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1', # Optional: defaults to AWS SDK default\n data_expiration_days=30, # Optional: items expire after 30 days\n debug_mode=False\n)\n```\n\n### Basic Operations\n\n```python\n# Save an item\nitem_data = {'name': 'John Doe', 'email': 'john@example.com', 'age': 30}\nsaved_item = repo.save('user-123', item_data)\n\n# Load an item\nuser = repo.load('user-123')\nif user:\n print(f\"User: {user['name']}\")\n\n# Load with exception if not found\ntry:\n user = repo.load_or_throw('user-123')\n print(f\"User: {user['name']}\")\nexcept ValueError as e:\n print(f\"User not found: {e}\")\n\n# Delete an item\nrepo.delete('user-123')\n```\n\n### Composite Key Operations\n\n```python\n# For tables with partition key + sort key\ncomposite_data = {\n 'partition_key': 'USER',\n 'sort_key': 'profile#123',\n 'name': 'John Doe',\n 'email': 'john@example.com'\n}\n\n# Save with composite key\nrepo.save_with_composite_key(composite_data)\n\n# Load with composite key\nkey_dict = {'partition_key': 'USER', 'sort_key': 'profile#123'}\nuser = repo.load_by_composite_key(key_dict)\n\n# Delete with composite key\nrepo.delete_by_composite_key(key_dict)\n```\n\n### Batch Operations\n\n```python\n# Batch save multiple items\nusers = [\n {'id': 'user-1', 'name': 'Alice', 'email': 'alice@example.com'},\n {'id': 'user-2', 'name': 'Bob', 'email': 'bob@example.com'},\n {'id': 'user-3', 'name': 'Charlie', 'email': 'charlie@example.com'}\n]\nrepo.save_batch(users)\n\n# Batch delete by keys\nkeys_to_delete = [\n {'id': 'user-1'},\n {'id': 'user-2'},\n {'id': 'user-3'}\n]\nrepo.delete_batch_by_keys(keys_to_delete)\n```\n\n### Query Operations\n\n```python\n# Find all items with a specific partition key\nitems = repo.find_all('USER')\n\n# Find items with filtering\nactive_users = repo.find_all('USER', filters={'status': 'active'})\n\n# Scan all items in the table (use carefully!)\nfor item in repo.load_all():\n print(f\"Item: {item}\")\n\n# Scan with filtering\nfor item in repo.load_all(filters={'age': {'gt': 18}}):\n print(f\"Adult: {item}\")\n\n# Count items in table\ntotal_items = repo.count()\nprint(f\"Total items: {total_items}\")\n```\n\n### Index-Based Queries\n\n```python\n# Query using Global Secondary Index (GSI)\nitems = repo.find_all_with_index(\n index_name='email-index',\n key_name='email', \n key_value='john@example.com'\n)\n\n# Query with additional filtering\nactive_admins = repo.find_all_with_index(\n index_name='role-index',\n key_name='role',\n key_value='admin',\n filters={'status': 'active', 'last_login': {'exists': True}}\n)\n\n# Find first matching item from index\nitem = repo.find_one_with_index(\n index_name='status-index',\n key_name='status',\n key_value='active'\n)\n\n# Find first item with filtering\nrecent_active = repo.find_one_with_index(\n index_name='status-index',\n key_name='status',\n key_value='active',\n filters={'last_activity': {'gt': '2024-01-01'}}\n)\n```\n\n## \ud83d\udd0d Advanced Filtering\n\nThe repository supports powerful filtering capabilities for refining query results. Filters can be applied to `load_all()`, `find_all()`, `find_all_with_index()`, and `find_one_with_index()` methods.\n\n### Filter Formats\n\n#### 1. Simple Equality\n```python\n# Find all active users\nactive_users = repo.find_all('USER', filters={'status': 'active'})\n\n# Scan for items with specific category\nasync for item in repo.load_all(filters={'category': 'electronics'}):\n print(item)\n```\n\n#### 2. Comparison Operators\n```python\n# Users older than 25\nfilters = {'age': {'gt': 25}}\nolder_users = repo.find_all('USER', filters=filters)\n\n# Products with price between $10 and $50\nfilters = {'price': {'between': [10, 50]}}\nproducts = repo.find_all('PRODUCT', filters=filters)\n\n# Items with score >= 90\nfilters = {'score': {'ge': 90}}\nhigh_scores = repo.find_all('SCORE', filters=filters)\n```\n\n#### 3. String Operations\n```python\n# Names containing \"John\"\nfilters = {'name': {'contains': 'John'}}\nusers = repo.find_all('USER', filters=filters)\n\n# Emails starting with \"admin\"\nfilters = {'email': {'begins_with': 'admin'}}\nadmins = repo.find_all('USER', filters=filters)\n```\n\n#### 4. List and Set Operations\n```python\n# Users in specific cities\nfilters = {'city': {'in': ['New York', 'Los Angeles', 'Chicago']}}\ncity_users = repo.find_all('USER', filters=filters)\n\n# Items with tags containing \"python\"\nfilters = {'tags': {'contains': 'python'}}\nitems = repo.find_all('ITEM', filters=filters)\n```\n\n#### 5. Existence Checks\n```python\n# Items that have an optional field\nfilters = {'optional_field': {'exists': True}}\nitems_with_field = repo.find_all('ITEM', filters=filters)\n\n# Items without deleted_at field (active items)\nfilters = {'deleted_at': {'not_exists': True}}\nactive_items = repo.find_all('ITEM', filters=filters)\n```\n\n#### 6. Multiple Conditions (AND Logic)\n```python\n# Active users older than 18 in New York\nfilters = {\n 'status': 'active',\n 'age': {'gt': 18},\n 'city': 'New York'\n}\nusers = repo.find_all('USER', filters=filters)\n```\n\n#### 7. Type-Explicit Filters\n```python\n# For precise numeric comparisons\nfilters = {\n 'price': {\n 'value': 19.99,\n 'type': 'N', # Numeric type\n 'operator': 'ge'\n }\n}\nproducts = repo.find_all('PRODUCT', filters=filters)\n```\n\n### Supported Operators\n\n| Operator | Description | Example |\n|----------|-------------|---------|\n| `eq` | Equals (default) | `{'status': 'active'}` |\n| `ne` | Not equals | `{'status': {'ne': 'deleted'}}` |\n| `lt` | Less than | `{'age': {'lt': 30}}` |\n| `le` | Less than or equal | `{'age': {'le': 30}}` |\n| `gt` | Greater than | `{'score': {'gt': 85}}` |\n| `ge` | Greater than or equal | `{'score': {'ge': 85}}` |\n| `between` | Between two values | `{'age': {'between': [18, 65]}}` |\n| `in` | In list of values | `{'status': {'in': ['active', 'pending']}}` |\n| `contains` | Contains substring/value | `{'name': {'contains': 'John'}}` |\n| `begins_with` | String begins with | `{'email': {'begins_with': 'admin'}}` |\n| `exists` | Attribute exists | `{'phone': {'exists': True}}` |\n| `not_exists` | Attribute doesn't exist | `{'deleted_at': {'not_exists': True}}` |\n\n### Filtering with Index Queries\n\n```python\n# Find active users in a specific index with additional filters\nactive_admins = repo.find_all_with_index(\n index_name='role-index',\n key_name='role',\n key_value='admin',\n filters={'status': 'active', 'last_login': {'exists': True}}\n)\n\n# Async version\nasync for user in repo.find_all_with_index(\n index_name='status-index',\n key_name='status', \n key_value='active',\n filters={'age': {'gt': 21}}\n):\n print(f\"Adult active user: {user['name']}\")\n```\n\n### Performance Notes\n\n- Filters are applied **after** the initial query/scan operation\n- For better performance, use proper indexing strategies rather than relying solely on filters\n- Filters work on the client side after data retrieval, so they don't reduce DynamoDB read costs\n- Consider using GSI/LSI for frequently filtered attributes\n\n## \ud83c\udfd7\ufe0f Advanced Configuration\n\n### Custom Logger\n\n```python\nimport logging\n\n# Setup custom logger\nlogger = logging.getLogger('my-app')\nlogger.setLevel(logging.INFO)\n\nrepo = GenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1',\n logger=logger\n)\n```\n\n### Debug Mode for Testing\n\n```python\n# Enable debug mode to skip actual database operations\nrepo = GenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1',\n debug_mode=True # Perfect for unit testing\n)\n```\n\n### Automatic Item Expiration\n\n```python\n# Items will automatically expire after 7 days\nrepo = GenericRepository(\n table_name='your-table-name',\n primary_key_name='id',\n region_name='us-east-1',\n data_expiration_days=7\n)\n```\n\n## \ud83e\uddea Testing\n\nThe package includes comprehensive test coverage. Run tests with:\n\n```bash\n# Install development dependencies\npip install -e .[dev]\n\n# Run tests\npython -m pytest tests/\n\n# Run with coverage\npython -m pytest tests/ --cov=generic_repo --cov-report=html\n```\n\n## \ud83e\udd1d Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.\n\n### Development Setup\n\n```bash\ngit clone https://github.com/subratamal/generic-repo.git\ncd generic-repo\npip install -e .[dev]\n```\n\n### Code Quality\n\nThis project uses:\n- **Ruff** for linting and formatting\n- **Type hints** for better code quality\n- **Comprehensive docstrings** for documentation\n\n```bash\n# Format code\nruff check --fix .\nruff format .\n```\n\n## \ud83d\udcc4 License\n\nThis project is licensed under the MIT License. See the LICENSE file for details.\n\n## \ud83d\udd17 Links\n\n- **GitHub Repository**: https://github.com/subratamal/generic-repo\n- **PyPI Package**: https://pypi.org/project/generic-repo/\n- **Documentation**: https://github.com/subratamal/generic-repo/wiki\n- **Issue Tracker**: https://github.com/subratamal/generic-repo/issues\n\n## \ud83d\udcde Support\n\n- **Email**: 06.subrat@gmail.com\n- **GitHub Issues**: https://github.com/subratamal/generic-repo/issues\n\n## \ud83c\udfaf Roadmap\n\n- [x] Async/await support for better performance\n- [x] Advanced filtering with multiple operators and conditions\n- [ ] More advanced query builders\n- [ ] OR logic support for filters\n- [ ] Built-in caching layer\n- [ ] CloudFormation templates for common DynamoDB setups\n- [ ] Integration with AWS CDK\n\n---\n\n**Made with \u2764\ufe0f by Subrat** ",
"bugtrack_url": null,
"license": "MIT",
"summary": "A powerful, production-ready Python package for DynamoDB operations with repository pattern (sync and async)",
"version": "2.0.2",
"project_urls": {
"Bug Tracker": "https://github.com/subratamal/generic-repo/issues",
"Changelog": "https://github.com/subratamal/generic-repo/blob/main/CHANGELOG.md",
"Documentation": "https://github.com/subratamal/generic-repo/wiki",
"Homepage": "https://github.com/subratamal/generic-repo",
"Repository": "https://github.com/subratamal/generic-repo.git"
},
"split_keywords": [
"async",
" asyncio",
" aws",
" batch-operations",
" boto3",
" crud",
" data-access",
" database",
" dynamodb",
" nosql",
" orm",
" repository"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "9a41da3b335f386651211ce06c25fe273c984bf497c6fb9ab6334c5681758aea",
"md5": "4a30c254268543dec6eb9894fcb1cd67",
"sha256": "148ce74ebed76f96f2e90ebff3280dc6aff820aec44b8a0088d2aed82aab3b4a"
},
"downloads": -1,
"filename": "generic_repo-2.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4a30c254268543dec6eb9894fcb1cd67",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 19385,
"upload_time": "2025-07-22T18:11:23",
"upload_time_iso_8601": "2025-07-22T18:11:23.900099Z",
"url": "https://files.pythonhosted.org/packages/9a/41/da3b335f386651211ce06c25fe273c984bf497c6fb9ab6334c5681758aea/generic_repo-2.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "52433b6d71e103e9fe6bef645ecdda2b8f5616b3451ccd8ebcaff8baa8f58bab",
"md5": "19c5063413b61706a4169e9f04824c4f",
"sha256": "157cd947f942a266828f12c0c8a742618089ce360094b3c72b346228b4073500"
},
"downloads": -1,
"filename": "generic_repo-2.0.2.tar.gz",
"has_sig": false,
"md5_digest": "19c5063413b61706a4169e9f04824c4f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 17676,
"upload_time": "2025-07-22T18:11:25",
"upload_time_iso_8601": "2025-07-22T18:11:25.575386Z",
"url": "https://files.pythonhosted.org/packages/52/43/3b6d71e103e9fe6bef645ecdda2b8f5616b3451ccd8ebcaff8baa8f58bab/generic_repo-2.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-22 18:11:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "subratamal",
"github_project": "generic-repo",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "generic-repo"
}