# python-serverless-crud
## The idea
Simple and powerful tool for quick serverless data management via API.
## Key concepts
- Don't Repeat Yourself - easy model definition with schema and cloud formation generation support
- Best practices applied by default (created with AWS LambdaPower Tools)
- Flexibility - enable, extend and modify what is needed
- One ring to rule them all - support for REST API, GraphQL (via API Gateway), AppSync GraphQL (direct resolvers)
## Features
- Full CRUD support with validation
- Native support for DynamoDB (including CloudFormation creation via troposphere)
- GlobalSecondaryIndex support
- LocalSecondaryIndex support
- Primary Key with and without sort keys
- Support for Scan, Query operations on the tables and indexes
- Virtual List method on the table or index
- Integrated record owner feature with KeyCondition and FilterCondition support (auto-detect)
# Documentation
## Sample service
```python
from aws_lambda_powertools import Tracer
from aws_lambda_powertools.logging import correlation_paths
from serverless_crud import api
from serverless_crud.dynamodb import annotation as db
from serverless_crud.model import BaseModel
from serverless_crud.logger import logger
tracer = Tracer()
@db.Model(
key=db.PrimaryKey(id=db.KeyFieldTypes.HASH),
indexes=(
db.GlobalSecondaryIndex("by_user", user=db.KeyFieldTypes.HASH, created=db.KeyFieldTypes.RANGE),
),
owner_field="user"
)
class Device(BaseModel):
id: str
created: int
user: str = None
api.rest.registry(Device, alias="device")
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
@tracer.capture_lambda_handler
def rest_handler(event, context):
return api.rest.handle(event, context)
```
With just a few lines of the code we are able to create `Device` model service which then can be extended.
In this example we:
1. Defined our `Device` model with some extra metadata, used by our generators. That includes:
1. Table key definition
2. GlobalSecondaryIndex
3. Definition of the field which will hold `owner` of the record (identity provided by cognito)
2. Registered our `Device` model into rest API under `device` alias
3. Created rest handler which then can be referred in our `serverless.yml` file
A few notes here:
- we need to define `rest_handler` function if we would like to use it as a target for local execution with serverless freamework
- Lambda Power Tools are build around functions and they don't work properly with object methods
- We use one function per API type, and we relay on internal router provided by each API implementation
### Serverless integration
If you use (serverless-builder)[https://github.com/epsyhealth/serverless-builder] you can create your `serverless.yml` with just a few lines of code (including DynamodbTables)
```python
from serverless import Service, Configuration
from serverless.aws.features import XRay
from serverless.aws.functions.http import HTTPFunction
from serverless.plugins import PythonRequirements, Prune
from serverless.provider import AWSProvider
from troposphere import dynamodb
from timemachine_api.handlers import api
service = Service(
"timemachine-api",
"Collect events in chronological order",
AWSProvider(),
config=Configuration(
domain="epsy.app"
)
)
service.provider.timeout = 5
service.plugins.add(Prune())
service.plugins.add(PythonRequirements(layer=False, useStaticCache=False, dockerSsh=True))
service.enable(XRay())
for name, table_specification in api.dynamodb_table_specifications().items():
service.resources.add(dynamodb.Table(name, **table_specification))
authorizer = dict(name="auth",
arn="arn:aws:cognito-idp:us-east-1:772962929486:userpool/us-east-1_FCl7gKtHC")
service.builder.function.http("rest", "Time machine REST API", "/rest/{proxy+}", HTTPFunction.ANY,
handler="timemachine_api.handlers.rest_handler", authorizer=authorizer)
service.render()
```
## Internals
### Annotations
`serverless-crud` project provides one annotation which must be used for all managed models.
```python
from serverless_crud.dynamodb import annotation as db
@db.Model(
key=db.PrimaryKey(name=db.KeyFieldTypes.HASH),
indexes=(
db.GlobalSecondaryIndex(...),
db.LocalSecondaryIndex(...)
),
owner_field="field"
)
```
Model annotation accepts:
- `key` - primary key definition, in form of `kwargs` where name of parameter would be a field name which should
be used in key, and value should be a value of `KeyFieldTypes` enum
- `indexes` - list of indexes GlobalSecondaryIndex|LocalSecondaryIndex. Indexes are defined in same way as primary key
- `owner_field` - name of the field which should be used for data filtering (based on the cognito identity)
### Data owner
`serverless-crud` can enforce some base data filtering on all kind of operations using Dynamodb conditional operations.
If you would like to use this feature you must set `owner_field` on each model you would like to use this feature.
Library will use this field for:
- setting its value on model creation / update (it will overwrite any value provided by user)
- as an extra `ConditionExpression` during `GET` and `DELETE` operations
- as a part of either `FilterExpression` or `KeyExpression` for Scan, Query and List operations
### Model registration
To be able to manage given model, you must first register it with specific API.
This can be done with a single line of code:
```python
api.rest.registry(Device, alias="device")
```
You need to provide only a model type to `registry` method, all other parameters are optional.
If you like, you can omit `alias` parameter, in that case framework will use model class name.
### Customizing endpoint behaviour
Framework defines a set of classes located in `serverless_crud.actions`:
- CreateAction
- DeleteAction
- GetAction
- ScanAction, ListAction, QueryAction
- UpdateAction
all those classes are subclasses of `serverless_crud.actions.base.Action` class and can be extended if needed.
You may need to execute custom logic after object creation, that can be done with custom `CreateAction` subclass
```python
from serverless_crud.actions import CreateAction
class CreateDeviceAction(CreateAction):
def handle(self, event: APIGatewayProxyEvent, context):
super().handle(event, context)
# custom logic
api.rest.registry(Device, create=CreateDeviceAction)
```
You can set custom handlers for each supported operation:
```python
def registry(self, model, alias=None, get=GetAction, create=CreateAction, update=UpdateAction, delete=DeleteAction,
lookup_list=ListAction, lookup_scan=ScanAction, lookup_query=QueryAction):
```
As you can see, all actions are defined by default. That also means that all actions are enabled by default, but
each action can be disabled.
If you need to disable action you need to set action handler to `None`, that will prevent framework from creating
route for given action, and it will disable whole logic behind it.
### Routes
REST API specific feature.
Framework will create multiple routes for each register model, using `alias` as a URL namespace.
Generated routes:
- GET /rest/{alias}/{pk} - fetch object by PK (see notes about PK below)
- POST /rest/{alias} - create new record
- PUT /rest/{alias}/{pk} - update record with given PK
- DELETE /rest/{alias}/{pk} - delete record with given PK
- GET /rest/lookup/{alias}/list - list all the records of given type using Query on the table
- GET /rest/lookup/{alias}/list/{index_name} - list all the records of the given type using Query on specific index
- POST /rest/lookup/{alias}/query - perform a query on given table
- POST /rest/lookup/{alias}/query/{index_name} - perform a query on given index
- POST /rest/lookup/{alias}/scan - perform a scan on given table
- POST /rest/lookup/{alias}/scan/{index_name} - perform a scan on given index
#### Primary Keys
> *Please remember that with DynamoDB key is a Partition Key with optional Sort Key.
In case you define Sort Key DynamoDB will require a value for it while getting / deleting key.
In that case framework will modify routes to include sort key as an extra path parameter*
## Endpoints
Raw data
{
"_id": null,
"home_page": "https://github.com/epsylabs/python-serverless-crud",
"name": "python-serverless-crud",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "library,serverless",
"author": "Epsy",
"author_email": "engineering@epsyhealth.com",
"download_url": "https://files.pythonhosted.org/packages/4f/17/470067b6f736ed06e882719456c7709762aa5e0c95de77d4455e6df739a0/python_serverless_crud-1.4.1.tar.gz",
"platform": null,
"description": "# python-serverless-crud\n\n## The idea \n\nSimple and powerful tool for quick serverless data management via API. \n\n## Key concepts\n\n- Don't Repeat Yourself - easy model definition with schema and cloud formation generation support\n- Best practices applied by default (created with AWS LambdaPower Tools)\n- Flexibility - enable, extend and modify what is needed\n- One ring to rule them all - support for REST API, GraphQL (via API Gateway), AppSync GraphQL (direct resolvers)\n\n\n## Features\n\n- Full CRUD support with validation\n- Native support for DynamoDB (including CloudFormation creation via troposphere)\n - GlobalSecondaryIndex support\n - LocalSecondaryIndex support\n - Primary Key with and without sort keys\n- Support for Scan, Query operations on the tables and indexes\n- Virtual List method on the table or index\n- Integrated record owner feature with KeyCondition and FilterCondition support (auto-detect)\n\n# Documentation\n\n## Sample service\n\n```python\nfrom aws_lambda_powertools import Tracer\nfrom aws_lambda_powertools.logging import correlation_paths\nfrom serverless_crud import api\nfrom serverless_crud.dynamodb import annotation as db\nfrom serverless_crud.model import BaseModel\nfrom serverless_crud.logger import logger\n\ntracer = Tracer()\n\n\n@db.Model(\n key=db.PrimaryKey(id=db.KeyFieldTypes.HASH),\n indexes=(\n db.GlobalSecondaryIndex(\"by_user\", user=db.KeyFieldTypes.HASH, created=db.KeyFieldTypes.RANGE),\n ),\n owner_field=\"user\"\n)\nclass Device(BaseModel):\n id: str\n created: int\n user: str = None\n\n\napi.rest.registry(Device, alias=\"device\")\n\n\n@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)\n@tracer.capture_lambda_handler\ndef rest_handler(event, context):\n return api.rest.handle(event, context)\n```\n\nWith just a few lines of the code we are able to create `Device` model service which then can be extended. \nIn this example we:\n\n1. Defined our `Device` model with some extra metadata, used by our generators. That includes:\n 1. Table key definition\n 2. GlobalSecondaryIndex\n 3. Definition of the field which will hold `owner` of the record (identity provided by cognito)\n2. Registered our `Device` model into rest API under `device` alias\n3. Created rest handler which then can be referred in our `serverless.yml` file \n\nA few notes here:\n- we need to define `rest_handler` function if we would like to use it as a target for local execution with serverless freamework\n- Lambda Power Tools are build around functions and they don't work properly with object methods\n- We use one function per API type, and we relay on internal router provided by each API implementation \n\n### Serverless integration\n\nIf you use (serverless-builder)[https://github.com/epsyhealth/serverless-builder] you can create your `serverless.yml` with just a few lines of code (including DynamodbTables)\n\n```python\nfrom serverless import Service, Configuration\nfrom serverless.aws.features import XRay\nfrom serverless.aws.functions.http import HTTPFunction\nfrom serverless.plugins import PythonRequirements, Prune\nfrom serverless.provider import AWSProvider\nfrom troposphere import dynamodb\n\nfrom timemachine_api.handlers import api\n\nservice = Service(\n \"timemachine-api\",\n \"Collect events in chronological order\",\n AWSProvider(),\n config=Configuration(\n domain=\"epsy.app\"\n )\n)\nservice.provider.timeout = 5\n\nservice.plugins.add(Prune())\nservice.plugins.add(PythonRequirements(layer=False, useStaticCache=False, dockerSsh=True))\n\nservice.enable(XRay())\n\nfor name, table_specification in api.dynamodb_table_specifications().items():\n service.resources.add(dynamodb.Table(name, **table_specification))\n\nauthorizer = dict(name=\"auth\",\n arn=\"arn:aws:cognito-idp:us-east-1:772962929486:userpool/us-east-1_FCl7gKtHC\")\n\nservice.builder.function.http(\"rest\", \"Time machine REST API\", \"/rest/{proxy+}\", HTTPFunction.ANY,\n handler=\"timemachine_api.handlers.rest_handler\", authorizer=authorizer)\n\n\nservice.render()\n```\n\n## Internals\n\n### Annotations\n\n`serverless-crud` project provides one annotation which must be used for all managed models.\n\n```python\nfrom serverless_crud.dynamodb import annotation as db\n@db.Model(\n key=db.PrimaryKey(name=db.KeyFieldTypes.HASH),\n indexes=(\n db.GlobalSecondaryIndex(...),\n db.LocalSecondaryIndex(...)\n ),\n owner_field=\"field\"\n)\n```\n\nModel annotation accepts:\n- `key` - primary key definition, in form of `kwargs` where name of parameter would be a field name which should \n be used in key, and value should be a value of `KeyFieldTypes` enum\n- `indexes` - list of indexes GlobalSecondaryIndex|LocalSecondaryIndex. Indexes are defined in same way as primary key\n- `owner_field` - name of the field which should be used for data filtering (based on the cognito identity)\n\n\n### Data owner \n\n`serverless-crud` can enforce some base data filtering on all kind of operations using Dynamodb conditional operations. \nIf you would like to use this feature you must set `owner_field` on each model you would like to use this feature.\n\nLibrary will use this field for:\n- setting its value on model creation / update (it will overwrite any value provided by user)\n- as an extra `ConditionExpression` during `GET` and `DELETE` operations\n- as a part of either `FilterExpression` or `KeyExpression` for Scan, Query and List operations\n\n\n### Model registration\n\nTo be able to manage given model, you must first register it with specific API. \nThis can be done with a single line of code:\n\n```python\napi.rest.registry(Device, alias=\"device\")\n```\n\nYou need to provide only a model type to `registry` method, all other parameters are optional. \nIf you like, you can omit `alias` parameter, in that case framework will use model class name.\n\n### Customizing endpoint behaviour\n\nFramework defines a set of classes located in `serverless_crud.actions`:\n- CreateAction\n- DeleteAction\n- GetAction\n- ScanAction, ListAction, QueryAction\n- UpdateAction\n\nall those classes are subclasses of `serverless_crud.actions.base.Action` class and can be extended if needed. \n\nYou may need to execute custom logic after object creation, that can be done with custom `CreateAction` subclass\n```python\n\nfrom serverless_crud.actions import CreateAction\n\nclass CreateDeviceAction(CreateAction):\n def handle(self, event: APIGatewayProxyEvent, context):\n super().handle(event, context)\n \n # custom logic\n\n\napi.rest.registry(Device, create=CreateDeviceAction)\n```\n\nYou can set custom handlers for each supported operation:\n\n```python\ndef registry(self, model, alias=None, get=GetAction, create=CreateAction, update=UpdateAction, delete=DeleteAction,\n lookup_list=ListAction, lookup_scan=ScanAction, lookup_query=QueryAction):\n```\n\nAs you can see, all actions are defined by default. That also means that all actions are enabled by default, but\neach action can be disabled.\n\nIf you need to disable action you need to set action handler to `None`, that will prevent framework from creating\nroute for given action, and it will disable whole logic behind it. \n\n### Routes\n\nREST API specific feature. \n\nFramework will create multiple routes for each register model, using `alias` as a URL namespace. \nGenerated routes: \n\n- GET /rest/{alias}/{pk} - fetch object by PK (see notes about PK below)\n- POST /rest/{alias} - create new record\n- PUT /rest/{alias}/{pk} - update record with given PK \n- DELETE /rest/{alias}/{pk} - delete record with given PK \n- GET /rest/lookup/{alias}/list - list all the records of given type using Query on the table\n- GET /rest/lookup/{alias}/list/{index_name} - list all the records of the given type using Query on specific index\n- POST /rest/lookup/{alias}/query - perform a query on given table\n- POST /rest/lookup/{alias}/query/{index_name} - perform a query on given index\n- POST /rest/lookup/{alias}/scan - perform a scan on given table\n- POST /rest/lookup/{alias}/scan/{index_name} - perform a scan on given index\n\n#### Primary Keys\n> *Please remember that with DynamoDB key is a Partition Key with optional Sort Key. \nIn case you define Sort Key DynamoDB will require a value for it while getting / deleting key.\nIn that case framework will modify routes to include sort key as an extra path parameter* \n\n\n## Endpoints\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Simple and powerful tool for quick serverless data management via API. ",
"version": "1.4.1",
"project_urls": {
"Homepage": "https://github.com/epsylabs/python-serverless-crud",
"Repository": "https://github.com/epsylabs/python-serverless-crud"
},
"split_keywords": [
"library",
"serverless"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2a79141cbaf3de8e50a0a58e7dd4a626ef17f06c37e56502a1393488c25b9f76",
"md5": "97f842d819672fc9289a511fbe9965d4",
"sha256": "a5aef395a2dff36f20a0ae5f1d8ac8a40af9a93116708661d7c0f0610c52a19e"
},
"downloads": -1,
"filename": "python_serverless_crud-1.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "97f842d819672fc9289a511fbe9965d4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 27500,
"upload_time": "2023-09-04T08:37:39",
"upload_time_iso_8601": "2023-09-04T08:37:39.774113Z",
"url": "https://files.pythonhosted.org/packages/2a/79/141cbaf3de8e50a0a58e7dd4a626ef17f06c37e56502a1393488c25b9f76/python_serverless_crud-1.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4f17470067b6f736ed06e882719456c7709762aa5e0c95de77d4455e6df739a0",
"md5": "cc884c8847e8fdd76b85b48f75547289",
"sha256": "57db31471134fc21f9746c86276440c64f69c3bc1c2834d14f55fdf285c75508"
},
"downloads": -1,
"filename": "python_serverless_crud-1.4.1.tar.gz",
"has_sig": false,
"md5_digest": "cc884c8847e8fdd76b85b48f75547289",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 20695,
"upload_time": "2023-09-04T08:37:40",
"upload_time_iso_8601": "2023-09-04T08:37:40.779848Z",
"url": "https://files.pythonhosted.org/packages/4f/17/470067b6f736ed06e882719456c7709762aa5e0c95de77d4455e6df739a0/python_serverless_crud-1.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-04 08:37:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "epsylabs",
"github_project": "python-serverless-crud",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "python-serverless-crud"
}
JSON
