[](https://travis-ci.com/github/neurobin/python-morm) [](https://coveralls.io/github/neurobin/python-morm?branch=release)
A minimal asynchronous database object relational mapper that supports transaction, connection pool and migration.
Currently supports *PostgreSQL* with `asyncpg`.
# Install
**Requires Python 3.10+**
```bash
pip install morm
```
# Init project
**Run `morm_admin init -p app` in your project directory to make some default files such as `_morm_config_.py`, `mgr.py`**
Edit *_morm_config_.py* to put the correct database credentials:
```python
from morm.db import Pool
DB_POOL = Pool(
dsn='postgres://',
host='localhost',
port=5432,
user='user',
password='pass',
database='db_name',
min_size=10,
max_size=90,
)
```
This will create and open an asyncpg pool which will be automatically closed at exit.
# Model
It's more than a good practice to define a Base model first:
```python
from morm.pg_models import BaseCommon as Model
# BaseCommon defines id, created_at and updated_at fields.
# While pg_models.Base defines only id.
class Base(Model):
class Meta:
abstract = True
```
Then a minimal model could look like this:
```python
from morm.fields import Field
class User(Base):
name = Field('varchar(65)')
email = Field('varchar(255)')
password = Field('varchar(255)')
```
Advanced models could look like this:
```python
import random
def get_rand():
return random.randint(1, 9)
class User(Base):
class Meta:
db_table = 'myapp_user'
abstract = False # default is False
proxy = False # default is False
# ... etc...
# see morm.meta.Meta for supported meta attributes.
name = Field('varchar(65)')
email = Field('varchar(255)')
password = Field('varchar(255)')
profession = Field('varchar(255)', default='Unknown')
random = Field('integer', default=get_rand) # function can be default
class UserProfile(User):
class Meta:
proxy = True
exclude_fields_down = ('password',) # exclude sensitive fields in retrieval
# this will also exclude this field from swagger docs if you are
# using our fastAPI framework
```
## Rules for field names
1. Must not start with an underscore (`_`). You can set arbitrary variables to the model instance with names starting with underscores; normally you can not set any variable to a model instance. Names not starting with an underscore are all expected to be field names, variables or methods that are defined during class definition.
2. `_<name>_` such constructions are reserved for pre-defined overridable methods such as `_pre_save_`, `_post_save_`, etc..
3. Name `Meta` is reserved to be a class that contains configuration of the model for both model and model instance.
## Initialize a model instance
keyword arguments initialize corresponding fields according to
the keys.
Positional arguments must be dictionaries of
keys and values.
Example:
```python
User(name='John Doe', profession='Teacher')
User({'name': 'John Doe', 'profession': 'Teacher'})
User({'name': 'John Doe', 'profession': 'Teacher'}, age=34)
```
## Validations
You can setup validation directly on the attribute or define a class method named `_clean_fieldname` to run a validation and change the value before it is inserted or updated into the db. These two types of validations work a bit differently:
1. **Validation on field attribute:** Can not change the value, must return True or False. It has more strict behavior than the `_clean_*` method for the attribute. This will run even when you are setting the value of an attribute by model instance, e.g `user.islive = 'live'` this would throw `ValueError` if you set the validator as `islive = Field('boolean', validator=lambda x: x is None or isinstance(x, bool))`.
2. **Validation with `_clean_{fieldName}` method:** Can change the value and must return the final value. It is only applied during insert or update using the model query handler (using `save` or `update` or `insert`).
Example:
```python
class User(Base):
class Meta:
db_table = 'myapp_user'
abstract = False # default is False
proxy = False # default is False
# ... etc...
# see morm.meta.Meta for supported meta attributes.
name = Field('varchar(65)')
email = Field('varchar(255)')
# restrict your devs to things such as user.password = '1234567' # <8 chars
password = Field('varchar(255)', validator=lambda x: x is None or len(x)>=8)
profession = Field('varchar(255)', default='Unknown')
random = Field('integer', default=get_rand) # function can be default
def _clean_password(self, v: str):
if not v: return v # password can be empty (e.g for third party login)
if len(v) < 8:
raise ValueError(f"Password must be at least 8 characters long.")
if len(v) > 100:
raise ValueError(f"Password must be at most 100 characters long.")
# password should contain at least one uppercase, one lowercase, one number, and one special character
if not any(c.isupper() for c in v):
raise ValueError(f"Password must contain at least one uppercase letter.")
if not any(c.islower() for c in v):
raise ValueError(f"Password must contain at least one lowercase letter.")
if not any(c.isdigit() for c in v):
raise ValueError(f"Password must contain at least one number.")
if not any(c in '!@#$%^&*()-_=+[]{}|;:,.<>?/~' for c in v):
raise ValueError(f"Password must contain at least one special character.")
return v
```
## Special Model Meta attribute `f`:
You can access field names from `ModelClass.Meta.f`.
This allows a spell-safe way to write the field names. If you
misspell the name, you will get `AttributeError`.
```python
f = User.Meta.f
my_data = {
f.name: 'John Doe', # safe from spelling mistake
f.profession: 'Teacher', # safe from spelling mistake
'hobby': 'Gardenning', # unsafe from spelling mistake
}
```
## Model Meta attributes
* `db_table` (*str*): db table name,
* `abstract` (*bool*): Whether it is an abstract model. Abstract models do not have db table and are used as base models.
* `pk` (*str*): Primary key. Defaults to 'id',
* `proxy` (*bool*): Whether it is a proxy model. Defaults to False. Proxy models inherit everything. This is only to have different pythonic behavior of a model. Proxy models can not define new fields and they do not have separate db table but share the same db table as their parents. Proxy setting is always inherited by child model, thus If you want to turn a child model non-proxy, set the proxy setting in its Meta class.
* `ordering` (*Tuple[str]*): Ordering. Example: `('name', '-price')`, where name is ascending and price is in descending order.
* `fields_up` (*Tuple[str]*): These fields only will be taken to update or save data onto db. Empty tuple means no restriction.
* `fields_down` (*Tuple[str]*): These fields only will be taken to select/retrieve data from db. Empty tuple means no restriction.
* `exclude_fields_up` (*Tuple[str]*): Exclude these fields when updating data to db. Empty tuple means no restriction.
* `exclude_fields_down` (*Tuple[str]*): Exclude these fields when retrieving data from db. Empty tuple means no restriction.
* `exclude_values_up` (*Dict[str, Tuple[Any]]*): Exclude fields with these values when updating. Empty dict and empty tuple means no restriction. Example: `{'': (None,), 'price': (0,)}` when field name is left empty ('') that criteria will be applied to all fields.
* `exclude_values_down` (*Dict[str, Tuple[Any]]*): Exclude fields with these values when retrieving data. Empty dict and empty tuple means no restriction. Example: `{'': (None,), 'price': (0,)}` when field name is left empty ('') that criteria will be applied to all fields.
* `f`: Access field names.
# CRUD
All available database operations are exposed through `DB` object.
Example:
```python
from morm.db import DB
db = DB(DB_POOL) # get a db handle.
# Create
user = User(name='John Doe', profession='Teacher')
await db.save(user)
# Read
user5 = await db(User).get(5)
# Update
user5.age = 30
await db.save(user5)
# Delete
await db.delete(user5)
```
## Get
The get method has the signature `get(*vals, col='', comp='=$1')`.
It gets the first row found by column and value. If `col` is not given, it defaults to the primary key (`pk`) of the model. If comparison is not given, it defaults to `=$1`
Example:
```python
from morm.db import DB
db = DB(DB_POOL) # get a db handle.
# get by pk:
user5 = await db(User).get(5)
# price between 5 and 2000
user = await db(User).get(5, 2000, col='price', comp='BETWEEN $1 AND $2')
```
## Filter
```python
from morm.db import DB
db = DB(DB_POOL) # get a db handle.
f = User.Meta.f
user_list = await db(User).qfilter().q(f'"{f.profession}"=$1', 'Teacher').fetch()
user_list = await db(User).qfilter().qc(f.profession, '=$1', 'Teacher').fetch()
```
It is safer to use `${qh.c}` instead of `$1`, `${qh.c+1}` instead of `$2`, etc.. :
```python
from morm.db import DB
db = DB(DB_POOL) # get a db handle.
qh = db(User)
user_list = await qh.qfilter()\
.q(f'{qh.f.profession} = ${qh.c} AND {qh.f.age} = ${qh.c+1}', 'Teacher', 30)\
.fetch()
```
# Query
Calling `db(Model)` gives you a model query handler which has several query methods to help you make queries.
Use `.q(query, *args)` method to make queries with positional arguments. If you want named arguments, use the uderscored version of these methods. For example, `q(query, *args)` has an underscored version `q_(query, *args, **kwargs)` that can take named arguments.
You can add a long query part by part:
```python
from morm.db import DB
db = DB(DB_POOL) # get a db handle.
qh = db(User) # get a query handle.
query, args = qh.q(f'SELECT * FROM {qh.db_table}')\
.q(f'WHERE {qh.f.profession} = ${qh.c}', 'Teacher')\
.q_(f'AND {qh.f.age} = :age', age=30)\
.getq()
print(query, args)
# fetch:
user_list = await qh.fetch()
```
The `q` family of methods (`q, qc, qu etc..`) can be used to
build a query step by step. These methods can be chained
together to break down the query building in multiple steps.
Several properties are available to get information of the model
such as:
1. `qh.db_table`: Quoted table name e.g `"my_user_table"`.
2. `qh.pk`: Quoted primary key name e.g `"id"`.
3. `qh.ordering`: ordering e.g `"price" ASC, "quantity" DESC`.
4. `qh.f.<field_name>`: quoted field names e.g`"profession"`.
5. `qh.c`: Current available position for positional argument (Instead of hardcoded `$1`, `$2`, use `f'${qh.c}'`, `f'${qh.c+1}'`).
`qh.c` is a counter that gives an integer representing the
last existing argument position plus 1.
`reset()` can be called to reset the query to start a new.
To execute a query, you need to run one of the execution methods
: `fetch, fetchrow, fetchval, execute`.
**Notable convenience methods:**
* `qupdate(data)`: Initialize a update query for data
* `qfilter()`: Initialize a filter query upto WHERE clasue.
* `get(pkval)`: Get an item by primary key.
# Transaction
```python
from morm.db import Transaction
async with Transaction(DB_POOL) as tdb:
# use tdb just like you use db
user6 = await tdb(User).get(6)
user6.age = 34
await tdb.save(user6)
user5 = await tdb(User).get(5)
user5.age = 34
await tdb.save(user5)
```
# Indexing
You can use the `index: Tuple[str] | str | None` parameter to define what type/s of indexing should be applied to the field. Examples:
```python
class User(Base):
parent_id = Field('integer', index='hash')
username = Field('varchar(65)', index='hash,btree') # two indexes
email = Field('varchar(255)', index=('hash', 'btree')) # tuple is allowed as well
perms = Field('integer[]', index='gin:gin__int_ops')
```
If you want to remove the indexing, Add a `-` minus sign to the specific index and then run migration. After that you can safely remove the index keyword, e.g:
```bash
--- parent_id = Field('integer', index='-hash')
===$ ./mgr makemigrations
===$ ./mgr migrate
>>> parent_id = Field('integer', index='') # now you can remove the hash
```
# Field/Model grouping
You can group your model fields, for example, you can define groups like `admin`, `mod`, `staff`, `normal` and make your model fields organized into these groups. This will enable you to implement complex field level organized access controls. You can say, that the `password` field belongs to the *admin* group, then `subscriptions` field to *mod* group and then `active_subscriptions` to *staff* group.
```python
class UserAdmin(Base):
class Meta:
groups = ('admin',) # this model belongs to the admin group
password = Field('varchar(100)', groups=('admin',))
subscriptions = Field('integer[]', groups=('mod',))
active_subscriptions = Field('integer[]', groups=('staff',))
```
# Sudo (Elevated access to fields)
We believe writing to certain fields or areas of your system should require elevated access.
`Field` can take an argument `sudo` that means **elevated access required**. IF `sudo` is set to true for some field, you will not be able to write to this field using the `ModelQuery` (direct raw query can still be performed) unless your db instance is set to have `sudo=True` as well:
```python
db = DB(DB_POOL, sudo=True)
```
# Migration
**Migration is a new feature and only forward migrations are supported as of now.**
You should have created the *_morm_config_.py* and *mgr.py* file with `morm_admin init`.
List all the models that you want migration for in *mgr.py*. You will know how to edit it once you open it.
Then, to make migration files, run:
```bash
python mgr.py makemigrations
```
This will ask you for confirmation on each changes, add `-y` flag to bypass this.
run
```bash
python mgr.py migrate
```
to apply the migrations.
## Adding data into migration
Go into migration directory after making the migration files and look for the `.py` files inside `queue` directory. Identify current migration files, open them for edit. You will find something similar to this:
```python
import morm
class MigrationRunner(morm.migration.MigrationRunner):
"""Run migration with pre and after steps.
"""
migration_query = """{migration_query}"""
# async def run_before(self):
# """Run before migration
# self.tdb is the db handle (transaction)
# self.model is the model class
# """
# dbm = self.tdb(self.model)
# # # Example
# # dbm.q('SOME QUERY TO SET "column_1"=$1', 'some_value')
# # await dbm.execute()
# # # etc..
# async def run_after(self):
# """Run after migration.
# self.tdb is the db handle (transaction)
# self.model is the model class
# """
# dbm = self.tdb(self.model)
# # # Example
# # dbm.q('SOME QUERY TO SET "column_1"=$1', 'some_value')
# # await dbm.execute()
# # # etc..
```
As you can see, there are `run_before` and `run_after` hooks. You can use them to make custom queries before and after the migration query. You can even modify the migration query itself.
Example:
```python
...
async def run_before(self):
"""Run before migration
self.tdb is the db handle (transaction)
self.model is the model class
"""
user0 = self.model(name='John Doe', profession='Software Engineer', age=45)
await self.tdb.save(user0)
...
```
# Do not do these
1. Do not delete migration files manually, use `python mgr.py delete_migration_files <start_index> <end_index>` command instead.
2. Do not modify mutable values in-place e.g `user.addresses.append('Some address')`, instead set the value: `user.addresses = [*user.addresses, 'Some address']` so that the `__setattr__` is called, on which `morm` depends for checking changed fields for the `db.save()` and related methods.
# Initialize a FastAPI project
Run `init_fap app` in your project root. It will create a directory structure like this:
```
├── app
│ ├── core
│ │ ├── __init__.py
│ │ ├── models
│ │ │ ├── base.py
│ │ │ ├── __init__.py
│ │ │ └── user.py
│ │ ├── schemas
│ │ │ └── __init__.py
│ │ └── settings.py
│ ├── __init__.py
│ ├── main.py
│ ├── tests
│ │ ├── __init__.py
│ │ └── v1
│ │ ├── __init__.py
│ │ └── test_sample.py
│ ├── v1
│ │ ├── dependencies
│ │ │ └── __init__.py
│ │ ├── __init__.py
│ │ ├── internal
│ │ │ └── __init__.py
│ │ └── routers
│ │ ├── __init__.py
│ │ └── root.py
│ └── workers.py
├── app.service
├── .gitignore
├── gunicorn.sh
├── mgr
├── mgr.py
├── _morm_config_.py
├── nginx
│ ├── app
│ └── default
├── requirements.txt
├── run
└── vact
```
You can run the dev app with `./run` or the production app with `./gunicorn.sh`.
To run the production app as a service with `systemctl start app`, copy the **app.service** to `/etc/systemd/system`
**Notes:**
* You can setup your venv path in the `vact` file. To activate the venv with all the environment vars, just run `. vact`.
* An environment file `.env_APP` is created in your home directory containing dev and production environments.
# Pydantic support
You can get pydantic model from any morm model using the `_pydantic_` method, e.g `User._pydantic_()` would give you the pydantic version of your `User` model. The `_pydantic_()` method supports a few parameters to customize the generated pydantic model:
* `up=False`: Defines if the model should be for up (update into database) or down (retrieval from database).
* `suffix=None`: You can add a suffix to the name of the generated pydantic model.
* `include_validators=None`: Whether the validators defined in each field (with validator parameter) should be added as pydantic validators. When `None` (which is default) validators will be included for data update into database (i.e for `up=True`). Note that, the model field validators return True or False, while pydantic validators return the value, this conversion is automatically added internally while generating the pydantic model.
If you are using our FastAPI framework, generating good docs for user data retrieval using the User model would be as simple as:
```python
@router.get('/crud/{model}', responses=Res.schema_all(User._pydantic_())
async def get(request: Request, model: str, vals = '', col: str='', comp: str='=$1'):
if some_authentication_error:
raise Res(status=Res.Status.unauthorized, errors=['Invalid Credentials!']) # throws a correct HTTP error with additional error message
...
return Res(user)
```
The above will define all common response types: 200, 401, 403, etc.. and the 200 success response will show an example with correct data types from your User model and will show only the fields that are allowed to be shown (controlled with `exclude_fields_down` or `fields_down` in the `User.Meta`).
# JSON handling
It may seem tempting to add json and jsonb support with `asyncpg.Connection.set_type_codec()` method, but we have not provided any option to use this method easily in `morm`, as it turned out to be making the queries very very slow. If you want to handle json, better add a `_clean_{field}` method in your model and do the conversion there:
```python
class User(Base):
settings = Field('jsonb')
...
def _clean_settings(self, v):
if not isinstance(v, str):
v = json.dumps(v)
return v
```
If you want to have it converted to json during data retrieval from database as well, pass a validator which should return False if it is not json, and then pass a modifier in the field to do the conversion. Do note that modifier only runs if validator fails. Thus you will set and get the value as json (list or dict) and the `_clean_settings` will covert it back to text during database insert or update.
```python
class User(Base):
settings = Field('jsonb', validator=lambda x: isinstance(x, list|dict), modifier=lambda x: json.loads(x))
...
def _clean_settings(self, v):
if not isinstance(v, str):
v = json.dumps(v)
return v
```
Raw data
{
"_id": null,
"home_page": "https://github.com/neurobin/python-morm",
"name": "morm",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10.0",
"maintainer_email": null,
"keywords": "async, orm, postgresql",
"author": "Md. Jahidul Hamid",
"author_email": "jahidulhamid@yahoo.com",
"download_url": "https://files.pythonhosted.org/packages/92/ff/16b4e032fae4b716ecb8ab39c4ce7f1eee4db735619dfd5af989139903dc/morm-2.7.1.tar.gz",
"platform": null,
"description": "[](https://travis-ci.com/github/neurobin/python-morm) [](https://coveralls.io/github/neurobin/python-morm?branch=release)\n\nA minimal asynchronous database object relational mapper that supports transaction, connection pool and migration.\n\nCurrently supports *PostgreSQL* with `asyncpg`.\n\n# Install\n\n**Requires Python 3.10+**\n\n```bash\npip install morm\n```\n\n# Init project\n\n**Run `morm_admin init -p app` in your project directory to make some default files such as `_morm_config_.py`, `mgr.py`**\n\nEdit *_morm_config_.py* to put the correct database credentials:\n\n```python\nfrom morm.db import Pool\n\nDB_POOL = Pool(\n dsn='postgres://',\n host='localhost',\n port=5432,\n user='user',\n password='pass',\n database='db_name',\n min_size=10,\n max_size=90,\n)\n```\n\nThis will create and open an asyncpg pool which will be automatically closed at exit.\n\n# Model\n\nIt's more than a good practice to define a Base model first:\n\n```python\nfrom morm.pg_models import BaseCommon as Model\n\n# BaseCommon defines id, created_at and updated_at fields.\n# While pg_models.Base defines only id.\n\nclass Base(Model):\n class Meta:\n abstract = True\n```\n\nThen a minimal model could look like this:\n\n```python\nfrom morm.fields import Field\n\nclass User(Base):\n name = Field('varchar(65)')\n email = Field('varchar(255)')\n password = Field('varchar(255)')\n```\n\nAdvanced models could look like this:\n\n```python\nimport random\n\ndef get_rand():\n return random.randint(1, 9)\n\nclass User(Base):\n class Meta:\n db_table = 'myapp_user'\n abstract = False # default is False\n proxy = False # default is False\n # ... etc...\n # see morm.meta.Meta for supported meta attributes.\n\n name = Field('varchar(65)')\n email = Field('varchar(255)')\n password = Field('varchar(255)')\n profession = Field('varchar(255)', default='Unknown')\n random = Field('integer', default=get_rand) # function can be default\n\nclass UserProfile(User):\n class Meta:\n proxy = True\n exclude_fields_down = ('password',) # exclude sensitive fields in retrieval\n # this will also exclude this field from swagger docs if you are\n # using our fastAPI framework\n```\n\n## Rules for field names\n\n1. Must not start with an underscore (`_`). You can set arbitrary variables to the model instance with names starting with underscores; normally you can not set any variable to a model instance. Names not starting with an underscore are all expected to be field names, variables or methods that are defined during class definition.\n2. `_<name>_` such constructions are reserved for pre-defined overridable methods such as `_pre_save_`, `_post_save_`, etc..\n3. Name `Meta` is reserved to be a class that contains configuration of the model for both model and model instance.\n\n\n## Initialize a model instance\n\nkeyword arguments initialize corresponding fields according to\nthe keys.\n\nPositional arguments must be dictionaries of\nkeys and values.\n\nExample:\n\n```python\nUser(name='John Doe', profession='Teacher')\nUser({'name': 'John Doe', 'profession': 'Teacher'})\nUser({'name': 'John Doe', 'profession': 'Teacher'}, age=34)\n```\n\n## Validations\n\nYou can setup validation directly on the attribute or define a class method named `_clean_fieldname` to run a validation and change the value before it is inserted or updated into the db. These two types of validations work a bit differently:\n\n1. **Validation on field attribute:** Can not change the value, must return True or False. It has more strict behavior than the `_clean_*` method for the attribute. This will run even when you are setting the value of an attribute by model instance, e.g `user.islive = 'live'` this would throw `ValueError` if you set the validator as `islive = Field('boolean', validator=lambda x: x is None or isinstance(x, bool))`.\n2. **Validation with `_clean_{fieldName}` method:** Can change the value and must return the final value. It is only applied during insert or update using the model query handler (using `save` or `update` or `insert`).\n\nExample:\n\n```python\nclass User(Base):\n class Meta:\n db_table = 'myapp_user'\n abstract = False # default is False\n proxy = False # default is False\n # ... etc...\n # see morm.meta.Meta for supported meta attributes.\n\n name = Field('varchar(65)')\n email = Field('varchar(255)')\n # restrict your devs to things such as user.password = '1234567' # <8 chars\n password = Field('varchar(255)', validator=lambda x: x is None or len(x)>=8)\n profession = Field('varchar(255)', default='Unknown')\n random = Field('integer', default=get_rand) # function can be default\n\n def _clean_password(self, v: str):\n if not v: return v # password can be empty (e.g for third party login)\n if len(v) < 8:\n raise ValueError(f\"Password must be at least 8 characters long.\")\n if len(v) > 100:\n raise ValueError(f\"Password must be at most 100 characters long.\")\n # password should contain at least one uppercase, one lowercase, one number, and one special character\n if not any(c.isupper() for c in v):\n raise ValueError(f\"Password must contain at least one uppercase letter.\")\n if not any(c.islower() for c in v):\n raise ValueError(f\"Password must contain at least one lowercase letter.\")\n if not any(c.isdigit() for c in v):\n raise ValueError(f\"Password must contain at least one number.\")\n if not any(c in '!@#$%^&*()-_=+[]{}|;:,.<>?/~' for c in v):\n raise ValueError(f\"Password must contain at least one special character.\")\n return v\n```\n\n## Special Model Meta attribute `f`:\n\nYou can access field names from `ModelClass.Meta.f`.\n\nThis allows a spell-safe way to write the field names. If you\nmisspell the name, you will get `AttributeError`.\n\n```python\nf = User.Meta.f\nmy_data = {\n f.name: 'John Doe', # safe from spelling mistake\n f.profession: 'Teacher', # safe from spelling mistake\n 'hobby': 'Gardenning', # unsafe from spelling mistake\n}\n```\n\n## Model Meta attributes\n\n\n* `db_table` (*str*): db table name,\n* `abstract` (*bool*): Whether it is an abstract model. Abstract models do not have db table and are used as base models.\n* `pk` (*str*): Primary key. Defaults to 'id',\n* `proxy` (*bool*): Whether it is a proxy model. Defaults to False. Proxy models inherit everything. This is only to have different pythonic behavior of a model. Proxy models can not define new fields and they do not have separate db table but share the same db table as their parents. Proxy setting is always inherited by child model, thus If you want to turn a child model non-proxy, set the proxy setting in its Meta class.\n* `ordering` (*Tuple[str]*): Ordering. Example: `('name', '-price')`, where name is ascending and price is in descending order.\n* `fields_up` (*Tuple[str]*): These fields only will be taken to update or save data onto db. Empty tuple means no restriction.\n* `fields_down` (*Tuple[str]*): These fields only will be taken to select/retrieve data from db. Empty tuple means no restriction.\n* `exclude_fields_up` (*Tuple[str]*): Exclude these fields when updating data to db. Empty tuple means no restriction.\n* `exclude_fields_down` (*Tuple[str]*): Exclude these fields when retrieving data from db. Empty tuple means no restriction.\n* `exclude_values_up` (*Dict[str, Tuple[Any]]*): Exclude fields with these values when updating. Empty dict and empty tuple means no restriction. Example: `{'': (None,), 'price': (0,)}` when field name is left empty ('') that criteria will be applied to all fields.\n* `exclude_values_down` (*Dict[str, Tuple[Any]]*): Exclude fields with these values when retrieving data. Empty dict and empty tuple means no restriction. Example: `{'': (None,), 'price': (0,)}` when field name is left empty ('') that criteria will be applied to all fields.\n* `f`: Access field names.\n\n# CRUD\n\nAll available database operations are exposed through `DB` object.\n\nExample:\n\n```python\nfrom morm.db import DB\n\ndb = DB(DB_POOL) # get a db handle.\n\n# Create\nuser = User(name='John Doe', profession='Teacher')\nawait db.save(user)\n\n# Read\nuser5 = await db(User).get(5)\n\n# Update\nuser5.age = 30\nawait db.save(user5)\n\n# Delete\nawait db.delete(user5)\n```\n\n## Get\n\nThe get method has the signature `get(*vals, col='', comp='=$1')`.\n\nIt gets the first row found by column and value. If `col` is not given, it defaults to the primary key (`pk`) of the model. If comparison is not given, it defaults to `=$1`\n\nExample:\n\n```python\nfrom morm.db import DB\n\ndb = DB(DB_POOL) # get a db handle.\n\n# get by pk:\nuser5 = await db(User).get(5)\n\n# price between 5 and 2000\nuser = await db(User).get(5, 2000, col='price', comp='BETWEEN $1 AND $2')\n```\n\n## Filter\n\n```python\nfrom morm.db import DB\n\ndb = DB(DB_POOL) # get a db handle.\n\nf = User.Meta.f\nuser_list = await db(User).qfilter().q(f'\"{f.profession}\"=$1', 'Teacher').fetch()\nuser_list = await db(User).qfilter().qc(f.profession, '=$1', 'Teacher').fetch()\n```\n\nIt is safer to use `${qh.c}` instead of `$1`, `${qh.c+1}` instead of `$2`, etc.. :\n\n```python\nfrom morm.db import DB\n\ndb = DB(DB_POOL) # get a db handle.\n\nqh = db(User)\nuser_list = await qh.qfilter()\\\n .q(f'{qh.f.profession} = ${qh.c} AND {qh.f.age} = ${qh.c+1}', 'Teacher', 30)\\\n .fetch()\n```\n\n# Query\n\nCalling `db(Model)` gives you a model query handler which has several query methods to help you make queries.\n\nUse `.q(query, *args)` method to make queries with positional arguments. If you want named arguments, use the uderscored version of these methods. For example, `q(query, *args)` has an underscored version `q_(query, *args, **kwargs)` that can take named arguments.\n\nYou can add a long query part by part:\n\n```python\nfrom morm.db import DB\n\ndb = DB(DB_POOL) # get a db handle.\nqh = db(User) # get a query handle.\n\nquery, args = qh.q(f'SELECT * FROM {qh.db_table}')\\\n .q(f'WHERE {qh.f.profession} = ${qh.c}', 'Teacher')\\\n .q_(f'AND {qh.f.age} = :age', age=30)\\\n .getq()\nprint(query, args)\n# fetch:\nuser_list = await qh.fetch()\n```\n\nThe `q` family of methods (`q, qc, qu etc..`) can be used to\nbuild a query step by step. These methods can be chained\ntogether to break down the query building in multiple steps.\n\nSeveral properties are available to get information of the model\nsuch as:\n\n1. `qh.db_table`: Quoted table name e.g `\"my_user_table\"`.\n2. `qh.pk`: Quoted primary key name e.g `\"id\"`.\n3. `qh.ordering`: ordering e.g `\"price\" ASC, \"quantity\" DESC`.\n4. `qh.f.<field_name>`: quoted field names e.g`\"profession\"`.\n5. `qh.c`: Current available position for positional argument (Instead of hardcoded `$1`, `$2`, use `f'${qh.c}'`, `f'${qh.c+1}'`).\n\n`qh.c` is a counter that gives an integer representing the\nlast existing argument position plus 1.\n\n`reset()` can be called to reset the query to start a new.\n\nTo execute a query, you need to run one of the execution methods\n: `fetch, fetchrow, fetchval, execute`.\n\n**Notable convenience methods:**\n\n* `qupdate(data)`: Initialize a update query for data\n* `qfilter()`: Initialize a filter query upto WHERE clasue.\n* `get(pkval)`: Get an item by primary key.\n\n\n# Transaction\n\n```python\nfrom morm.db import Transaction\n\nasync with Transaction(DB_POOL) as tdb:\n # use tdb just like you use db\n user6 = await tdb(User).get(6)\n user6.age = 34\n await tdb.save(user6)\n user5 = await tdb(User).get(5)\n user5.age = 34\n await tdb.save(user5)\n```\n\n# Indexing\n\nYou can use the `index: Tuple[str] | str | None` parameter to define what type/s of indexing should be applied to the field. Examples:\n\n```python\nclass User(Base):\n parent_id = Field('integer', index='hash')\n username = Field('varchar(65)', index='hash,btree') # two indexes\n email = Field('varchar(255)', index=('hash', 'btree')) # tuple is allowed as well\n perms = Field('integer[]', index='gin:gin__int_ops')\n```\n\nIf you want to remove the indexing, Add a `-` minus sign to the specific index and then run migration. After that you can safely remove the index keyword, e.g:\n\n```bash\n--- parent_id = Field('integer', index='-hash')\n===$ ./mgr makemigrations\n===$ ./mgr migrate\n>>> parent_id = Field('integer', index='') # now you can remove the hash\n```\n\n# Field/Model grouping\n\nYou can group your model fields, for example, you can define groups like `admin`, `mod`, `staff`, `normal` and make your model fields organized into these groups. This will enable you to implement complex field level organized access controls. You can say, that the `password` field belongs to the *admin* group, then `subscriptions` field to *mod* group and then `active_subscriptions` to *staff* group.\n\n```python\nclass UserAdmin(Base):\n class Meta:\n groups = ('admin',) # this model belongs to the admin group\n password = Field('varchar(100)', groups=('admin',))\n subscriptions = Field('integer[]', groups=('mod',))\n active_subscriptions = Field('integer[]', groups=('staff',))\n```\n\n# Sudo (Elevated access to fields)\n\nWe believe writing to certain fields or areas of your system should require elevated access.\n\n`Field` can take an argument `sudo` that means **elevated access required**. IF `sudo` is set to true for some field, you will not be able to write to this field using the `ModelQuery` (direct raw query can still be performed) unless your db instance is set to have `sudo=True` as well:\n\n```python\ndb = DB(DB_POOL, sudo=True)\n```\n\n# Migration\n\n**Migration is a new feature and only forward migrations are supported as of now.**\n\nYou should have created the *_morm_config_.py* and *mgr.py* file with `morm_admin init`.\n\nList all the models that you want migration for in *mgr.py*. You will know how to edit it once you open it.\n\nThen, to make migration files, run:\n\n```bash\npython mgr.py makemigrations\n```\n\nThis will ask you for confirmation on each changes, add `-y` flag to bypass this.\n\nrun\n\n```bash\npython mgr.py migrate\n```\n\nto apply the migrations.\n\n\n## Adding data into migration\n\nGo into migration directory after making the migration files and look for the `.py` files inside `queue` directory. Identify current migration files, open them for edit. You will find something similar to this:\n\n```python\nimport morm\n\nclass MigrationRunner(morm.migration.MigrationRunner):\n \"\"\"Run migration with pre and after steps.\n \"\"\"\n migration_query = \"\"\"{migration_query}\"\"\"\n\n # async def run_before(self):\n # \"\"\"Run before migration\n\n # self.tdb is the db handle (transaction)\n # self.model is the model class\n # \"\"\"\n # dbm = self.tdb(self.model)\n # # # Example\n # # dbm.q('SOME QUERY TO SET \"column_1\"=$1', 'some_value')\n # # await dbm.execute()\n # # # etc..\n\n # async def run_after(self):\n # \"\"\"Run after migration.\n\n # self.tdb is the db handle (transaction)\n # self.model is the model class\n # \"\"\"\n # dbm = self.tdb(self.model)\n # # # Example\n # # dbm.q('SOME QUERY TO SET \"column_1\"=$1', 'some_value')\n # # await dbm.execute()\n # # # etc..\n```\n\nAs you can see, there are `run_before` and `run_after` hooks. You can use them to make custom queries before and after the migration query. You can even modify the migration query itself.\n\nExample:\n\n```python\n...\n async def run_before(self):\n \"\"\"Run before migration\n\n self.tdb is the db handle (transaction)\n self.model is the model class\n \"\"\"\n user0 = self.model(name='John Doe', profession='Software Engineer', age=45)\n await self.tdb.save(user0)\n...\n```\n\n# Do not do these\n\n1. Do not delete migration files manually, use `python mgr.py delete_migration_files <start_index> <end_index>` command instead.\n2. Do not modify mutable values in-place e.g `user.addresses.append('Some address')`, instead set the value: `user.addresses = [*user.addresses, 'Some address']` so that the `__setattr__` is called, on which `morm` depends for checking changed fields for the `db.save()` and related methods.\n\n# Initialize a FastAPI project\n\nRun `init_fap app` in your project root. It will create a directory structure like this:\n\n```\n\u251c\u2500\u2500 app\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 core\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 models\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 base.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 user.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 schemas\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 settings.py\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 main.py\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 tests\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 v1\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 test_sample.py\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 v1\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 dependencies\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 internal\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 routers\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 root.py\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 workers.py\n\u251c\u2500\u2500 app.service\n\u251c\u2500\u2500 .gitignore\n\u251c\u2500\u2500 gunicorn.sh\n\u251c\u2500\u2500 mgr\n\u251c\u2500\u2500 mgr.py\n\u251c\u2500\u2500 _morm_config_.py\n\u251c\u2500\u2500 nginx\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 app\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 default\n\u251c\u2500\u2500 requirements.txt\n\u251c\u2500\u2500 run\n\u2514\u2500\u2500 vact\n```\n\nYou can run the dev app with `./run` or the production app with `./gunicorn.sh`.\n\nTo run the production app as a service with `systemctl start app`, copy the **app.service** to `/etc/systemd/system`\n\n**Notes:**\n\n* You can setup your venv path in the `vact` file. To activate the venv with all the environment vars, just run `. vact`.\n* An environment file `.env_APP` is created in your home directory containing dev and production environments.\n\n\n# Pydantic support\n\nYou can get pydantic model from any morm model using the `_pydantic_` method, e.g `User._pydantic_()` would give you the pydantic version of your `User` model. The `_pydantic_()` method supports a few parameters to customize the generated pydantic model:\n\n* `up=False`: Defines if the model should be for up (update into database) or down (retrieval from database).\n* `suffix=None`: You can add a suffix to the name of the generated pydantic model.\n* `include_validators=None`: Whether the validators defined in each field (with validator parameter) should be added as pydantic validators. When `None` (which is default) validators will be included for data update into database (i.e for `up=True`). Note that, the model field validators return True or False, while pydantic validators return the value, this conversion is automatically added internally while generating the pydantic model.\n\nIf you are using our FastAPI framework, generating good docs for user data retrieval using the User model would be as simple as:\n\n```python\n@router.get('/crud/{model}', responses=Res.schema_all(User._pydantic_())\nasync def get(request: Request, model: str, vals = '', col: str='', comp: str='=$1'):\n if some_authentication_error:\n raise Res(status=Res.Status.unauthorized, errors=['Invalid Credentials!']) # throws a correct HTTP error with additional error message\n ...\n return Res(user)\n```\n\nThe above will define all common response types: 200, 401, 403, etc.. and the 200 success response will show an example with correct data types from your User model and will show only the fields that are allowed to be shown (controlled with `exclude_fields_down` or `fields_down` in the `User.Meta`).\n\n\n# JSON handling\n\nIt may seem tempting to add json and jsonb support with `asyncpg.Connection.set_type_codec()` method, but we have not provided any option to use this method easily in `morm`, as it turned out to be making the queries very very slow. If you want to handle json, better add a `_clean_{field}` method in your model and do the conversion there:\n\n```python\nclass User(Base):\n settings = Field('jsonb')\n ...\n\n def _clean_settings(self, v):\n if not isinstance(v, str):\n v = json.dumps(v)\n return v\n```\n\nIf you want to have it converted to json during data retrieval from database as well, pass a validator which should return False if it is not json, and then pass a modifier in the field to do the conversion. Do note that modifier only runs if validator fails. Thus you will set and get the value as json (list or dict) and the `_clean_settings` will covert it back to text during database insert or update.\n\n```python\nclass User(Base):\n settings = Field('jsonb', validator=lambda x: isinstance(x, list|dict), modifier=lambda x: json.loads(x))\n ...\n\n def _clean_settings(self, v):\n if not isinstance(v, str):\n v = json.dumps(v)\n return v\n```\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "A minimal asynchronous database object relational mapper",
"version": "2.7.1",
"project_urls": {
"Homepage": "https://github.com/neurobin/python-morm"
},
"split_keywords": [
"async",
" orm",
" postgresql"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1e3b79daec57512dd699bc97d9c28aefa791b211f0a916cc71dbfbc5982b6ed1",
"md5": "5424e5af1b0bf86193ac9871915942c9",
"sha256": "277e7560d3b6c9335636e69c9c949aad09e3fa8136eb3fd9fe210519d8b497e1"
},
"downloads": -1,
"filename": "morm-2.7.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5424e5af1b0bf86193ac9871915942c9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10.0",
"size": 56275,
"upload_time": "2024-11-23T11:50:55",
"upload_time_iso_8601": "2024-11-23T11:50:55.923928Z",
"url": "https://files.pythonhosted.org/packages/1e/3b/79daec57512dd699bc97d9c28aefa791b211f0a916cc71dbfbc5982b6ed1/morm-2.7.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "92ff16b4e032fae4b716ecb8ab39c4ce7f1eee4db735619dfd5af989139903dc",
"md5": "033370ab50f4582d72ee05ba49e915b3",
"sha256": "655314c19e922b4dab28b607c4ad6ee4e645f8aba68111db62f1fbeaff0b834b"
},
"downloads": -1,
"filename": "morm-2.7.1.tar.gz",
"has_sig": false,
"md5_digest": "033370ab50f4582d72ee05ba49e915b3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10.0",
"size": 67340,
"upload_time": "2024-11-23T11:50:57",
"upload_time_iso_8601": "2024-11-23T11:50:57.576655Z",
"url": "https://files.pythonhosted.org/packages/92/ff/16b4e032fae4b716ecb8ab39c4ce7f1eee4db735619dfd5af989139903dc/morm-2.7.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-23 11:50:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "neurobin",
"github_project": "python-morm",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "morm"
}
JSON
