Django RQL
==========
[](https://pypi.org/project/django-rql/)
[](https://pypi.org/project/django-rql/)
[](https://pypi.org/project/django-rql/)
[](https://readthedocs.org/projects/django-rql)
[](https://github.com/cloudblue/django-rql/actions)
[](https://sonarcloud.io/summary/new_code?id=django-rql)
[](https://sonarcloud.io/summary/new_code?id=django-rql)
[](https://pypi.org/project/django-rql/)
`django-rql` is the Django app, that adds RQL filtering to your application.
This library is based on core [lib-rql](https://github.com/cloudblue/lib-rql) library.
RQL
---
RQL (Resource query language) is designed for modern application development. It is built for the web, ready for NoSQL, and highly extensible with simple syntax.
This is a query language fast and convenient database interaction. RQL was designed for use in URLs to request object-style data structures.
[RQL Reference](https://connect.cloudblue.com/community/api/rql/)
Currently supported operators
=============================
1. Comparison (eq, ne, gt, ge, lt, le, like, ilike, search)
2. List (in, out)
3. Logical (and, or, not)
4. Constants (null(), empty())
5. Ordering (ordering)
6. Select (select)
7. Tuple (t)
Documentation
=============
Full documentation is available at [https://django-rql.readthedocs.org](https://django-rql.readthedocs.org).
Example
=======
```python
from dj_rql.filter_cls import RQLFilterClass, RQL_NULL
from py_rql.constants import FilterLookups
class ModelFilterClass(RQLFilterClass):
"""
MODEL - Django ORM model
FILTERS - List of filters
EXTENDED_SEARCH_ORM_ROUTES - List of additional Django ORM fields for search
DISTINCT - Boolean flag, that specifies if queryset must always be DISTINCT
SELECT - Boolean flag, that specifies if Filter Class supports select operations and queryset optimizations
OPENAPI_SPECIFICATION - Python class that renders OpenAPI specification
MAX_ORDERING_LENGTH_IN_QUERY - Integer max allowed number of provided ordering filters in query ordering expression
ALLOWED_ORDERING_PERMUTATIONS_IN_QUERY - Set of tuples of strings to specify a set of allowed ordering permutations
Filters can be set in two ways:
1) string (default settings are calculated from ORM)
2) dict (overriding settings for specific cases)
Filter Dict Structure
{
'filter': str
# or
'namespace': str
'source': str
# or
'sources': iterable
# or
'custom': bool
# or
'dynamic': bool
'field': obj
'lookups': set
'qs': obj
'use_repr': bool # can't be used in namespaces
'ordering': bool # can't be true if 'use_repr=True'
'search': bool # can't be true if 'use_repr=True'
'hidden': bool
}
"""
MODEL = Model
FILTERS = ['id', {
# `null_values` can be set to override ORM is_null behaviour
# RQL_NULL is the default value if NULL lookup is supported by field
'filter': 'title',
'null_values': {RQL_NULL, 'NULL_ID'},
'ordering': False,
}, {
# `ordering` can be set to True, if filter must support ordering (sorting)
# `ordering` can't be applied to non-db fields
'filter': 'status',
'ordering': True,
}, {
# `search` must be set to True for filter to be used in searching
# `search` must be applied only to text db-fields, which have ilike lookup
'filter': 'author__email',
'search': True,
}, {
# `source` must be set when filter name doesn't match ORM path
'filter': 'name',
'source': 'author__name',
}, {
# `namespace` is useful for API consistency, when dealing with related models
'namespace': 'author',
'filters': ['id', 'name'], # will be converted to `author.id` and `author.name`
},{
# `distinct` needs to be setup for filters that require QS to work in DISTINCT mode
# `openapi` configuration is automatically collected by OpenAPI autogenerator
'filter': 'published.at',
'source': 'published_at',
'distinct': True,
'openapi': {
'required': True,
'deprecated': True,
'description': 'Good description',
'hidden': False, # can be set to avoid collecting by autogenerator
# type and format are collected automatically and shouldn't be setup, in general
'type': 'string',
'format': 'date',
},
}, {
# `use_repr` flag is used to filter by choice representations
'filter': 'rating.blog',
'source': 'blog_rating',
'use_repr': True,
}, {
# `hidden` flag is used to set default select behaviour for associated field
'filter': 'rating.blog_int',
'source': 'blog_rating',
'use_repr': False,
'ordering': True,
'hidden': True,
}, {
# We can change default lookups for a certain filter
'filter': 'amazon_rating',
'lookups': {FilterLookups.GE, FilterLookups.LT},
}, {
# Sometimes it's needed to filter by several sources at once (distinct is always True).
# F.e. this could be helpful for searching.
'filter': 'd_id',
'sources': {'id', 'author__id'},
'ordering': True,
}, {
# Some fields may have no DB representation or non-typical ORM filtering
# `custom` option must be set to True for such fields
'filter': 'custom_filter',
'custom': True,
'lookups': {FilterLookups.EQ, FilterLookups.IN, FilterLookups.I_LIKE},
'ordering': True,
'search': True,
# Optional ORM field for query parameter value validation
'field': IntegerField(),
'custom_data': [1],
}]
from dj_rql.drf.backend import RQLFilterBackend
from dj_rql.drf.paginations import RQLContentRangeLimitOffsetPagination
class DRFViewSet(mixins.ListModelMixin, GenericViewSet):
queryset = MODEL.objects.all()
serializer_class = ModelSerializer
rql_filter_class = ModelFilterClass
pagination_class = RQLContentRangeLimitOffsetPagination
filter_backends = (RQLFilterBackend,)
```
Notes
=====
0. Values with whitespaces or special characters, like ',' need to have "" or ''
1. Supported date format is ISO8601: 2019-02-12
2. Supported datetime format is ISO8601: 2019-02-12T10:02:00 / 2019-02-12T10:02Z / 2019-02-12T10:02:00+03:00
3. Support for Choices() fields from [Django Model Utilities](https://django-model-utils.readthedocs.io/en/latest/utilities.html#choices) is added
4. Library supports [caching with different strategies](https://cachetools.readthedocs.io/en/stable/#cache-implementations) for queryset building, which can be very useful for collections, which use `select()`.
> Queryset execution result (filtered data) is NOT cached (!), only queryset building is cached.
```python
from dj_rql.filter_cls import RQLFilterClass
from cachetools import LRUCache
class MyFilterClass(RQLFilterClass):
SELECT = True
QUERIES_CACHE_BACKEND = LRUCache
QUERIES_CACHE_SIZE = 100
```
Helpers
================================
There is a Django command `generate_rql_class` to decrease development and integration efforts for filtering.
This command automatically generates a filter class for a given model with all relations and all optimizations (!) to the specified depth.
Example
-------
```commandline
django-admin generate_rql_class --settings=tests.dj_rf.settings tests.dj_rf.models.Publisher --depth=1 --exclude=authors,fk2
```
This command for the model `Publisher` from tests package will produce the following output to stdout:
```python
from tests.dj_rf.models import Publisher
from dj_rql.filter_cls import RQLFilterClass
from dj_rql.qs import NSR
class PublisherFilters(RQLFilterClass):
MODEL = Publisher
SELECT = True
EXCLUDE_FILTERS = ['authors', 'fk2']
FILTERS = [
{
"filter": "id",
"ordering": True,
"search": False
},
{
"filter": "name",
"ordering": True,
"search": True
},
{
"namespace": "fk1",
"filters": [
{
"filter": "id",
"ordering": True,
"search": False
}
],
"qs": NSR('fk1')
}
]
```
Django Rest Framework Extensions
================================
1. Pagination (limit, offset)
2. Support for custom fields, inherited at any depth from basic model fields, like CharField().
3. Backend `DjangoFiltersRQLFilterBackend` with automatic conversion of [Django-Filters](https://django-filter.readthedocs.io/en/master/) query to RQL query.
4. OpenAPI docs are autogenerated for filter classes.
Best Practices
==============
1. Use `dj_rql.utils.assert_filter_cls` to test your API view filters. If the mappings are correct and there is no custom filtering logic, then it's practically guaranteed, that filtering will work correctly.
2. Prefer using `custom=True` with `RQLFilterClass.build_q_for_custom_filter` overriding over overriding `RQLFilterClass.build_q_for_filter`.
3. Custom filters may support ordering (`ordering=True`) with `build_name_for_custom_ordering`.
4. Django JSON fields can't be used as namespaces currently, but can be supported via `dynamic=True`, for example:
```python
{
'filter': 'json_data.key',
'source': 'json_data__key',
'dynamic': True,
'field': CharField(null=True),
},
```
Development
===========
1. Python 3.8+
2. Install poetry: `pip install poetry`
3. Install dependencies: `poetry install`
4. We use `isort` library to order and format our imports, and we check it using `flake8-isort` library (automatically on `flake8` run).
For convenience you may run `isort .` to order imports.
5. Run flake8: `poetry run flake8`
Testing
=======
1. Python 3.8+
2. Install poetry: `pip install poetry`
3. Install dependencies: `poetry install`
Check code style: `poetry run flake8`
Run tests: `poetry run pytest`
Tests reports are generated in `tests/reports`.
* `out.xml` - JUnit test results
* `coverage.xml` - Coverage xml results
To generate HTML coverage reports use:
`--cov-report html:tests/reports/cov_html`
Raw data
{
"_id": null,
"home_page": "https://connect.cloudblue.com/community/api/rql/",
"name": "django-rql",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4",
"maintainer_email": "",
"keywords": "django,rql,filter,rest,api",
"author": "Ingram Micro",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/c7/84/914c73100cd34071fd81a109ebafef37272d8db1b39495f48b8020b0dc5a/django_rql-4.4.0.tar.gz",
"platform": null,
"description": "Django RQL\n==========\n[](https://pypi.org/project/django-rql/)\n[](https://pypi.org/project/django-rql/)\n[](https://pypi.org/project/django-rql/)\n[](https://readthedocs.org/projects/django-rql) \n[](https://github.com/cloudblue/django-rql/actions)\n[](https://sonarcloud.io/summary/new_code?id=django-rql)\n[](https://sonarcloud.io/summary/new_code?id=django-rql)\n[](https://pypi.org/project/django-rql/)\n\n`django-rql` is the Django app, that adds RQL filtering to your application.\nThis library is based on core [lib-rql](https://github.com/cloudblue/lib-rql) library.\n\n\nRQL\n---\n\nRQL (Resource query language) is designed for modern application development. It is built for the web, ready for NoSQL, and highly extensible with simple syntax.\nThis is a query language fast and convenient database interaction. RQL was designed for use in URLs to request object-style data structures.\n\n[RQL Reference](https://connect.cloudblue.com/community/api/rql/)\n\n\nCurrently supported operators\n=============================\n1. Comparison (eq, ne, gt, ge, lt, le, like, ilike, search)\n2. List (in, out)\n3. Logical (and, or, not)\n4. Constants (null(), empty())\n5. Ordering (ordering)\n6. Select (select)\n7. Tuple (t)\n\n\nDocumentation\n=============\n\nFull documentation is available at [https://django-rql.readthedocs.org](https://django-rql.readthedocs.org).\n\n\nExample\n=======\n\n```python\nfrom dj_rql.filter_cls import RQLFilterClass, RQL_NULL\n\nfrom py_rql.constants import FilterLookups\n\n\nclass ModelFilterClass(RQLFilterClass):\n \"\"\"\n MODEL - Django ORM model\n FILTERS - List of filters\n EXTENDED_SEARCH_ORM_ROUTES - List of additional Django ORM fields for search\n DISTINCT - Boolean flag, that specifies if queryset must always be DISTINCT\n SELECT - Boolean flag, that specifies if Filter Class supports select operations and queryset optimizations\n OPENAPI_SPECIFICATION - Python class that renders OpenAPI specification\n MAX_ORDERING_LENGTH_IN_QUERY - Integer max allowed number of provided ordering filters in query ordering expression\n ALLOWED_ORDERING_PERMUTATIONS_IN_QUERY - Set of tuples of strings to specify a set of allowed ordering permutations\n\n Filters can be set in two ways:\n 1) string (default settings are calculated from ORM)\n 2) dict (overriding settings for specific cases)\n\n Filter Dict Structure\n {\n 'filter': str\n # or\n 'namespace': str\n\n 'source': str\n # or\n 'sources': iterable\n # or\n 'custom': bool\n # or\n 'dynamic': bool\n 'field': obj\n\n 'lookups': set\n\n 'qs': obj\n\n 'use_repr': bool # can't be used in namespaces\n 'ordering': bool # can't be true if 'use_repr=True'\n 'search': bool # can't be true if 'use_repr=True'\n 'hidden': bool\n }\n\n \"\"\"\n MODEL = Model\n FILTERS = ['id', {\n # `null_values` can be set to override ORM is_null behaviour\n # RQL_NULL is the default value if NULL lookup is supported by field\n 'filter': 'title',\n 'null_values': {RQL_NULL, 'NULL_ID'},\n 'ordering': False,\n }, {\n # `ordering` can be set to True, if filter must support ordering (sorting)\n # `ordering` can't be applied to non-db fields\n 'filter': 'status',\n 'ordering': True,\n }, {\n # `search` must be set to True for filter to be used in searching\n # `search` must be applied only to text db-fields, which have ilike lookup\n 'filter': 'author__email',\n 'search': True,\n }, {\n # `source` must be set when filter name doesn't match ORM path\n 'filter': 'name',\n 'source': 'author__name',\n }, {\n # `namespace` is useful for API consistency, when dealing with related models\n 'namespace': 'author',\n 'filters': ['id', 'name'], # will be converted to `author.id` and `author.name`\n },{\n # `distinct` needs to be setup for filters that require QS to work in DISTINCT mode\n # `openapi` configuration is automatically collected by OpenAPI autogenerator\n 'filter': 'published.at',\n 'source': 'published_at',\n 'distinct': True,\n 'openapi': {\n 'required': True,\n 'deprecated': True,\n 'description': 'Good description',\n 'hidden': False, # can be set to avoid collecting by autogenerator\n # type and format are collected automatically and shouldn't be setup, in general\n 'type': 'string',\n 'format': 'date',\n },\n }, {\n # `use_repr` flag is used to filter by choice representations\n 'filter': 'rating.blog',\n 'source': 'blog_rating',\n 'use_repr': True,\n }, {\n # `hidden` flag is used to set default select behaviour for associated field\n 'filter': 'rating.blog_int',\n 'source': 'blog_rating',\n 'use_repr': False,\n 'ordering': True,\n 'hidden': True,\n }, {\n # We can change default lookups for a certain filter\n 'filter': 'amazon_rating',\n 'lookups': {FilterLookups.GE, FilterLookups.LT},\n }, {\n # Sometimes it's needed to filter by several sources at once (distinct is always True).\n # F.e. this could be helpful for searching.\n 'filter': 'd_id',\n 'sources': {'id', 'author__id'},\n 'ordering': True,\n }, {\n # Some fields may have no DB representation or non-typical ORM filtering\n # `custom` option must be set to True for such fields\n 'filter': 'custom_filter',\n 'custom': True,\n 'lookups': {FilterLookups.EQ, FilterLookups.IN, FilterLookups.I_LIKE},\n 'ordering': True,\n 'search': True,\n # Optional ORM field for query parameter value validation\n 'field': IntegerField(), \n\n 'custom_data': [1],\n }]\n\n\nfrom dj_rql.drf.backend import RQLFilterBackend\nfrom dj_rql.drf.paginations import RQLContentRangeLimitOffsetPagination\n\n\nclass DRFViewSet(mixins.ListModelMixin, GenericViewSet):\n queryset = MODEL.objects.all()\n serializer_class = ModelSerializer\n rql_filter_class = ModelFilterClass\n pagination_class = RQLContentRangeLimitOffsetPagination\n filter_backends = (RQLFilterBackend,)\n```\n\nNotes\n=====\n0. Values with whitespaces or special characters, like ',' need to have \"\" or ''\n1. Supported date format is ISO8601: 2019-02-12\n2. Supported datetime format is ISO8601: 2019-02-12T10:02:00 / 2019-02-12T10:02Z / 2019-02-12T10:02:00+03:00\n3. Support for Choices() fields from [Django Model Utilities](https://django-model-utils.readthedocs.io/en/latest/utilities.html#choices) is added\n4. Library supports [caching with different strategies](https://cachetools.readthedocs.io/en/stable/#cache-implementations) for queryset building, which can be very useful for collections, which use `select()`.\n> Queryset execution result (filtered data) is NOT cached (!), only queryset building is cached.\n\n```python\nfrom dj_rql.filter_cls import RQLFilterClass\n\nfrom cachetools import LRUCache\n\nclass MyFilterClass(RQLFilterClass):\n SELECT = True\n QUERIES_CACHE_BACKEND = LRUCache\n QUERIES_CACHE_SIZE = 100\n```\n\nHelpers\n================================\nThere is a Django command `generate_rql_class` to decrease development and integration efforts for filtering.\nThis command automatically generates a filter class for a given model with all relations and all optimizations (!) to the specified depth.\n\nExample\n-------\n```commandline\ndjango-admin generate_rql_class --settings=tests.dj_rf.settings tests.dj_rf.models.Publisher --depth=1 --exclude=authors,fk2\n```\nThis command for the model `Publisher` from tests package will produce the following output to stdout:\n```python\nfrom tests.dj_rf.models import Publisher\n\nfrom dj_rql.filter_cls import RQLFilterClass\nfrom dj_rql.qs import NSR\n\n\nclass PublisherFilters(RQLFilterClass):\n MODEL = Publisher\n SELECT = True\n EXCLUDE_FILTERS = ['authors', 'fk2']\n FILTERS = [\n {\n \"filter\": \"id\",\n \"ordering\": True,\n \"search\": False\n },\n {\n \"filter\": \"name\",\n \"ordering\": True,\n \"search\": True\n },\n {\n \"namespace\": \"fk1\",\n \"filters\": [\n {\n \"filter\": \"id\",\n \"ordering\": True,\n \"search\": False\n }\n ],\n \"qs\": NSR('fk1')\n }\n]\n\n```\n\n\nDjango Rest Framework Extensions\n================================\n1. Pagination (limit, offset)\n2. Support for custom fields, inherited at any depth from basic model fields, like CharField().\n3. Backend `DjangoFiltersRQLFilterBackend` with automatic conversion of [Django-Filters](https://django-filter.readthedocs.io/en/master/) query to RQL query.\n4. OpenAPI docs are autogenerated for filter classes.\n\nBest Practices\n==============\n1. Use `dj_rql.utils.assert_filter_cls` to test your API view filters. If the mappings are correct and there is no custom filtering logic, then it's practically guaranteed, that filtering will work correctly.\n2. Prefer using `custom=True` with `RQLFilterClass.build_q_for_custom_filter` overriding over overriding `RQLFilterClass.build_q_for_filter`.\n3. Custom filters may support ordering (`ordering=True`) with `build_name_for_custom_ordering`.\n4. Django JSON fields can't be used as namespaces currently, but can be supported via `dynamic=True`, for example:\n```python\n{\n 'filter': 'json_data.key',\n 'source': 'json_data__key',\n 'dynamic': True,\n 'field': CharField(null=True),\n},\n```\n\nDevelopment\n===========\n\n1. Python 3.8+\n2. Install poetry: `pip install poetry`\n3. Install dependencies: `poetry install`\n4. We use `isort` library to order and format our imports, and we check it using `flake8-isort` library (automatically on `flake8` run). \nFor convenience you may run `isort .` to order imports.\n5. Run flake8: `poetry run flake8`\n\nTesting\n=======\n\n1. Python 3.8+\n2. Install poetry: `pip install poetry`\n3. Install dependencies: `poetry install`\n\nCheck code style: `poetry run flake8`\nRun tests: `poetry run pytest`\n\nTests reports are generated in `tests/reports`.\n* `out.xml` - JUnit test results\n* `coverage.xml` - Coverage xml results\n\nTo generate HTML coverage reports use:\n`--cov-report html:tests/reports/cov_html`\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Django RQL Filtering",
"version": "4.4.0",
"project_urls": {
"Homepage": "https://connect.cloudblue.com/community/api/rql/",
"Repository": "https://github.com/cloudblue/django-rql"
},
"split_keywords": [
"django",
"rql",
"filter",
"rest",
"api"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e25b5a64e3bed3bef62b4484e924e8190d47ad0d569563ca4d4f0aea47a831d1",
"md5": "c69ab5b29679236043c6f5b876b59b99",
"sha256": "b9b7ff13f373860148e40e8a2372da8cfeb8490538e350fe6bd37dd1dbff1d6e"
},
"downloads": -1,
"filename": "django_rql-4.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c69ab5b29679236043c6f5b876b59b99",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4",
"size": 36907,
"upload_time": "2023-06-01T06:51:11",
"upload_time_iso_8601": "2023-06-01T06:51:11.171167Z",
"url": "https://files.pythonhosted.org/packages/e2/5b/5a64e3bed3bef62b4484e924e8190d47ad0d569563ca4d4f0aea47a831d1/django_rql-4.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c784914c73100cd34071fd81a109ebafef37272d8db1b39495f48b8020b0dc5a",
"md5": "9ef9a81bc0c1b2779c53db8299408c92",
"sha256": "a579f0964f63a0fcbc4728a765c7327d6d59d5f7760b972ff872431e0d3002a2"
},
"downloads": -1,
"filename": "django_rql-4.4.0.tar.gz",
"has_sig": false,
"md5_digest": "9ef9a81bc0c1b2779c53db8299408c92",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4",
"size": 35243,
"upload_time": "2023-06-01T06:51:13",
"upload_time_iso_8601": "2023-06-01T06:51:13.044396Z",
"url": "https://files.pythonhosted.org/packages/c7/84/914c73100cd34071fd81a109ebafef37272d8db1b39495f48b8020b0dc5a/django_rql-4.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-01 06:51:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cloudblue",
"github_project": "django-rql",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "django-rql"
}
JSON
