=======================================
Datalookup - Deep nested data filtering
=======================================
.. image:: https://img.shields.io/badge/python-3.9-blue.svg
:target: https://github.com/pyshare/datalookup
.. image:: https://github.com/pyshare/datalookup/actions/workflows/tests.yml/badge.svg
:target: https://github.com/pyshare/datalookup/actions?query=workflow%3APython%20testing
.. image:: https://codecov.io/gh/pyshare/datalookup/branch/master/graph/badge.svg
:target: https://codecov.io/gh/pyshare/datalookup
:alt: Code coverage Status
.. image:: https://github.com/pyshare/datalookup/actions/workflows/linters.yml/badge.svg
:target: https://github.com/pyshare/datalookup/actions?query=workflow%3APython%20linting
.. image:: https://readthedocs.org/projects/datalookup/badge/?version=latest
:target: https://datalookup.readthedocs.io/en/latest/
:alt: Documentation Status
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
----
The **Datalookup** library makes it easier to filter and manipulate your data. The
module is inspired by the Django Queryset Api and it's lookups.
Installation
============
.. code-block:: console
$ pip install datalookup
Example
=======
Throughout the below examples, we'll refer to the following data,
which comprise a list of authors with the books they wrote.
.. code-block:: python
data = [
{
"id": 1,
"author": "J. K. Rowling",
"books": [
{
"name": "Harry Potter and the Chamber of Secrets",
"genre": "Fantasy",
"published": "1998"
},
{
"name": "Harry Potter and the Prisoner of Azkaban",
"genre": "Fantasy",
"published": "1999"
}
]
},
{
"id": 2,
"author": "Agatha Christie",
"books": [
{
"name": "And Then There Were None",
"genre": "Mystery",
"published": "1939"
}
]
}
]
**Datalookup** makes it easy to find an author by calling one of the methods
of the ``Dataset`` class like ``filter()`` or ``exclude()``. There
are multiple ways to retrieve an author.
Basic filtering
---------------
Use one of the ``field`` of your author dictionary to filter your data.
.. code-block:: python
from datalookup import Dataset
# Use Dataset to manipulate and filter your data
books = Dataset(data)
# Retrieve an author using the author name
authors = books.filter(author="J. K. Rowling")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
# Retrieve an author using '__in' lookup
authors = books.filter(id__in=[2, 3])
assert len(authors) == 1
assert authors[0].author == "Agatha Christie"
# Retrieve an author using 'exclude' and '__contains' lookup
authors = books.exclude(author__contains="Christie")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
Related field filtering
-----------------------
Use a related field like ``books`` separated by a ``__`` (double-underscore)
and a field of the books. Something like ``books__name``.
.. code-block:: python
# Retrieve an author using the date when the book was published
authors = books.filter(books__published="1939")
assert len(authors) == 1
assert authors[0].author == "Agatha Christie"
# Retrieve an author using '__regex' lookup
authors = books.filter(books__name__regex=".*Potter.*")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
AND, OR - filtering
-------------------
Keyword argument queries - in filter(), etc. - are "AND"ed together.
If you need to execute more complex queries (for example, queries with OR statements),
you can combine two filter request with "|".
.. code-block:: python
# Retrieve an author using multiple filters with a single request (AND). This
# filter use the '__icontains' lookup. Same as '__contains' but case-insensitive
authors = books.filter(books__name__icontains="and", books__genre="Fantasy")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
# Retrieve an author by combining filters (OR)
authors = books.filter(author="Stephane Capponi") | books.filter(
author="J. K. Rowling"
)
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
Filter nested related field
----------------------------
The library provides also a way to filter nested relationship. This means that you
can make requests to only retrieve ``books`` in the author collection. Or you can
use that output to filter the authors.
.. code-block:: python
# filter_related is the method to use to filter all related nodes
related_books = books.filter_related('books', genre="Mystery")
assert len(related_books) == 1
assert related_books[0].name == "And Then There Were None"
# You can also use filter_related to filter authors.
authors = books.filter(
books=books.filter_related('books', name__regex=".*Potter.*")
)
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
Cascade filtering
-----------------
Sometimes you will want to filter the author but also the related books.
It is possible to do that by calling the ``on_cascade()`` method before filtering.
.. code-block:: python
# Filter the author but also the books of the author
authors = books.on_cascade().filter(
books__name="Harry Potter and the Chamber of Secrets"
)
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
# The books are also filtered
assert len(authors[0].books) == 1
assert authors[0].books[0].name == "Harry Potter and the Chamber of Secrets"
List of available lookups
=========================
Field lookups are used to specify how a the dataset should query the results it returns.
They're specified as keyword arguments to the ``Dataset`` methods
``filter()`` and ``exclude()``. Basic lookups keyword arguments
take the form "field__lookuptype=value". (That's a double-underscore).
As a convenience when no lookup type is provided (like in
``books.filter(id=1)``) the lookup type is assumed to be ``exact``.
.. code-block:: python
# author is one of the field of the dictionary
# '__contains' is the lookup
books.filter(author__contains="Row")
+--------------+-------------------------+-----------------------------------------------------------------+
| Lookup | Case-insensitive lookup | Description |
+==============+=========================+=================================================================+
| exact | iexact | Exact match |
+--------------+-------------------------+-----------------------------------------------------------------+
| contains | icontains | Containment test |
+--------------+-------------------------+-----------------------------------------------------------------+
| startswtih | istartswith | Starts with a specific string |
+--------------+-------------------------+-----------------------------------------------------------------+
| endswith | iendswith | Ends with a specific string |
+--------------+-------------------------+-----------------------------------------------------------------+
| regex | iregex | Regular expression match |
+--------------+-------------------------+-----------------------------------------------------------------+
| in | | In a given iterable; strings (being iterables) are accepted |
+--------------+-------------------------+-----------------------------------------------------------------+
| gt | | Grater than |
+--------------+-------------------------+-----------------------------------------------------------------+
| gte | | Greater that or equal |
+--------------+-------------------------+-----------------------------------------------------------------+
| lt | | Lower than |
+--------------+-------------------------+-----------------------------------------------------------------+
| lte | | Lower than or equal to |
+--------------+-------------------------+-----------------------------------------------------------------+
| range | | Range between two values. Integer only |
+--------------+-------------------------+-----------------------------------------------------------------+
| isnull | | Check that a field is null. Takes either True or False |
+--------------+-------------------------+-----------------------------------------------------------------+
| contained_by | | Check data is a subset of the passed values. ArrayField only |
+--------------+-------------------------+-----------------------------------------------------------------+
| overlap | | Data shares any results with the passed values. ArrayField only |
+--------------+-------------------------+-----------------------------------------------------------------+
| len | | Check length of the array. ArrayField only |
+--------------+-------------------------+-----------------------------------------------------------------+
Documentation
=============
Datalookup does not stop here. The full documentation is in the ``docs``
directory or online at `https://datalookup.readthedocs.io/en/latest/`
Contribution
============
Anyone can contribute to Datalookup's development. Checkout our documentation on how to
get involved: `https://datalookup.readthedocs.io/en/latest/internals/contributing.html`
License
=======
Copyright Stephane Capponi and others, 2023
Distributed under the terms of the `MIT`_ license, Datalookup is free and
open source software.
Datalookup was inspired by Django and only the `RegisterLookupMixin`_ was
copied. Everything else was inspired and re-interpreted.
You can find the license of Django in the ``licenses`` folder.
.. _`MIT`: https://github.com/pyshare/datalookup/blob/master/LICENCE
.. _`RegisterLookupMixin`: https://github.com/pyshare/datalookup/blob/78d315e474d842d82a392127e835cb304940d1c7/datalookup/utils.py#LL20C10-L20C10
Raw data
{
"_id": null,
"home_page": null,
"name": "datalookup",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "filter, dictionary, nested, datalookup, data-filtering, deep-filtering, lookup",
"author": null,
"author_email": "Stephane Capponi <stephane.capponi@proton.me>",
"download_url": "https://files.pythonhosted.org/packages/d4/ef/3f64222107848557885d77507d3eec2cf278b049e0610a37363abff71920/datalookup-1.0.1.tar.gz",
"platform": null,
"description": "=======================================\r\nDatalookup - Deep nested data filtering\r\n=======================================\r\n\r\n.. image:: https://img.shields.io/badge/python-3.9-blue.svg\r\n :target: https://github.com/pyshare/datalookup\r\n\r\n.. image:: https://github.com/pyshare/datalookup/actions/workflows/tests.yml/badge.svg\r\n :target: https://github.com/pyshare/datalookup/actions?query=workflow%3APython%20testing\r\n\r\n.. image:: https://codecov.io/gh/pyshare/datalookup/branch/master/graph/badge.svg\r\n :target: https://codecov.io/gh/pyshare/datalookup\r\n :alt: Code coverage Status\r\n\r\n.. image:: https://github.com/pyshare/datalookup/actions/workflows/linters.yml/badge.svg\r\n :target: https://github.com/pyshare/datalookup/actions?query=workflow%3APython%20linting\r\n\r\n.. image:: https://readthedocs.org/projects/datalookup/badge/?version=latest\r\n :target: https://datalookup.readthedocs.io/en/latest/\r\n :alt: Documentation Status\r\n\r\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\r\n :target: https://github.com/psf/black\r\n\r\n----\r\n\r\nThe **Datalookup** library makes it easier to filter and manipulate your data. The\r\nmodule is inspired by the Django Queryset Api and it's lookups.\r\n\r\nInstallation\r\n============\r\n\r\n.. code-block:: console\r\n\r\n $ pip install datalookup\r\n\r\nExample\r\n=======\r\n\r\nThroughout the below examples, we'll refer to the following data,\r\nwhich comprise a list of authors with the books they wrote.\r\n\r\n.. code-block:: python\r\n\r\n data = [\r\n {\r\n \"id\": 1,\r\n \"author\": \"J. K. Rowling\",\r\n \"books\": [\r\n {\r\n \"name\": \"Harry Potter and the Chamber of Secrets\",\r\n \"genre\": \"Fantasy\",\r\n \"published\": \"1998\"\r\n },\r\n {\r\n \"name\": \"Harry Potter and the Prisoner of Azkaban\",\r\n \"genre\": \"Fantasy\",\r\n \"published\": \"1999\"\r\n }\r\n ]\r\n },\r\n {\r\n \"id\": 2,\r\n \"author\": \"Agatha Christie\",\r\n \"books\": [\r\n {\r\n \"name\": \"And Then There Were None\",\r\n \"genre\": \"Mystery\",\r\n \"published\": \"1939\"\r\n }\r\n ]\r\n }\r\n ]\r\n\r\n**Datalookup** makes it easy to find an author by calling one of the methods\r\nof the ``Dataset`` class like ``filter()`` or ``exclude()``. There\r\nare multiple ways to retrieve an author.\r\n\r\nBasic filtering\r\n---------------\r\n\r\nUse one of the ``field`` of your author dictionary to filter your data.\r\n\r\n.. code-block:: python\r\n\r\n from datalookup import Dataset\r\n\r\n # Use Dataset to manipulate and filter your data\r\n books = Dataset(data)\r\n\r\n # Retrieve an author using the author name\r\n authors = books.filter(author=\"J. K. Rowling\")\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\n # Retrieve an author using '__in' lookup\r\n authors = books.filter(id__in=[2, 3])\r\n assert len(authors) == 1\r\n assert authors[0].author == \"Agatha Christie\"\r\n\r\n # Retrieve an author using 'exclude' and '__contains' lookup\r\n authors = books.exclude(author__contains=\"Christie\")\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\nRelated field filtering\r\n-----------------------\r\n\r\nUse a related field like ``books`` separated by a ``__`` (double-underscore)\r\nand a field of the books. Something like ``books__name``.\r\n\r\n.. code-block:: python\r\n\r\n # Retrieve an author using the date when the book was published\r\n authors = books.filter(books__published=\"1939\")\r\n assert len(authors) == 1\r\n assert authors[0].author == \"Agatha Christie\"\r\n\r\n # Retrieve an author using '__regex' lookup\r\n authors = books.filter(books__name__regex=\".*Potter.*\")\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\nAND, OR - filtering\r\n-------------------\r\n\r\nKeyword argument queries - in filter(), etc. - are \"AND\"ed together.\r\nIf you need to execute more complex queries (for example, queries with OR statements),\r\nyou can combine two filter request with \"|\".\r\n\r\n.. code-block:: python\r\n\r\n # Retrieve an author using multiple filters with a single request (AND). This\r\n # filter use the '__icontains' lookup. Same as '__contains' but case-insensitive\r\n authors = books.filter(books__name__icontains=\"and\", books__genre=\"Fantasy\")\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\n # Retrieve an author by combining filters (OR)\r\n authors = books.filter(author=\"Stephane Capponi\") | books.filter(\r\n author=\"J. K. Rowling\"\r\n )\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\nFilter nested related field\r\n----------------------------\r\n\r\nThe library provides also a way to filter nested relationship. This means that you\r\ncan make requests to only retrieve ``books`` in the author collection. Or you can\r\nuse that output to filter the authors.\r\n\r\n.. code-block:: python\r\n\r\n # filter_related is the method to use to filter all related nodes\r\n related_books = books.filter_related('books', genre=\"Mystery\")\r\n assert len(related_books) == 1\r\n assert related_books[0].name == \"And Then There Were None\"\r\n\r\n # You can also use filter_related to filter authors.\r\n authors = books.filter(\r\n books=books.filter_related('books', name__regex=\".*Potter.*\")\r\n )\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\nCascade filtering\r\n-----------------\r\n\r\nSometimes you will want to filter the author but also the related books.\r\nIt is possible to do that by calling the ``on_cascade()`` method before filtering.\r\n\r\n.. code-block:: python\r\n\r\n # Filter the author but also the books of the author\r\n authors = books.on_cascade().filter(\r\n books__name=\"Harry Potter and the Chamber of Secrets\"\r\n )\r\n assert len(authors) == 1\r\n assert authors[0].author == \"J. K. Rowling\"\r\n\r\n # The books are also filtered\r\n assert len(authors[0].books) == 1\r\n assert authors[0].books[0].name == \"Harry Potter and the Chamber of Secrets\"\r\n\r\nList of available lookups\r\n=========================\r\n\r\nField lookups are used to specify how a the dataset should query the results it returns.\r\nThey're specified as keyword arguments to the ``Dataset`` methods\r\n``filter()`` and ``exclude()``. Basic lookups keyword arguments\r\ntake the form \"field__lookuptype=value\". (That's a double-underscore).\r\n\r\nAs a convenience when no lookup type is provided (like in\r\n``books.filter(id=1)``) the lookup type is assumed to be ``exact``.\r\n\r\n.. code-block:: python\r\n\r\n # author is one of the field of the dictionary\r\n # '__contains' is the lookup\r\n books.filter(author__contains=\"Row\")\r\n\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| Lookup | Case-insensitive lookup | Description |\r\n+==============+=========================+=================================================================+\r\n| exact | iexact | Exact match |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| contains | icontains | Containment test |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| startswtih | istartswith | Starts with a specific string |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| endswith | iendswith | Ends with a specific string |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| regex | iregex | Regular expression match |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| in | | In a given iterable; strings (being iterables) are accepted |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| gt | | Grater than |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| gte | | Greater that or equal |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| lt | | Lower than |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| lte | | Lower than or equal to |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| range | | Range between two values. Integer only |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| isnull | | Check that a field is null. Takes either True or False |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| contained_by | | Check data is a subset of the passed values. ArrayField only |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| overlap | | Data shares any results with the passed values. ArrayField only |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n| len | | Check length of the array. ArrayField only |\r\n+--------------+-------------------------+-----------------------------------------------------------------+\r\n\r\nDocumentation\r\n=============\r\n\r\nDatalookup does not stop here. The full documentation is in the ``docs``\r\ndirectory or online at `https://datalookup.readthedocs.io/en/latest/`\r\n\r\nContribution\r\n============\r\n\r\nAnyone can contribute to Datalookup's development. Checkout our documentation on how to\r\nget involved: `https://datalookup.readthedocs.io/en/latest/internals/contributing.html`\r\n\r\nLicense\r\n=======\r\n\r\nCopyright Stephane Capponi and others, 2023\r\nDistributed under the terms of the `MIT`_ license, Datalookup is free and\r\nopen source software.\r\n\r\nDatalookup was inspired by Django and only the `RegisterLookupMixin`_ was\r\ncopied. Everything else was inspired and re-interpreted.\r\nYou can find the license of Django in the ``licenses`` folder.\r\n\r\n.. _`MIT`: https://github.com/pyshare/datalookup/blob/master/LICENCE\r\n.. _`RegisterLookupMixin`: https://github.com/pyshare/datalookup/blob/78d315e474d842d82a392127e835cb304940d1c7/datalookup/utils.py#LL20C10-L20C10\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Deep nested data filtering library",
"version": "1.0.1",
"project_urls": {
"changelog": "https://datalookup.readthedocs.io/en/latest/internals/changelog.html",
"homepage": "https://datalookup.readthedocs.io/en/latest/",
"repository": "https://github.com/pyshare/datalookup"
},
"split_keywords": [
"filter",
" dictionary",
" nested",
" datalookup",
" data-filtering",
" deep-filtering",
" lookup"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "45f3d00588353777b9308acbb384219e8f905a48f58aa26a87ec473875757b70",
"md5": "266c12df8f49661167ca1b0f4be54913",
"sha256": "6c454dc23a482947e632380c5f022574565c65059cd718fa417fa2fb9de01c80"
},
"downloads": -1,
"filename": "datalookup-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "266c12df8f49661167ca1b0f4be54913",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 14694,
"upload_time": "2024-04-19T09:33:57",
"upload_time_iso_8601": "2024-04-19T09:33:57.875394Z",
"url": "https://files.pythonhosted.org/packages/45/f3/d00588353777b9308acbb384219e8f905a48f58aa26a87ec473875757b70/datalookup-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d4ef3f64222107848557885d77507d3eec2cf278b049e0610a37363abff71920",
"md5": "0ce3282be8ee2db138f57e3ce26daa1e",
"sha256": "85119cc0bbefca1c85b33185bdd580b2d099e5e918a7e92c1df2b47d00cc94c4"
},
"downloads": -1,
"filename": "datalookup-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "0ce3282be8ee2db138f57e3ce26daa1e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 41301,
"upload_time": "2024-04-19T09:34:00",
"upload_time_iso_8601": "2024-04-19T09:34:00.157829Z",
"url": "https://files.pythonhosted.org/packages/d4/ef/3f64222107848557885d77507d3eec2cf278b049e0610a37363abff71920/datalookup-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-19 09:34:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pyshare",
"github_project": "datalookup",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "datalookup"
}
JSON
