QueryableList
=============
What
----
QueryableList allows you to "filter" a list of items of varying types, simplifing code by replacing tedious for-loops with simple chaining.
It uses an interface common to some ORMs like Django, Flask, and IndexedRedis.
You can perform single filters on lists of data, or you can build queries and execute that query on any number of arbitrary data sets.
QueryableList also implements the boolean logic operators for lists (AND, OR, XOR) which can simplify your code.
**What pattern does it replace?**
Constant loops/getters to drill down data. If you are filtering data, displaying data by criteria, etc, your code will be FULL of these.
QueryableList simplifies and makes generic this common pattern of filtering.
*Before*
def getOlderThan(people, minAge):
ret = []
for person in people:
if person.age and person.age > minAge:
ret.append(person)
return ret
...
people = getAllPeople() # Get your data here
oldEnoughToRide = getOlderThan(people, 13)
notOldEnough = [person for person in people if person not in oldEnoughToRide]
*After*
people = QueryableListObjs( getAllPeople() ) # Transform data into QueryableList
oldEnoughToRide = people.filter(age__gt=13)
notOldEnough = people ^ oldEnoughToRide # The XOR of the filtered list to the parent is the NOT of the filter criteria
No function, no loop, and list comprehensions can get very messy or impossible with a large number of complicated filters applied.
The above example shows a one-time filtering of a list. You can also build reusable queries, and append different criteria based on conditions or through passing the query around different functions. See "Building Reusable Queries" section below for more info.
How?
----
Types
-----
Perform one-time filters through one of the list-type extending classes:
**QueryableListObjs** - This assumes each item extends object [or implements \_\_getattribute\_\_].
**QueryableListDicts** - This assumes that each item is a dict [or implements \_\_getitem\_\_].
**QueryableListMixed** - QueryableList which can contain dict-like items or object-like item. (This is somewhat slower than using QueryableListObjs or QueryableListDicts directly, but use it if you need to mix, or need to support either type.)
The items within these lists do not need to be of the same type. If any fields are missing on the filtered objects, it will be assigned a value of "None" for filtering purposes.
Filter Methods
--------------
You can filter the data within these objects through one of the following methods:
*filterAnd* - returns a QueryableList where each item matches ALL of the provided criteria.
*filter* - Alias for filterAnd
*filterOr* - returns a QueryableList where each item matches ANY of the provided criteria.
*customFilter* - Takes a lambda or a function as a parameter. Each element in the list is passed into this function, and if it returns True, that element is retained.
The QueryableList types support all the operations of a list, and return the same QueryableList types so you can perform chaining.
Additionally, you can use ADD(+), SUB(-), AND(&), OR(|), and XOR(^) operators against other QueryableLists as another powerful means of filtering.
You specify the filter operations by passing arguments of $fieldName\_\_$operation.
Example: e.x. results = objs.filter(name\_\_ne='Tim') # get all objects where the 'name' field does not equal 'Tim'
For all available operations, see the "Operations" section below.
Other Methods
-------------
QueryableList collections have several other methods to make them as closely api-compatible with server-side filtering ORMs as possible.
This allows you to use the same functions regardless of if you are server-side or client-side filtering.
* count - Returns the number of items in this collection ( same as len(..) )
* all - Returns a copy of this collection, same elements but a new collection
Building Reusable Queries
-------------------------
You can build a reusable query, out of several chains of filters (either AND or OR) by using the **QueryBuilder** class.
The QueryBuilder class stores a "chain" of filters, which are applied in order. Each link in the chain contains a filter type (AND or OR), and the filters themselves (same as the filter methods on the QueryableList).
Use the *addFilter(filterType, ..filters..)* method to add a link to the chain.
To execute the query, call *execute(lst)* , where "lst" is your list of items. You can execute a query multiple times on any number of datasets.
Use the *copy* method to create a copy of the current set of filters.
If you know the type in advance, you can pass a QueryableListObjs or QueryableListDicts when calling *execute* to slightly speed up access times, otherwise a *QueryableListMixed* (supports both dict and object style access) will be used.
Example:
myQuery = QueryBuilder()
myQuery.addFilter(age__gt=21) # Age must be greater than 21
myQuery.addFilter('OR', job__ieq='Manager', numSubordinates__gt=0) # Is a manager, or has more than 0 subordinates
managerPartyCompany1 = myQuery.execute(company1Persons) # Filter from all company1Persons those that meet above criteria
managerPartyCompany2 = myQuery.execute(company2Persons) # use same filter to apply same query to company2Persons
Extending QueryableList for your own data sets
----------------------------------------------
One of the powerful aspects of QueryableList is that it is easily extendable.
Generally, you won't need to do this, as QueryableListDicts or QueryableListObjs will handle your needs.
But sometimes, you have more advanced requirements than can be satisfied by properties and matching on them.
For these cases, you can extend QueryableList.QueryableListBase to create your own QueryableList type.
You only need to implement a single method,
@staticmethod
def _get_item_value(item, fieldName)
"item" will be an item in your collection, and "fieldName" is the field being queried.
For example, say you have a series of objects, "Job", which contain some attributes and a "queue".
You want to be able to filter on both the attributes on the object and various special attributes of it's queue (like size, item ids, etc).
You can implement like this:
class MyJobCollection(QueryableList.QueryableListBase):
@staticmethod
def _get_item_value(item, fieldName):
if fieldName == 'queueSize':
# queueSize is the number of items in the queue
return len(item.queue)
elif fieldName == 'queueItemIds':
# queueItemIds is a list of the ids in the item queue,
# so a "contains" query can check if an id is in this item's queue
return [qi.id for qi in item.queue]
elif hasattr(item, fieldName):
# Otherwise, if this is an attribute on the item, return it's value
if fieldName == 'queue':
raise KeyError('Cannot query queue directly. Try queueSize or queueItemIds.')
return getattr(item, fieldName)
else:
raise KeyError('Invalid attribute "%s" on %s' %(fieldName, item.__class__.__name__))
The init method takes a list of items (and it contains all the methods a list has, like *.append*), so you can create it like:
myJob1 = MyQueue(...)
myJob2 = MyQueue(...)
myJobs = MyQueueCollection([myJob1, myJob2])
and use it like:
largeJobs = myJobs.filter(queueSize__gt=10)
So just by implementing that one method, you now have all the powerful filter capabilities that QueryableList provides!
Operations
----------
* eq - Test equality ( = operator )
* ieq - Test equality, ignoring case (must be strings, or at least implement the .lower() method)
* ne - Test inequality ( != operator )
* ine - Test inequality, ignoring case (must be strings, or at least implement the .lower() method)
* lt - The item's field value must be less than the provided value
* lte - The item's field value must be less than or equal to the provided value
* gt - The item's field value must be greater than the provided value
* gte - The item's field value must be greater than or equal to the provided value
* isnull - Provided value must be True/False. If True, the item's field value must be None, otherwise it must not be None.
* is - Test identity equality ( is operator )
* isnot - Test identity inequality ( is not operator )
* in - Test that the item's field value is contained in the provided list of items
* notin - Test that the item's field value is not contained in the provided list of items
* contains - Test that the item's field value contains the provided value ( using "in" )
* icontains - Case-insensitive "contains"
* notcontains - Test that the item's field value does not contain the provided value ( using "not in" )
* icontains - Case-insensitive "notcontains"
* containsAny - Test that the item's field value contains any of the items in the provided list ( using "in" )
* notcontainsAny - Test that the item's field value does not contain any of the items in the provided list ( using "not in" )
* splitcontains - Takes a tuple, (splitBy<str>, containsThis<str>). Use for a string that represents a list. The field will be split by the first, "splitBy", param, and the result tested that it contains an item matching the second, "containsThis", param. E.x. item\_\_splitcontains=(' ', 'someValue')
* splitnotcontains - Takes a tuple, (splitBy<str>, containsThis<str>). Use for a string that represents a list. The field will be split by the first, "splitBy", param, and the result tested that it does not contain an item matching the second, "containsThis", param.
* splitcontainsAny - Takes a tuple, (splitBy<str>, possibleMatches <list<str>>). Use for a string that represents a list. The field will be split by the first, "splitBy", param, and the result tested that it contains any of the items in the provided list.
* splitnotcontainsAny - Takes a tuple, (splitBy<str>, possibleMatches <list<str>>). Use for a string that represents a list. The field will be split by the first, "splitBy", param, and the result tested that it does not contains any of the items in the provided list.
* customMatch - Takes a lambda or function, which is passed in the value of the given field. If it returns True, the element is a match, otherwise it is not.
Full PyDoc Documentation
------------------------
Pydoc documentation can be found at: http://htmlpreview.github.io/?https://github.com/kata198/QueryableList/blob/master/doc/QueryableList.html?vers=4
Raw data
{
"_id": null,
"home_page": "https://github.com/kata198/QueryableList",
"name": "QueryableList",
"maintainer": null,
"docs_url": "https://pythonhosted.org/QueryableList/",
"requires_python": null,
"maintainer_email": null,
"keywords": "queryablelist,query,list,filter,objects,eq,ne,lt,gt,equals,not,compare,comprehension,orm,queryable,django,flask,indexedredis,contains,icontains",
"author": "Tim Savannah",
"author_email": "kata198@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/59/cd/ff38fb62fc81def97f50a98fc4bdbbe258351b5be8be6a6b69fec7a25c81/QueryableList-3.1.0.tar.gz",
"platform": "UNKNOWN",
"description": "QueryableList\n=============\n\nWhat\n----\n\nQueryableList allows you to \"filter\" a list of items of varying types, simplifing code by replacing tedious for-loops with simple chaining.\n\nIt uses an interface common to some ORMs like Django, Flask, and IndexedRedis.\n\nYou can perform single filters on lists of data, or you can build queries and execute that query on any number of arbitrary data sets.\n\nQueryableList also implements the boolean logic operators for lists (AND, OR, XOR) which can simplify your code.\n\n\n**What pattern does it replace?**\n\nConstant loops/getters to drill down data. If you are filtering data, displaying data by criteria, etc, your code will be FULL of these.\n\nQueryableList simplifies and makes generic this common pattern of filtering.\n\n*Before*\n\n def getOlderThan(people, minAge):\n\n\n ret = []\n\n for person in people:\n\n if person.age and person.age > minAge:\n\n ret.append(person)\n\n return ret\n\n\n ...\n\n people = getAllPeople() # Get your data here\n\n oldEnoughToRide = getOlderThan(people, 13)\n\n notOldEnough = [person for person in people if person not in oldEnoughToRide]\n\n\n*After*\n\n people = QueryableListObjs( getAllPeople() ) # Transform data into QueryableList\n\n oldEnoughToRide = people.filter(age__gt=13)\n\n notOldEnough = people ^ oldEnoughToRide # The XOR of the filtered list to the parent is the NOT of the filter criteria\n\n\n\nNo function, no loop, and list comprehensions can get very messy or impossible with a large number of complicated filters applied.\n\nThe above example shows a one-time filtering of a list. You can also build reusable queries, and append different criteria based on conditions or through passing the query around different functions. See \"Building Reusable Queries\" section below for more info.\n\n\nHow?\n----\n\n\nTypes\n-----\n\nPerform one-time filters through one of the list-type extending classes:\n\n\n**QueryableListObjs** - This assumes each item extends object [or implements \\_\\_getattribute\\_\\_].\n\n**QueryableListDicts** - This assumes that each item is a dict [or implements \\_\\_getitem\\_\\_].\n\n**QueryableListMixed** - QueryableList which can contain dict-like items or object-like item. (This is somewhat slower than using QueryableListObjs or QueryableListDicts directly, but use it if you need to mix, or need to support either type.)\n\n\nThe items within these lists do not need to be of the same type. If any fields are missing on the filtered objects, it will be assigned a value of \"None\" for filtering purposes.\n\n\nFilter Methods\n--------------\n\nYou can filter the data within these objects through one of the following methods:\n\n*filterAnd* - returns a QueryableList where each item matches ALL of the provided criteria.\n\n*filter* - Alias for filterAnd\n\n*filterOr* - returns a QueryableList where each item matches ANY of the provided criteria.\n\n*customFilter* - Takes a lambda or a function as a parameter. Each element in the list is passed into this function, and if it returns True, that element is retained.\n\n\nThe QueryableList types support all the operations of a list, and return the same QueryableList types so you can perform chaining.\n\nAdditionally, you can use ADD(+), SUB(-), AND(&), OR(|), and XOR(^) operators against other QueryableLists as another powerful means of filtering.\n\n\nYou specify the filter operations by passing arguments of $fieldName\\_\\_$operation.\n\nExample: e.x. results = objs.filter(name\\_\\_ne='Tim') # get all objects where the 'name' field does not equal 'Tim'\n\n\nFor all available operations, see the \"Operations\" section below.\n\n\nOther Methods\n-------------\n\nQueryableList collections have several other methods to make them as closely api-compatible with server-side filtering ORMs as possible.\n\nThis allows you to use the same functions regardless of if you are server-side or client-side filtering.\n\n* count - Returns the number of items in this collection ( same as len(..) )\n\n* all - Returns a copy of this collection, same elements but a new collection\n\n\nBuilding Reusable Queries\n-------------------------\n\nYou can build a reusable query, out of several chains of filters (either AND or OR) by using the **QueryBuilder** class.\n\nThe QueryBuilder class stores a \"chain\" of filters, which are applied in order. Each link in the chain contains a filter type (AND or OR), and the filters themselves (same as the filter methods on the QueryableList).\n\n\nUse the *addFilter(filterType, ..filters..)* method to add a link to the chain.\n\nTo execute the query, call *execute(lst)* , where \"lst\" is your list of items. You can execute a query multiple times on any number of datasets.\n\nUse the *copy* method to create a copy of the current set of filters.\n\n\nIf you know the type in advance, you can pass a QueryableListObjs or QueryableListDicts when calling *execute* to slightly speed up access times, otherwise a *QueryableListMixed* (supports both dict and object style access) will be used.\n\nExample:\n\n myQuery = QueryBuilder()\n\n myQuery.addFilter(age__gt=21) # Age must be greater than 21\n\n myQuery.addFilter('OR', job__ieq='Manager', numSubordinates__gt=0) # Is a manager, or has more than 0 subordinates\n\n\n managerPartyCompany1 = myQuery.execute(company1Persons) # Filter from all company1Persons those that meet above criteria\n\n managerPartyCompany2 = myQuery.execute(company2Persons) # use same filter to apply same query to company2Persons\n\n\nExtending QueryableList for your own data sets\n----------------------------------------------\n\nOne of the powerful aspects of QueryableList is that it is easily extendable.\n\nGenerally, you won't need to do this, as QueryableListDicts or QueryableListObjs will handle your needs.\n\nBut sometimes, you have more advanced requirements than can be satisfied by properties and matching on them.\n\nFor these cases, you can extend QueryableList.QueryableListBase to create your own QueryableList type.\n\nYou only need to implement a single method,\n\n\n @staticmethod\n\n def _get_item_value(item, fieldName)\n\n\n\"item\" will be an item in your collection, and \"fieldName\" is the field being queried.\n\nFor example, say you have a series of objects, \"Job\", which contain some attributes and a \"queue\".\n\nYou want to be able to filter on both the attributes on the object and various special attributes of it's queue (like size, item ids, etc).\n\nYou can implement like this:\n\n class MyJobCollection(QueryableList.QueryableListBase):\n\n\n @staticmethod\n\n def _get_item_value(item, fieldName):\n\n\n if fieldName == 'queueSize':\n\n # queueSize is the number of items in the queue\n\n return len(item.queue)\n\n elif fieldName == 'queueItemIds':\n\n # queueItemIds is a list of the ids in the item queue,\n\n # so a \"contains\" query can check if an id is in this item's queue\n\n return [qi.id for qi in item.queue]\n\n elif hasattr(item, fieldName):\n\n # Otherwise, if this is an attribute on the item, return it's value\n\n if fieldName == 'queue':\n\n raise KeyError('Cannot query queue directly. Try queueSize or queueItemIds.')\n\n return getattr(item, fieldName)\n\n else:\n\n raise KeyError('Invalid attribute \"%s\" on %s' %(fieldName, item.__class__.__name__))\n\n\nThe init method takes a list of items (and it contains all the methods a list has, like *.append*), so you can create it like:\n\n myJob1 = MyQueue(...)\n\n myJob2 = MyQueue(...)\n\n\n myJobs = MyQueueCollection([myJob1, myJob2])\n\nand use it like:\n\n largeJobs = myJobs.filter(queueSize__gt=10)\n\nSo just by implementing that one method, you now have all the powerful filter capabilities that QueryableList provides!\n\n\n\nOperations\n----------\n\n* eq - Test equality ( = operator )\n\n\n* ieq - Test equality, ignoring case (must be strings, or at least implement the .lower() method)\n\n\n* ne - Test inequality ( != operator )\n\n\n* ine - Test inequality, ignoring case (must be strings, or at least implement the .lower() method)\n\n\n* lt - The item's field value must be less than the provided value\n\n\n* lte - The item's field value must be less than or equal to the provided value\n\n\n* gt - The item's field value must be greater than the provided value\n\n\n* gte - The item's field value must be greater than or equal to the provided value\n\n\n* isnull - Provided value must be True/False. If True, the item's field value must be None, otherwise it must not be None.\n\n\n* is - Test identity equality ( is operator )\n\n\n* isnot - Test identity inequality ( is not operator )\n\n\n* in - Test that the item's field value is contained in the provided list of items\n\n\n* notin - Test that the item's field value is not contained in the provided list of items\n\n\n* contains - Test that the item's field value contains the provided value ( using \"in\" )\n\n\n* icontains - Case-insensitive \"contains\"\n\n\n* notcontains - Test that the item's field value does not contain the provided value ( using \"not in\" )\n\n\n* icontains - Case-insensitive \"notcontains\"\n\n\n* containsAny - Test that the item's field value contains any of the items in the provided list ( using \"in\" )\n\n\n* notcontainsAny - Test that the item's field value does not contain any of the items in the provided list ( using \"not in\" )\n\n\n* splitcontains - Takes a tuple, (splitBy<str>, containsThis<str>). Use for a string that represents a list. The field will be split by the first, \"splitBy\", param, and the result tested that it contains an item matching the second, \"containsThis\", param. E.x. item\\_\\_splitcontains=(' ', 'someValue')\n\n\n* splitnotcontains - Takes a tuple, (splitBy<str>, containsThis<str>). Use for a string that represents a list. The field will be split by the first, \"splitBy\", param, and the result tested that it does not contain an item matching the second, \"containsThis\", param.\n\n\n* splitcontainsAny - Takes a tuple, (splitBy<str>, possibleMatches <list<str>>). Use for a string that represents a list. The field will be split by the first, \"splitBy\", param, and the result tested that it contains any of the items in the provided list.\n\n\n* splitnotcontainsAny - Takes a tuple, (splitBy<str>, possibleMatches <list<str>>). Use for a string that represents a list. The field will be split by the first, \"splitBy\", param, and the result tested that it does not contains any of the items in the provided list.\n\n* customMatch - Takes a lambda or function, which is passed in the value of the given field. If it returns True, the element is a match, otherwise it is not.\n\n\n\nFull PyDoc Documentation\n------------------------\n\nPydoc documentation can be found at: http://htmlpreview.github.io/?https://github.com/kata198/QueryableList/blob/master/doc/QueryableList.html?vers=4",
"bugtrack_url": null,
"license": "LGPLv2",
"summary": "Python module to add support for ORM-style filtering to any list of items",
"version": "3.1.0",
"project_urls": {
"Homepage": "https://github.com/kata198/QueryableList"
},
"split_keywords": [
"queryablelist",
"query",
"list",
"filter",
"objects",
"eq",
"ne",
"lt",
"gt",
"equals",
"not",
"compare",
"comprehension",
"orm",
"queryable",
"django",
"flask",
"indexedredis",
"contains",
"icontains"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "59cdff38fb62fc81def97f50a98fc4bdbbe258351b5be8be6a6b69fec7a25c81",
"md5": "e9be982545ecda0d15f2b3ff718d758f",
"sha256": "8891dccbadc69a35f5944e1826d8f8db224522aa3af913e301a7a448f5b411e9"
},
"downloads": -1,
"filename": "QueryableList-3.1.0.tar.gz",
"has_sig": false,
"md5_digest": "e9be982545ecda0d15f2b3ff718d758f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 55602,
"upload_time": "2017-04-24T03:27:12",
"upload_time_iso_8601": "2017-04-24T03:27:12.618316Z",
"url": "https://files.pythonhosted.org/packages/59/cd/ff38fb62fc81def97f50a98fc4bdbbe258351b5be8be6a6b69fec7a25c81/QueryableList-3.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2017-04-24 03:27:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kata198",
"github_project": "QueryableList",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "queryablelist"
}
JSON
