# pyspecter
[](https://img.shields.io/pypi/pyversions/pyspecter)
[](https://badge.fury.io/py/pyspecter)
[](https://img.shields.io/pypi/dm/pyspecter)
A Python library to query nested structure, inspired by [specter](https://github.com/redplanetlabs/specter)
If you are dealing with nested python structure and it require complex rule to search the data underneath like:
* Key start with a pattern
* Value to be filter
* Conditional path to walk into
* ...
this is the right library fit the use case as it extract the `Navigation` rule to a list, saving your a lot of trouble writting your own logic to navigate the nested data.
## Get started
## Examples
m = {"A":{"B1":[10,20],"B2":2,"B3":3}
,"C":{"B1":[1,2],"B2":[3,4]}
,"D":[1,2,3,4]
,"E":[None,2,3,4]
,"F":{"G":1}
,"H":["A1","A2","A3"]}
### Navigate to specific item
#### FIRST/LAST
assert query(m, ["D", S.FIRST]) == 1
assert query(m, ["A", S.FIRST]) == ("B1", [10, 20]) # first element from dict.items()
assert query(m, ["D", S.LAST]) == 4
assert query(m, ["A", S.LAST]) == ("B3", 3) # last element from dict.items()
#### Nth
Navigate to the Nth element
assert query(m, ["D", (S.NTH, 1)]) == 2
assert query(m, ["D",(S.NTH, 1, 2)]) == [ 2, 3 ]
#### Operation on dict
Navigate to `values` or `keys` of current position
assert query(m, ["C", S.MVALS]) == [[1, 2], [3, 4]]
assert query(m, ["C", S.MKEYS]) == ['B1', 'B2']
Navigate to a sub map of current position
assert query(m, ["A", (S.SUB_MAP, "B1")]) == {"B1":[10,20]}
Annotate with index with current position
assert query(m, ["A", S.INDEXED_VALS]) == [(0, ("B1", [10,20])), (1, ("B2", 2)), (2, ("B3", 3))]
assert query(m, ["C", "B1", S.INDEXED_VALS]) == [(0, 1), (1, 2)]
#### Filtering
filter elements by supplying a function
assert query(m, ["A", "B1", (S.FILTER, lambda x: x > 10)]) == [20]
assert query(m, ["C", (S.FILTER, lambda k, v: k.endswith("2"))]) == [[3, 4]]
Navigate to map which its key or value satisify a custom function
assert query(m, ["A", (S.MKEY_IF, lambda x:x.endswith("1"))]) == [[10, 20]]
assert query(m, ["A", (S.MVAL_IF, lambda x:x==[10,20])]) == [[10, 20]]
#### Branching paths
Branching by multiple paths
mpath = {"A":{"B":1,
"C":[2,3,4,5]}}
assert query(mpath,["A",
(S.MULTI_PATH,["B"]
,["C", S.LAST])]) == [1,5]
#### Conditional Navigation
Navigate to a specifie path
assert query(m, ["D", (S.NTH_PATH, 2)]) == [3]
Navigate to a position if and only if the path exists
assert query(m, [(S.MUST,"F","G")]) == 1
assert query(m, [(S.MUST,"F","G","NOT_EXISTS")]) == None
Navigate to a range
assert query(m, ["D",(S.SRANGE,2,3)]) == [3]
Navigate to a `2nd path` if `1st path` exists, else return `None`
assert query(m,[(S.IF_PATH,["C","B1"],["E"])]) == [None,2,3,4]
assert query(m,[(S.IF_PATH,["C","B1"],["NOT_EXISTS"])]) == None
Navigate to a `2nd path` if `1st path` exists, else navigate to `3rd path`
assert query(m,[(S.IF_PATH,["C","B3"],["E"],["F"])]) == {'G': 1}
Navigate to values of dict if key satisfy a regex expression:
assert query(m, ["A",(S.REGEX,r"B[23]")]) == [ 2,3 ]
Navigate to values of list if elements satisfy a regex expression:
assert query(m, ["H",(S.REGEX,r"\S1")]) == [ "A1" ]
Navigate with optional path node
assert query(m, [(S.MAYBE,"F","G")]) == 1
#### Handling None value
Return default value if current position is `None`
assert query(None,[(S.NONE_VAL,10)]) == 10
If current position is not a None,then return the value of current position
assert query(5,[(S.NONE_VAL,10)]) == 5
### Operation on results
user can operate on the query result by
* REUDCE -> perform `reduce` on the `list` or `dict` result ,from left to right.
<!-- -->
m2 = {"A":{"B":["C1","C2","C3"]}}
assert query(m2, ["A","B",(H.REDUCE,lambda acc,x:acc+"|"+x, ">>>")]) == ">>>|C1|C2|C3"
assert query(m2, ["A","B",(H.REDUCE,lambda acc,x:acc+"|"+x)]) == "C1|C2|C3"
m2 = {"A":{"B":1,"D":2}}
assert query(m2, ["A",(H.REDUCE,lambda acc,x:acc+"|"+str(x[1]),">>")]) == ">>|1|2"
assert query(m2, ["A",(H.REDUCE,lambda acc,x:acc+"|"+str(x[0]),">>")]) == ">>|B|D"
* MAP -> performn a `transformation` on the `list` or `dict` result
<!-- -->
m2 = {"A":{"B":["C1","C2","C3"]}}
assert query(m2, ["A","B",(H.MAP,lambda x:x+"!")]) == ["C1!","C2!","C3!"]
m2 = {"A":{"B":"C1","D":"C2"}}
assert query(m2, ["A",(H.MAP,(lambda k,v: v+"!"))]) == ["C1!","C2!"]
* SUM -> `sum()` the result list, with optional custom function passed before summing.
<!-- -->
m2 = {"A":{"B":[1,2]}}
assert query(m2, ["A","B",H.SUM]) == 3
m2 = {"A":{"B":"1","D":"2"}}
assert query(m2, ["A",S.MVALS,(H.SUM,lambda x: int(x))]) == 3
* MAX/MIN -> Using built-in `max()`/`min()` on the `list` result, with optioanl custom function passed before max/min
<!-- -->
m2 = {"A":{"B":[3,5,2]}}
assert query(m2, ["A","B",H.MIN]) == 2
assert query(m2, ["A","B",(H.MIN,str)]) == "2"
assert query(m2, ["A","B",H.MAX]) == 5
assert query(m2, ["A","B",(H.MAX, str)]) == "5"
* ORDER -> using `sorted()` to sort the result list, with optional custom function passed before sorting.
<!-- -->
m2 = {"A":{"B":[3,5,2]}}
m2 = {"A":{"B":[3,5,2]}}
assert query(m2, ["A","B",H.ORDER]) == [2,3,5]
assert query(m2, ["A","B",(H.ORDER, str)]) == ["2","3","5"]
Raw data
{
"_id": null,
"home_page": "https://github.com/yellowbean/pyspecter",
"name": "pyspecter",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "",
"author": "Xiaoyu Zhang",
"author_email": "always.zhang@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/69/3e/9cf1083e08aada33115d2deb5215626a38c32325f76f1a26e29a005af047/pyspecter-0.1.1.tar.gz",
"platform": null,
"description": "# pyspecter\n\n[](https://img.shields.io/pypi/pyversions/pyspecter)\n[](https://badge.fury.io/py/pyspecter)\n[](https://img.shields.io/pypi/dm/pyspecter)\n\nA Python library to query nested structure, inspired by [specter](https://github.com/redplanetlabs/specter)\n\nIf you are dealing with nested python structure and it require complex rule to search the data underneath like:\n\n* Key start with a pattern\n* Value to be filter\n* Conditional path to walk into\n* ...\n\nthis is the right library fit the use case as it extract the `Navigation` rule to a list, saving your a lot of trouble writting your own logic to navigate the nested data.\n\n## Get started\n\n\n## Examples\n\n m = {\"A\":{\"B1\":[10,20],\"B2\":2,\"B3\":3}\n ,\"C\":{\"B1\":[1,2],\"B2\":[3,4]}\n ,\"D\":[1,2,3,4]\n ,\"E\":[None,2,3,4]\n ,\"F\":{\"G\":1}\n ,\"H\":[\"A1\",\"A2\",\"A3\"]}\n \n### Navigate to specific item \n#### FIRST/LAST \n\n assert query(m, [\"D\", S.FIRST]) == 1\n assert query(m, [\"A\", S.FIRST]) == (\"B1\", [10, 20]) # first element from dict.items()\n\n assert query(m, [\"D\", S.LAST]) == 4\n assert query(m, [\"A\", S.LAST]) == (\"B3\", 3) # last element from dict.items()\n \n#### Nth \nNavigate to the Nth element\n\n assert query(m, [\"D\", (S.NTH, 1)]) == 2\n assert query(m, [\"D\",(S.NTH, 1, 2)]) == [ 2, 3 ]\n \n#### Operation on dict \n \nNavigate to `values` or `keys` of current position\n \n assert query(m, [\"C\", S.MVALS]) == [[1, 2], [3, 4]]\n assert query(m, [\"C\", S.MKEYS]) == ['B1', 'B2']\n\nNavigate to a sub map of current position\n\n assert query(m, [\"A\", (S.SUB_MAP, \"B1\")]) == {\"B1\":[10,20]}\n\nAnnotate with index with current position\n \n assert query(m, [\"A\", S.INDEXED_VALS]) == [(0, (\"B1\", [10,20])), (1, (\"B2\", 2)), (2, (\"B3\", 3))]\n assert query(m, [\"C\", \"B1\", S.INDEXED_VALS]) == [(0, 1), (1, 2)]\n \n#### Filtering \n \nfilter elements by supplying a function\n\n assert query(m, [\"A\", \"B1\", (S.FILTER, lambda x: x > 10)]) == [20]\n assert query(m, [\"C\", (S.FILTER, lambda k, v: k.endswith(\"2\"))]) == [[3, 4]]\n\nNavigate to map which its key or value satisify a custom function \n\n assert query(m, [\"A\", (S.MKEY_IF, lambda x:x.endswith(\"1\"))]) == [[10, 20]]\n assert query(m, [\"A\", (S.MVAL_IF, lambda x:x==[10,20])]) == [[10, 20]]\n \n#### Branching paths\n\nBranching by multiple paths\n\n mpath = {\"A\":{\"B\":1,\n \"C\":[2,3,4,5]}}\n assert query(mpath,[\"A\",\n (S.MULTI_PATH,[\"B\"]\n ,[\"C\", S.LAST])]) == [1,5]\n\n\n\n#### Conditional Navigation\n\nNavigate to a specifie path\n\n assert query(m, [\"D\", (S.NTH_PATH, 2)]) == [3]\n\nNavigate to a position if and only if the path exists\n\n assert query(m, [(S.MUST,\"F\",\"G\")]) == 1\n assert query(m, [(S.MUST,\"F\",\"G\",\"NOT_EXISTS\")]) == None \n\nNavigate to a range\n\n assert query(m, [\"D\",(S.SRANGE,2,3)]) == [3]\n\nNavigate to a `2nd path` if `1st path` exists, else return `None`\n\n assert query(m,[(S.IF_PATH,[\"C\",\"B1\"],[\"E\"])]) == [None,2,3,4]\n assert query(m,[(S.IF_PATH,[\"C\",\"B1\"],[\"NOT_EXISTS\"])]) == None\n\nNavigate to a `2nd path` if `1st path` exists, else navigate to `3rd path`\n\n assert query(m,[(S.IF_PATH,[\"C\",\"B3\"],[\"E\"],[\"F\"])]) == {'G': 1}\n\nNavigate to values of dict if key satisfy a regex expression:\n\n assert query(m, [\"A\",(S.REGEX,r\"B[23]\")]) == [ 2,3 ]\n\nNavigate to values of list if elements satisfy a regex expression:\n\n assert query(m, [\"H\",(S.REGEX,r\"\\S1\")]) == [ \"A1\" ]\n\nNavigate with optional path node \n\n assert query(m, [(S.MAYBE,\"F\",\"G\")]) == 1\n\n#### Handling None value\n\nReturn default value if current position is `None`\n\n assert query(None,[(S.NONE_VAL,10)]) == 10\n\nIf current position is not a None,then return the value of current position\n\n assert query(5,[(S.NONE_VAL,10)]) == 5\n \n \n### Operation on results\n\nuser can operate on the query result by\n\n* REUDCE -> perform `reduce` on the `list` or `dict` result ,from left to right.\n\n<!-- -->\n\n m2 = {\"A\":{\"B\":[\"C1\",\"C2\",\"C3\"]}}\n assert query(m2, [\"A\",\"B\",(H.REDUCE,lambda acc,x:acc+\"|\"+x, \">>>\")]) == \">>>|C1|C2|C3\"\n assert query(m2, [\"A\",\"B\",(H.REDUCE,lambda acc,x:acc+\"|\"+x)]) == \"C1|C2|C3\"\n\n m2 = {\"A\":{\"B\":1,\"D\":2}}\n assert query(m2, [\"A\",(H.REDUCE,lambda acc,x:acc+\"|\"+str(x[1]),\">>\")]) == \">>|1|2\"\n assert query(m2, [\"A\",(H.REDUCE,lambda acc,x:acc+\"|\"+str(x[0]),\">>\")]) == \">>|B|D\"\n\n* MAP -> performn a `transformation` on the `list` or `dict` result \n\n<!-- -->\n\n m2 = {\"A\":{\"B\":[\"C1\",\"C2\",\"C3\"]}}\n assert query(m2, [\"A\",\"B\",(H.MAP,lambda x:x+\"!\")]) == [\"C1!\",\"C2!\",\"C3!\"]\n\n m2 = {\"A\":{\"B\":\"C1\",\"D\":\"C2\"}}\n assert query(m2, [\"A\",(H.MAP,(lambda k,v: v+\"!\"))]) == [\"C1!\",\"C2!\"]\n\n* SUM -> `sum()` the result list, with optional custom function passed before summing.\n\n<!-- -->\n\n m2 = {\"A\":{\"B\":[1,2]}}\n assert query(m2, [\"A\",\"B\",H.SUM]) == 3\n\n m2 = {\"A\":{\"B\":\"1\",\"D\":\"2\"}}\n assert query(m2, [\"A\",S.MVALS,(H.SUM,lambda x: int(x))]) == 3\n \n\n* MAX/MIN -> Using built-in `max()`/`min()` on the `list` result, with optioanl custom function passed before max/min\n\n<!-- -->\n\n m2 = {\"A\":{\"B\":[3,5,2]}}\n assert query(m2, [\"A\",\"B\",H.MIN]) == 2\n assert query(m2, [\"A\",\"B\",(H.MIN,str)]) == \"2\"\n assert query(m2, [\"A\",\"B\",H.MAX]) == 5\n assert query(m2, [\"A\",\"B\",(H.MAX, str)]) == \"5\"\n \n\n\n* ORDER -> using `sorted()` to sort the result list, with optional custom function passed before sorting.\n\n<!-- -->\n\n m2 = {\"A\":{\"B\":[3,5,2]}}\n m2 = {\"A\":{\"B\":[3,5,2]}}\n assert query(m2, [\"A\",\"B\",H.ORDER]) == [2,3,5]\n assert query(m2, [\"A\",\"B\",(H.ORDER, str)]) == [\"2\",\"3\",\"5\"]\n\n \n",
"bugtrack_url": null,
"license": "",
"summary": "A library to query nested data in Python",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/yellowbean/pyspecter"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1d0fae4b663ae3324346bec9ff6a7a3b46583f4f94252b533fb4cc1f0af8167c",
"md5": "c0ce680aeb69d0fdd4cd4fe69ea70b08",
"sha256": "1101f053408fd7927eb7bf6cb89c6aeff3c0ba6de756195e773ada01f4a6ad43"
},
"downloads": -1,
"filename": "pyspecter-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c0ce680aeb69d0fdd4cd4fe69ea70b08",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 6165,
"upload_time": "2023-07-18T17:11:09",
"upload_time_iso_8601": "2023-07-18T17:11:09.605299Z",
"url": "https://files.pythonhosted.org/packages/1d/0f/ae4b663ae3324346bec9ff6a7a3b46583f4f94252b533fb4cc1f0af8167c/pyspecter-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "693e9cf1083e08aada33115d2deb5215626a38c32325f76f1a26e29a005af047",
"md5": "777a515674ca18bb069a265a1ba13bac",
"sha256": "aeb2a4254f55c3863b5004cbd08f82f4b1eea1428d80f2a4553ab3db2dd01a55"
},
"downloads": -1,
"filename": "pyspecter-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "777a515674ca18bb069a265a1ba13bac",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 6127,
"upload_time": "2023-07-18T17:11:10",
"upload_time_iso_8601": "2023-07-18T17:11:10.892909Z",
"url": "https://files.pythonhosted.org/packages/69/3e/9cf1083e08aada33115d2deb5215626a38c32325f76f1a26e29a005af047/pyspecter-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-18 17:11:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yellowbean",
"github_project": "pyspecter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyspecter"
}
JSON
