# pyflattener
One useful utility for making Swimlane Bundles is the Flattener, which helps to simplify complex JSON.
## Basic Usage
```python
from pyflattener import Flattener, do_flatten
data = {
"outer_key": {
"inner_key1": "inner val 1",
"inner_key2": "inner val 2"
},
"basic_key": "value",
"basic_list": ["1", "2", "3"],
"mixed_list": [1, "2", "3"],
"basic_list2": [1, 2, 3]
}
# Simple flatten
flat_data = do_flatten(data)
# Instance analogue
flat_data = Flattener(prefix=None, stringify_lists=True, shallow_flatten=False,
keep_simple_lists=True, autoparse_dt_strs=True).flatten(data)
```
flat_data will now look like:
```python
{
'outer_key_inner_key1': 'inner val 1',
'basic_list': ['1', '2', '3'], # Simple list kept as a list
'basic_key': 'value',
'outer_key_inner_key2': 'inner val 2',
'basic_list2': [1, 2, 3], # Simple list kept as list
'mixed_list': '1,2,3' # Nonsimple list CVSV'd
}
```
Here is a description of the params you can pass to a Flattener() object or do_flatten
### Prefix
Prefix to add to the data after flattening
```python
do_flatten({"a": 5}, prefix="my_prefix")
# {"my_prefix_a": 5}
```
### Stringify Lists
Turn lists with basic types into CSV, defaults to True. This option is ignored for simple lists if keep_simple_lists is True
```python
stringify = <True or False>
do_flatten({"a": [1,2,3]}, stringify_lists=stringify, keep_simple_lists=False)
# True -> {"a": "1,2,3"}
# False -> {"a": [1,2,3]}
```
### Shallow Flatten
Ignore the first level of nesting, and only flatten each element within it. Used for lists of dictionaries
```python
data = [
{"a": { "sub_a": 1 }, "b": 5},
{"a": { "sub_a": 2 }, "b": 6},
]
shallow = <True or False>
do_flatten(data, shallow_flatten=shallow)
# True -> [
# {"a_sub_a": 1, "b": 5},
# {"a_sub_a": 2, "b": 6}
# ]
# False -> {"a_sub_a": [1,2], "b": [5,6]}
```
### Keep Simple Lists
If a list in the resulting flattened dict is only integers or only strings, even if stringify_lists is True, keep this list
```python
simple = <True or False>
do_flatten({"a": [1,2,3], "b": ["c", 4]}, keep_simple_lists=simple)
# True -> {"a": [1,2,3], "b": "c,4"}
# False -> {"a": "1,2,3", "b": "c,4"}
```
## Misc Flattening Functions
There are many useful flattening functions for more complicated data
### Hoist Key(s)
Grab keys from a list of dicts
```python
hoist_key("a", [{"a": 5}, {"a": 6}])
# -> [5, 6]
hoist_keys(["a", "b"], [{"a": 5, "b": 1}, {"a": 6, "b": 2}])
# -> [[5, 6], [1, 2]]
```
### Replace Dict Prefix
Replace a prefix in a dictionary
```python
replace_dict_prefix("aaa", "bbbb", {"aaa_data": 5})
# -> {'bbbb_data': 5}
# Or more commonly like:
replace_dict_prefix("aaa_", "", {"aaa_data": 5})
# -> {'data': 5}
```
### Merge Dicts
Merge two dictionaries together, regardless if they share keys or not.
If they share keys, it uses combine_listdict
```python
merge_dicts({"a": 1}, {"b": 2})
# -> {"a": 1, "b": 2}
merge_dicts({"a": 1}, {"a": 2})
# -> {"a": [1, 2]})
```
### Is SimpleList
Check if a list is purely of integers or purely of strings
```python
is_simplelist([1,2,3])
# -> True
is_simplelist([1,2,"3"])
# -> False
```
### Flatten Single Lists
Flatten all keys in a dict that are lists with a single entry
```python
flatten_single_lists({"a": [1,2,3], "b": [5]})
# -> {"b": 5,"a": [1, 2, 3]}
```
### Combine ListDict
Combine a list of dictionaries into a single dictionary
```python
combine_listdict([{"a": 1},{"a": 2}, {"a": 3}])
# -> {"a": [1, 2, 3]}
complicated_data = [
{
"a": "entry 1",
"b": "v1"
},
{
"b": "v2"
},
{
"a": "entry 2"
}
]
combine_listdict(complicated_data)
# -> {
# 'a': ['entry 1', None, 'entry 2'],
# 'b': ['v1', 'v2', None]
#}
```
Note how the missing entries were filled in with None This is to ensure the ordering of elements can be
obtained in the flattened dict result.
Also note that attempting to combine a list of dictionaries with nonbasic keys (subdicts or lists) can
lead to odd results, or not be possible to combine in that form
### Clean XMLToDict Result
XMLToDict returns very ugly data, this helps clean it up. It only cleans top-level keys, so it is most
effective after flattening
```python
import xmltodict
ugly_xml = "<xml><key attr=\"5\">val</key></xml>"
xml_dict = xmltodict.parse(ugly_xml)
clean_xmltodict_result(do_flatten(xml_dict))
# -> {u'xml_key_text': u'val', u'xml_key_attr': "5"}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/swimlane/pyflattener",
"name": "pyflattener",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Swimlane",
"author_email": "info@swimlane.com",
"download_url": "https://files.pythonhosted.org/packages/46/74/105c2a8422666c1c78b53172b25c1370b11937c03944c95ad008127f0337/pyflattener-1.1.0.tar.gz",
"platform": null,
"description": "# pyflattener\nOne useful utility for making Swimlane Bundles is the Flattener, which helps to simplify complex JSON.\n\n## Basic Usage\n```python\nfrom pyflattener import Flattener, do_flatten\n\ndata = {\n \"outer_key\": {\n \"inner_key1\": \"inner val 1\",\n \"inner_key2\": \"inner val 2\"\n },\n \"basic_key\": \"value\",\n \"basic_list\": [\"1\", \"2\", \"3\"],\n \"mixed_list\": [1, \"2\", \"3\"],\n \"basic_list2\": [1, 2, 3]\n}\n\n# Simple flatten\nflat_data = do_flatten(data)\n\n# Instance analogue\nflat_data = Flattener(prefix=None, stringify_lists=True, shallow_flatten=False,\n keep_simple_lists=True, autoparse_dt_strs=True).flatten(data)\n```\nflat_data will now look like:\n```python\n{\n'outer_key_inner_key1': 'inner val 1',\n'basic_list': ['1', '2', '3'], # Simple list kept as a list\n'basic_key': 'value',\n'outer_key_inner_key2': 'inner val 2',\n'basic_list2': [1, 2, 3], # Simple list kept as list\n'mixed_list': '1,2,3' # Nonsimple list CVSV'd\n}\n```\nHere is a description of the params you can pass to a Flattener() object or do_flatten\n\n### Prefix\nPrefix to add to the data after flattening\n```python\ndo_flatten({\"a\": 5}, prefix=\"my_prefix\")\n# {\"my_prefix_a\": 5}\n```\n\n### Stringify Lists\nTurn lists with basic types into CSV, defaults to True. This option is ignored for simple lists if keep_simple_lists is True\n```python\nstringify = <True or False>\ndo_flatten({\"a\": [1,2,3]}, stringify_lists=stringify, keep_simple_lists=False)\n# True -> {\"a\": \"1,2,3\"}\n# False -> {\"a\": [1,2,3]}\n```\n\n### Shallow Flatten\nIgnore the first level of nesting, and only flatten each element within it. Used for lists of dictionaries\n```python\ndata = [\n {\"a\": { \"sub_a\": 1 }, \"b\": 5},\n {\"a\": { \"sub_a\": 2 }, \"b\": 6},\n]\nshallow = <True or False>\ndo_flatten(data, shallow_flatten=shallow)\n# True -> [\n# {\"a_sub_a\": 1, \"b\": 5},\n# {\"a_sub_a\": 2, \"b\": 6}\n# ]\n\n# False -> {\"a_sub_a\": [1,2], \"b\": [5,6]}\n```\n\n### Keep Simple Lists\nIf a list in the resulting flattened dict is only integers or only strings, even if stringify_lists is True, keep this list\n```python\nsimple = <True or False>\ndo_flatten({\"a\": [1,2,3], \"b\": [\"c\", 4]}, keep_simple_lists=simple)\n# True -> {\"a\": [1,2,3], \"b\": \"c,4\"}\n# False -> {\"a\": \"1,2,3\", \"b\": \"c,4\"}\n```\n\n## Misc Flattening Functions\nThere are many useful flattening functions for more complicated data\n\n### Hoist Key(s)\n\nGrab keys from a list of dicts\n```python\nhoist_key(\"a\", [{\"a\": 5}, {\"a\": 6}])\n# -> [5, 6]\n\nhoist_keys([\"a\", \"b\"], [{\"a\": 5, \"b\": 1}, {\"a\": 6, \"b\": 2}])\n# -> [[5, 6], [1, 2]]\n```\n\n### Replace Dict Prefix\nReplace a prefix in a dictionary\n```python\nreplace_dict_prefix(\"aaa\", \"bbbb\", {\"aaa_data\": 5})\n# -> {'bbbb_data': 5}\n# Or more commonly like:\n\nreplace_dict_prefix(\"aaa_\", \"\", {\"aaa_data\": 5})\n# -> {'data': 5}\n```\n### Merge Dicts\nMerge two dictionaries together, regardless if they share keys or not.\nIf they share keys, it uses combine_listdict\n```python\nmerge_dicts({\"a\": 1}, {\"b\": 2})\n# -> {\"a\": 1, \"b\": 2}\n\nmerge_dicts({\"a\": 1}, {\"a\": 2})\n# -> {\"a\": [1, 2]})\n```\n### Is SimpleList\n\nCheck if a list is purely of integers or purely of strings\n```python\nis_simplelist([1,2,3])\n# -> True\nis_simplelist([1,2,\"3\"])\n# -> False\n```\n\n### Flatten Single Lists\n\nFlatten all keys in a dict that are lists with a single entry\n```python\nflatten_single_lists({\"a\": [1,2,3], \"b\": [5]})\n# -> {\"b\": 5,\"a\": [1, 2, 3]}\n```\n\n### Combine ListDict\n\nCombine a list of dictionaries into a single dictionary\n```python\ncombine_listdict([{\"a\": 1},{\"a\": 2}, {\"a\": 3}])\n# -> {\"a\": [1, 2, 3]}\n\ncomplicated_data = [\n {\n \"a\": \"entry 1\",\n \"b\": \"v1\"\n },\n {\n \"b\": \"v2\"\n },\n {\n \"a\": \"entry 2\"\n }\n]\ncombine_listdict(complicated_data)\n# -> {\n# 'a': ['entry 1', None, 'entry 2'],\n# 'b': ['v1', 'v2', None]\n#}\n```\nNote how the missing entries were filled in with None This is to ensure the ordering of elements can be \nobtained in the flattened dict result.\n\nAlso note that attempting to combine a list of dictionaries with nonbasic keys (subdicts or lists) can\nlead to odd results, or not be possible to combine in that form\n\n### Clean XMLToDict Result\n\nXMLToDict returns very ugly data, this helps clean it up. It only cleans top-level keys, so it is most \neffective after flattening\n\n```python\nimport xmltodict\n\nugly_xml = \"<xml><key attr=\\\"5\\\">val</key></xml>\"\nxml_dict = xmltodict.parse(ugly_xml)\nclean_xmltodict_result(do_flatten(xml_dict))\n# -> {u'xml_key_text': u'val', u'xml_key_attr': \"5\"}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Flatten nested python objects.",
"version": "1.1.0",
"project_urls": {
"Bug Tracker": "https://github.com/swimlane/pyflattener/issues",
"Homepage": "https://github.com/swimlane/pyflattener"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b414e4d311ba011b6037592bc9dd1d50e509296ca325b4993a6eaf9b30bedf11",
"md5": "cc30f01fb3c5ab4a7e263f29061645b9",
"sha256": "10d248c1c00de7160353650f2736671c0698c7634fb25d798f866bfe30979ca1"
},
"downloads": -1,
"filename": "pyflattener-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cc30f01fb3c5ab4a7e263f29061645b9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 11226,
"upload_time": "2024-07-03T17:17:43",
"upload_time_iso_8601": "2024-07-03T17:17:43.410821Z",
"url": "https://files.pythonhosted.org/packages/b4/14/e4d311ba011b6037592bc9dd1d50e509296ca325b4993a6eaf9b30bedf11/pyflattener-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4674105c2a8422666c1c78b53172b25c1370b11937c03944c95ad008127f0337",
"md5": "c626a287790831e268aa6a15a49a5086",
"sha256": "22c2041f872666e05bc7e24a0f279a3412b004ea0d3b09bb9217118ac1432ef3"
},
"downloads": -1,
"filename": "pyflattener-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "c626a287790831e268aa6a15a49a5086",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 14628,
"upload_time": "2024-07-03T17:17:44",
"upload_time_iso_8601": "2024-07-03T17:17:44.428734Z",
"url": "https://files.pythonhosted.org/packages/46/74/105c2a8422666c1c78b53172b25c1370b11937c03944c95ad008127f0337/pyflattener-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-03 17:17:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "swimlane",
"github_project": "pyflattener",
"github_not_found": true,
"lcname": "pyflattener"
}
JSON
