| Name | compress-json-python JSON |
| Version |
3.0.0
JSON |
| download |
| home_page | |
| Summary | convert JSON data to space efficient format |
| upload_time | 2023-12-17 00:42:43 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | |
| license | BSD 2-Clause License Copyright (c) [2023], [Beeno Tung (Tung Cheung Leong)] All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
| keywords |
json
compress
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# compress-json-python
Store JSON data in space efficient manner.
[](https://pypi.org/project/compress-json-python)
This library is optimized to compress json object in compact format, which can save network bandwidth and disk space.
## Features
- Supports all JSON types
- Object key order is preserved
- Repeated values are stored only once
- Numbers are encoded in base62 format (0-9A-Za-z)
- Support custom backend for memory store and cache
## Multi Language Implementation
This package is a python implementation of [compress-json](https://github.com/beenotung/compress-json). It is fully compatible with the npm package so the data compressed by either side can be decompressed by another side.
### All Implementations
- Javascript/Typescript: [source](https://github.com/beenotung/compress-json) / [package](https://www.npmjs.com/package/compress-json)
- PHP: [source](https://github.com/inkrot/php-compress-json) / [package](https://packagist.org/packages/inkrot/php-compress-json)
- Python: [source](https://github.com/beenotung/compress-json/tree/master/python) / [package](https://pypi.org/project/compress-json-python)
- C#: (TODO)
## Installation
```bash
pip install compress-json-python
```
## Usage Example
```python
# Import functions from the Python package
from compress_json import compress, decompress
data = {
'user': 'Alice',
# more fields of any json values (string, number, array, object, e.t.c.)
}
compressed = compress(data) # the result is a list (array)
import requests
requests.post('https://example.com/submit', json=compressed) # used as json value
import json
with open("data.json", "w") as fd:
fd.write(json.dumps(compressed)) # convert into string if needed
reversed = decompress(compressed)
data === reversed # will be true
```
Detail example can refer to the demo [cli.py](./src/compress_json/cli.py) and tests in [core_test.py](./src/compress_json/core_test.py)
## Compression Format
**Sample data**:
```python
long_str = 'A very very long string, that is repeated'
data = {
'int': 42,
'float': 12.34,
'str': 'Alice',
'long_str': long_str,
'longNum': 9876543210.123455,
'bool': True,
'bool2': False,
'arr': [42, long_str],
'arr2': [42, long_str], # identical values will be deduplidated, including array and object
'obj': { # nested values are supported
'id': 123,
'name': 'Alice',
'role': [ 'Admin', 'User', 'Guest' ],
'long_str': 'A very very long string, that is repeated',
'longNum': 9876543210.123455
},
'escape': [ 's|str', 'n|123', 'o|1', 'a|1', 'b|T', 'b|F' ]
}
```
**Compressed data**:
```python
# [ encoded value array, root value index ]
compressed = [
[ # encoded value array
'int', # string
'float',
'str',
'long_str',
'longNum',
'bool',
'bool2',
'arr',
'arr2',
'obj',
'escape',
'a|0|1|2|3|4|5|6|7|8|9|A',
'n|g', # number (integer) (base62-encoded)
'n|C.h', # number (float) (integer part and decimals are base62-encoded separately)
'Alice',
'A very very long string, that is repeated',
'n|AmOy42.2KCf',
'b|T', # boolean (True)
'b|F', # boolean (False)
'a|C|F', # array
'id',
'name',
'role',
'a|K|L|M|3|4',
'n|1z',
'Admin',
'User',
'Guest',
'a|P|Q|R',
'o|N|O|E|S|F|G', # object
's|s|str', # escaped string
's|n|123', # escaped number
's|o|1',
's|a|1',
's|b|T', # escaped boolean
's|b|F',
'a|U|V|W|X|Y|Z',
'o|B|C|D|E|F|G|H|I|J|J|T|a'
],
'b' # root value index
]
```
## Example structure for efficient compression
**Original JSON data**: (749 characters without white-spaces)
```json
{
"count": 5,
"names": ["New York", "London", "Paris", "Beijing", "Moscow"],
"cities": [
{
"id": 1,
"name": "New York",
"countryName": "USA",
"location": { "latitude": 40.714606, "longitude": -74.0028 },
"localityType": "BIG_CITY"
},
{
"id": 2,
"name": "London",
"countryName": "UK",
"location": { "latitude": 51.507351, "longitude": -0.127696 },
"localityType": "COUNTRY_CAPITAL"
},
{
"id": 3,
"name": "Paris",
"countryName": "France",
"location": { "latitude": 48.856663, "longitude": 2.351556 },
"localityType": "COUNTRY_CAPITAL"
},
{
"id": 4,
"name": "Beijing",
"countryName": "China",
"location": { "latitude": 39.90185, "longitude": 116.391441 },
"localityType": "COUNTRY_CAPITAL"
},
{
"id": 5,
"name": "Moscow",
"countryName": "Russia",
"location": { "latitude": 55.755864, "longitude": 37.617698 },
"localityType": "COUNTRY_CAPITAL"
}
]
}
```
**Compressed json**: (562 characters without white-spaces)
```json
[["count", "names", "cities", "a|0|1|2", "n|5", "New York", "London", "Paris", "Beijing", "Moscow", "a|5|6|7|8|9", "id", "name", "countryName", "location", "localityType", "a|B|C|D|E|F", "n|1", "USA", "latitude", "longitude", "a|J|K", "n|e.2Xkv", "n|-1C.28G", "o|L|M|N", "BIG_CITY", "o|G|H|5|I|O|P", "n|2", "UK", "n|p.dz7", "n|-0.2vFR", "o|L|T|U", "COUNTRY_CAPITAL", "o|G|R|6|S|V|W", "n|3", "France", "n|m.1XNq", "n|2.2kQz", "o|L|a|b", "o|G|Y|7|Z|c|W", "n|4", "China", "n|d.F7F", "n|1s.bVh", "o|L|g|h", "o|G|e|8|f|i|W", "Russia", "n|t.1xtN", "n|b.3lHA", "o|L|l|m", "o|G|4|9|k|n|W", "a|Q|X|d|j|o", "o|3|4|A|p"], "q"]
```
In this example, compression saves 25% of characters. However, the more complex and repetitive the structure, the more characters can be saved.
## License
This project is licensed with [BSD-2-Clause](./LICENSE)
This is free, libre, and open-source software. It comes down to four essential freedoms [[ref]](https://seirdy.one/2021/01/27/whatsapp-and-the-domestication-of-users.html#fnref:2):
- The freedom to run the program as you wish, for any purpose
- The freedom to study how the program works, and change it so it does your computing as you wish
- The freedom to redistribute copies so you can help others
- The freedom to distribute copies of your modified versions to others
Raw data
{
"_id": null,
"home_page": "",
"name": "compress-json-python",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "json,compress",
"author": "",
"author_email": "Beeno Tung <aabbcc1241@yahoo.com.hk>",
"download_url": "https://files.pythonhosted.org/packages/12/93/29df7bf79b8384520823a987e5f4e4d330393c1f250c5e51701d98fd1506/compress-json-python-3.0.0.tar.gz",
"platform": null,
"description": "# compress-json-python\n\nStore JSON data in space efficient manner.\n\n[](https://pypi.org/project/compress-json-python)\n\nThis library is optimized to compress json object in compact format, which can save network bandwidth and disk space.\n\n## Features\n\n- Supports all JSON types\n- Object key order is preserved\n- Repeated values are stored only once\n- Numbers are encoded in base62 format (0-9A-Za-z)\n- Support custom backend for memory store and cache\n\n## Multi Language Implementation\n\nThis package is a python implementation of [compress-json](https://github.com/beenotung/compress-json). It is fully compatible with the npm package so the data compressed by either side can be decompressed by another side.\n\n### All Implementations\n\n- Javascript/Typescript: [source](https://github.com/beenotung/compress-json) / [package](https://www.npmjs.com/package/compress-json)\n- PHP: [source](https://github.com/inkrot/php-compress-json) / [package](https://packagist.org/packages/inkrot/php-compress-json)\n- Python: [source](https://github.com/beenotung/compress-json/tree/master/python) / [package](https://pypi.org/project/compress-json-python)\n- C#: (TODO)\n\n## Installation\n\n```bash\npip install compress-json-python\n```\n\n## Usage Example\n\n```python\n# Import functions from the Python package\nfrom compress_json import compress, decompress\n\ndata = {\n 'user': 'Alice',\n # more fields of any json values (string, number, array, object, e.t.c.)\n}\n\ncompressed = compress(data) # the result is a list (array)\n\nimport requests\nrequests.post('https://example.com/submit', json=compressed) # used as json value\n\nimport json\nwith open(\"data.json\", \"w\") as fd:\n\tfd.write(json.dumps(compressed)) # convert into string if needed\n\nreversed = decompress(compressed)\ndata === reversed # will be true\n```\n\nDetail example can refer to the demo [cli.py](./src/compress_json/cli.py) and tests in [core_test.py](./src/compress_json/core_test.py)\n\n## Compression Format\n\n**Sample data**:\n\n```python\nlong_str = 'A very very long string, that is repeated'\ndata = {\n 'int': 42,\n 'float': 12.34,\n 'str': 'Alice',\n 'long_str': long_str,\n 'longNum': 9876543210.123455,\n 'bool': True,\n 'bool2': False,\n 'arr': [42, long_str],\n 'arr2': [42, long_str], # identical values will be deduplidated, including array and object\n 'obj': { # nested values are supported\n 'id': 123,\n 'name': 'Alice',\n 'role': [ 'Admin', 'User', 'Guest' ],\n 'long_str': 'A very very long string, that is repeated',\n 'longNum': 9876543210.123455\n },\n 'escape': [ 's|str', 'n|123', 'o|1', 'a|1', 'b|T', 'b|F' ]\n}\n```\n\n**Compressed data**:\n\n```python\n# [ encoded value array, root value index ]\ncompressed = [\n [ # encoded value array\n 'int', # string\n 'float',\n 'str',\n 'long_str',\n 'longNum',\n 'bool',\n 'bool2',\n 'arr',\n 'arr2',\n 'obj',\n 'escape',\n 'a|0|1|2|3|4|5|6|7|8|9|A',\n 'n|g', # number (integer) (base62-encoded)\n 'n|C.h', # number (float) (integer part and decimals are base62-encoded separately)\n 'Alice',\n 'A very very long string, that is repeated',\n 'n|AmOy42.2KCf',\n 'b|T', # boolean (True)\n 'b|F', # boolean (False)\n 'a|C|F', # array\n 'id',\n 'name',\n 'role',\n 'a|K|L|M|3|4',\n 'n|1z',\n 'Admin',\n 'User',\n 'Guest',\n 'a|P|Q|R',\n 'o|N|O|E|S|F|G', # object\n 's|s|str', # escaped string\n 's|n|123', # escaped number\n 's|o|1',\n 's|a|1',\n 's|b|T', # escaped boolean\n 's|b|F',\n 'a|U|V|W|X|Y|Z',\n 'o|B|C|D|E|F|G|H|I|J|J|T|a'\n ],\n 'b' # root value index\n]\n```\n\n## Example structure for efficient compression\n\n**Original JSON data**: (749 characters without white-spaces)\n\n```json\n{\n \"count\": 5,\n \"names\": [\"New York\", \"London\", \"Paris\", \"Beijing\", \"Moscow\"],\n \"cities\": [\n {\n \"id\": 1,\n \"name\": \"New York\",\n \"countryName\": \"USA\",\n \"location\": { \"latitude\": 40.714606, \"longitude\": -74.0028 },\n \"localityType\": \"BIG_CITY\"\n },\n {\n \"id\": 2,\n \"name\": \"London\",\n \"countryName\": \"UK\",\n \"location\": { \"latitude\": 51.507351, \"longitude\": -0.127696 },\n \"localityType\": \"COUNTRY_CAPITAL\"\n },\n {\n \"id\": 3,\n \"name\": \"Paris\",\n \"countryName\": \"France\",\n \"location\": { \"latitude\": 48.856663, \"longitude\": 2.351556 },\n \"localityType\": \"COUNTRY_CAPITAL\"\n },\n {\n \"id\": 4,\n \"name\": \"Beijing\",\n \"countryName\": \"China\",\n \"location\": { \"latitude\": 39.90185, \"longitude\": 116.391441 },\n \"localityType\": \"COUNTRY_CAPITAL\"\n },\n {\n \"id\": 5,\n \"name\": \"Moscow\",\n \"countryName\": \"Russia\",\n \"location\": { \"latitude\": 55.755864, \"longitude\": 37.617698 },\n \"localityType\": \"COUNTRY_CAPITAL\"\n }\n ]\n}\n```\n\n**Compressed json**: (562 characters without white-spaces)\n\n```json\n[[\"count\", \"names\", \"cities\", \"a|0|1|2\", \"n|5\", \"New York\", \"London\", \"Paris\", \"Beijing\", \"Moscow\", \"a|5|6|7|8|9\", \"id\", \"name\", \"countryName\", \"location\", \"localityType\", \"a|B|C|D|E|F\", \"n|1\", \"USA\", \"latitude\", \"longitude\", \"a|J|K\", \"n|e.2Xkv\", \"n|-1C.28G\", \"o|L|M|N\", \"BIG_CITY\", \"o|G|H|5|I|O|P\", \"n|2\", \"UK\", \"n|p.dz7\", \"n|-0.2vFR\", \"o|L|T|U\", \"COUNTRY_CAPITAL\", \"o|G|R|6|S|V|W\", \"n|3\", \"France\", \"n|m.1XNq\", \"n|2.2kQz\", \"o|L|a|b\", \"o|G|Y|7|Z|c|W\", \"n|4\", \"China\", \"n|d.F7F\", \"n|1s.bVh\", \"o|L|g|h\", \"o|G|e|8|f|i|W\", \"Russia\", \"n|t.1xtN\", \"n|b.3lHA\", \"o|L|l|m\", \"o|G|4|9|k|n|W\", \"a|Q|X|d|j|o\", \"o|3|4|A|p\"], \"q\"]\n```\n\nIn this example, compression saves 25% of characters. However, the more complex and repetitive the structure, the more characters can be saved.\n\n## License\n\nThis project is licensed with [BSD-2-Clause](./LICENSE)\n\nThis is free, libre, and open-source software. It comes down to four essential freedoms [[ref]](https://seirdy.one/2021/01/27/whatsapp-and-the-domestication-of-users.html#fnref:2):\n\n- The freedom to run the program as you wish, for any purpose\n- The freedom to study how the program works, and change it so it does your computing as you wish\n- The freedom to redistribute copies so you can help others\n- The freedom to distribute copies of your modified versions to others\n",
"bugtrack_url": null,
"license": "BSD 2-Clause License Copyright (c) [2023], [Beeno Tung (Tung Cheung Leong)] All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ",
"summary": "convert JSON data to space efficient format",
"version": "3.0.0",
"project_urls": {
"Homepage": "https://github.com/beenotung/compress-json"
},
"split_keywords": [
"json",
"compress"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "57c240b825be74262c82152fdda0370acfbd36024effc1b07675a4592773c341",
"md5": "94846120da6ef2710a7d1b17f631b680",
"sha256": "0e55fe713439c9f351a53570481c32095ab9e3285cec29a7f67a557881f35474"
},
"downloads": -1,
"filename": "compress_json_python-3.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "94846120da6ef2710a7d1b17f631b680",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 12888,
"upload_time": "2023-12-17T00:42:41",
"upload_time_iso_8601": "2023-12-17T00:42:41.913861Z",
"url": "https://files.pythonhosted.org/packages/57/c2/40b825be74262c82152fdda0370acfbd36024effc1b07675a4592773c341/compress_json_python-3.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "129329df7bf79b8384520823a987e5f4e4d330393c1f250c5e51701d98fd1506",
"md5": "5a775a6321f1c55cfbc6cbfefa781c05",
"sha256": "78c8740ab9174f93a95c20e71a4ed8d424917f9b2567c3bfcd86896fba48bbe8"
},
"downloads": -1,
"filename": "compress-json-python-3.0.0.tar.gz",
"has_sig": false,
"md5_digest": "5a775a6321f1c55cfbc6cbfefa781c05",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 12976,
"upload_time": "2023-12-17T00:42:43",
"upload_time_iso_8601": "2023-12-17T00:42:43.976332Z",
"url": "https://files.pythonhosted.org/packages/12/93/29df7bf79b8384520823a987e5f4e4d330393c1f250c5e51701d98fd1506/compress-json-python-3.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-17 00:42:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "beenotung",
"github_project": "compress-json",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "compress-json-python"
}
