serializejson
=============
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **Authors** | `Baptiste de La Gorce <contact@smartaudiotools.com>`_ |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **PyPI** | https://pypi.org/project/serializejson |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **Documentation** | https://smartaudiotools.github.io/serializejson |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **Sources** | https://github.com/SmartAudioTools/serializejson |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **Issues** | https://github.com/SmartAudioTools/serializejson/issues |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **Noncommercial license** | `Prosperity Public License 3.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PROSPERITY.rst>`_ |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
| **Commercial license** | `Patron License 1.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PATRON.rst>`_ |
| | ⇒ `Sponsor me ! <https://github.com/sponsors/SmartAudioTools>`_ or `contact me ! <contact@smartaudiotools.com>`_ |
+---------------------------+--------------------------------------------------------------------------------------------------------------------------+
**serializejson** is a python library for fast serialization and deserialization
of python objects in `JSON <http://json.org>`_ designed as a safe, interoperable and human-readable drop-in replacement for the Python `pickle <https://docs.python.org/3/library/pickle.html>`_ package.
Complex python object hierarchies are serializable, deserializable or updatable in once, allowing for example to save or restore a complete application state in few lines of code.
The library is build upon
`python-rapidjson <https://github.com/python-rapidjson/python-rapidjson>`_,
`pybase64 <https://github.com/mayeut/pybase64>`_ and
`blosc <https://github.com/Blosc/python-blosc>`_ for optional `zstandard <https://github.com/facebook/zstd>`_ compression.
Some of the main features:
- supports Python 3.7 (maybe lower) or greater.
- serializes arbitrary python objects into a dictionary by adding `__class__` ,and eventually `__init__`, `__new__`, `__state__`, `__items__` keys.
- calls the same objects methods as pickle. Therefore almost all pickable objects are serializable with serializejson without any modification.
- for not already pickable object, you will allways be able to serialize it by adding methodes to the object or creating plugins for pickle or serializejson.
- generally 2x slower than pickle for dumping and 3x slower than pickle for loading (on your benchmark) except for big arrays (optimisation will soon be done).
- serializes and deserializes bytes and bytearray very quickly in base64 thanks to `pybase64 <https://github.com/mayeut/pybase64>`_ and lossless `blosc <https://github.com/Blosc/python-blosc>`_ compression.
- serialize properties and attributes with getters and setters if wanted (unlike pickle).
- json data will still be directly loadable if you have transform some attributes in slots or properties in your code since your last serialization. (unlike pickle)
- can serialize `__init__(self,..)` arguments by name instead of positions, allowing to skip arguments with defauts values and making json datas robust to a change of `__init__` parameters order.
- serialized objects take generally less space than when serialized with pickle: for binary data, the 30% increase due to base64 encoding is in general largely compensated using the lossless `blosc <https://github.com/Blosc/python-blosc>`_ compression.
- serialized objects are human-readable and easy to read. Unlike pickled data, your data will never become unreadable if your code evolves: you will always be able to modify your datas with a text editor (with find & replace for example if you change an attribut name).
- serialized objects are text and therefore versionable and comparable with versionning and comparaison tools.
- can safely load untrusted / unauthenticated sources if authorized_classes list parameter is set carefully with strictly necessary objects (unlike pickle).
- can update existing objects recursively instead of override them. serializejson can be used to save and restore in place a complete application state (⚠ not yet well tested).
- filters attribute starting with "_" by default (unlike pickle). You can keep them if wanted with `filter_ = False`.
- numpy arrays can be serialized as lists with automatic conversion in both ways or in a conservative way.
- supports circular references and serialize only once duplicated objects, using "$ref" key an path to the first occurance in the json : `{"$ref": "root.xxx.elt"}` (⚠ not yet if the object is a list or dictionary).
- accepts json with comment (// and /\* \*/) if `accept_comments = True`.
- can automatically recognize objects in json from keys names and recreate them, without the need of `__class__` key, if passed in `recognized_classes`.
- serializejson is easly interoperable outside of the Python ecosystem with this recognition of objects from keys names or with `__class__` translation between python and other language classes.
- dump and load support string path.
- can iteratively encode (with append) and decode (with iterator) a list in json file, which helps saving memory space during the process of serialization and deserialization and useful for logs.
.. warning::
**⚠** Do not load serializejson files from untrusted / unauthenticated sources without carefully setting the load authorized_classes parameter.
**⚠** Never dump a dictionary with the `__class__` key, otherwise serializejson will attempt to reconstruct an object when loading the json.
Be careful not to allow a user to manually enter a dictionary key somewhere without checking that it is not `__class__`.
Due to current limitation of rapidjson we cannot we cannot at the moment efficiently detect dictionaries with the `__class__` key to raise an error.
Installation
============
**Last offical release**
.. code-block::
pip install serializejson
**Developpement version unreleased**
.. code-block::
pip install git+https://github.com/SmartAudioTools/serializejson.git
Examples
================
**Serialization with fonctions API**
.. code-block:: python
import serializejson
# serialize in string
object1 = set([1,2])
dumped1 = serializejson.dumps(object1)
loaded1 = serializejson.loads(dumped1)
print(dumped1)
>{
> "__class__": "set",
> "__init__": [1,2]
>}
# serialize in file
object2 = set([3,4])
serializejson.dump(object2,"dumped2.json")
loaded2 = serializejson.load("dumped2.json")
**Serialization with classes based API.**
.. code-block:: python
import serializejson
encoder = serializejson.Encoder()
decoder = serializejson.Decoder()
# serialize in string
object1 = set([1,2])
dumped1 = encoder.dumps(object1)
loaded1 = decoder.loads(dumped1)
print(dumped1)
# serialize in file
object2 = set([3,4])
encoder.dump(object2,"dumped2.json")
loaded2 = decoder.load("dumped2.json")
**Update existing object**
.. code-block:: python
import serializejson
object1 = set([1,2])
object2 = set([3,4])
dumped1 = serializejson.dumps(object1)
print(f"id {id(object2)} : {object2}")
serializejson.loads(dumped1,obj = object2, updatables_classes = [set])
print(f"id {id(object2)} : {object2}")
**Iterative serialization and deserialization**
.. code-block:: python
import serializejson
encoder = serializejson.Encoder("my_list.json",indent = None)
for elt in range(3):
encoder.append(elt)
print(open("my_list.json").read())
for elt in serializejson.Decoder("my_list.json"):
print(elt)
>[0,1,2]
>0
>1
>2
More examples and complete documentation `here <https://smartaudiotools.github.io/serializejson/>`_
License
=======
Copyright 2020 Baptiste de La Gorce
For noncommercial use or thirty-day limited free-trial period commercial use, this project is licensed under the `Prosperity Public License 3.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PROSPERITY.rst>`_.
For non limited commercial use, this project is licensed under the `Patron License 1.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PATRON.rst>`_.
To acquire a license please `contact me <mailto:contact@smartaudiotools.com>`_, or just `sponsor me on GitHub <https://github.com/sponsors/SmartAudioTools>`_ under the appropriate tier ! This funding model helps me making my work sustainable and compensates me for the work it took to write this crate!
Third-party contributions are licensed under `Apache License, Version 2.0 <http://www.apache.org/licenses/LICENSE-2.0>`_ and belong to their respective authors.
History
=======
Version 0.3.4
-------------
:Date: 2023-06-11
* Restore ducumentation
Version 0.3.3
-------------
:Date: 2022-10-18
* Big speed improvement for bytes and numpy array serialization
Version 0.3.2
-------------
:Date: 2022-10-01
* API changed
* add better support for cicular reférences and duplicates with {"$ref": ...}
Version 0.2.0
-------------
:Date: 2021-02-18
* API changed
* can serialize dict with no-string keys
* add support for cicular reférences and duplicates with {"$ref": ...}
Version 0.1.0
-------------
:Date: 2020-11-28
* change description for pipy
* add license for pipy
* enable load of tuple, time.struct_time, Counter, OrderedDict and defaultdict
Version 0.0.4
-------------
:Date: 2020-11-24
* API changed
* add plugins support
* add bytes, bytearray and numpy.array compression with blosc zstd
* fix itertive append and decode (not fully tested).
* fix dump of numpy types without conversion to python types(not yet numpy.float64)
Raw data
{
"_id": null,
"home_page": "https://github.com/SmartAudioTools/serializejson",
"name": "serializejson",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3",
"maintainer_email": "",
"keywords": "pickle json serialize dump dumps rapidjson base64",
"author": "Baptiste de La Gorce",
"author_email": "baptiste.delagorce@smartaudiotools.com",
"download_url": "https://files.pythonhosted.org/packages/9a/8c/d4e859aca9bf04452fb06bd74e6033f47e80934b47e7529eb635d9075a5e/serializejson-0.3.4.tar.gz",
"platform": null,
"description": "serializejson\n=============\n\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **Authors** | `Baptiste de La Gorce <contact@smartaudiotools.com>`_ |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **PyPI** | https://pypi.org/project/serializejson |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **Documentation** | https://smartaudiotools.github.io/serializejson |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **Sources** | https://github.com/SmartAudioTools/serializejson |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **Issues** | https://github.com/SmartAudioTools/serializejson/issues |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **Noncommercial license** | `Prosperity Public License 3.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PROSPERITY.rst>`_ |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n| **Commercial license** | `Patron License 1.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PATRON.rst>`_ |\n| | \u21d2 `Sponsor me ! <https://github.com/sponsors/SmartAudioTools>`_ or `contact me ! <contact@smartaudiotools.com>`_ |\n+---------------------------+--------------------------------------------------------------------------------------------------------------------------+\n\n\n**serializejson** is a python library for fast serialization and deserialization\nof python objects in `JSON <http://json.org>`_ designed as a safe, interoperable and human-readable drop-in replacement for the Python `pickle <https://docs.python.org/3/library/pickle.html>`_ package.\nComplex python object hierarchies are serializable, deserializable or updatable in once, allowing for example to save or restore a complete application state in few lines of code.\nThe library is build upon\n`python-rapidjson <https://github.com/python-rapidjson/python-rapidjson>`_,\n`pybase64 <https://github.com/mayeut/pybase64>`_ and\n`blosc <https://github.com/Blosc/python-blosc>`_ for optional `zstandard <https://github.com/facebook/zstd>`_ compression.\n\nSome of the main features:\n\n- supports Python 3.7 (maybe lower) or greater.\n- serializes arbitrary python objects into a dictionary\u00a0by adding `__class__` ,and eventually `__init__`, `__new__`, `__state__`, `__items__` keys.\n- calls the same objects methods as pickle. Therefore almost all pickable objects are serializable with serializejson without any modification.\n- for not already pickable object, you will allways be able to serialize it by adding methodes to the object or creating plugins for pickle or serializejson.\n- generally 2x slower than pickle for dumping and 3x slower than pickle for loading (on your benchmark) except for big arrays (optimisation will soon be done).\n- serializes and deserializes bytes and bytearray very quickly in base64 thanks to `pybase64 <https://github.com/mayeut/pybase64>`_ and lossless `blosc <https://github.com/Blosc/python-blosc>`_ compression.\n- serialize properties and attributes with getters and setters if wanted (unlike pickle).\n- json data will still be directly loadable if you have transform some attributes in slots or properties in your code since your last serialization. (unlike pickle)\n- can serialize `__init__(self,..)` arguments by name instead of positions, allowing to skip arguments with defauts values and making json datas robust to a change of `__init__` parameters order.\n- serialized objects take generally less space than when serialized with pickle: for binary data, the 30% increase due to base64 encoding is in general largely compensated using the lossless `blosc <https://github.com/Blosc/python-blosc>`_ compression.\n- serialized objects are human-readable and easy to read. Unlike pickled data, your data will never become unreadable if your code evolves: you will always be able to modify your datas with a text editor (with find & replace for example if you change an attribut name).\n- serialized objects are text and therefore versionable and comparable with versionning and comparaison tools.\n- can safely load untrusted / unauthenticated sources if authorized_classes list parameter is set carefully with strictly necessary objects (unlike pickle).\n- can update existing objects recursively instead of override them. serializejson can be used to save and restore in place a complete application state (\u26a0 not yet well tested).\n- filters attribute starting with \"_\" by default (unlike pickle). You can keep them if wanted with `filter_ = False`.\n- numpy arrays can be serialized as lists with automatic conversion in both ways or in a conservative way.\n- supports circular references and serialize only once duplicated objects, using \"$ref\" key an path to the first occurance in the json : `{\"$ref\": \"root.xxx.elt\"}` (\u26a0 not yet if the object is a list or dictionary).\n- accepts json with comment (// and /\\* \\*/) if `accept_comments = True`.\n- can automatically recognize objects in json from keys names and recreate them, without the need of `__class__` key, if passed\u00a0in `recognized_classes`.\n- serializejson is easly interoperable outside of the Python ecosystem with this recognition of objects from keys names or with `__class__` translation between python and other language classes.\n- dump and load support string path.\n- can iteratively encode (with append) and decode (with iterator) a list in json file, which helps saving memory space during the process of serialization and deserialization and useful for logs.\n\n.. warning::\n\n **\u26a0** Do not load serializejson files from untrusted / unauthenticated sources without carefully setting the load authorized_classes parameter.\n\n **\u26a0** Never dump a dictionary with the `__class__` key, otherwise serializejson will attempt to reconstruct an object when loading the json.\n Be careful not to allow a user to manually enter a dictionary key somewhere without checking that it is not `__class__`.\n Due to current limitation of rapidjson we cannot we cannot at the moment efficiently detect dictionaries with the `__class__` key to raise an error.\n\n\nInstallation\n============\n\n**Last offical release**\n\n.. code-block::\n\n pip install serializejson\n\n**Developpement version unreleased**\n\n.. code-block::\n\n pip install git+https://github.com/SmartAudioTools/serializejson.git\n\nExamples\n================\n\n**Serialization with fonctions API**\n\n.. code-block:: python\n\n import serializejson\n\n # serialize in string\n object1 = set([1,2])\n dumped1 = serializejson.dumps(object1)\n loaded1 = serializejson.loads(dumped1)\n print(dumped1)\n >{\n > \"__class__\": \"set\",\n > \"__init__\": [1,2]\n >}\n\n\n # serialize in file\n object2 = set([3,4])\n serializejson.dump(object2,\"dumped2.json\")\n loaded2 = serializejson.load(\"dumped2.json\")\n\n**Serialization with classes based API.**\n\n.. code-block:: python\n\n import serializejson\n encoder = serializejson.Encoder()\n decoder = serializejson.Decoder()\n\n # serialize in string\n\n object1 = set([1,2])\n dumped1 = encoder.dumps(object1)\n loaded1 = decoder.loads(dumped1)\n print(dumped1)\n\n # serialize in file\n object2 = set([3,4])\n encoder.dump(object2,\"dumped2.json\")\n loaded2 = decoder.load(\"dumped2.json\")\n\n**Update existing object**\n\n.. code-block:: python\n\n import serializejson\n object1 = set([1,2])\n object2 = set([3,4])\n dumped1 = serializejson.dumps(object1)\n print(f\"id {id(object2)} : {object2}\")\n serializejson.loads(dumped1,obj = object2, updatables_classes = [set])\n print(f\"id {id(object2)} : {object2}\")\n\n**Iterative serialization and deserialization**\n\n.. code-block:: python\n\n import serializejson\n encoder = serializejson.Encoder(\"my_list.json\",indent = None)\n for elt in range(3):\n encoder.append(elt)\n print(open(\"my_list.json\").read())\n for elt in serializejson.Decoder(\"my_list.json\"):\n print(elt)\n >[0,1,2]\n >0\n >1\n >2\n\nMore examples and complete documentation `here <https://smartaudiotools.github.io/serializejson/>`_\n\nLicense\n=======\n\nCopyright 2020 Baptiste de La Gorce\n\nFor noncommercial use or thirty-day limited free-trial period commercial use, this project is licensed under the `Prosperity Public License 3.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PROSPERITY.rst>`_.\n\nFor non limited commercial use, this project is licensed under the `Patron License 1.0.0 <https://github.com/SmartAudioTools/serializejson/blob/master/LICENSE-PATRON.rst>`_.\nTo acquire a license please `contact me <mailto:contact@smartaudiotools.com>`_, or just `sponsor me on GitHub <https://github.com/sponsors/SmartAudioTools>`_ under the appropriate tier ! This funding model helps me making my work sustainable and compensates me for the work it took to write this crate!\n\nThird-party contributions are licensed under `Apache License, Version 2.0 <http://www.apache.org/licenses/LICENSE-2.0>`_ and belong to their respective authors.\nHistory\n=======\n\nVersion 0.3.4\n-------------\n:Date: 2023-06-11\n\n* Restore ducumentation\n\n\nVersion 0.3.3\n-------------\n:Date: 2022-10-18\n\n* Big speed improvement for bytes and numpy array serialization\n\nVersion 0.3.2\n-------------\n:Date: 2022-10-01\n\n* API changed\n* add better support for cicular ref\u00e9rences and duplicates with {\"$ref\": ...}\n\nVersion 0.2.0\n-------------\n:Date: 2021-02-18\n\n* API changed\n* can serialize dict with no-string keys\n* add support for cicular ref\u00e9rences and duplicates with {\"$ref\": ...}\n\n\nVersion 0.1.0\n-------------\n:Date: 2020-11-28\n\n* change description for pipy\n* add license for pipy\n* enable load of tuple, time.struct_time, Counter, OrderedDict and defaultdict\n\nVersion 0.0.4\n-------------\n:Date: 2020-11-24\n\t\n* API changed\n* add plugins support\n* add bytes, bytearray and numpy.array compression with blosc zstd\n* fix itertive append and decode (not fully tested).\n* fix dump of numpy types without conversion to python types(not yet numpy.float64)\n",
"bugtrack_url": null,
"license": "Prosperity Public License 3.0.0 and Patron License 1.0.0",
"summary": "A python library for fast serialization and deserialization of complex Python objects into JSON.",
"version": "0.3.4",
"project_urls": {
"Documentation": "https://smartaudiotools.github.io/serializejson",
"Download": "https://github.com/SmartAudioTools/serializejson/tarball/master",
"Funding": "https://github.com/sponsors/SmartAudioTools",
"Homepage": "https://github.com/SmartAudioTools/serializejson",
"Source": "https://github.com/SmartAudioTools/serializejson",
"Tracker": "https://github.com/SmartAudioTools/serializejson/issues"
},
"split_keywords": [
"pickle",
"json",
"serialize",
"dump",
"dumps",
"rapidjson",
"base64"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7f5669bbb66c468fb5206c376b1b1aefd8c94a2e3dce8cc4a46bd993e4534e42",
"md5": "1e1e0e68202eaec9d3a1d5b45b18d1c9",
"sha256": "61a8f6c008042042bb8dd3488ac8208b5ebf181a6038d0a27a81e65fac78869a"
},
"downloads": -1,
"filename": "serializejson-0.3.4-py3.11-linux-x86_64.egg",
"has_sig": false,
"md5_digest": "1e1e0e68202eaec9d3a1d5b45b18d1c9",
"packagetype": "bdist_egg",
"python_version": "0.3.4",
"requires_python": ">=3",
"size": 1507179,
"upload_time": "2023-06-11T21:15:17",
"upload_time_iso_8601": "2023-06-11T21:15:17.664200Z",
"url": "https://files.pythonhosted.org/packages/7f/56/69bbb66c468fb5206c376b1b1aefd8c94a2e3dce8cc4a46bd993e4534e42/serializejson-0.3.4-py3.11-linux-x86_64.egg",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9a8cd4e859aca9bf04452fb06bd74e6033f47e80934b47e7529eb635d9075a5e",
"md5": "8164d47edb33c4860074e0719fe1cbf3",
"sha256": "bccd3d7b72ed7945cadb4a9e97dec45173cc171227df92592e73df41d6e72732"
},
"downloads": -1,
"filename": "serializejson-0.3.4.tar.gz",
"has_sig": false,
"md5_digest": "8164d47edb33c4860074e0719fe1cbf3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3",
"size": 93916,
"upload_time": "2023-06-11T21:15:20",
"upload_time_iso_8601": "2023-06-11T21:15:20.483037Z",
"url": "https://files.pythonhosted.org/packages/9a/8c/d4e859aca9bf04452fb06bd74e6033f47e80934b47e7529eb635d9075a5e/serializejson-0.3.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-11 21:15:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SmartAudioTools",
"github_project": "serializejson",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "serializejson"
}
JSON
