sqlalchemy-json
###############
SQLAlchemy-JSON provides mutation-tracked JSON types to SQLAlchemy_:
* ``MutableJson`` is a straightforward implementation for keeping track of top-level changes to JSON objects;
* ``NestedMutableJson`` is an extension of this which tracks changes even when these happen in nested objects or arrays (Python ``dicts`` and ``lists``).
Examples
========
Basic change tracking
---------------------
This is essentially the SQLAlchemy `mutable JSON recipe`_. We define a simple author model which list the author's name and a property ``handles`` for various social media handles used:
.. code-block:: python
class Author(Base):
name = Column(Text)
handles = Column(MutableJson)
Or, using the declarative mapping style:
.. code-block:: python
class Category(Base):
__tablename__ = "categories"
id = mapped_column(Integer, primary_key=True)
created_at: Mapped[DateTime] = mapped_column(DateTime, default=datetime.now)
updated_at: Mapped[DateTime] = mapped_column(
DateTime, default=datetime.now, onupdate=datetime.now
)
keywords: Mapped[list[str]] = mapped_column(MutableJson)
The example below loads one of the existing authors and retrieves the mapping of social media handles. The error in the twitter handle is then corrected and committed. The change is detected by SQLAlchemy and the appropriate ``UPDATE`` statement is generated.
.. code-block:: python
>>> author = session.query(Author).first()
>>> author.handles
{'twitter': '@JohnDoe', 'facebook': 'JohnDoe'}
>>> author.handles['twitter'] = '@JDoe'
>>> session.commit()
>>> author.handles
{'twitter': '@JDoe', 'facebook': 'JohnDoe'}
Nested change tracking
----------------------
The example below defines a simple model for articles. One of the properties on this model is a mutable JSON structure called ``references`` which includes a count of links that the article contains, grouped by domain:
.. code-block:: python
class Article(Base):
author = Column(ForeignKey('author.name'))
content = Column(Text)
references = Column(NestedMutableJson)
With this in place, an existing article is loaded and its current references inspected. Following that, the count for one of these is increased by ten, and the session is committed:
.. code-block:: python
>>> article = session.query(Article).first()
>>> article.references
{'github.com': {'edelooff/sqlalchemy-json': 4, 'zzzeek/sqlalchemy': 7}}
>>> article.references['github.com']['edelooff/sqlalchemy-json'] += 10
>>> session.commit()
>>> article.references
{'github.com': {'edelooff/sqlalchemy-json': 14, 'zzzeek/sqlalchemy': 7}}
Had the articles model used ``MutableJson`` like in the previous example this code would have failed. This is because the top level dictionary is never altered directly. The *nested* mutable ensures the change happening at the lower level *bubbles up* to the outermost container.
Non-native JSON / other serialization types
===========================================
By default, sqlalchemy-json uses the JSON column type provided by SQLAlchemy (specifically ``sqlalchemy.types.JSON``.)
If you wish to use another type (e.g. PostgreSQL's ``JSONB``), your database does not natively support JSON (e.g. versions of SQLite before 3.37.2/), or you wish to serialize to a format other than JSON, you'll need to provide a different backing type.
This is done by using the utility function ``mutable_json_type``. This type creator function accepts two parameters:
* ``dbtype`` controls the database type used. This can be an existing type provided by SQLAlchemy or SQLALchemy-utils_, or an `augmented type`_ to provide serialization to any other format;
* ``nested`` controls whether the created type is made mutable based on ``MutableDict`` or ``NestedMutable`` (defaults to ``False`` for ``MutableDict``).
.. code-block:: python
import json
from sqlalchemy import JSON, String, TypeDecorator
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy_json import mutable_json_type
class JsonString(TypeDecorator):
"""Enables JSON storage by encoding and decoding on the fly."""
impl = String
def process_bind_param(self, value, dialect):
return json.dumps(value)
def process_result_value(self, value, dialect):
return json.loads(value)
postgres_jsonb_mutable = mutable_json_type(dbtype=JSONB)
string_backed_nested_mutable = mutable_json_type(dbtype=JsonString, nested=True)
Dependencies
============
* ``sqlalchemy``
Development
===========
Here's how to setup your development environment:
.. code-block:: shell
python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
# run tests
pytest
Changelog
=========
0.7.0
-----
* Adds support for top-level list for ``MutableJson``, rather than having that support only be available in the *nested* variant (https://github.com/edelooff/sqlalchemy-json/pull/51)
* Adds ``pytest`` as development dependency
0.6.0
-----
* Fixes pickling support (https://github.com/edelooff/sqlalchemy-json/issues/36)
* Drops python 2.x support (previously claimed, but already broken for some time)
* Removes test runners for CPython 3.6 since Github actions support has been dropped
0.5.0
-----
* Fixes a lingering Python 3 compatibility issue (``cmp`` parameter for ``TrackedList.sort``)
* Adds pickling and unpickling support (https://github.com/edelooff/sqlalchemy-json/pull/28)
* Adds tracking for dictionary in-place updates (https://github.com/edelooff/sqlalchemy-json/pull/33)
0.4.0
-----
* Adds a type creation function to allow for custom or alternate serialization types. This allows for a way around the regression in SQLite compatibility introduced by v0.3.0.
0.3.0
-----
* Switches JSON base type to ``sqlalchemy.types.JSON`` from deprecated JSON type provided by SQLAlchemy-utils.
0.2.2
-----
* Fixes a bug where assigning ``None`` to the column resulted in an error (https://github.com/edelooff/sqlalchemy-json/issues/10)
0.2.1
-----
* Fixes a typo in the README found after uploading 0.2.0 to PyPI.
0.2.0 (unreleased)
------------------
* Now uses ``JSONType`` provided by SQLAlchemy-utils_ to handle backend storage;
* **Backwards incompatible**: Changed class name ``JsonObject`` to ``MutableJson`` and ``NestedJsonObject`` to ``NestedMutableJson``
* Outermost container for ``NestedMutableJson`` can now be an ``array`` (Python ``list``)
0.1.0 (unreleased)
------------------
Initial version. This initially carried a 1.0.0 version number but has never been released on PyPI.
.. _augmented type: https://docs.sqlalchemy.org/en/13/core/custom_types.html#augmenting-existing-types
.. _mutable json recipe: http://docs.sqlalchemy.org/en/latest/core/custom_types.html#marshal-json-strings
.. _sqlalchemy: https://www.sqlalchemy.org/
.. _sqlalchemy-utils: https://sqlalchemy-utils.readthedocs.io/
Raw data
{
"_id": null,
"home_page": "https://github.com/edelooff/sqlalchemy-json",
"name": "sqlalchemy-json",
"maintainer": "",
"docs_url": null,
"requires_python": ">= 3.6",
"maintainer_email": "",
"keywords": "sqlalchemy json mutable",
"author": "Elmer de Looff",
"author_email": "elmer.delooff@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/3e/57/73ba3d0ee5efbec5a0d15ee3e21606edd33b1d1fd11b5d64e581c8b8a3f6/sqlalchemy-json-0.7.0.tar.gz",
"platform": null,
"description": "sqlalchemy-json\n###############\n\nSQLAlchemy-JSON provides mutation-tracked JSON types to SQLAlchemy_:\n\n* ``MutableJson`` is a straightforward implementation for keeping track of top-level changes to JSON objects;\n* ``NestedMutableJson`` is an extension of this which tracks changes even when these happen in nested objects or arrays (Python ``dicts`` and ``lists``).\n\n\nExamples\n========\n\nBasic change tracking\n---------------------\n\nThis is essentially the SQLAlchemy `mutable JSON recipe`_. We define a simple author model which list the author's name and a property ``handles`` for various social media handles used:\n\n.. code-block:: python\n\n class Author(Base):\n name = Column(Text)\n handles = Column(MutableJson)\n\nOr, using the declarative mapping style:\n\n.. code-block:: python\n\n class Category(Base):\n __tablename__ = \"categories\"\n\n id = mapped_column(Integer, primary_key=True)\n created_at: Mapped[DateTime] = mapped_column(DateTime, default=datetime.now)\n updated_at: Mapped[DateTime] = mapped_column(\n DateTime, default=datetime.now, onupdate=datetime.now\n )\n keywords: Mapped[list[str]] = mapped_column(MutableJson)\n\nThe example below loads one of the existing authors and retrieves the mapping of social media handles. The error in the twitter handle is then corrected and committed. The change is detected by SQLAlchemy and the appropriate ``UPDATE`` statement is generated.\n\n.. code-block:: python\n\n >>> author = session.query(Author).first()\n >>> author.handles\n {'twitter': '@JohnDoe', 'facebook': 'JohnDoe'}\n >>> author.handles['twitter'] = '@JDoe'\n >>> session.commit()\n >>> author.handles\n {'twitter': '@JDoe', 'facebook': 'JohnDoe'}\n\n\nNested change tracking\n----------------------\n\nThe example below defines a simple model for articles. One of the properties on this model is a mutable JSON structure called ``references`` which includes a count of links that the article contains, grouped by domain:\n\n.. code-block:: python\n\n class Article(Base):\n author = Column(ForeignKey('author.name'))\n content = Column(Text)\n references = Column(NestedMutableJson)\n\nWith this in place, an existing article is loaded and its current references inspected. Following that, the count for one of these is increased by ten, and the session is committed:\n\n.. code-block:: python\n\n >>> article = session.query(Article).first()\n >>> article.references\n {'github.com': {'edelooff/sqlalchemy-json': 4, 'zzzeek/sqlalchemy': 7}}\n >>> article.references['github.com']['edelooff/sqlalchemy-json'] += 10\n >>> session.commit()\n >>> article.references\n {'github.com': {'edelooff/sqlalchemy-json': 14, 'zzzeek/sqlalchemy': 7}}\n\nHad the articles model used ``MutableJson`` like in the previous example this code would have failed. This is because the top level dictionary is never altered directly. The *nested* mutable ensures the change happening at the lower level *bubbles up* to the outermost container.\n\n\nNon-native JSON / other serialization types\n===========================================\n\nBy default, sqlalchemy-json uses the JSON column type provided by SQLAlchemy (specifically ``sqlalchemy.types.JSON``.)\nIf you wish to use another type (e.g. PostgreSQL's ``JSONB``), your database does not natively support JSON (e.g. versions of SQLite before 3.37.2/), or you wish to serialize to a format other than JSON, you'll need to provide a different backing type.\n\nThis is done by using the utility function ``mutable_json_type``. This type creator function accepts two parameters:\n\n* ``dbtype`` controls the database type used. This can be an existing type provided by SQLAlchemy or SQLALchemy-utils_, or an `augmented type`_ to provide serialization to any other format;\n* ``nested`` controls whether the created type is made mutable based on ``MutableDict`` or ``NestedMutable`` (defaults to ``False`` for ``MutableDict``).\n\n.. code-block:: python\n\n import json\n\n from sqlalchemy import JSON, String, TypeDecorator\n from sqlalchemy.dialects.postgresql import JSONB\n from sqlalchemy_json import mutable_json_type\n\n class JsonString(TypeDecorator):\n \"\"\"Enables JSON storage by encoding and decoding on the fly.\"\"\"\n\n impl = String\n\n def process_bind_param(self, value, dialect):\n return json.dumps(value)\n\n def process_result_value(self, value, dialect):\n return json.loads(value)\n\n\n postgres_jsonb_mutable = mutable_json_type(dbtype=JSONB)\n string_backed_nested_mutable = mutable_json_type(dbtype=JsonString, nested=True)\n\n\nDependencies\n============\n\n* ``sqlalchemy``\n\nDevelopment\n===========\n\nHere's how to setup your development environment:\n\n.. code-block:: shell\n\n python -m venv .venv\n . .venv/bin/activate\n pip install -e \".[dev]\"\n # run tests\n pytest\n\n\nChangelog\n=========\n\n0.7.0\n-----\n\n* Adds support for top-level list for ``MutableJson``, rather than having that support only be available in the *nested* variant (https://github.com/edelooff/sqlalchemy-json/pull/51)\n* Adds ``pytest`` as development dependency\n\n0.6.0\n-----\n\n* Fixes pickling support (https://github.com/edelooff/sqlalchemy-json/issues/36)\n* Drops python 2.x support (previously claimed, but already broken for some time)\n* Removes test runners for CPython 3.6 since Github actions support has been dropped\n\n0.5.0\n-----\n* Fixes a lingering Python 3 compatibility issue (``cmp`` parameter for ``TrackedList.sort``)\n* Adds pickling and unpickling support (https://github.com/edelooff/sqlalchemy-json/pull/28)\n* Adds tracking for dictionary in-place updates (https://github.com/edelooff/sqlalchemy-json/pull/33)\n\n0.4.0\n-----\n\n* Adds a type creation function to allow for custom or alternate serialization types. This allows for a way around the regression in SQLite compatibility introduced by v0.3.0.\n\n0.3.0\n-----\n\n* Switches JSON base type to ``sqlalchemy.types.JSON`` from deprecated JSON type provided by SQLAlchemy-utils.\n\n0.2.2\n-----\n\n* Fixes a bug where assigning ``None`` to the column resulted in an error (https://github.com/edelooff/sqlalchemy-json/issues/10)\n\n\n0.2.1\n-----\n\n* Fixes a typo in the README found after uploading 0.2.0 to PyPI.\n\n\n0.2.0 (unreleased)\n------------------\n\n* Now uses ``JSONType`` provided by SQLAlchemy-utils_ to handle backend storage;\n* **Backwards incompatible**: Changed class name ``JsonObject`` to ``MutableJson`` and ``NestedJsonObject`` to ``NestedMutableJson``\n* Outermost container for ``NestedMutableJson`` can now be an ``array`` (Python ``list``)\n\n\n0.1.0 (unreleased)\n------------------\n\nInitial version. This initially carried a 1.0.0 version number but has never been released on PyPI.\n\n\n.. _augmented type: https://docs.sqlalchemy.org/en/13/core/custom_types.html#augmenting-existing-types\n.. _mutable json recipe: http://docs.sqlalchemy.org/en/latest/core/custom_types.html#marshal-json-strings\n.. _sqlalchemy: https://www.sqlalchemy.org/\n.. _sqlalchemy-utils: https://sqlalchemy-utils.readthedocs.io/\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "JSON type with nested change tracking for SQLAlchemy",
"version": "0.7.0",
"project_urls": {
"Homepage": "https://github.com/edelooff/sqlalchemy-json"
},
"split_keywords": [
"sqlalchemy",
"json",
"mutable"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e1791fc7309ecf75756e9ad9280f19cd83ca7b79a0dae36cd025f668e8e2741f",
"md5": "e31bbf1a5b38444c2bcb044b4e9ca9d3",
"sha256": "27881d662ca18363a4ac28175cc47ea2a6f2bef997ae1159c151026b741818e6"
},
"downloads": -1,
"filename": "sqlalchemy_json-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e31bbf1a5b38444c2bcb044b4e9ca9d3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">= 3.6",
"size": 7688,
"upload_time": "2023-08-30T19:52:56",
"upload_time_iso_8601": "2023-08-30T19:52:56.219213Z",
"url": "https://files.pythonhosted.org/packages/e1/79/1fc7309ecf75756e9ad9280f19cd83ca7b79a0dae36cd025f668e8e2741f/sqlalchemy_json-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3e5773ba3d0ee5efbec5a0d15ee3e21606edd33b1d1fd11b5d64e581c8b8a3f6",
"md5": "8636d699180972f2ca9080e2973b8fdd",
"sha256": "620d0b26f648f21a8fa9127df66f55f83a5ab4ae010e5397a5c6989a08238561"
},
"downloads": -1,
"filename": "sqlalchemy-json-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "8636d699180972f2ca9080e2973b8fdd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">= 3.6",
"size": 8848,
"upload_time": "2023-08-30T19:52:57",
"upload_time_iso_8601": "2023-08-30T19:52:57.367473Z",
"url": "https://files.pythonhosted.org/packages/3e/57/73ba3d0ee5efbec5a0d15ee3e21606edd33b1d1fd11b5d64e581c8b8a3f6/sqlalchemy-json-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-30 19:52:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "edelooff",
"github_project": "sqlalchemy-json",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "sqlalchemy-json"
}
JSON
