*************************************************************
:fire:toastedmarshmallow:fire:: Makes Marshmallow Toasty Fast
*************************************************************
Toasted Marshmallow implements a JIT for marshmallow that speeds up dumping
objects 10-25X (depending on your schema). Toasted Marshmallow allows you to
have the great API that
`Marshmallow <https://github.com/marshmallow-code/marshmallow>`_ provides
without having to sacrifice performance!
::
Benchmark Result:
Original Time: 2682.61 usec/dump
Optimized Time: 176.38 usec/dump
Speed up: 15.21x
Even ``PyPy`` benefits from ``toastedmarshmallow``!
::
Benchmark Result:
Original Time: 189.78 usec/dump
Optimized Time: 20.03 usec/dump
Speed up: 9.48x
Installing toastedmarshmallow
-----------------------------
.. code-block:: bash
pip install toastedmarshmallow
This will *also* install a slightly-forked ``marshmallow`` that includes some
hooks Toastedmarshmallow needs enable the JIT to run before falling back
to the original marshmallow code. These changes are minimal making it easier
to track upstream. You can find the changes
`Here <https://github.com/marshmallow-code/marshmallow/pull/629/files>`_.
This means you should **remove** ``marshmallow`` from your requirements and
replace it with ``toastedmarshmallow``. By default there is no
difference unless you explicitly enable Toasted Marshmallow.
Enabling Toasted Marshmallow
----------------------------
Enabling Toasted Marshmallow on an existing Schema is just one line of code,
set the ``jit`` property on any ``Schema`` instance to
``toastedmarshmallow.Jit``. For example:
.. code-block:: python
from datetime import date
import toastedmarshmallow
from marshmallow import Schema, fields, pprint
class ArtistSchema(Schema):
name = fields.Str()
class AlbumSchema(Schema):
title = fields.Str()
release_date = fields.Date()
artist = fields.Nested(ArtistSchema())
schema = AlbumSchema()
# Specify the jit method as toastedmarshmallow's jit
schema.jit = toastedmarshmallow.Jit
# And that's it! Your dump methods are 15x faster!
It's also possible to use the ``Meta`` class on the ``Marshmallow`` schema
to specify all instances of a given ``Schema`` should be optimized:
.. code-block:: python
import toastedmarshmallow
from marshmallow import Schema, fields, pprint
class ArtistSchema(Schema):
class Meta:
jit = toastedMarshmallow.Jit
name = fields.Str()
You can also enable Toasted Marshmallow globally by setting the environment
variable ``MARSHMALLOW_SCHEMA_DEFAULT_JIT`` to ``toastedmarshmallow.Jit`` .
Future versions of Toasted Marshmallow may make this the default.
How it works
------------
Toasted Marshmallow works by generating code at runtime to optimize dumping
objects without going through layers and layers of reflection. The generated
code optimistically assumes the objects being passed in are schematically valid,
falling back to the original marshmallow code on failure.
For example, taking ``AlbumSchema`` from above, Toastedmarshmallow will
generate the following 3 methods:
.. code-block:: python
def InstanceSerializer(obj):
res = {}
value = obj.release_date; value = value() if callable(value) else value; res["release_date"] = _field_release_date__serialize(value, "release_date", obj)
value = obj.artist; value = value() if callable(value) else value; res["artist"] = _field_artist__serialize(value, "artist", obj)
value = obj.title; value = value() if callable(value) else value; value = str(value) if value is not None else None; res["title"] = value
return res
def DictSerializer(obj):
res = {}
if "release_date" in obj:
value = obj["release_date"]; value = value() if callable(value) else value; res["release_date"] = _field_release_date__serialize(value, "release_date", obj)
if "artist" in obj:
value = obj["artist"]; value = value() if callable(value) else value; res["artist"] = _field_artist__serialize(value, "artist", obj)
if "title" in obj:
value = obj["title"]; value = value() if callable(value) else value; value = str(value) if value is not None else None; res["title"] = value
return res
def HybridSerializer(obj):
res = {}
try:
value = obj["release_date"]
except (KeyError, AttributeError, IndexError, TypeError):
value = obj.release_date
value = value; value = value() if callable(value) else value; res["release_date"] = _field_release_date__serialize(value, "release_date", obj)
try:
value = obj["artist"]
except (KeyError, AttributeError, IndexError, TypeError):
value = obj.artist
value = value; value = value() if callable(value) else value; res["artist"] = _field_artist__serialize(value, "artist", obj)
try:
value = obj["title"]
except (KeyError, AttributeError, IndexError, TypeError):
value = obj.title
value = value; value = value() if callable(value) else value; value = str(value) if value is not None else None; res["title"] = value
return res
Toastedmarshmallow will invoke the proper serializer based upon the input.
Since Toastedmarshmallow is generating code at runtime, it's critical you
re-use Schema objects. If you're creating a new Schema object every time you
serialize/deserialize an object you'll likely have much worse performance.
Raw data
{
"_id": null,
"home_page": "https://github.com/lyft/toastedmarshmallow",
"name": "toastedmarshmallow",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "serialization,rest,json,api,marshal,marshalling,deserialization,validation,schema",
"author": "Roy Williams",
"author_email": "rwilliams@lyft.com",
"download_url": "https://files.pythonhosted.org/packages/17/32/183250558a858f74bbce5faa46054feac73723708ae8f9d9e4c554f62be3/toastedmarshmallow-2.15.2.post1.tar.gz",
"platform": "",
"description": "*************************************************************\n:fire:toastedmarshmallow:fire:: Makes Marshmallow Toasty Fast\n*************************************************************\n\nToasted Marshmallow implements a JIT for marshmallow that speeds up dumping\nobjects 10-25X (depending on your schema). Toasted Marshmallow allows you to\nhave the great API that\n`Marshmallow <https://github.com/marshmallow-code/marshmallow>`_ provides\nwithout having to sacrifice performance!\n\n::\n\n Benchmark Result:\n Original Time: 2682.61 usec/dump\n Optimized Time: 176.38 usec/dump\n Speed up: 15.21x\n\nEven ``PyPy`` benefits from ``toastedmarshmallow``!\n\n::\n\n Benchmark Result:\n \tOriginal Time: 189.78 usec/dump\n \tOptimized Time: 20.03 usec/dump\n \tSpeed up: 9.48x\n\nInstalling toastedmarshmallow\n-----------------------------\n\n.. code-block:: bash\n\n pip install toastedmarshmallow\n\nThis will *also* install a slightly-forked ``marshmallow`` that includes some\nhooks Toastedmarshmallow needs enable the JIT to run before falling back\nto the original marshmallow code. These changes are minimal making it easier\nto track upstream. You can find the changes\n`Here <https://github.com/marshmallow-code/marshmallow/pull/629/files>`_.\n\nThis means you should **remove** ``marshmallow`` from your requirements and\nreplace it with ``toastedmarshmallow``. By default there is no\ndifference unless you explicitly enable Toasted Marshmallow.\n\nEnabling Toasted Marshmallow\n----------------------------\n\nEnabling Toasted Marshmallow on an existing Schema is just one line of code,\nset the ``jit`` property on any ``Schema`` instance to \n``toastedmarshmallow.Jit``. For example:\n\n.. code-block:: python\n\n from datetime import date\n import toastedmarshmallow\n from marshmallow import Schema, fields, pprint\n\n class ArtistSchema(Schema):\n name = fields.Str()\n\n class AlbumSchema(Schema):\n title = fields.Str()\n release_date = fields.Date()\n artist = fields.Nested(ArtistSchema())\n\n schema = AlbumSchema()\n # Specify the jit method as toastedmarshmallow's jit\n schema.jit = toastedmarshmallow.Jit\n # And that's it! Your dump methods are 15x faster!\n\nIt's also possible to use the ``Meta`` class on the ``Marshmallow`` schema\nto specify all instances of a given ``Schema`` should be optimized:\n\n.. code-block:: python\n\n import toastedmarshmallow\n from marshmallow import Schema, fields, pprint\n\n class ArtistSchema(Schema):\n class Meta:\n jit = toastedMarshmallow.Jit\n name = fields.Str()\n\nYou can also enable Toasted Marshmallow globally by setting the environment\nvariable ``MARSHMALLOW_SCHEMA_DEFAULT_JIT`` to ``toastedmarshmallow.Jit`` .\nFuture versions of Toasted Marshmallow may make this the default.\n\nHow it works\n------------\n\nToasted Marshmallow works by generating code at runtime to optimize dumping\nobjects without going through layers and layers of reflection. The generated\ncode optimistically assumes the objects being passed in are schematically valid,\nfalling back to the original marshmallow code on failure.\n\nFor example, taking ``AlbumSchema`` from above, Toastedmarshmallow will\ngenerate the following 3 methods:\n\n.. code-block:: python\n\n def InstanceSerializer(obj):\n res = {}\n value = obj.release_date; value = value() if callable(value) else value; res[\"release_date\"] = _field_release_date__serialize(value, \"release_date\", obj)\n value = obj.artist; value = value() if callable(value) else value; res[\"artist\"] = _field_artist__serialize(value, \"artist\", obj)\n value = obj.title; value = value() if callable(value) else value; value = str(value) if value is not None else None; res[\"title\"] = value\n return res\n\n def DictSerializer(obj):\n res = {}\n if \"release_date\" in obj:\n value = obj[\"release_date\"]; value = value() if callable(value) else value; res[\"release_date\"] = _field_release_date__serialize(value, \"release_date\", obj)\n if \"artist\" in obj:\n value = obj[\"artist\"]; value = value() if callable(value) else value; res[\"artist\"] = _field_artist__serialize(value, \"artist\", obj)\n if \"title\" in obj:\n value = obj[\"title\"]; value = value() if callable(value) else value; value = str(value) if value is not None else None; res[\"title\"] = value\n return res\n\n def HybridSerializer(obj):\n res = {}\n try:\n value = obj[\"release_date\"]\n except (KeyError, AttributeError, IndexError, TypeError):\n value = obj.release_date\n value = value; value = value() if callable(value) else value; res[\"release_date\"] = _field_release_date__serialize(value, \"release_date\", obj)\n try:\n value = obj[\"artist\"]\n except (KeyError, AttributeError, IndexError, TypeError):\n value = obj.artist\n value = value; value = value() if callable(value) else value; res[\"artist\"] = _field_artist__serialize(value, \"artist\", obj)\n try:\n value = obj[\"title\"]\n except (KeyError, AttributeError, IndexError, TypeError):\n value = obj.title\n value = value; value = value() if callable(value) else value; value = str(value) if value is not None else None; res[\"title\"] = value\n return res\n\nToastedmarshmallow will invoke the proper serializer based upon the input.\n\nSince Toastedmarshmallow is generating code at runtime, it's critical you\nre-use Schema objects. If you're creating a new Schema object every time you\nserialize/deserialize an object you'll likely have much worse performance.",
"bugtrack_url": null,
"license": "apache2",
"summary": "A JIT implementation for Marshmallow to speed up dumping and loading objects.",
"version": "2.15.2.post1",
"split_keywords": [
"serialization",
"rest",
"json",
"api",
"marshal",
"marshalling",
"deserialization",
"validation",
"schema"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "9ac2dfaed2d9600452b97099d3f2935d",
"sha256": "aad7c89720a27d6362e1362394ceaedf5e1648aba4506c1971da35b3642810e9"
},
"downloads": -1,
"filename": "toastedmarshmallow-2.15.2.post1.tar.gz",
"has_sig": false,
"md5_digest": "9ac2dfaed2d9600452b97099d3f2935d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 62254,
"upload_time": "2019-06-19T15:18:10",
"upload_time_iso_8601": "2019-06-19T15:18:10.810510Z",
"url": "https://files.pythonhosted.org/packages/17/32/183250558a858f74bbce5faa46054feac73723708ae8f9d9e4c554f62be3/toastedmarshmallow-2.15.2.post1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2019-06-19 15:18:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "lyft",
"github_project": "toastedmarshmallow",
"lcname": "toastedmarshmallow"
}
JSON
