| Name | pydantic-cereal JSON |
| Version |
0.0.6
JSON |
| download |
| home_page | |
| Summary | Advanced serialization for Pydantic models |
| upload_time | 2024-02-16 09:06:07 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.9 |
| license | MIT License Copyright (c) 2023 Anatoly Makarevich Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
pydantic
serialize
fsspec
yaml
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# `pydantic-cereal`
## Advanced serialization for Pydantic models
[Pydantic](https://docs.pydantic.dev/latest/) is the most widely used data validation library for Python.
It uses type hints/type annotations to define data models and has quite a nice "feel" to it.
Pydantic V2 was [released in June 2023](https://docs.pydantic.dev/2.0/blog/pydantic-v2-final/) and
brings many changes and improvements, including a
[new Rust-based engine for serializing and validating data](https://github.com/pydantic/pydantic-core).
This package, `pydantic-cereal`, is a small extension package that enables users to serialize Pydantic
models with "arbitrary" (non-JSON-fiendly) types to "arbitrary" file-system-like locations.
It uses [`fsspec`](https://filesystem-spec.readthedocs.io/en/latest/) to support generic file systems.
Writing a custom writer (serializer) and reader (loader) with `fsspec` URIs is quite straightforward.
You can also use [`universal-pathlib`](https://pypi.org/project/universal-pathlib/)'s
`UPath` with `pydantic-cereal`.
📘 See the [full documentation here](https://pydantic-cereal.readthedocs.io/). 📘
## Usage Example
See the [minimal pure-Python example](./docs/examples/minimal.ipynb) to learn how to wrap your own type.
Below is a preview of this example.
```python
from fsspec import AbstractFileSystem
from pydantic import BaseModel, ConfigDict
from pydantic_cereal import Cereal
cereal = Cereal() # This is a global variable
# Create and "register" a custom type
class MyType(object):
"""My custom type, which isn't a Pydantic model."""
def __init__(self, value: str):
self.value = str(value)
def __repr__(self) -> str:
return f"MyType({self.value})"
def my_reader(fs: AbstractFileSystem, path: str) -> MyType:
"""Read a MyType from an fsspec URI."""
return MyType(value=fs.read_text(path)) # type: ignore
def my_writer(obj: MyType, fs: AbstractFileSystem, path: str) -> None:
"""Write a MyType object to an fsspec URI."""
fs.write_text(path, obj.value)
MyWrappedType = cereal.wrap_type(MyType, reader=my_reader, writer=my_writer)
# Use type within Pydantic model
class MyModel(BaseModel):
"""My custom Pydantic model."""
config = ConfigDict(arbitrary_types_allowed=True) # Pydantic configuration
fld: MyWrappedType
mdl = MyModel(fld=MyType("my_field"))
# We can save the whole model to an fsspec URI, such as this MemoryFileSystem
uri = "memory://my_model"
cereal.write_model(mdl, uri)
# And we can read it back later
obj = cereal.read_model(uri)
assert isinstance(obj, MyModel)
assert isinstance(obj.fld, MyType)
```
For wrapping 3rd-party libraries, see the [Pandas dataframe example](./docs/examples/pandas.ipynb).
Raw data
{
"_id": null,
"home_page": "",
"name": "pydantic-cereal",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "pydantic,serialize,fsspec,yaml",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/33/84/3c67e76d3d77f2603a2598150e0ebabe2fb380c8552d4e727b333fcec3ce/pydantic-cereal-0.0.6.tar.gz",
"platform": null,
"description": "# `pydantic-cereal`\n\n## Advanced serialization for Pydantic models\n\n[Pydantic](https://docs.pydantic.dev/latest/) is the most widely used data validation library for Python.\nIt uses type hints/type annotations to define data models and has quite a nice \"feel\" to it.\nPydantic V2 was [released in June 2023](https://docs.pydantic.dev/2.0/blog/pydantic-v2-final/) and\nbrings many changes and improvements, including a\n[new Rust-based engine for serializing and validating data](https://github.com/pydantic/pydantic-core).\n\nThis package, `pydantic-cereal`, is a small extension package that enables users to serialize Pydantic\nmodels with \"arbitrary\" (non-JSON-fiendly) types to \"arbitrary\" file-system-like locations.\nIt uses [`fsspec`](https://filesystem-spec.readthedocs.io/en/latest/) to support generic file systems.\nWriting a custom writer (serializer) and reader (loader) with `fsspec` URIs is quite straightforward.\nYou can also use [`universal-pathlib`](https://pypi.org/project/universal-pathlib/)'s\n`UPath` with `pydantic-cereal`.\n\n\ud83d\udcd8 See the [full documentation here](https://pydantic-cereal.readthedocs.io/). \ud83d\udcd8\n\n## Usage Example\n\nSee the [minimal pure-Python example](./docs/examples/minimal.ipynb) to learn how to wrap your own type.\nBelow is a preview of this example.\n\n```python\nfrom fsspec import AbstractFileSystem\nfrom pydantic import BaseModel, ConfigDict\n\nfrom pydantic_cereal import Cereal\n\ncereal = Cereal() # This is a global variable\n\n\n# Create and \"register\" a custom type\n\nclass MyType(object):\n \"\"\"My custom type, which isn't a Pydantic model.\"\"\"\n\n def __init__(self, value: str):\n self.value = str(value)\n\n def __repr__(self) -> str:\n return f\"MyType({self.value})\"\n\n\ndef my_reader(fs: AbstractFileSystem, path: str) -> MyType:\n \"\"\"Read a MyType from an fsspec URI.\"\"\"\n return MyType(value=fs.read_text(path)) # type: ignore\n\n\ndef my_writer(obj: MyType, fs: AbstractFileSystem, path: str) -> None:\n \"\"\"Write a MyType object to an fsspec URI.\"\"\"\n fs.write_text(path, obj.value)\n\nMyWrappedType = cereal.wrap_type(MyType, reader=my_reader, writer=my_writer)\n\n\n# Use type within Pydantic model\n\nclass MyModel(BaseModel):\n \"\"\"My custom Pydantic model.\"\"\"\n\n config = ConfigDict(arbitrary_types_allowed=True) # Pydantic configuration\n fld: MyWrappedType\n\n\nmdl = MyModel(fld=MyType(\"my_field\"))\n\n# We can save the whole model to an fsspec URI, such as this MemoryFileSystem\nuri = \"memory://my_model\"\ncereal.write_model(mdl, uri)\n\n# And we can read it back later\nobj = cereal.read_model(uri)\nassert isinstance(obj, MyModel)\nassert isinstance(obj.fld, MyType)\n```\n\nFor wrapping 3rd-party libraries, see the [Pandas dataframe example](./docs/examples/pandas.ipynb).\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2023 Anatoly Makarevich Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Advanced serialization for Pydantic models",
"version": "0.0.6",
"project_urls": null,
"split_keywords": [
"pydantic",
"serialize",
"fsspec",
"yaml"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "33daec4811d5b21ff2b54f126265a2c6cdae65c6faed69b86fa7705e0556a95e",
"md5": "2f1a67f8ab497207efea34596cda5981",
"sha256": "334ecc669687a415e1bb02c943393def6401b32d001fb2c7313f2923c439874b"
},
"downloads": -1,
"filename": "pydantic_cereal-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2f1a67f8ab497207efea34596cda5981",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 13605,
"upload_time": "2024-02-16T09:06:05",
"upload_time_iso_8601": "2024-02-16T09:06:05.613181Z",
"url": "https://files.pythonhosted.org/packages/33/da/ec4811d5b21ff2b54f126265a2c6cdae65c6faed69b86fa7705e0556a95e/pydantic_cereal-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "33843c67e76d3d77f2603a2598150e0ebabe2fb380c8552d4e727b333fcec3ce",
"md5": "ff3f3295ab9d1ae0fe4872a396701dfd",
"sha256": "aac44edc80de142706f765eb550b241e7be16ced713cea2f2fc981d248a8fdca"
},
"downloads": -1,
"filename": "pydantic-cereal-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "ff3f3295ab9d1ae0fe4872a396701dfd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 25132,
"upload_time": "2024-02-16T09:06:07",
"upload_time_iso_8601": "2024-02-16T09:06:07.031463Z",
"url": "https://files.pythonhosted.org/packages/33/84/3c67e76d3d77f2603a2598150e0ebabe2fb380c8552d4e727b333fcec3ce/pydantic-cereal-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-16 09:06:07",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pydantic-cereal"
}
