# `dagster-serde`
[![image](https://img.shields.io/pypi/v/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)
[![image](https://img.shields.io/pypi/l/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)
[![image](https://img.shields.io/pypi/pyversions/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)
[![CI](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml/badge.svg)](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
[![Checked with pyright](https://microsoft.github.io/pyright/img/pyright_badge.svg)](https://microsoft.github.io/pyright/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
Type-aware ser/de library for [Dagster](https://github.com/dagster-io/dagster).
```python
@dataclass
class MyStruct:
foo: str
my_struct = MyStruct(foo="bar")
@asset(
io_manager_key="json_io_manager",
)
def upstream() -> MyStruct:
return my_struct # my_struct is serialized to a json file
@asset
def downstream(
upstream: MyStruct,
): # my_struct is loaded as MyStruct based on the type hint
assert upstream == my_struct
```
# Installation
```shell
pip install dagster-serde
```
# IOManagers
## `BaseSerdeUPathIOManager`
Base class for IOManagers that ser/de to/from filesystems.
## `JsonIOManager`
Implements `BaseSerdeUPathIOManager` for JSON files.
# Examples
See [examples](docs/examples.md).
## Development
### Installation
```shell
poetry install
poetry run pre-commit install
```
### Testing
```shell
poetry run pytest
```
Raw data
{
"_id": null,
"home_page": "https://github.com/danielgafni/dagster-serde",
"name": "dagster-serde",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "dagster,serde,ETL",
"author": "Daniel Gafni",
"author_email": "danielgafni16@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ad/c9/c4367dea412ce33db6ecd230d9f6099cfbe4b307c61a4cff79545acc3622/dagster_serde-0.0.0.tar.gz",
"platform": null,
"description": "\n\n# `dagster-serde`\n\n[![image](https://img.shields.io/pypi/v/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)\n[![image](https://img.shields.io/pypi/l/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)\n[![image](https://img.shields.io/pypi/pyversions/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)\n[![CI](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml/badge.svg)](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)\n[![Checked with pyright](https://microsoft.github.io/pyright/img/pyright_badge.svg)](https://microsoft.github.io/pyright/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n\nType-aware ser/de library for [Dagster](https://github.com/dagster-io/dagster).\n\n```python\n@dataclass\nclass MyStruct:\n foo: str\n\n\nmy_struct = MyStruct(foo=\"bar\")\n\n\n@asset(\n io_manager_key=\"json_io_manager\",\n)\ndef upstream() -> MyStruct:\n return my_struct # my_struct is serialized to a json file\n\n\n@asset\ndef downstream(\n upstream: MyStruct,\n): # my_struct is loaded as MyStruct based on the type hint\n assert upstream == my_struct\n```\n\n# Installation\n\n```shell\npip install dagster-serde\n```\n\n# IOManagers\n\n## `BaseSerdeUPathIOManager`\nBase class for IOManagers that ser/de to/from filesystems.\n\n## `JsonIOManager`\n\nImplements `BaseSerdeUPathIOManager` for JSON files.\n\n# Examples\nSee [examples](docs/examples.md).\n\n## Development\n\n### Installation\n```shell\npoetry install\npoetry run pre-commit install\n```\n\n### Testing\n```shell\npoetry run pytest\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Dagster library for ser/de of data",
"version": "0.0.0",
"project_urls": {
"Homepage": "https://github.com/danielgafni/dagster-serde",
"Repository": "https://github.com/danielgafni/dagster-serde"
},
"split_keywords": [
"dagster",
"serde",
"etl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c2afca12183a49e76ca3c0c8b4fc351af1bec370b76a3b029341fd465253fc1f",
"md5": "16756c1a199fb402be89b3a169a4f463",
"sha256": "d9bdf6d0a943b9455572a259b7dcdf39359d3e50206626f45fb1d817d03d6057"
},
"downloads": -1,
"filename": "dagster_serde-0.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "16756c1a199fb402be89b3a169a4f463",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 8911,
"upload_time": "2023-10-11T20:48:21",
"upload_time_iso_8601": "2023-10-11T20:48:21.149555Z",
"url": "https://files.pythonhosted.org/packages/c2/af/ca12183a49e76ca3c0c8b4fc351af1bec370b76a3b029341fd465253fc1f/dagster_serde-0.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "adc9c4367dea412ce33db6ecd230d9f6099cfbe4b307c61a4cff79545acc3622",
"md5": "a8d108b500fd4a1fddfb09390b11699f",
"sha256": "3a3767def9393668b03042b49e637dff3a2b6a3b633822d48ed17ea627b4620d"
},
"downloads": -1,
"filename": "dagster_serde-0.0.0.tar.gz",
"has_sig": false,
"md5_digest": "a8d108b500fd4a1fddfb09390b11699f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 8268,
"upload_time": "2023-10-11T20:48:22",
"upload_time_iso_8601": "2023-10-11T20:48:22.687311Z",
"url": "https://files.pythonhosted.org/packages/ad/c9/c4367dea412ce33db6ecd230d9f6099cfbe4b307c61a4cff79545acc3622/dagster_serde-0.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-11 20:48:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "danielgafni",
"github_project": "dagster-serde",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dagster-serde"
}