dagster-serde


Namedagster-serde JSON
Version 0.0.0 PyPI version JSON
download
home_pagehttps://github.com/danielgafni/dagster-serde
SummaryDagster library for ser/de of data
upload_time2023-10-11 20:48:22
maintainer
docs_urlNone
authorDaniel Gafni
requires_python>=3.8,<4.0
licenseApache-2.0
keywords dagster serde etl
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            

# `dagster-serde`

[![image](https://img.shields.io/pypi/v/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)
[![image](https://img.shields.io/pypi/l/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)
[![image](https://img.shields.io/pypi/pyversions/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)
[![CI](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml/badge.svg)](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
[![Checked with pyright](https://microsoft.github.io/pyright/img/pyright_badge.svg)](https://microsoft.github.io/pyright/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)


Type-aware ser/de library for [Dagster](https://github.com/dagster-io/dagster).

```python
@dataclass
class MyStruct:
    foo: str


my_struct = MyStruct(foo="bar")


@asset(
    io_manager_key="json_io_manager",
)
def upstream() -> MyStruct:
    return my_struct  # my_struct is serialized to a json file


@asset
def downstream(
    upstream: MyStruct,
):  # my_struct is loaded as MyStruct based on the type hint
    assert upstream == my_struct
```

# Installation

```shell
pip install dagster-serde
```

# IOManagers

## `BaseSerdeUPathIOManager`
Base class for IOManagers that ser/de to/from filesystems.

## `JsonIOManager`

Implements `BaseSerdeUPathIOManager` for JSON files.

# Examples
See [examples](docs/examples.md).

## Development

### Installation
```shell
poetry install
poetry run pre-commit install
```

### Testing
```shell
poetry run pytest
```

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/danielgafni/dagster-serde",
    "name": "dagster-serde",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.8,<4.0",
    "maintainer_email": "",
    "keywords": "dagster,serde,ETL",
    "author": "Daniel Gafni",
    "author_email": "danielgafni16@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/ad/c9/c4367dea412ce33db6ecd230d9f6099cfbe4b307c61a4cff79545acc3622/dagster_serde-0.0.0.tar.gz",
    "platform": null,
    "description": "\n\n# `dagster-serde`\n\n[![image](https://img.shields.io/pypi/v/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)\n[![image](https://img.shields.io/pypi/l/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)\n[![image](https://img.shields.io/pypi/pyversions/dagster-serde.svg)](https://pypi.python.org/pypi/dagster-serde)\n[![CI](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml/badge.svg)](https://github.com/danielgafni/dagster-serde/actions/workflows/ci.yml)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)\n[![Checked with pyright](https://microsoft.github.io/pyright/img/pyright_badge.svg)](https://microsoft.github.io/pyright/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n\nType-aware ser/de library for [Dagster](https://github.com/dagster-io/dagster).\n\n```python\n@dataclass\nclass MyStruct:\n    foo: str\n\n\nmy_struct = MyStruct(foo=\"bar\")\n\n\n@asset(\n    io_manager_key=\"json_io_manager\",\n)\ndef upstream() -> MyStruct:\n    return my_struct  # my_struct is serialized to a json file\n\n\n@asset\ndef downstream(\n    upstream: MyStruct,\n):  # my_struct is loaded as MyStruct based on the type hint\n    assert upstream == my_struct\n```\n\n# Installation\n\n```shell\npip install dagster-serde\n```\n\n# IOManagers\n\n## `BaseSerdeUPathIOManager`\nBase class for IOManagers that ser/de to/from filesystems.\n\n## `JsonIOManager`\n\nImplements `BaseSerdeUPathIOManager` for JSON files.\n\n# Examples\nSee [examples](docs/examples.md).\n\n## Development\n\n### Installation\n```shell\npoetry install\npoetry run pre-commit install\n```\n\n### Testing\n```shell\npoetry run pytest\n```\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "Dagster library for ser/de of data",
    "version": "0.0.0",
    "project_urls": {
        "Homepage": "https://github.com/danielgafni/dagster-serde",
        "Repository": "https://github.com/danielgafni/dagster-serde"
    },
    "split_keywords": [
        "dagster",
        "serde",
        "etl"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c2afca12183a49e76ca3c0c8b4fc351af1bec370b76a3b029341fd465253fc1f",
                "md5": "16756c1a199fb402be89b3a169a4f463",
                "sha256": "d9bdf6d0a943b9455572a259b7dcdf39359d3e50206626f45fb1d817d03d6057"
            },
            "downloads": -1,
            "filename": "dagster_serde-0.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "16756c1a199fb402be89b3a169a4f463",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8,<4.0",
            "size": 8911,
            "upload_time": "2023-10-11T20:48:21",
            "upload_time_iso_8601": "2023-10-11T20:48:21.149555Z",
            "url": "https://files.pythonhosted.org/packages/c2/af/ca12183a49e76ca3c0c8b4fc351af1bec370b76a3b029341fd465253fc1f/dagster_serde-0.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "adc9c4367dea412ce33db6ecd230d9f6099cfbe4b307c61a4cff79545acc3622",
                "md5": "a8d108b500fd4a1fddfb09390b11699f",
                "sha256": "3a3767def9393668b03042b49e637dff3a2b6a3b633822d48ed17ea627b4620d"
            },
            "downloads": -1,
            "filename": "dagster_serde-0.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "a8d108b500fd4a1fddfb09390b11699f",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8,<4.0",
            "size": 8268,
            "upload_time": "2023-10-11T20:48:22",
            "upload_time_iso_8601": "2023-10-11T20:48:22.687311Z",
            "url": "https://files.pythonhosted.org/packages/ad/c9/c4367dea412ce33db6ecd230d9f6099cfbe4b307c61a4cff79545acc3622/dagster_serde-0.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-10-11 20:48:22",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "danielgafni",
    "github_project": "dagster-serde",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "dagster-serde"
}
Daniel Gafni
Elapsed time: 0.12985s