pydantic_numpy


Namepydantic_numpy JSON
Version 7.0.0 PyPI version JSON
download
home_pagehttps://github.com/caniko/pydantic-numpy
SummaryPydantic Model integration of the NumPy array
upload_time2024-11-16 07:23:09
maintainerCan H. Tartanoglu
docs_urlNone
authorCan H. Tartanoglu
requires_python<3.14,>=3.10
licenseBSD-4
keywords pydantic numpy typing validation
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # pydantic-numpy

![Python 3.10-3.13](https://img.shields.io/badge/python-3.9--3.13-blue.svg)
[![Packaged with Poetry](https://img.shields.io/badge/packaging-poetry-cyan.svg)](https://python-poetry.org/)
![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)
![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)
![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)


## Usage

Package that integrates NumPy Arrays into Pydantic!

- `pydantic_numpy.typing` provides many typings such as `NpNDArrayFp64`, `Np3DArrayFp64` (float64 that must be 3D)! Works with both `pydantic.BaseModel` and `pydantic.dataclass`
- `NumpyModel` (derived from `pydantic.BaseModel`) make it possible to dump and load `np.ndarray` within model fields alongside other fields that are not instances of `np.ndarray`!

See the [`test.helper.testing_groups`](https://github.com/caniko/pydantic-numpy/blob/trunk/tests/helper/testing_groups.py) to see types that are defined explicitly.

### Examples

For more examples see [test_ndarray.py](./tests/test_typing.py)

```python
import numpy as np
from pydantic import BaseModel

import pydantic_numpy.typing as pnd
from pydantic_numpy import np_array_pydantic_annotated_typing
from pydantic_numpy.model import NumpyModel, MultiArrayNumpyFile


class MyBaseModelDerivedModel(BaseModel):
    any_array_dtype_and_dimension: pnd.NpNDArray

    # Must be numpy float32 as dtype
    k: np_array_pydantic_annotated_typing(data_type=np.float32)
    shorthand_for_k: pnd.NpNDArrayFp32

    must_be_1d_np_array: np_array_pydantic_annotated_typing(dimensions=1)


class MyDemoNumpyModel(NumpyModel):
    k: np_array_pydantic_annotated_typing(data_type=np.float32)


# Instantiate from array
cfg = MyDemoModel(k=[1, 2])
# Instantiate from numpy file
cfg = MyDemoModel(k="path_to/array.npy")
# Instantiate from npz file with key
cfg = MyDemoModel(k=MultiArrayNumpyFile(path="path_to/array.npz", key="k"))

cfg.k   # np.ndarray[np.float32]

cfg.dump("path_to_dump_dir", "object_id")
cfg.load("path_to_dump_dir", "object_id")
```

`NumpyModel.load` requires the original model:
```python
MyNumpyModel.load(<path>)
```
Use `model_agnostic_load` when you have several models that may be the correct model:

```python
from pydantic_numpy.model import model_agnostic_load

cfg.dump("path_to_dump_dir", "object_id")
equals_cfg = model_agnostic_load("path_to_dump_dir", "object_id", models=[MyNumpyModel, MyDemoModel])
```

### Custom type
There are two ways to define. Function derived types with `pydantic_numpy.helper.annotation.np_array_pydantic_annotated_typing`.

Function derived types don't work with static type checkers like Pyright and MyPy. In case you need the support,
just create the types yourself:
    
```python
NpStrict1DArrayInt64 = Annotated[
    np.ndarray[tuple[int], np.dtype[np.int64]],
    NpArrayPydanticAnnotation.factory(data_type=np.int64, dimensions=1, strict_data_typing=True),
]
```

#### Custom serialization

If the default serialization of NumpyDataDict, as outlined in [typing.py](https://github.com/caniko/pydantic-numpy/blob/trunk/pydantic_numpy/helper/typing.py), doesn't meet your requirements, you have the option to define a custom type with its own serializer. This can be achieved using the NpArrayPydanticAnnotation.factory method, which accepts a custom serialization function through its serialize_numpy_array_to_json parameter. This parameter expects a function of the form `Callable[[npt.ArrayLike], Iterable]`, allowing you to tailor the serialization process to your specific needs.

Example below illustrates definition of 1d-array of `float32` type that serializes to flat Python list (without nested dict as in default `NumpyDataDict` case):

```python
def _serialize_numpy_array_to_float_list(array_like: npt.ArrayLike) -> Iterable:
    return np.array(array_like).astype(float).tolist()


Np1DArrayFp32 = Annotated[
    np.ndarray[tuple[int], np.dtype[np.float32]],
    NpArrayPydanticAnnotation.factory(
        data_type=np.float32,
        dimensions=1,
        strict_data_typing=False,
        serialize_numpy_array_to_json=_serialize_numpy_array_to_float_list,
    ),
]
```

### Install
```shell
pip install pydantic-numpy
```

### History
The original idea originates from [this discussion](https://gist.github.com/danielhfrank/00e6b8556eed73fb4053450e602d2434), and forked from [cheind's](https://github.com/cheind/pydantic-numpy) repository.

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/caniko/pydantic-numpy",
    "name": "pydantic_numpy",
    "maintainer": "Can H. Tartanoglu",
    "docs_url": null,
    "requires_python": "<3.14,>=3.10",
    "maintainer_email": "python@rotas.mozmail.com",
    "keywords": "pydantic, numpy, typing, validation",
    "author": "Can H. Tartanoglu",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/d6/69/8b528273fabaebcfcb3f864e237b9a187863ecb3ad4cacfc446cfd7c879a/pydantic_numpy-7.0.0.tar.gz",
    "platform": null,
    "description": "# pydantic-numpy\n\n![Python 3.10-3.13](https://img.shields.io/badge/python-3.9--3.13-blue.svg)\n[![Packaged with Poetry](https://img.shields.io/badge/packaging-poetry-cyan.svg)](https://python-poetry.org/)\n![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)\n![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)\n![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)\n\n\n## Usage\n\nPackage that integrates NumPy Arrays into Pydantic!\n\n- `pydantic_numpy.typing` provides many typings such as `NpNDArrayFp64`, `Np3DArrayFp64` (float64 that must be 3D)! Works with both `pydantic.BaseModel` and `pydantic.dataclass`\n- `NumpyModel` (derived from `pydantic.BaseModel`) make it possible to dump and load `np.ndarray` within model fields alongside other fields that are not instances of `np.ndarray`!\n\nSee the [`test.helper.testing_groups`](https://github.com/caniko/pydantic-numpy/blob/trunk/tests/helper/testing_groups.py) to see types that are defined explicitly.\n\n### Examples\n\nFor more examples see [test_ndarray.py](./tests/test_typing.py)\n\n```python\nimport numpy as np\nfrom pydantic import BaseModel\n\nimport pydantic_numpy.typing as pnd\nfrom pydantic_numpy import np_array_pydantic_annotated_typing\nfrom pydantic_numpy.model import NumpyModel, MultiArrayNumpyFile\n\n\nclass MyBaseModelDerivedModel(BaseModel):\n    any_array_dtype_and_dimension: pnd.NpNDArray\n\n    # Must be numpy float32 as dtype\n    k: np_array_pydantic_annotated_typing(data_type=np.float32)\n    shorthand_for_k: pnd.NpNDArrayFp32\n\n    must_be_1d_np_array: np_array_pydantic_annotated_typing(dimensions=1)\n\n\nclass MyDemoNumpyModel(NumpyModel):\n    k: np_array_pydantic_annotated_typing(data_type=np.float32)\n\n\n# Instantiate from array\ncfg = MyDemoModel(k=[1, 2])\n# Instantiate from numpy file\ncfg = MyDemoModel(k=\"path_to/array.npy\")\n# Instantiate from npz file with key\ncfg = MyDemoModel(k=MultiArrayNumpyFile(path=\"path_to/array.npz\", key=\"k\"))\n\ncfg.k   # np.ndarray[np.float32]\n\ncfg.dump(\"path_to_dump_dir\", \"object_id\")\ncfg.load(\"path_to_dump_dir\", \"object_id\")\n```\n\n`NumpyModel.load` requires the original model:\n```python\nMyNumpyModel.load(<path>)\n```\nUse `model_agnostic_load` when you have several models that may be the correct model:\n\n```python\nfrom pydantic_numpy.model import model_agnostic_load\n\ncfg.dump(\"path_to_dump_dir\", \"object_id\")\nequals_cfg = model_agnostic_load(\"path_to_dump_dir\", \"object_id\", models=[MyNumpyModel, MyDemoModel])\n```\n\n### Custom type\nThere are two ways to define. Function derived types with `pydantic_numpy.helper.annotation.np_array_pydantic_annotated_typing`.\n\nFunction derived types don't work with static type checkers like Pyright and MyPy. In case you need the support,\njust create the types yourself:\n    \n```python\nNpStrict1DArrayInt64 = Annotated[\n    np.ndarray[tuple[int], np.dtype[np.int64]],\n    NpArrayPydanticAnnotation.factory(data_type=np.int64, dimensions=1, strict_data_typing=True),\n]\n```\n\n#### Custom serialization\n\nIf the default serialization of NumpyDataDict, as outlined in [typing.py](https://github.com/caniko/pydantic-numpy/blob/trunk/pydantic_numpy/helper/typing.py), doesn't meet your requirements, you have the option to define a custom type with its own serializer. This can be achieved using the NpArrayPydanticAnnotation.factory method, which accepts a custom serialization function through its serialize_numpy_array_to_json parameter. This parameter expects a function of the form `Callable[[npt.ArrayLike], Iterable]`, allowing you to tailor the serialization process to your specific needs.\n\nExample below illustrates definition of 1d-array of `float32` type that serializes to flat Python list (without nested dict as in default `NumpyDataDict` case):\n\n```python\ndef _serialize_numpy_array_to_float_list(array_like: npt.ArrayLike) -> Iterable:\n    return np.array(array_like).astype(float).tolist()\n\n\nNp1DArrayFp32 = Annotated[\n    np.ndarray[tuple[int], np.dtype[np.float32]],\n    NpArrayPydanticAnnotation.factory(\n        data_type=np.float32,\n        dimensions=1,\n        strict_data_typing=False,\n        serialize_numpy_array_to_json=_serialize_numpy_array_to_float_list,\n    ),\n]\n```\n\n### Install\n```shell\npip install pydantic-numpy\n```\n\n### History\nThe original idea originates from [this discussion](https://gist.github.com/danielhfrank/00e6b8556eed73fb4053450e602d2434), and forked from [cheind's](https://github.com/cheind/pydantic-numpy) repository.\n",
    "bugtrack_url": null,
    "license": "BSD-4",
    "summary": "Pydantic Model integration of the NumPy array",
    "version": "7.0.0",
    "project_urls": {
        "Homepage": "https://github.com/caniko/pydantic-numpy"
    },
    "split_keywords": [
        "pydantic",
        " numpy",
        " typing",
        " validation"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "50e7d2876afc8d2a3ef839c1428919063c4a0a3608e4aac821b9c57d1181233d",
                "md5": "86f4139dc1f371a344a390cbbdfb80e0",
                "sha256": "51602b946fc0f4aab4940faef7046a93676e0d94bb9f59ea4bd18a712a58a434"
            },
            "downloads": -1,
            "filename": "pydantic_numpy-7.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "86f4139dc1f371a344a390cbbdfb80e0",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<3.14,>=3.10",
            "size": 19804,
            "upload_time": "2024-11-16T07:23:08",
            "upload_time_iso_8601": "2024-11-16T07:23:08.226958Z",
            "url": "https://files.pythonhosted.org/packages/50/e7/d2876afc8d2a3ef839c1428919063c4a0a3608e4aac821b9c57d1181233d/pydantic_numpy-7.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d6698b528273fabaebcfcb3f864e237b9a187863ecb3ad4cacfc446cfd7c879a",
                "md5": "0688b264076f66addaef53292282aa90",
                "sha256": "c70138afd8443dc593a0ef5c8f88f140f51f3ad38f4ebdb1e05f3a0b784ca6b3"
            },
            "downloads": -1,
            "filename": "pydantic_numpy-7.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "0688b264076f66addaef53292282aa90",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.14,>=3.10",
            "size": 15703,
            "upload_time": "2024-11-16T07:23:09",
            "upload_time_iso_8601": "2024-11-16T07:23:09.884703Z",
            "url": "https://files.pythonhosted.org/packages/d6/69/8b528273fabaebcfcb3f864e237b9a187863ecb3ad4cacfc446cfd7c879a/pydantic_numpy-7.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-11-16 07:23:09",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "caniko",
    "github_project": "pydantic-numpy",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "pydantic_numpy"
}
        
Elapsed time: 2.35883s