| Name | numbin JSON |
| Version |
0.5.0
JSON |
| download |
| home_page | |
| Summary | an efficient binary serialization format for numerical data |
| upload_time | 2024-02-28 17:26:02 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | Apache-2.0 |
| keywords |
machine learning
deep learning
serialization
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# NumBin
<p align="center">
<a href="https://pypi.org/project/numbin/">
<img src="https://badge.fury.io/py/numbin.svg" alt="PyPI version" height="20">
</a>
<a href="https://pypi.org/project/numbin">
<img src="https://img.shields.io/pypi/pyversions/numbin" alt="Python Version" />
</a>
<a href="https://tldrlegal.com/license/apache-license-2.0-(apache-2.0)">
<img src="https://img.shields.io/github/license/mosecorg/numbin" alt="License" height="20">
</a>
<a href="https://github.com/mosecorg/numbin/actions/workflows/python-check.yml">
<img src="https://github.com/mosecorg/numbin/actions/workflows/python-check.yml/badge.svg" alt="Check status" height="20">
</a>
</p>
An efficient binary serialization format for numerical data.
## Install
```sh
pip install numbin
```
## Usage
Work with pure NumPy data:
```python
import numbin as nb
import numpy as np
arr = np.random.rand(5, 3)
# in memory
binary = nb.dumps(arr)
print(nb.loads(binary))
# file
with open("num.bin", "wb") as f:
nb.dump(arr, f)
with open("num.bin", "rb") as f:
print(nb.load(f))
```
Work with complex data:
```python
from numbin.msg_ext import NumBinMessage
nbm = NumBinMessage()
data = {"tensor": arr, "labels": ["dog", "cat"], "safe": True}
# in memory
binary = nbm.dumps(data)
print(nbm.loads(binary))
# file
with open("data.bin", "wb") as f:
nbm.dump(data, f)
with open("data.bin", "rb") as f:
print(nbm.load(f))
```
## Benchmark
The code can be found in [bench.py](benchmark/bench.py)
Tested with Intel(R) Core(TM) i7-13700K Python 3.11.0.
```shell
pip install .[bench]
python benchmark/bench.py
```
```console
>>> benchmark for numpy array
========================================================================================================================
pickle_serde size: 1 times: min(3.33e-06) mid(3.782e-06) max(5.6893e-05) 95%(3.491e-06) Std.(2.1728e-07)
numbin_serde size: 1 times: min(9.9101e-07) mid(1.106e-06) max(0.00016518) 95%(1.032e-06) Std.(1.9601e-07)
numpy_serde size: 1 times: min(4.9589e-05) mid(5.2873e-05) max(0.0010263) 95%(5.0937e-05) Std.(7.0191e-06)
safets_serde size: 1 times: min(3.558e-06) mid(4.141e-06) max(0.00016262) 95%(3.841e-06) Std.(3.9577e-07)
msg_np_serde size: 1 times: min(1.743e-06) mid(1.937e-06) max(3.4253e-05) 95%(1.83e-06) Std.(1.3042e-07)
========================================================================================================================
pickle_serde size: 1024 times: min(3.555e-06) mid(4.158e-06) max(9.9592e-05) 95%(3.813e-06) Std.(5.7795e-07)
numbin_serde size: 1024 times: min(1.204e-06) mid(1.355e-06) max(2.9116e-05) 95%(1.256e-06) Std.(1.668e-07)
numpy_serde size: 1024 times: min(5.0394e-05) mid(5.4297e-05) max(0.00019953) 95%(5.2156e-05) Std.(2.0507e-06)
safets_serde size: 1024 times: min(4.08e-06) mid(4.667e-06) max(4.5634e-05) 95%(4.342e-06) Std.(2.6851e-07)
msg_np_serde size: 1024 times: min(2.081e-06) mid(2.339e-06) max(3.0831e-05) 95%(2.194e-06) Std.(2.1181e-07)
========================================================================================================================
pickle_serde size: 65536 times: min(1.9884e-05) mid(2.1078e-05) max(9.3203e-05) 95%(2.024e-05) Std.(1.2878e-06)
numbin_serde size: 65536 times: min(1.6847e-05) mid(1.7845e-05) max(5.5421e-05) 95%(1.7083e-05) Std.(1.0197e-06)
numpy_serde size: 65536 times: min(0.00010117) mid(0.00010785) max(0.00022275) 95%(0.00010237) Std.(4.3429e-06)
safets_serde size: 65536 times: min(4.0613e-05) mid(4.2319e-05) max(0.00010681) 95%(4.0948e-05) Std.(2.1643e-06)
msg_np_serde size: 65536 times: min(2.4801e-05) mid(2.6234e-05) max(7.0627e-05) 95%(2.5042e-05) Std.(1.2072e-06)
========================================================================================================================
pickle_serde size: 3145728 times: min(0.0077576) mid(0.0080867) max(0.016288) 95%(0.0077705) Std.(0.00068357)
numbin_serde size: 3145728 times: min(0.0093903) mid(0.013968) max(0.014586) 95%(0.013006) Std.(0.00054932)
numpy_serde size: 3145728 times: min(0.016239) mid(0.017057) max(0.017629) 95%(0.01627) Std.(0.00038771)
safets_serde size: 3145728 times: min(0.01532) mid(0.016254) max(0.022971) 95%(0.015348) Std.(0.00083347)
msg_np_serde size: 3145728 times: min(0.016298) mid(0.021077) max(0.021851) 95%(0.019673) Std.(0.00062183)
========================================================================================================================
pickle_serde size: 201326592 times: min(0.89339) mid(0.89483) max(0.89901) 95%(0.89343) Std.(0.0020278)
numbin_serde size: 201326592 times: min(0.87285) mid(0.87507) max(0.87934) 95%(0.87292) Std.(0.0021327)
numpy_serde size: 201326592 times: min(0.76402) mid(0.76939) max(0.8509) 95%(0.76415) Std.(0.032678)
safets_serde size: 201326592 times: min(1.7488) mid(1.7555) max(1.8294) 95%(1.7489) Std.(0.030627)
msg_np_serde size: 201326592 times: min(1.3325) mid(1.3386) max(1.343) 95%(1.3325) Std.(0.004391)
```
Raw data
{
"_id": null,
"home_page": "",
"name": "numbin",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "machine learning,deep learning,serialization",
"author": "",
"author_email": "Keming <kemingy94@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/a6/6b/3c37c967513180e61cea6b9ddfaf1022607ce6c4c112305b83e38849fcd2/numbin-0.5.0.tar.gz",
"platform": null,
"description": "# NumBin\n\n<p align=\"center\">\n <a href=\"https://pypi.org/project/numbin/\">\n <img src=\"https://badge.fury.io/py/numbin.svg\" alt=\"PyPI version\" height=\"20\">\n </a>\n <a href=\"https://pypi.org/project/numbin\">\n <img src=\"https://img.shields.io/pypi/pyversions/numbin\" alt=\"Python Version\" />\n </a>\n <a href=\"https://tldrlegal.com/license/apache-license-2.0-(apache-2.0)\">\n <img src=\"https://img.shields.io/github/license/mosecorg/numbin\" alt=\"License\" height=\"20\">\n </a>\n <a href=\"https://github.com/mosecorg/numbin/actions/workflows/python-check.yml\">\n <img src=\"https://github.com/mosecorg/numbin/actions/workflows/python-check.yml/badge.svg\" alt=\"Check status\" height=\"20\">\n </a>\n</p>\n\nAn efficient binary serialization format for numerical data.\n\n## Install\n\n```sh\npip install numbin\n```\n\n## Usage\n\nWork with pure NumPy data:\n\n```python\nimport numbin as nb\nimport numpy as np\n\n\narr = np.random.rand(5, 3)\n\n# in memory\nbinary = nb.dumps(arr)\nprint(nb.loads(binary))\n\n# file\nwith open(\"num.bin\", \"wb\") as f:\n nb.dump(arr, f)\n\nwith open(\"num.bin\", \"rb\") as f:\n print(nb.load(f))\n```\n\nWork with complex data:\n\n```python\nfrom numbin.msg_ext import NumBinMessage\n\n\nnbm = NumBinMessage()\ndata = {\"tensor\": arr, \"labels\": [\"dog\", \"cat\"], \"safe\": True}\n\n# in memory\nbinary = nbm.dumps(data)\nprint(nbm.loads(binary))\n\n# file\nwith open(\"data.bin\", \"wb\") as f:\n nbm.dump(data, f)\n\nwith open(\"data.bin\", \"rb\") as f:\n print(nbm.load(f))\n```\n\n## Benchmark\n\nThe code can be found in [bench.py](benchmark/bench.py)\n\nTested with Intel(R) Core(TM) i7-13700K Python 3.11.0.\n\n```shell\npip install .[bench]\npython benchmark/bench.py\n```\n\n```console\n>>> benchmark for numpy array\n========================================================================================================================\npickle_serde\tsize: 1\ttimes: min(3.33e-06)\tmid(3.782e-06)\tmax(5.6893e-05)\t95%(3.491e-06)\tStd.(2.1728e-07)\nnumbin_serde\tsize: 1\ttimes: min(9.9101e-07)\tmid(1.106e-06)\tmax(0.00016518)\t95%(1.032e-06)\tStd.(1.9601e-07)\nnumpy_serde\tsize: 1\ttimes: min(4.9589e-05)\tmid(5.2873e-05)\tmax(0.0010263)\t95%(5.0937e-05)\tStd.(7.0191e-06)\nsafets_serde\tsize: 1\ttimes: min(3.558e-06)\tmid(4.141e-06)\tmax(0.00016262)\t95%(3.841e-06)\tStd.(3.9577e-07)\nmsg_np_serde\tsize: 1\ttimes: min(1.743e-06)\tmid(1.937e-06)\tmax(3.4253e-05)\t95%(1.83e-06)\tStd.(1.3042e-07)\n========================================================================================================================\npickle_serde\tsize: 1024\ttimes: min(3.555e-06)\tmid(4.158e-06)\tmax(9.9592e-05)\t95%(3.813e-06)\tStd.(5.7795e-07)\nnumbin_serde\tsize: 1024\ttimes: min(1.204e-06)\tmid(1.355e-06)\tmax(2.9116e-05)\t95%(1.256e-06)\tStd.(1.668e-07)\nnumpy_serde\tsize: 1024\ttimes: min(5.0394e-05)\tmid(5.4297e-05)\tmax(0.00019953)\t95%(5.2156e-05)\tStd.(2.0507e-06)\nsafets_serde\tsize: 1024\ttimes: min(4.08e-06)\tmid(4.667e-06)\tmax(4.5634e-05)\t95%(4.342e-06)\tStd.(2.6851e-07)\nmsg_np_serde\tsize: 1024\ttimes: min(2.081e-06)\tmid(2.339e-06)\tmax(3.0831e-05)\t95%(2.194e-06)\tStd.(2.1181e-07)\n========================================================================================================================\npickle_serde\tsize: 65536\ttimes: min(1.9884e-05)\tmid(2.1078e-05)\tmax(9.3203e-05)\t95%(2.024e-05)\tStd.(1.2878e-06)\nnumbin_serde\tsize: 65536\ttimes: min(1.6847e-05)\tmid(1.7845e-05)\tmax(5.5421e-05)\t95%(1.7083e-05)\tStd.(1.0197e-06)\nnumpy_serde\tsize: 65536\ttimes: min(0.00010117)\tmid(0.00010785)\tmax(0.00022275)\t95%(0.00010237)\tStd.(4.3429e-06)\nsafets_serde\tsize: 65536\ttimes: min(4.0613e-05)\tmid(4.2319e-05)\tmax(0.00010681)\t95%(4.0948e-05)\tStd.(2.1643e-06)\nmsg_np_serde\tsize: 65536\ttimes: min(2.4801e-05)\tmid(2.6234e-05)\tmax(7.0627e-05)\t95%(2.5042e-05)\tStd.(1.2072e-06)\n========================================================================================================================\npickle_serde\tsize: 3145728\ttimes: min(0.0077576)\tmid(0.0080867)\tmax(0.016288)\t95%(0.0077705)\tStd.(0.00068357)\nnumbin_serde\tsize: 3145728\ttimes: min(0.0093903)\tmid(0.013968)\tmax(0.014586)\t95%(0.013006)\tStd.(0.00054932)\nnumpy_serde\tsize: 3145728\ttimes: min(0.016239)\tmid(0.017057)\tmax(0.017629)\t95%(0.01627)\tStd.(0.00038771)\nsafets_serde\tsize: 3145728\ttimes: min(0.01532)\tmid(0.016254)\tmax(0.022971)\t95%(0.015348)\tStd.(0.00083347)\nmsg_np_serde\tsize: 3145728\ttimes: min(0.016298)\tmid(0.021077)\tmax(0.021851)\t95%(0.019673)\tStd.(0.00062183)\n========================================================================================================================\npickle_serde\tsize: 201326592\ttimes: min(0.89339)\tmid(0.89483)\tmax(0.89901)\t95%(0.89343)\tStd.(0.0020278)\nnumbin_serde\tsize: 201326592\ttimes: min(0.87285)\tmid(0.87507)\tmax(0.87934)\t95%(0.87292)\tStd.(0.0021327)\nnumpy_serde\tsize: 201326592\ttimes: min(0.76402)\tmid(0.76939)\tmax(0.8509)\t95%(0.76415)\tStd.(0.032678)\nsafets_serde\tsize: 201326592\ttimes: min(1.7488)\tmid(1.7555)\tmax(1.8294)\t95%(1.7489)\tStd.(0.030627)\nmsg_np_serde\tsize: 201326592\ttimes: min(1.3325)\tmid(1.3386)\tmax(1.343)\t95%(1.3325)\tStd.(0.004391)\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "an efficient binary serialization format for numerical data",
"version": "0.5.0",
"project_urls": {
"changelog": "https://github.com/mosecorg/numbin/releases",
"documentation": "https://mosecorg.github.io/numbin",
"homepage": "https://mosecorg.github.io/numbin",
"repository": "https://github.com/mosecorg/numbin"
},
"split_keywords": [
"machine learning",
"deep learning",
"serialization"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c81724bef191255e24b839db65a34b673a1221cfd590223de5423e9dd9416d27",
"md5": "88666797a5194c0c29f43bb5e2763eb0",
"sha256": "c0842e4e1d99478621d854ce055d788114897dbf3057e0bafa3ccc35fd18e452"
},
"downloads": -1,
"filename": "numbin-0.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "88666797a5194c0c29f43bb5e2763eb0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 9268,
"upload_time": "2024-02-28T17:26:00",
"upload_time_iso_8601": "2024-02-28T17:26:00.566983Z",
"url": "https://files.pythonhosted.org/packages/c8/17/24bef191255e24b839db65a34b673a1221cfd590223de5423e9dd9416d27/numbin-0.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a66b3c37c967513180e61cea6b9ddfaf1022607ce6c4c112305b83e38849fcd2",
"md5": "7084bafec3cb223e19beac52a802a23a",
"sha256": "897b546a6ddbea2f337658388f1820d45e42f89bd0211e8945cbc61dd898b1d5"
},
"downloads": -1,
"filename": "numbin-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "7084bafec3cb223e19beac52a802a23a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 8914,
"upload_time": "2024-02-28T17:26:02",
"upload_time_iso_8601": "2024-02-28T17:26:02.382983Z",
"url": "https://files.pythonhosted.org/packages/a6/6b/3c37c967513180e61cea6b9ddfaf1022607ce6c4c112305b83e38849fcd2/numbin-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-28 17:26:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mosecorg",
"github_project": "numbin",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "numbin"
}
