Overview
========
ndarray + sharing = ndsharray
Introduction
------------
This python module let you share a numpy ndarray within different processes (either via python's multiprocessing or
sharing between different python instances). The library behind this package is the lib
[mmap](https://docs.python.org/3/library/mmap.html) from official python - no extra library, except numpy, is needed.
The mmap approach is using the shared memory, which can be accessed by different CPUs/python instances. Using shared
memory is much
faster than the pickle approach - you can even do a video streaming on a Raspberry Pi / Jetson Nano over multiple
python processes.\
This library is easy-to-use, just initialize the shared array with a unique tag and write/read! You can even change the
numpy array dimension/shape/dtype during runtime - the mmap will be silently rebuild if there is a change in the
numpy array size/shape/dtype.
Small Example Code:
```python
import numpy as np
from ndsharray import NdShArray
shared_array = NdShArray("my_unique_tag", r_w='w') # r_w must be specified
my_array = np.random.random((400, 800))
shared_array.write(my_array)
print(my_array)
```
That's all for writing into shared memory. How to read? Open a second python instance:
```python
import numpy as np
from ndsharray import NdShArray
shared_array = NdShArray("my_unique_tag", r_w='r') # r_w must be specified
status, my_array = shared_array.read()
print(my_array)
```
Documentation
-------------
Can be found here: https://ndsharray.readthedocs.io/en/latest/
Requirements
------------
- Python ≥ 3.6
- numpy
Tested with example codes on
- Windows 10, amd64, Python 3.6 / 3.8
- Ubuntu 20, amd64, Python 3.6 /3.8
- NVIDIA Jetson Nano with Ubuntu 18.04, ARM64-bit (aarch64), Python 3.6
Some technical notes
--------------------
This library shall be an easy-to-use library and also shall be faster than pickling numpy arrays to another process.
Please note that the python's provided
[shared_memory](https://docs.python.org/3/library/multiprocessing.shared_memory.html) does the same as ndsharray, but
is using byte array instead of numpy array! However, shared_memory is available since python 3.8 and not
supported for python 3.6.
Installation from Github
------------------------
Make sure to have git, python and pip in your environment path or activate your python environment.\
To install enter in cmd/shell:
```console
git clone https://github.com/monzelr/ndsharray.git
cd ndsharray
pip install .
```
Alternative with python:
```console
python setup.py install
```
Raw data
{
"_id": null,
"home_page": "https://github.com/monzelr/ndsharray",
"name": "ndsharray",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "sharing numpy arrays, inter process communication, subprocess, multiple process reading a numpy ndarray",
"author": "Rune Monzel",
"author_email": "runemonzel@googlemail.com",
"download_url": null,
"platform": null,
"description": "Overview\r\n========\r\nndarray + sharing = ndsharray\r\n\r\nIntroduction\r\n------------\r\nThis python module let you share a numpy ndarray within different processes (either via python's multiprocessing or \r\nsharing between different python instances). The library behind this package is the lib \r\n[mmap](https://docs.python.org/3/library/mmap.html) from official python - no extra library, except numpy, is needed. \r\nThe mmap approach is using the shared memory, which can be accessed by different CPUs/python instances. Using shared \r\nmemory is much \r\nfaster than the pickle approach - you can even do a video streaming on a Raspberry Pi / Jetson Nano over multiple \r\npython processes.\\\r\nThis library is easy-to-use, just initialize the shared array with a unique tag and write/read! You can even change the \r\nnumpy array dimension/shape/dtype during runtime - the mmap will be silently rebuild if there is a change in the \r\nnumpy array size/shape/dtype.\r\n\r\nSmall Example Code:\r\n```python\r\nimport numpy as np\r\nfrom ndsharray import NdShArray\r\n \r\nshared_array = NdShArray(\"my_unique_tag\", r_w='w') # r_w must be specified\r\n\r\nmy_array = np.random.random((400, 800))\r\nshared_array.write(my_array)\r\n\r\nprint(my_array)\r\n```\r\n\r\nThat's all for writing into shared memory. How to read? Open a second python instance:\r\n```python\r\nimport numpy as np\r\nfrom ndsharray import NdShArray\r\n\r\nshared_array = NdShArray(\"my_unique_tag\", r_w='r') # r_w must be specified\r\n\r\nstatus, my_array = shared_array.read()\r\n\r\nprint(my_array)\r\n```\r\n\r\nDocumentation\r\n-------------\r\nCan be found here: https://ndsharray.readthedocs.io/en/latest/\r\n\r\n\r\nRequirements\r\n------------ \r\n- Python \u00e2\u2030\u00a5 3.6\r\n- numpy\r\n\r\nTested with example codes on \r\n- Windows 10, amd64, Python 3.6 / 3.8\r\n- Ubuntu 20, amd64, Python 3.6 /3.8\r\n- NVIDIA Jetson Nano with Ubuntu 18.04, ARM64-bit (aarch64), Python 3.6\r\n\r\nSome technical notes\r\n--------------------\r\nThis library shall be an easy-to-use library and also shall be faster than pickling numpy arrays to another process. \r\nPlease note that the python's provided \r\n[shared_memory](https://docs.python.org/3/library/multiprocessing.shared_memory.html) does the same as ndsharray, but \r\nis using byte array instead of numpy array! However, shared_memory is available since python 3.8 and not \r\nsupported for python 3.6.\r\n\r\n\r\nInstallation from Github\r\n------------------------\r\nMake sure to have git, python and pip in your environment path or activate your python environment.\\\r\nTo install enter in cmd/shell:\r\n```console\r\ngit clone https://github.com/monzelr/ndsharray.git\r\n\r\ncd ndsharray\r\n\r\npip install .\r\n```\r\n\r\nAlternative with python:\r\n```console\r\npython setup.py install\r\n```\r\n\r\n",
"bugtrack_url": null,
"license": "BSD 3-Clause License",
"summary": "Sharing numpy ndarray with a simple API between different python processes with shared memory.",
"version": "1.1.3",
"project_urls": {
"Homepage": "https://github.com/monzelr/ndsharray"
},
"split_keywords": [
"sharing numpy arrays",
" inter process communication",
" subprocess",
" multiple process reading a numpy ndarray"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8ccf1a48fc3263cd66150425fe3b09e3c2ea07e57403398b52c5d654c4d8b129",
"md5": "0bf7b3c5776aca6c5bae1c54a339594b",
"sha256": "2a8e1fa5328eb160b5bf34f7bf433f91d520ef9b7780707911fe26a5689f84a7"
},
"downloads": -1,
"filename": "ndsharray-1.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0bf7b3c5776aca6c5bae1c54a339594b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 8485,
"upload_time": "2024-06-24T13:39:36",
"upload_time_iso_8601": "2024-06-24T13:39:36.659465Z",
"url": "https://files.pythonhosted.org/packages/8c/cf/1a48fc3263cd66150425fe3b09e3c2ea07e57403398b52c5d654c4d8b129/ndsharray-1.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-24 13:39:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "monzelr",
"github_project": "ndsharray",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "ndsharray"
}
JSON
