| Name | array-split JSON |
| Version |
0.6.5
JSON |
| download |
| home_page | None |
| Summary | The array_split python package is an enhancement to existing numpy.ndarray functions (such as numpy.array_split) which sub-divide a multi-dimensional array into a number of multi-dimensional sub-arrays (slices) |
| upload_time | 2024-08-04 05:42:04 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | Copyright (C) 2017 The Australian National University. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
multi-dimensional-array
array
sub-array
tile
tiling
splitting
split
partition
partitioning
scipy
numpy
ndarray
domain-decomposition
array-decomposition
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
=============
`array_split`
=============
.. Start of sphinx doc include.
.. start long description.
.. start badges.
.. image:: https://img.shields.io/pypi/v/array_split.svg
:target: https://pypi.python.org/pypi/array_split/
:alt: array_split python package
.. image:: https://github.com/array-split/array_split/actions/workflows/python-test.yml/badge.svg
:target: https://github.com/array-split/array_split/actions/workflows/python-test.yml
:alt: array_split python package
.. image:: https://readthedocs.org/projects/array-split/badge/?version=stable
:target: http://array-split.readthedocs.io/en/stable
:alt: Documentation Status
.. image:: https://coveralls.io/repos/github/array-split/array_split/badge.svg
:target: https://coveralls.io/github/array-split/array_split
:alt: Coveralls Status
.. image:: https://img.shields.io/pypi/l/array_split.svg
:target: https://pypi.python.org/pypi/array_split/
:alt: MIT License
.. image:: https://img.shields.io/pypi/pyversions/array_split.svg
:target: https://pypi.python.org/pypi/array_split/
:alt: array_split python package
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.889078.svg
:target: https://doi.org/10.5281/zenodo.889078
.. image:: http://joss.theoj.org/papers/10.21105/joss.00373/status.svg
:target: http://joss.theoj.org/papers/4b59c7430176ef78c80c6a1100031eb6
.. end badges.
The `array_split <http://array-split.readthedocs.io/en/latest>`_ python package is
an enhancement to existing
`numpy.ndarray <http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html>`_ functions,
such as
`numpy.array_split <http://docs.scipy.org/doc/numpy/reference/generated/numpy.array_split.html>`_,
`skimage.util.view_as_blocks <http://scikit-image.org/docs/0.13.x/api/skimage.util.html#view-as-blocks>`_
and
`skimage.util.view_as_windows <http://scikit-image.org/docs/0.13.x/api/skimage.util.html#view-as-windows>`_,
which sub-divide a multi-dimensional array into a number of multi-dimensional sub-arrays (slices).
Example application areas include:
**Parallel Processing**
A large (dense) array is partitioned into smaller sub-arrays which can be
processed concurrently by multiple processes
(`multiprocessing <https://docs.python.org/3/library/multiprocessing.html>`_
or `mpi4py <http://pythonhosted.org/mpi4py/>`_) or other memory-limited hardware
(e.g. GPGPU using `pyopencl <https://mathema.tician.de/software/pyopencl/>`_,
`pycuda <https://mathema.tician.de/software/pycuda/>`_, etc).
For GPGPU, it is necessary for sub-array not to exceed the GPU memory and
desirable for the sub-array shape to be a multiple of the *work-group*
(`OpenCL <https://en.wikipedia.org/wiki/OpenCL>`_)
or *thread-block* (`CUDA <https://en.wikipedia.org/wiki/CUDA>`_) size.
**File I/O**
A large (dense) array is partitioned into smaller sub-arrays which can be
written to individual files
(as, for example, a
`HDF5 Virtual Dataset <https://support.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html>`_).
It is often desirable for the individual files not to exceed a specified number
of (Giga) bytes and, for `HDF5 <https://support.hdfgroup.org/HDF5/>`_, it is desirable
to have the individual file sub-array shape a multiple of
the `chunk shape <https://support.hdfgroup.org/HDF5/doc1.8/Advanced/Chunking/index.html>`_.
Similarly, `out of core <https://en.wikipedia.org/wiki/Out-of-core_algorithm>`_
algorithms for large dense arrays often involve processing the entire data-set as
a series of *in-core* sub-arrays. Again, it is desirable for the individual sub-array shape
to be a multiple of the
`chunk shape <https://support.hdfgroup.org/HDF5/doc1.8/Advanced/Chunking/index.html>`_.
The `array_split <http://array-split.readthedocs.io/en/latest>`_ package provides the
means to partition an array (or array shape) using any of the following criteria:
- Per-axis indices indicating the *cut* positions.
- Per-axis number of sub-arrays.
- Total number of sub-arrays (with optional per-axis *number of sections* constraints).
- Specific sub-array shape.
- Specification of *halo* (*ghost*) elements for sub-arrays.
- Arbitrary *start index* for the shape to be partitioned.
- Maximum number of bytes for a sub-array with constraints:
- sub-arrays are an even multiple of a specified sub-tile shape
- upper limit on the per-axis sub-array shape
Quick Start Example
===================
>>> from array_split import array_split, shape_split
>>> import numpy as np
>>>
>>> ary = np.arange(0, 4*9)
>>>
>>> array_split(ary, 4) # 1D split into 4 sections (like numpy.array_split)
[array([0, 1, 2, 3, 4, 5, 6, 7, 8]),
array([ 9, 10, 11, 12, 13, 14, 15, 16, 17]),
array([18, 19, 20, 21, 22, 23, 24, 25, 26]),
array([27, 28, 29, 30, 31, 32, 33, 34, 35])]
>>>
>>> shape_split(ary.shape, 4) # 1D split into 4 parts, returns slice objects
array([(slice(0, 9, None),), (slice(9, 18, None),), (slice(18, 27, None),), (slice(27, 36, None),)],
dtype=[('0', 'O')])
>>>
>>> ary = ary.reshape(4, 9) # Make ary 2D
>>> split = shape_split(ary.shape, axis=(2, 3)) # 2D split into 2*3=6 sections
>>> split.shape
(2, 3)
>>> split
array([[(slice(0, 2, None), slice(0, 3, None)),
(slice(0, 2, None), slice(3, 6, None)),
(slice(0, 2, None), slice(6, 9, None))],
[(slice(2, 4, None), slice(0, 3, None)),
(slice(2, 4, None), slice(3, 6, None)),
(slice(2, 4, None), slice(6, 9, None))]],
dtype=[('0', 'O'), ('1', 'O')])
>>> sub_arys = [ary[tup] for tup in split.flatten()] # Create sub-array views from slice tuples.
>>> sub_arys
[array([[ 0, 1, 2], [ 9, 10, 11]]),
array([[ 3, 4, 5], [12, 13, 14]]),
array([[ 6, 7, 8], [15, 16, 17]]),
array([[18, 19, 20], [27, 28, 29]]),
array([[21, 22, 23], [30, 31, 32]]),
array([[24, 25, 26], [33, 34, 35]])]
Latest sphinx documentation (including more examples)
at http://array-split.readthedocs.io/en/latest/.
.. end long description.
Installation
============
Using ``pip`` (root access required):
``pip install array_split``
or local user install (no root access required):
``pip install --user array_split``
or local user install from latest github source:
``pip install --user git+git://github.com/array-split/array_split.git#egg=array_split``
Requirements
============
Requires `numpy <http://docs.scipy.org/doc/numpy/>`_ version `>= 1.6`,
python-2 version `>= 2.6` or python-3 version `>= 3.2`.
Testing
=======
Run tests (unit-tests and doctest module docstring tests) using::
python -m array_split.tests
or, from the source tree, run::
python setup.py test
Travis CI at:
https://travis-ci.org/array-split/array_split/
and AppVeyor at:
https://ci.appveyor.com/project/array-split/array-split
Documentation
=============
Latest sphinx generated documentation is at:
http://array-split.readthedocs.io/en/latest
and at github *gh-pages*:
https://array-split.github.io/array_split/
Sphinx documentation can be built from the source::
python setup.py build_sphinx
with the HTML generated in ``docs/_build/html``.
Latest source code
==================
Source at github:
https://github.com/array-split/array_split
Bug Reports
===========
To search for bugs or report them, please use the bug tracker at:
https://github.com/array-split/array_split/issues
Contributing
============
Check out the `CONTRIBUTING doc <https://github.com/array-split/array_split/blob/dev/CONTRIBUTING.rst>`_.
License information
===================
See the file `LICENSE.txt <https://github.com/array-split/array_split/blob/dev/LICENSE.txt>`_
for terms & conditions, for usage and a DISCLAIMER OF ALL WARRANTIES.
Raw data
{
"_id": null,
"home_page": null,
"name": "array-split",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "multi-dimensional-array, array, sub-array, tile, tiling, splitting, split, partition, partitioning, scipy, numpy, ndarray, domain-decomposition, array-decomposition",
"author": null,
"author_email": "Shane-J-Latham <array.split@gmail.com>, array-split <array.split@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/47/b4/f88e32eeee2ee6172244f8d19b374f1f5d6c3a1de9ba7c1712fbc7a2e176/array_split-0.6.5.zip",
"platform": null,
"description": "\n=============\n`array_split`\n=============\n\n.. Start of sphinx doc include.\n.. start long description.\n.. start badges.\n\n.. image:: https://img.shields.io/pypi/v/array_split.svg\n :target: https://pypi.python.org/pypi/array_split/\n :alt: array_split python package\n.. image:: https://github.com/array-split/array_split/actions/workflows/python-test.yml/badge.svg\n :target: https://github.com/array-split/array_split/actions/workflows/python-test.yml\n :alt: array_split python package\n.. image:: https://readthedocs.org/projects/array-split/badge/?version=stable\n :target: http://array-split.readthedocs.io/en/stable\n :alt: Documentation Status\n.. image:: https://coveralls.io/repos/github/array-split/array_split/badge.svg\n :target: https://coveralls.io/github/array-split/array_split\n :alt: Coveralls Status\n.. image:: https://img.shields.io/pypi/l/array_split.svg\n :target: https://pypi.python.org/pypi/array_split/\n :alt: MIT License\n.. image:: https://img.shields.io/pypi/pyversions/array_split.svg\n :target: https://pypi.python.org/pypi/array_split/\n :alt: array_split python package\n.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.889078.svg\n :target: https://doi.org/10.5281/zenodo.889078\n.. image:: http://joss.theoj.org/papers/10.21105/joss.00373/status.svg\n :target: http://joss.theoj.org/papers/4b59c7430176ef78c80c6a1100031eb6\n\n.. end badges.\n\nThe `array_split <http://array-split.readthedocs.io/en/latest>`_ python package is\nan enhancement to existing\n`numpy.ndarray <http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html>`_ functions,\nsuch as\n`numpy.array_split <http://docs.scipy.org/doc/numpy/reference/generated/numpy.array_split.html>`_,\n`skimage.util.view_as_blocks <http://scikit-image.org/docs/0.13.x/api/skimage.util.html#view-as-blocks>`_\nand\n`skimage.util.view_as_windows <http://scikit-image.org/docs/0.13.x/api/skimage.util.html#view-as-windows>`_,\nwhich sub-divide a multi-dimensional array into a number of multi-dimensional sub-arrays (slices).\nExample application areas include:\n\n**Parallel Processing**\n A large (dense) array is partitioned into smaller sub-arrays which can be\n processed concurrently by multiple processes\n (`multiprocessing <https://docs.python.org/3/library/multiprocessing.html>`_\n or `mpi4py <http://pythonhosted.org/mpi4py/>`_) or other memory-limited hardware\n (e.g. GPGPU using `pyopencl <https://mathema.tician.de/software/pyopencl/>`_,\n `pycuda <https://mathema.tician.de/software/pycuda/>`_, etc).\n For GPGPU, it is necessary for sub-array not to exceed the GPU memory and\n desirable for the sub-array shape to be a multiple of the *work-group*\n (`OpenCL <https://en.wikipedia.org/wiki/OpenCL>`_)\n or *thread-block* (`CUDA <https://en.wikipedia.org/wiki/CUDA>`_) size.\n\n**File I/O**\n A large (dense) array is partitioned into smaller sub-arrays which can be\n written to individual files\n (as, for example, a\n `HDF5 Virtual Dataset <https://support.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html>`_).\n It is often desirable for the individual files not to exceed a specified number\n of (Giga) bytes and, for `HDF5 <https://support.hdfgroup.org/HDF5/>`_, it is desirable\n to have the individual file sub-array shape a multiple of\n the `chunk shape <https://support.hdfgroup.org/HDF5/doc1.8/Advanced/Chunking/index.html>`_.\n Similarly, `out of core <https://en.wikipedia.org/wiki/Out-of-core_algorithm>`_\n algorithms for large dense arrays often involve processing the entire data-set as\n a series of *in-core* sub-arrays. Again, it is desirable for the individual sub-array shape\n to be a multiple of the\n `chunk shape <https://support.hdfgroup.org/HDF5/doc1.8/Advanced/Chunking/index.html>`_. \n\n\nThe `array_split <http://array-split.readthedocs.io/en/latest>`_ package provides the\nmeans to partition an array (or array shape) using any of the following criteria:\n\n- Per-axis indices indicating the *cut* positions.\n- Per-axis number of sub-arrays.\n- Total number of sub-arrays (with optional per-axis *number of sections* constraints).\n- Specific sub-array shape.\n- Specification of *halo* (*ghost*) elements for sub-arrays.\n- Arbitrary *start index* for the shape to be partitioned.\n- Maximum number of bytes for a sub-array with constraints:\n\n - sub-arrays are an even multiple of a specified sub-tile shape\n - upper limit on the per-axis sub-array shape\n\n\nQuick Start Example\n===================\n\n\n >>> from array_split import array_split, shape_split\n >>> import numpy as np\n >>>\n >>> ary = np.arange(0, 4*9)\n >>> \n >>> array_split(ary, 4) # 1D split into 4 sections (like numpy.array_split)\n [array([0, 1, 2, 3, 4, 5, 6, 7, 8]),\n array([ 9, 10, 11, 12, 13, 14, 15, 16, 17]),\n array([18, 19, 20, 21, 22, 23, 24, 25, 26]),\n array([27, 28, 29, 30, 31, 32, 33, 34, 35])]\n >>> \n >>> shape_split(ary.shape, 4) # 1D split into 4 parts, returns slice objects \n array([(slice(0, 9, None),), (slice(9, 18, None),), (slice(18, 27, None),), (slice(27, 36, None),)], \n dtype=[('0', 'O')])\n >>> \n >>> ary = ary.reshape(4, 9) # Make ary 2D\n >>> split = shape_split(ary.shape, axis=(2, 3)) # 2D split into 2*3=6 sections\n >>> split.shape\n (2, 3)\n >>> split\n array([[(slice(0, 2, None), slice(0, 3, None)),\n (slice(0, 2, None), slice(3, 6, None)),\n (slice(0, 2, None), slice(6, 9, None))],\n [(slice(2, 4, None), slice(0, 3, None)),\n (slice(2, 4, None), slice(3, 6, None)),\n (slice(2, 4, None), slice(6, 9, None))]], \n dtype=[('0', 'O'), ('1', 'O')])\n >>> sub_arys = [ary[tup] for tup in split.flatten()] # Create sub-array views from slice tuples.\n >>> sub_arys\n [array([[ 0, 1, 2], [ 9, 10, 11]]),\n array([[ 3, 4, 5], [12, 13, 14]]),\n array([[ 6, 7, 8], [15, 16, 17]]),\n array([[18, 19, 20], [27, 28, 29]]),\n array([[21, 22, 23], [30, 31, 32]]),\n array([[24, 25, 26], [33, 34, 35]])]\n\n\nLatest sphinx documentation (including more examples)\nat http://array-split.readthedocs.io/en/latest/.\n\n.. end long description.\n\nInstallation\n============\n\nUsing ``pip`` (root access required):\n\n ``pip install array_split``\n \nor local user install (no root access required):\n \n ``pip install --user array_split``\n\nor local user install from latest github source:\n\n ``pip install --user git+git://github.com/array-split/array_split.git#egg=array_split``\n\n\nRequirements\n============\n\nRequires `numpy <http://docs.scipy.org/doc/numpy/>`_ version `>= 1.6`,\npython-2 version `>= 2.6` or python-3 version `>= 3.2`.\n\nTesting\n=======\n\nRun tests (unit-tests and doctest module docstring tests) using::\n\n python -m array_split.tests\n\nor, from the source tree, run::\n\n python setup.py test\n\n\nTravis CI at:\n\n https://travis-ci.org/array-split/array_split/\n\nand AppVeyor at:\n\n https://ci.appveyor.com/project/array-split/array-split\n\nDocumentation\n=============\n\nLatest sphinx generated documentation is at:\n\n http://array-split.readthedocs.io/en/latest\n\nand at github *gh-pages*:\n\n https://array-split.github.io/array_split/\n\nSphinx documentation can be built from the source::\n\n python setup.py build_sphinx\n\nwith the HTML generated in ``docs/_build/html``.\n\n\nLatest source code\n==================\n\nSource at github:\n\n https://github.com/array-split/array_split\n\n\nBug Reports\n===========\n\nTo search for bugs or report them, please use the bug tracker at:\n\n https://github.com/array-split/array_split/issues\n\n\nContributing\n============\n\nCheck out the `CONTRIBUTING doc <https://github.com/array-split/array_split/blob/dev/CONTRIBUTING.rst>`_.\n\n\nLicense information\n===================\n\nSee the file `LICENSE.txt <https://github.com/array-split/array_split/blob/dev/LICENSE.txt>`_\nfor terms & conditions, for usage and a DISCLAIMER OF ALL WARRANTIES.\n",
"bugtrack_url": null,
"license": "Copyright (C) 2017 The Australian National University. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "The array_split python package is an enhancement to existing numpy.ndarray functions (such as numpy.array_split) which sub-divide a multi-dimensional array into a number of multi-dimensional sub-arrays (slices)",
"version": "0.6.5",
"project_urls": {
"Documentation": "https://array-split.readthedocs.io/en/latest/",
"Homepage": "https://github.com/array-split/array_split",
"Issues": "https://github.com/array-split/array_split/issues",
"Repository": "https://github.com/array-split/array_split"
},
"split_keywords": [
"multi-dimensional-array",
" array",
" sub-array",
" tile",
" tiling",
" splitting",
" split",
" partition",
" partitioning",
" scipy",
" numpy",
" ndarray",
" domain-decomposition",
" array-decomposition"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "67f53558e3be19b839b321443f0421a7e1b83965e1ec18060c48bb546c41ce0d",
"md5": "78dc1e0817004e9cd4cd4dfdf9c31e41",
"sha256": "b33614b9af58c5ae8154b2fb34f68ad75786882d8e0abd6ea25b970a2b11eb92"
},
"downloads": -1,
"filename": "array_split-0.6.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "78dc1e0817004e9cd4cd4dfdf9c31e41",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.8",
"size": 33058,
"upload_time": "2024-08-04T05:42:03",
"upload_time_iso_8601": "2024-08-04T05:42:03.062115Z",
"url": "https://files.pythonhosted.org/packages/67/f5/3558e3be19b839b321443f0421a7e1b83965e1ec18060c48bb546c41ce0d/array_split-0.6.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "47b4f88e32eeee2ee6172244f8d19b374f1f5d6c3a1de9ba7c1712fbc7a2e176",
"md5": "4d577654b74367afabc2ff5643bc606d",
"sha256": "267953eb84808a70bba631a5ca1d516cfbb732d6e03ffbf7fa979c4f3ca69673"
},
"downloads": -1,
"filename": "array_split-0.6.5.zip",
"has_sig": false,
"md5_digest": "4d577654b74367afabc2ff5643bc606d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 41756,
"upload_time": "2024-08-04T05:42:04",
"upload_time_iso_8601": "2024-08-04T05:42:04.713056Z",
"url": "https://files.pythonhosted.org/packages/47/b4/f88e32eeee2ee6172244f8d19b374f1f5d6c3a1de9ba7c1712fbc7a2e176/array_split-0.6.5.zip",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-04 05:42:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "array-split",
"github_project": "array_split",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "array-split"
}
