| Name | cyclebane JSON |
| Version |
24.10.0
JSON |
| download |
| home_page | None |
| Summary | Transform directed acyclic graphs using map-reduce and groupby operations |
| upload_time | 2024-10-11 07:49:52 |
| maintainer | None |
| docs_url | None |
| author | Scipp contributors |
| requires_python | >=3.10 |
| license | BSD 3-Clause License Copyright (c) 2024, Scipp contributors (https://github.com/scipp) All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](CODE_OF_CONDUCT.md)
[](https://pypi.python.org/pypi/cyclebane)
[](https://anaconda.org/scipp/cyclebane)
[](LICENSE)
# Cyclebane
## About
Transform directed acyclic graphs using map-reduce and groupby operations
This library is an attempt to merge the concepts of directed acyclic graphs (DAG) with array-like objects such as NumPy arrays, Pandas DataFrames, or Xarray/Scipp DataArrays.
This could be useful for describing tasks graphs, e.g., when a series of tasks is applied to chunks of an array.
These tasks also have an array structure.
After an reduction operation of chunks, the graph loses this structure, i.e., only a subset of the graph's nodes has array structure.
What if we could work with this structure, even though only parts of the graph follows it?
And what if we could use the power of array slicing with named dimensions, or select by label?
This is what Cyclebane tries to do.
Our initial goal is to support:
- `map` operations of a DAG's source nodes over an array-like (https://docs.dask.org/en/latest/high-level-graphs.html).
Cyclebane will effectively copy all descendants of those nodes, once for each array element.
Cyclebane will support joint mappings of multiple source nodes by mapping over, e.g., a DataFrame with multiple columns, as well as chaining independent map operations at different source nodes.
In the latter case this will effectively broadcast at descendant nodes that depend on multiple such source nodes.
- `reduce` operations at descendants of mapped nodes.
This will add a new node with edges to all copies of the mapped node being reduced.
Cyclebane will support reducing only individual axes or all axes, similar to Numpy.
- `groupby` operations similar to Pandas and Xarray (albeit more limited).
- Positional and label-based indexing.
Cyclebane will support selecting branches that were creating during `map` (or `groupby`) operations based on their indices.
The graph structure will be left untouched, i.e., nodes after a `reduce` operation will be preserved, but fewer edges will lead to the reduce node.
See also Dask's [High Level Graphs](https://docs.dask.org/en/latest/high-level-graphs.html) for a related concept (without the direct support for any such operations).
## Installation
```sh
python -m pip install cyclebane
```
Raw data
{
"_id": null,
"home_page": null,
"name": "cyclebane",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Scipp contributors",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/22/ed/0f1c567a755b9db849e9bcef78983c4a3dcf23c846e4aa5709ff096f363e/cyclebane-24.10.0.tar.gz",
"platform": null,
"description": "[](CODE_OF_CONDUCT.md)\n[](https://pypi.python.org/pypi/cyclebane)\n[](https://anaconda.org/scipp/cyclebane)\n[](LICENSE)\n\n# Cyclebane\n\n## About\n\nTransform directed acyclic graphs using map-reduce and groupby operations\n\nThis library is an attempt to merge the concepts of directed acyclic graphs (DAG) with array-like objects such as NumPy arrays, Pandas DataFrames, or Xarray/Scipp DataArrays.\nThis could be useful for describing tasks graphs, e.g., when a series of tasks is applied to chunks of an array.\nThese tasks also have an array structure.\nAfter an reduction operation of chunks, the graph loses this structure, i.e., only a subset of the graph's nodes has array structure.\nWhat if we could work with this structure, even though only parts of the graph follows it?\nAnd what if we could use the power of array slicing with named dimensions, or select by label?\nThis is what Cyclebane tries to do.\n\nOur initial goal is to support:\n\n- `map` operations of a DAG's source nodes over an array-like (https://docs.dask.org/en/latest/high-level-graphs.html).\n Cyclebane will effectively copy all descendants of those nodes, once for each array element.\n Cyclebane will support joint mappings of multiple source nodes by mapping over, e.g., a DataFrame with multiple columns, as well as chaining independent map operations at different source nodes.\n In the latter case this will effectively broadcast at descendant nodes that depend on multiple such source nodes.\n- `reduce` operations at descendants of mapped nodes.\n This will add a new node with edges to all copies of the mapped node being reduced.\n Cyclebane will support reducing only individual axes or all axes, similar to Numpy.\n- `groupby` operations similar to Pandas and Xarray (albeit more limited).\n- Positional and label-based indexing.\n Cyclebane will support selecting branches that were creating during `map` (or `groupby`) operations based on their indices.\n The graph structure will be left untouched, i.e., nodes after a `reduce` operation will be preserved, but fewer edges will lead to the reduce node.\n\nSee also Dask's [High Level Graphs](https://docs.dask.org/en/latest/high-level-graphs.html) for a related concept (without the direct support for any such operations).\n\n## Installation\n\n```sh\npython -m pip install cyclebane\n```\n",
"bugtrack_url": null,
"license": "BSD 3-Clause License Copyright (c) 2024, Scipp contributors (https://github.com/scipp) All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ",
"summary": "Transform directed acyclic graphs using map-reduce and groupby operations",
"version": "24.10.0",
"project_urls": {
"Bug Tracker": "https://github.com/scipp/cyclebane/issues",
"Documentation": "https://scipp.github.io/cyclebane",
"Source": "https://github.com/scipp/cyclebane"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "42d927b13bc9419bf5dae02905b348f16ca827646cd76244ddd326f1a8139a6a",
"md5": "1efca101aa6f23d1194cf97312f544c9",
"sha256": "902dd318667e4a222afc270cc5bc72c67d5d6047d2e0e1c36018885fb80f5e5d"
},
"downloads": -1,
"filename": "cyclebane-24.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1efca101aa6f23d1194cf97312f544c9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 13645,
"upload_time": "2024-10-11T07:49:51",
"upload_time_iso_8601": "2024-10-11T07:49:51.074799Z",
"url": "https://files.pythonhosted.org/packages/42/d9/27b13bc9419bf5dae02905b348f16ca827646cd76244ddd326f1a8139a6a/cyclebane-24.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "22ed0f1c567a755b9db849e9bcef78983c4a3dcf23c846e4aa5709ff096f363e",
"md5": "bb14de8031ac3054525671c08493941b",
"sha256": "aa1aeb24f6915c8c55c85fcebdb3235544ab4e319a2a3c17aca634f6a0f0360b"
},
"downloads": -1,
"filename": "cyclebane-24.10.0.tar.gz",
"has_sig": false,
"md5_digest": "bb14de8031ac3054525671c08493941b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 42358,
"upload_time": "2024-10-11T07:49:52",
"upload_time_iso_8601": "2024-10-11T07:49:52.739694Z",
"url": "https://files.pythonhosted.org/packages/22/ed/0f1c567a755b9db849e9bcef78983c4a3dcf23c846e4aa5709ff096f363e/cyclebane-24.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-11 07:49:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "scipp",
"github_project": "cyclebane",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "cyclebane"
}
