| Name | icflow JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | None |
| Summary | A collection of simple utilities for machine learning workflows. |
| upload_time | 2024-10-08 16:25:24 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | None |
| keywords |
machine learning
workflow
hpc
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
`icflow` is a Python package with some prototype 'workflow' tools for use in ICHEC.
It is intended to help standardize some of our workflows in areas like Machine Learning by using common utilities, data-formats and data-structures.
See the project documentation on [ReadTheDocs](https://icflow.readthedocs.io/en/latest/).
# Installation #
It is available on PyPI:
``` shell
pip install icflow
```
# Features #
`icflow` is mostly intended to be used as a library for building workflow tooling - however it has a few CLI applications.
## Parameter Sweep ##
You can define a parameter sweep in a `yaml` file as follows:
``` yaml
title: "my_parameter_sweep"
program: "launch_program"
parameters:
param_0 : 1
param_1 : "my_value"
param_2 : [1, 2]
param_3 : ["a", "b"]
```
Running:
``` shell
icflow sweep --config my_config.yaml
```
with this fill will launch the program or script defined by `program`, which should be in the system `PATH`.
The listed parameters are passed as command line arguments to the `program` in the form `--key value`. Parameter value lists are expanded such that there is a program launch for each combination of values in the list. In the above example this will result in the following program launches:
``` shell
launch_program --param_0 1 --param_1 my_value --param_2 1 --param_3 a
launch_program --param_0 1 --param_1 my_value --param_2 2 --param_3 a
launch_program --param_0 1 --param_1 my_value --param_2 1 --param_3 b
launch_program --param_0 1 --param_1 my_value --param_2 2 --param_3 b
```
Program launching is handled internally by ICHEC's [ictasks](https://git.ichec.ie/performance/toolshed/ictasks) library, with each of these program launches handled as a 'task'.
# Copyright
This software is Copyright of the Irish Centre for High End Computing 2024. You can use it under the terms of the GPLv3+. See the included `LICENSE` file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "icflow",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Machine Learning, Workflow, HPC",
"author": null,
"author_email": "\"James Grogan, Irish Centre for High End Computing\" <james.grogan@ichec.ie>",
"download_url": "https://files.pythonhosted.org/packages/7f/43/1882fdb2d995c7fa93a1d2c759647193219ffcc77f0e39e21b46952356e6/icflow-0.1.0.tar.gz",
"platform": null,
"description": "`icflow` is a Python package with some prototype 'workflow' tools for use in ICHEC.\n\nIt is intended to help standardize some of our workflows in areas like Machine Learning by using common utilities, data-formats and data-structures. \n\nSee the project documentation on [ReadTheDocs](https://icflow.readthedocs.io/en/latest/).\n\n# Installation #\n\nIt is available on PyPI:\n\n``` shell\npip install icflow\n```\n\n# Features #\n\n`icflow` is mostly intended to be used as a library for building workflow tooling - however it has a few CLI applications.\n\n## Parameter Sweep ##\n\nYou can define a parameter sweep in a `yaml` file as follows:\n\n``` yaml\ntitle: \"my_parameter_sweep\"\nprogram: \"launch_program\" \n\nparameters:\n\tparam_0 : 1\n\tparam_1 : \"my_value\"\n\tparam_2 : [1, 2]\n\tparam_3 : [\"a\", \"b\"]\n```\n\nRunning:\n\n``` shell\nicflow sweep --config my_config.yaml\n```\n\nwith this fill will launch the program or script defined by `program`, which should be in the system `PATH`.\n\nThe listed parameters are passed as command line arguments to the `program` in the form `--key value`. Parameter value lists are expanded such that there is a program launch for each combination of values in the list. In the above example this will result in the following program launches:\n\n``` shell\nlaunch_program --param_0 1 --param_1 my_value --param_2 1 --param_3 a\nlaunch_program --param_0 1 --param_1 my_value --param_2 2 --param_3 a\nlaunch_program --param_0 1 --param_1 my_value --param_2 1 --param_3 b\nlaunch_program --param_0 1 --param_1 my_value --param_2 2 --param_3 b\n```\n\nProgram launching is handled internally by ICHEC's [ictasks](https://git.ichec.ie/performance/toolshed/ictasks) library, with each of these program launches handled as a 'task'.\n\n# Copyright\n\nThis software is Copyright of the Irish Centre for High End Computing 2024. You can use it under the terms of the GPLv3+. See the included `LICENSE` file for details.\n",
"bugtrack_url": null,
"license": null,
"summary": "A collection of simple utilities for machine learning workflows.",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://git.ichec.ie/performance/toolshed/icflow",
"Repository": "https://git.ichec.ie/performance/toolshed/icflow"
},
"split_keywords": [
"machine learning",
" workflow",
" hpc"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dd7c5bb30d648966351ffa75973150af6e2cba59f2a9b77a803957218246ff9f",
"md5": "211e2c6400d8d1fe25fa8948558c2b49",
"sha256": "1dcb69f76a66e686da6383fb1bcb0a06be13c12fb6df370d6f53865126199b63"
},
"downloads": -1,
"filename": "icflow-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "211e2c6400d8d1fe25fa8948558c2b49",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 26609,
"upload_time": "2024-10-08T16:25:23",
"upload_time_iso_8601": "2024-10-08T16:25:23.214226Z",
"url": "https://files.pythonhosted.org/packages/dd/7c/5bb30d648966351ffa75973150af6e2cba59f2a9b77a803957218246ff9f/icflow-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7f431882fdb2d995c7fa93a1d2c759647193219ffcc77f0e39e21b46952356e6",
"md5": "775dcb2d538228e5f9cbab455bacc3e9",
"sha256": "2a63cd2cfcf74f4a82a8b12bbfb663018ce24f684eccdf18eb5f490948532d25"
},
"downloads": -1,
"filename": "icflow-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "775dcb2d538228e5f9cbab455bacc3e9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 25110,
"upload_time": "2024-10-08T16:25:24",
"upload_time_iso_8601": "2024-10-08T16:25:24.975344Z",
"url": "https://files.pythonhosted.org/packages/7f/43/1882fdb2d995c7fa93a1d2c759647193219ffcc77f0e39e21b46952356e6/icflow-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-08 16:25:24",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "icflow"
}
