icflow

Name	icflow JSON
Version	0.0.2 JSON
	download
home_page	None
Summary	A collection of simple utilities for machine learning workflows.
upload_time	2024-05-13 16:33:22
maintainer	None
docs_url	None
author	None
requires_python	>=3.8
license	None
keywords	machine learning workflow hpc
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            `icflow` is a Python package with some prototype 'workflow' tools for use in ICHEC.

# Goals
The goal is to use it to help convert tools and approaches used by ML 'domain experts' into a more standardized workflow by:

* Analysing incoming Juyter notebooks, models, datasets and runtime environments (conda/container)
* Identifying 'hard-coded' data that can be moved into config files
* Adding utility scripts and methods for fetching data and models as needed
* Describing a study as a workflow, using some scripts here to 'stitch' the workflow together

Ultimately we will end up using some common workflow tools across ICHEC, likely something established and open-source (eg MLFlow) - this package is intended to understand and flesh-out our workflow needs and start transforming how we set up studies to ultimately move to these more standard tools.

# Tests

In a Python virtual environment do:

```sh
pip install .'[test]'
```

## Unit Tests

```sh
pytest
```

## Linting and Static Analysis

```sh
black src test
mypy src test
```

## All Tests

Requires `tox`:

```sh
tox
```


            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "icflow",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "Machine Learning, Workflow, HPC",
    "author": null,
    "author_email": "\"James Grogan, Irish Centre for High End Computing\" <james.grogan@ichec.ie>",
    "download_url": "https://files.pythonhosted.org/packages/f6/71/01dd8cba7c78fc7a0805e50b637400c4b0c85c4c9110372da6c9f7b26258/icflow-0.0.2.tar.gz",
    "platform": null,
    "description": "`icflow` is a Python package with some prototype 'workflow' tools for use in ICHEC.\n\n# Goals\nThe goal is to use it to help convert tools and approaches used by ML 'domain experts' into a more standardized workflow by:\n\n* Analysing incoming Juyter notebooks, models, datasets and runtime environments (conda/container)\n* Identifying 'hard-coded' data that can be moved into config files\n* Adding utility scripts and methods for fetching data and models as needed\n* Describing a study as a workflow, using some scripts here to 'stitch' the workflow together\n\nUltimately we will end up using some common workflow tools across ICHEC, likely something established and open-source (eg MLFlow) - this package is intended to understand and flesh-out our workflow needs and start transforming how we set up studies to ultimately move to these more standard tools.\n\n# Tests\n\nIn a Python virtual environment do:\n\n```sh\npip install .'[test]'\n```\n\n## Unit Tests\n\n```sh\npytest\n```\n\n## Linting and Static Analysis\n\n```sh\nblack src test\nmypy src test\n```\n\n## All Tests\n\nRequires `tox`:\n\n```sh\ntox\n```\n\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "A collection of simple utilities for machine learning workflows.",
    "version": "0.0.2",
    "project_urls": {
        "Homepage": "https://git.ichec.ie/performance/toolshed/icflow",
        "Repository": "https://git.ichec.ie/performance/toolshed/icflow"
    },
    "split_keywords": [
        "machine learning",
        " workflow",
        " hpc"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "63b860e519c307604c5d00ae56d3d314b74ab6e31d54586cda649b311c239d96",
                "md5": "508bec5ab2f3b9badd3b71e53398aed1",
                "sha256": "2f569534847a72226ddac9488487494f49c98c4acb8f295c4f7abeba190d0189"
            },
            "downloads": -1,
            "filename": "icflow-0.0.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "508bec5ab2f3b9badd3b71e53398aed1",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 33300,
            "upload_time": "2024-05-13T16:33:21",
            "upload_time_iso_8601": "2024-05-13T16:33:21.635587Z",
            "url": "https://files.pythonhosted.org/packages/63/b8/60e519c307604c5d00ae56d3d314b74ab6e31d54586cda649b311c239d96/icflow-0.0.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "f67101dd8cba7c78fc7a0805e50b637400c4b0c85c4c9110372da6c9f7b26258",
                "md5": "196bcda57458f5255b3128cfb889923d",
                "sha256": "227de47289860988b533357f0ca390514278c807a6dc6d9a9d3aca57b2f84859"
            },
            "downloads": -1,
            "filename": "icflow-0.0.2.tar.gz",
            "has_sig": false,
            "md5_digest": "196bcda57458f5255b3128cfb889923d",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 27903,
            "upload_time": "2024-05-13T16:33:22",
            "upload_time_iso_8601": "2024-05-13T16:33:22.883997Z",
            "url": "https://files.pythonhosted.org/packages/f6/71/01dd8cba7c78fc7a0805e50b637400c4b0c85c4c9110372da6c9f7b26258/icflow-0.0.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-05-13 16:33:22",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "icflow"
}