tinypipeline


Nametinypipeline JSON
Version 0.3.0 PyPI version JSON
download
home_page
SummaryA tiny mlops library for building machine learning pipelines on your local machine.
upload_time2023-03-21 03:14:48
maintainer
docs_urlNone
author
requires_python>=3.9
licenseMIT
keywords mlops machine learning pipelines
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # tinypipeline

## Overview

`tinypipeline` is a tiny mlops library that provides a simple framework for organizing your machine learning pipeline code into a series of steps. It does not handle networking, I/O, or compute resources. You do the rest in your pipeline steps.

## Installation

```
$ pip install tinypipeline
```

## Usage

`tinypipeline` exposes two main objects:
- `pipeline`: a decorator for defining your pipeline. Returns a `Pipeline` instance.
- `step`: a decorator that is used to define individual pipeline steps. Returns a `Step` instance.

Each object requires you provide a `name`, `version`, and `description` to explicitly define what pipeline you're creating.

The `Pipeline` object that is returned from the decorator has a single method: `run()`.

## API

If you'd like to use this package, you can follow the `example.py` below:

```python
from tinypipeline import pipeline, step

# define all of the steps
@step(name='step_one', version='0.0.1', description='first step')
def step_one():
    print("Step function one")

@step(name='step_two', version='0.0.1', description='second step')
def step_two():
    print("Step function two")

@step(name='step_three', version='0.0.1', description='third step')
def step_three():
    print("Step function three")

@step(name='step_four', version='0.0.1', description='fourth step')
def step_four():
    print("Step function four")


# define the pipeline
@pipeline(
    name='test-pipeline', 
    version='0.0.1', 
    description='a test tinypipeline',
)
def pipe():
    # run the steps in the defined order
    return [
        step_one, 
        step_two, 
        step_three, 
        step_four,
    ]

pipe = pipe()
pipe.run()
```

You can also define steps using a dictionary, where each key of the dictionary
is a step to run, and the values are steps that run after the step named in the key.

```python
# define the pipeline
@pipeline(
    name='test-pipeline', 
    version='0.0.1', 
    description='a test tinypipeline',
)
def pipe():
    # run the steps in the defined order of the graph
    return {
        step_one: [step_two, step_four],
        step_two: [step_three, step_four]
    }
```


**Output**:

You can run the `example.py` like so:

```console
$ python example.py
+-------------------------------------------------------------------+
| Running pipeline: Pipeline(name='test-pipeline', version='0.0.1') |
+-------------------------------------------------------------------+

Running step [step_one]...
Step function one
Step [step_one] completed in 0.000325 seconds

Running step [step_two]...
Step function two
Step [step_two] completed in 0.000286 seconds

Running step [step_three]...
Step function three
Step [step_three] completed in 0.000251 seconds

Running step [step_four]...
Step function four
Step [step_four] completed in 0.000313 seconds
```

## Running tests

Tests are run using pytest. To run the tests you can do:

```console
$ pip install pytest
$ pytest
```

            

Raw data

            {
    "_id": null,
    "home_page": "",
    "name": "tinypipeline",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.9",
    "maintainer_email": "",
    "keywords": "mlops,machine learning,pipelines",
    "author": "",
    "author_email": "Freddie Vargus <fjv41995@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/3b/bc/18e476df1af97ea7b8fff22c59826d9b27f1e64a20c739f7ce7658e4e45f/tinypipeline-0.3.0.tar.gz",
    "platform": null,
    "description": "# tinypipeline\n\n## Overview\n\n`tinypipeline` is a tiny mlops library that provides a simple framework for organizing your machine learning pipeline code into a series of steps. It does not handle networking, I/O, or compute resources. You do the rest in your pipeline steps.\n\n## Installation\n\n```\n$ pip install tinypipeline\n```\n\n## Usage\n\n`tinypipeline` exposes two main objects:\n- `pipeline`: a decorator for defining your pipeline. Returns a `Pipeline` instance.\n- `step`: a decorator that is used to define individual pipeline steps. Returns a `Step` instance.\n\nEach object requires you provide a `name`, `version`, and `description` to explicitly define what pipeline you're creating.\n\nThe `Pipeline` object that is returned from the decorator has a single method: `run()`.\n\n## API\n\nIf you'd like to use this package, you can follow the `example.py` below:\n\n```python\nfrom tinypipeline import pipeline, step\n\n# define all of the steps\n@step(name='step_one', version='0.0.1', description='first step')\ndef step_one():\n    print(\"Step function one\")\n\n@step(name='step_two', version='0.0.1', description='second step')\ndef step_two():\n    print(\"Step function two\")\n\n@step(name='step_three', version='0.0.1', description='third step')\ndef step_three():\n    print(\"Step function three\")\n\n@step(name='step_four', version='0.0.1', description='fourth step')\ndef step_four():\n    print(\"Step function four\")\n\n\n# define the pipeline\n@pipeline(\n    name='test-pipeline', \n    version='0.0.1', \n    description='a test tinypipeline',\n)\ndef pipe():\n    # run the steps in the defined order\n    return [\n        step_one, \n        step_two, \n        step_three, \n        step_four,\n    ]\n\npipe = pipe()\npipe.run()\n```\n\nYou can also define steps using a dictionary, where each key of the dictionary\nis a step to run, and the values are steps that run after the step named in the key.\n\n```python\n# define the pipeline\n@pipeline(\n    name='test-pipeline', \n    version='0.0.1', \n    description='a test tinypipeline',\n)\ndef pipe():\n    # run the steps in the defined order of the graph\n    return {\n        step_one: [step_two, step_four],\n        step_two: [step_three, step_four]\n    }\n```\n\n\n**Output**:\n\nYou can run the `example.py` like so:\n\n```console\n$ python example.py\n+-------------------------------------------------------------------+\n| Running pipeline: Pipeline(name='test-pipeline', version='0.0.1') |\n+-------------------------------------------------------------------+\n\nRunning step [step_one]...\nStep function one\nStep [step_one] completed in 0.000325 seconds\n\nRunning step [step_two]...\nStep function two\nStep [step_two] completed in 0.000286 seconds\n\nRunning step [step_three]...\nStep function three\nStep [step_three] completed in 0.000251 seconds\n\nRunning step [step_four]...\nStep function four\nStep [step_four] completed in 0.000313 seconds\n```\n\n## Running tests\n\nTests are run using pytest. To run the tests you can do:\n\n```console\n$ pip install pytest\n$ pytest\n```\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "A tiny mlops library for building machine learning pipelines on your local machine.",
    "version": "0.3.0",
    "split_keywords": [
        "mlops",
        "machine learning",
        "pipelines"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "ad3125e9d8d142295e5a2500a4e34b5054dfffbff6a55191bfd860ea1016081d",
                "md5": "9992befbbc76701339c677cf5c1bc8fd",
                "sha256": "9cae51b77a50f4f6705fedfe97232f964bf660b2955aad422f3027dcb1025b78"
            },
            "downloads": -1,
            "filename": "tinypipeline-0.3.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "9992befbbc76701339c677cf5c1bc8fd",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9",
            "size": 5789,
            "upload_time": "2023-03-21T03:14:47",
            "upload_time_iso_8601": "2023-03-21T03:14:47.175668Z",
            "url": "https://files.pythonhosted.org/packages/ad/31/25e9d8d142295e5a2500a4e34b5054dfffbff6a55191bfd860ea1016081d/tinypipeline-0.3.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "3bbc18e476df1af97ea7b8fff22c59826d9b27f1e64a20c739f7ce7658e4e45f",
                "md5": "f190e36541dee74ea4ff182a5a52510c",
                "sha256": "cec9f38f6afeef9b2708649c40b60165eb2df62cf85db9885b023dd7c154b144"
            },
            "downloads": -1,
            "filename": "tinypipeline-0.3.0.tar.gz",
            "has_sig": false,
            "md5_digest": "f190e36541dee74ea4ff182a5a52510c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9",
            "size": 8455,
            "upload_time": "2023-03-21T03:14:48",
            "upload_time_iso_8601": "2023-03-21T03:14:48.676239Z",
            "url": "https://files.pythonhosted.org/packages/3b/bc/18e476df1af97ea7b8fff22c59826d9b27f1e64a20c739f7ce7658e4e45f/tinypipeline-0.3.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-03-21 03:14:48",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "lcname": "tinypipeline"
}
        
Elapsed time: 0.04862s