| Name | tpdp JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | |
| Summary | The Pipeline Design Pattern realization in python. |
| upload_time | 2023-05-31 18:32:09 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | |
| keywords |
pipeline
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# The Pipeline Design Pattern
**tpdp** it is the Pipeline Design Pattern realization in Python.
## Quickstart
Install with pip
```bash
pip install tpdp
```
and creating your first pipeline
```python
from tpdp import Pipeline, State, Step
class SimpleState(State):
pipeline_name: str
step_1_count: int = 0
step_2_count: int = 0
init_state = SimpleState(pipeline_name="simple_pipeline")
pipeline = Pipeline(name="simple_pipeline")
class Step1(Step):
def run(self, state: SimpleState, **kwargs) -> SimpleState:
state.step_1_count += 1
self.logger.info("step_1_count increased by 1")
return state
class Step2(Step):
def run(self, state: SimpleState, **kwargs) -> SimpleState:
state.step_2_count += 1
self.logger.info("step_2_count increased by 1")
return state
step_1 = Step1(step_name="step_1")
step_2 = Step2(step_name="step_2")
pipeline.registry_step(step_1)
pipeline.registry_step(step_2)
pipeline.registry_step(step_1)
pipeline.registry_step(step_2)
pipeline.registry_step(step_1)
pipeline.registry_step(step_2)
pipeline.registry_step(step_1)
result = pipeline.run(init_state=init_state)
final_state = pipeline.get_state()
print(f"final_state.step_1_count = {final_state.step_1_count}")
print(f"final_state.step_2_count = {final_state.step_2_count}")
```
You will see the following logs
```bash
[...: INFO] Pipeline: Step registered: step_name="step_1"
[...: INFO] Pipeline: Step registered: step_name="step_2"
[...: INFO] Pipeline: Step registered: step_name="step_1"
[...: INFO] Pipeline: Step registered: step_name="step_2"
[...: INFO] Pipeline: Step registered: step_name="step_1"
[...: INFO] Pipeline: Step registered: step_name="step_2"
[...: INFO] Pipeline: Step registered: step_name="step_1"
[...: INFO] Pipeline: Pipeline start: pipeline_name="simple_pipeline", start_at="..."
[...: INFO] Pipeline: Step start: step_name="step_1"
[...: INFO] Step1: step_1_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_1", step_duration="0.000024"
[...: INFO] Pipeline: Step start: step_name="step_2"
[...: INFO] Step2: step_2_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_2", step_duration="0.000020"
[...: INFO] Pipeline: Step start: step_name="step_1"
[...: INFO] Step1: step_1_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_1", step_duration="0.000016"
[...: INFO] Pipeline: Step start: step_name="step_2"
[...: INFO] Step2: step_2_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_2", step_duration="0.000016"
[...: INFO] Pipeline: Step start: step_name="step_1"
[...: INFO] Step1: step_1_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_1", step_duration="0.000016"
[...: INFO] Pipeline: Step start: step_name="step_2"
[...: INFO] Step2: step_2_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_2", step_duration="0.000016"
[...: INFO] Pipeline: Step start: step_name="step_1"
[...: INFO] Step1: step_1_count increased by 1
[...: INFO] Pipeline: Step finish: step_name="step_1", step_duration="0.000023"
[...: INFO] Pipeline: Pipeline finish: pipeline_name="simple_pipeline", finish_at="...", duration="0.0005033016204833984"
final_state.step_1_count = 4
final_state.step_2_count = 3
```
Raw data
{
"_id": null,
"home_page": "",
"name": "tpdp",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "pipeline",
"author": "",
"author_email": "\"Denis A. Artyushin\" <artyushinden@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/d4/08/840b4616243c500b9cd19e0194b078153797f4d8c847aebbc4c48a1430d1/tpdp-0.1.0.tar.gz",
"platform": null,
"description": "# The Pipeline Design Pattern\n\n**tpdp** it is the Pipeline Design Pattern realization in Python.\n\n## Quickstart\n\nInstall with pip\n\n```bash\npip install tpdp\n```\n\nand creating your first pipeline\n\n```python\nfrom tpdp import Pipeline, State, Step\n\nclass SimpleState(State):\n pipeline_name: str\n step_1_count: int = 0\n step_2_count: int = 0\n\ninit_state = SimpleState(pipeline_name=\"simple_pipeline\")\n\npipeline = Pipeline(name=\"simple_pipeline\")\n\nclass Step1(Step):\n def run(self, state: SimpleState, **kwargs) -> SimpleState:\n state.step_1_count += 1\n self.logger.info(\"step_1_count increased by 1\")\n return state\n\nclass Step2(Step):\n def run(self, state: SimpleState, **kwargs) -> SimpleState:\n state.step_2_count += 1\n self.logger.info(\"step_2_count increased by 1\")\n return state\n\nstep_1 = Step1(step_name=\"step_1\")\nstep_2 = Step2(step_name=\"step_2\")\n\npipeline.registry_step(step_1)\npipeline.registry_step(step_2)\npipeline.registry_step(step_1)\npipeline.registry_step(step_2)\npipeline.registry_step(step_1)\npipeline.registry_step(step_2)\npipeline.registry_step(step_1)\n\nresult = pipeline.run(init_state=init_state)\nfinal_state = pipeline.get_state()\n\nprint(f\"final_state.step_1_count = {final_state.step_1_count}\")\nprint(f\"final_state.step_2_count = {final_state.step_2_count}\")\n```\n\nYou will see the following logs\n\n```bash\n[...: INFO] Pipeline: Step registered: step_name=\"step_1\"\n[...: INFO] Pipeline: Step registered: step_name=\"step_2\"\n[...: INFO] Pipeline: Step registered: step_name=\"step_1\"\n[...: INFO] Pipeline: Step registered: step_name=\"step_2\"\n[...: INFO] Pipeline: Step registered: step_name=\"step_1\"\n[...: INFO] Pipeline: Step registered: step_name=\"step_2\"\n[...: INFO] Pipeline: Step registered: step_name=\"step_1\"\n[...: INFO] Pipeline: Pipeline start: pipeline_name=\"simple_pipeline\", start_at=\"...\"\n[...: INFO] Pipeline: Step start: step_name=\"step_1\"\n[...: INFO] Step1: step_1_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_1\", step_duration=\"0.000024\"\n[...: INFO] Pipeline: Step start: step_name=\"step_2\"\n[...: INFO] Step2: step_2_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_2\", step_duration=\"0.000020\"\n[...: INFO] Pipeline: Step start: step_name=\"step_1\"\n[...: INFO] Step1: step_1_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_1\", step_duration=\"0.000016\"\n[...: INFO] Pipeline: Step start: step_name=\"step_2\"\n[...: INFO] Step2: step_2_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_2\", step_duration=\"0.000016\"\n[...: INFO] Pipeline: Step start: step_name=\"step_1\"\n[...: INFO] Step1: step_1_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_1\", step_duration=\"0.000016\"\n[...: INFO] Pipeline: Step start: step_name=\"step_2\"\n[...: INFO] Step2: step_2_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_2\", step_duration=\"0.000016\"\n[...: INFO] Pipeline: Step start: step_name=\"step_1\"\n[...: INFO] Step1: step_1_count increased by 1\n[...: INFO] Pipeline: Step finish: step_name=\"step_1\", step_duration=\"0.000023\"\n[...: INFO] Pipeline: Pipeline finish: pipeline_name=\"simple_pipeline\", finish_at=\"...\", duration=\"0.0005033016204833984\"\nfinal_state.step_1_count = 4\nfinal_state.step_2_count = 3\n```",
"bugtrack_url": null,
"license": "",
"summary": "The Pipeline Design Pattern realization in python.",
"version": "0.1.0",
"project_urls": {
"Documentation": "https://denisart.github.io/tpdp",
"Homepage": "https://github.com/denisart/tpdp",
"Source": "https://github.com/denisart/tpdp"
},
"split_keywords": [
"pipeline"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a7d42f6736df50fb7349529e4f528671d95568224e340e6d3370ed27f327a6e1",
"md5": "a129f655f87f07cee10ba53698c99d87",
"sha256": "006e2b2bb5b6bc4493fae114f3bfa5196ad000cb3c31a0fe4cd9cb123c043c5d"
},
"downloads": -1,
"filename": "tpdp-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a129f655f87f07cee10ba53698c99d87",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 5815,
"upload_time": "2023-05-31T18:32:07",
"upload_time_iso_8601": "2023-05-31T18:32:07.825274Z",
"url": "https://files.pythonhosted.org/packages/a7/d4/2f6736df50fb7349529e4f528671d95568224e340e6d3370ed27f327a6e1/tpdp-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d408840b4616243c500b9cd19e0194b078153797f4d8c847aebbc4c48a1430d1",
"md5": "56035479365e528d3ef034d57e41f0c2",
"sha256": "b8c1c9299b0c1019d4d2742ae92363a96b6e69bec53008d074172e74163b1ecd"
},
"downloads": -1,
"filename": "tpdp-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "56035479365e528d3ef034d57e41f0c2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 9774,
"upload_time": "2023-05-31T18:32:09",
"upload_time_iso_8601": "2023-05-31T18:32:09.052624Z",
"url": "https://files.pythonhosted.org/packages/d4/08/840b4616243c500b9cd19e0194b078153797f4d8c847aebbc4c48a1430d1/tpdp-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-31 18:32:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "denisart",
"github_project": "tpdp",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tpdp"
}
