| Name | polidoro-pipeline JSON |
| Version |
0.1
JSON |
| download |
| home_page | None |
| Summary | A powerful Python library for parallel data processing through a sequence of steps. |
| upload_time | 2025-08-01 19:28:11 |
| maintainer | None |
| docs_url | None |
| author | Heitor Luis Polidoro |
| requires_python | >=3.9 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Polidoro Pipeline
A powerful Python library for parallel data processing through a sequence of steps.
[](https://github.com/heitorpolidoro/polidoro-pipeline/actions/workflows/code_quality.yml)
[](https://github.com/heitorpolidoro/polidoro-pipeline/actions/workflows/pypi-publish.yml)
<br>
[](https://github.com/heitorpolidoro/polidoro-pipeline/releases/latest)
<br>
[](https://github.com/heitorpolidoro/polidoro-pipeline/issues)
[](https://github.com/heitorpolidoro/polidoro-pipeline/pulls)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
<br>
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)
</br>
[](https://app.deepsource.com/gh/heitorpolidoro/polidoro-pipeline/)
</br>
| Python Versions |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| &logo=python&label=3.10)<br/>&logo=python&label=3.11)<br/>&logo=python&label=3.12)<br/>&logo=python&label=3.13) |
## 🚀 Overview
Polidoro Pipeline is a Python library that simplifies parallel data processing through a sequence of steps. It automatically handles parallelization using Python's ThreadPoolExecutor, making it easy to process both single values and lists of values efficiently.
## ✨ Features
- 🔄 Process data through a sequence of steps
- ⚡ Automatic parallelization of processing
- 🧩 Simple and intuitive API
- 🔌 Easy to integrate with existing code
- 📦 Handles both single values and lists of values
## 📋 Installation
```bash
pip install polidoro_pipeline
```
## 🔧 Usage
### Basic Example
```python
from ppipeline import Pipeline
# Define processing steps
def add_1(x):
return x + 1
def multiply_by_2(x):
return x * 2
# Create a pipeline with steps
pipeline = Pipeline([add_1, multiply_by_2])
# Process a single value
result = list(pipeline.run(1)) # [4]
# Process multiple values in parallel
results = list(pipeline.run([1, 2, 3])) # [4, 6, 8]
```
### Adding Steps Incrementally
```python
pipeline = Pipeline()
pipeline.add_step(add_1)
pipeline.add_step(multiply_by_2)
result = list(pipeline.run(2)) # [6]
```
### Controlling Thread Count
```python
# Limit the number of worker threads
pipeline = Pipeline([add_1, multiply_by_2], thread_count=4)
results = list(pipeline.run([1, 2, 3, 4, 5]))
```
## 🧠 How It Works
1. The Pipeline class takes a list of callable functions as steps
2. When you call `run()` with input data, each item is processed through all steps in sequence
3. If the input is a list, items are processed in parallel using ThreadPoolExecutor
4. Each step can return a single value or a list of values
5. If a step returns multiple values (as a list), each value is processed independently in subsequent steps
## 🤝 Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## 📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "polidoro-pipeline",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Heitor Luis Polidoro",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/75/94/2df5db78f37cb06db00081ce2216a4d0ca82363f98bd44fc42626538c277/polidoro_pipeline-0.1.tar.gz",
"platform": null,
"description": "# Polidoro Pipeline\n\nA powerful Python library for parallel data processing through a sequence of steps.\n\n[](https://github.com/heitorpolidoro/polidoro-pipeline/actions/workflows/code_quality.yml)\n[](https://github.com/heitorpolidoro/polidoro-pipeline/actions/workflows/pypi-publish.yml)\n<br>\n[](https://github.com/heitorpolidoro/polidoro-pipeline/releases/latest)\n\n\n\n<br>\n[](https://github.com/heitorpolidoro/polidoro-pipeline/issues)\n[](https://github.com/heitorpolidoro/polidoro-pipeline/pulls)\n\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n<br>\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n[](https://sonarcloud.io/summary/new_code?id=heitorpolidoro_polidoro-pipeline)\n</br>\n[](https://app.deepsource.com/gh/heitorpolidoro/polidoro-pipeline/)\n</br>\n\n\n\n| Python Versions |\n|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| &logo=python&label=3.10)<br/>&logo=python&label=3.11)<br/>&logo=python&label=3.12)<br/>&logo=python&label=3.13) |\n\n## \ud83d\ude80 Overview\n\nPolidoro Pipeline is a Python library that simplifies parallel data processing through a sequence of steps. It automatically handles parallelization using Python's ThreadPoolExecutor, making it easy to process both single values and lists of values efficiently.\n\n## \u2728 Features\n\n- \ud83d\udd04 Process data through a sequence of steps\n- \u26a1 Automatic parallelization of processing\n- \ud83e\udde9 Simple and intuitive API\n- \ud83d\udd0c Easy to integrate with existing code\n- \ud83d\udce6 Handles both single values and lists of values\n\n## \ud83d\udccb Installation\n\n```bash\npip install polidoro_pipeline\n```\n\n## \ud83d\udd27 Usage\n\n### Basic Example\n\n```python\nfrom ppipeline import Pipeline\n\n# Define processing steps\ndef add_1(x):\n return x + 1\n\ndef multiply_by_2(x):\n return x * 2\n\n# Create a pipeline with steps\npipeline = Pipeline([add_1, multiply_by_2])\n\n# Process a single value\nresult = list(pipeline.run(1)) # [4]\n\n# Process multiple values in parallel\nresults = list(pipeline.run([1, 2, 3])) # [4, 6, 8]\n```\n\n### Adding Steps Incrementally\n\n```python\npipeline = Pipeline()\npipeline.add_step(add_1)\npipeline.add_step(multiply_by_2)\nresult = list(pipeline.run(2)) # [6]\n```\n\n### Controlling Thread Count\n\n```python\n# Limit the number of worker threads\npipeline = Pipeline([add_1, multiply_by_2], thread_count=4)\nresults = list(pipeline.run([1, 2, 3, 4, 5]))\n```\n\n## \ud83e\udde0 How It Works\n\n1. The Pipeline class takes a list of callable functions as steps\n2. When you call `run()` with input data, each item is processed through all steps in sequence\n3. If the input is a list, items are processed in parallel using ThreadPoolExecutor\n4. Each step can return a single value or a list of values\n5. If a step returns multiple values (as a list), each value is processed independently in subsequent steps\n\n## \ud83e\udd1d Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## \ud83d\udcc4 License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A powerful Python library for parallel data processing through a sequence of steps.",
"version": "0.1",
"project_urls": {
"Homepage": "https://github.com/heitorpolidoro/polidoro_pipeline"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "2e6ad9800f34dc4caf18494caf78d97bcdb67908a93a527b7b236362052fb08d",
"md5": "7f3df9ad269c1f28ea28cc0c29ee0c23",
"sha256": "bda4d17adafc427786e0249ed4ccb121c88c5165f4021ade123ad18b1edf90eb"
},
"downloads": -1,
"filename": "polidoro_pipeline-0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7f3df9ad269c1f28ea28cc0c29ee0c23",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 6639,
"upload_time": "2025-08-01T19:28:10",
"upload_time_iso_8601": "2025-08-01T19:28:10.699638Z",
"url": "https://files.pythonhosted.org/packages/2e/6a/d9800f34dc4caf18494caf78d97bcdb67908a93a527b7b236362052fb08d/polidoro_pipeline-0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "75942df5db78f37cb06db00081ce2216a4d0ca82363f98bd44fc42626538c277",
"md5": "8cadf3719ceffd138bd97594bb01de99",
"sha256": "01bd90ad0929ebd5a0ad538f370c689c0dbe0b125e959a98331b67e5277ef77a"
},
"downloads": -1,
"filename": "polidoro_pipeline-0.1.tar.gz",
"has_sig": false,
"md5_digest": "8cadf3719ceffd138bd97594bb01de99",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 7233,
"upload_time": "2025-08-01T19:28:11",
"upload_time_iso_8601": "2025-08-01T19:28:11.830461Z",
"url": "https://files.pythonhosted.org/packages/75/94/2df5db78f37cb06db00081ce2216a4d0ca82363f98bd44fc42626538c277/polidoro_pipeline-0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-01 19:28:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "heitorpolidoro",
"github_project": "polidoro_pipeline",
"github_not_found": true,
"lcname": "polidoro-pipeline"
}
