# Rivusio
[](https://codecov.io/gh/zbytealchemy/rivusio)
[](https://badge.fury.io/py/rivusio)
[](https://rivusio.readthedocs.io/en/latest/?badge=latest)
[](https://www.python.org)
[](https://github.com/zbytealchemy/rivusio/blob/main/LICENSE)
[](https://github.com/zbytealchemy/rivusio/actions/workflows/test.yml)
[](https://pepy.tech/project/rivusio)
[](https://github.com/astral-sh/ruff)
[](http://mypy-lang.org/)
[](https://github.com/PyCQA/bandit)
A blazing-fast, type-safe data processing pipeline framework in Python, designed for seamless integration of both synchronous and asynchronous operations, complete with robust parallel execution for ultimate performance and scalability.
## 🚀 Key Features
- **Type Safety**: Comprehensive type hints and runtime validation
- **Flexible Processing**: Support for both sync and async operations
- **High Performance**: Parallel execution strategies and optimized processing
- **Stream Processing**: Built-in support for various windowing strategies
- **Robust Error Handling**: Configurable retries and error recovery
- **Monitoring**: Built-in metrics collection and monitoring
- **Configuration**: Type-safe configuration using Pydantic
- **Extensible**: Plugin system for custom components
- **Composable**: Easy pipeline composition with the `>>` operator
- **Comprehensive Documentation**: Extensive guides and examples
## 🛠️ Installation
```bash
pip install rivusio
```
## 🎯 Quick Start
```python
from typing import Dict, List
from rivusio import AsyncBasePipe, SyncBasePipe, AsyncPipeline
# Define a simple transformation pipe
class NumberFilterPipe(AsyncBasePipe[Dict[str, int], List[int]]):
async def __call__(self, data: Dict[str, int]) -> List[int]:
return [v for v in data.values() if v > 100]
# Create and use a pipeline
async def main():
pipeline = AsyncPipeline([NumberFilterPipe()])
# Process data
data = {"a": 150, "b": 50, "c": 200}
result = await pipeline(data) # [150, 200]
print(result)
if __name__ == "__main__":
import asyncio
asyncio.run(main())
```
## 📚 Documentation
Visit our [comprehensive documentation](https://rivusio.readthedocs.io) for:
- Detailed guides and tutorials
- API reference
- Best practices
- Advanced examples
- Configuration options
## 🔧 Development Setup
```bash
# Clone the repository
git clone https://github.com/zbytealchemy/rivusio.git
cd rivusio
# Install dependencies
poetry install
# Run tests
poetry run pytest
# Build documentation
poetry install --with docs
make docs
```
## 🤝 Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details on:
- Code of Conduct
- Development process
- Pull request guidelines
- Testing requirements
## 📋 Requirements
- Python 3.10+
- Poetry for dependency management
- Git for version control
## 📈 Performance
Rivusio is designed for high performance:
- Efficient batch processing
- Parallel execution capabilities
- Optimized stream processing
- Minimal overhead
## 🔒 Security
- Regular security audits
- Type-safe operations
- Input validation
- Comprehensive testing
## 📄 License
MIT License - see [LICENSE](LICENSE) file for details.
## 🙏 Acknowledgments
Special thanks to all [contributors](https://github.com/zbytealchemy/rivusio/graphs/contributors) who have helped make Rivusio better.
---
Built with ❤️ by [ZbyteAlchemy](https://github.com/zbytealchemy)
Raw data
{
"_id": null,
"home_page": "https://github.com/zbytealchemy/rivusio",
"name": "rivusio",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "pipeline, data processing, async, sync, type-safe, streaming, etl, data engineering, fast, parallel execution",
"author": "Zbigniew Mastylo",
"author_email": "zmastylo@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c6/76/59884c1572d6e49f950810fe3c111661635c1de8f18a6647915198c11a3a/rivusio-0.2.0.tar.gz",
"platform": null,
"description": "# Rivusio\n\n[](https://codecov.io/gh/zbytealchemy/rivusio)\n[](https://badge.fury.io/py/rivusio)\n[](https://rivusio.readthedocs.io/en/latest/?badge=latest)\n[](https://www.python.org)\n[](https://github.com/zbytealchemy/rivusio/blob/main/LICENSE)\n[](https://github.com/zbytealchemy/rivusio/actions/workflows/test.yml)\n[](https://pepy.tech/project/rivusio)\n[](https://github.com/astral-sh/ruff)\n[](http://mypy-lang.org/)\n[](https://github.com/PyCQA/bandit)\n\nA blazing-fast, type-safe data processing pipeline framework in Python, designed for seamless integration of both synchronous and asynchronous operations, complete with robust parallel execution for ultimate performance and scalability.\n\n## \ud83d\ude80 Key Features\n\n- **Type Safety**: Comprehensive type hints and runtime validation\n- **Flexible Processing**: Support for both sync and async operations\n- **High Performance**: Parallel execution strategies and optimized processing\n- **Stream Processing**: Built-in support for various windowing strategies\n- **Robust Error Handling**: Configurable retries and error recovery\n- **Monitoring**: Built-in metrics collection and monitoring\n- **Configuration**: Type-safe configuration using Pydantic\n- **Extensible**: Plugin system for custom components\n- **Composable**: Easy pipeline composition with the `>>` operator\n- **Comprehensive Documentation**: Extensive guides and examples\n\n## \ud83d\udee0\ufe0f Installation\n\n```bash\npip install rivusio\n```\n\n## \ud83c\udfaf Quick Start\n\n```python\nfrom typing import Dict, List\nfrom rivusio import AsyncBasePipe, SyncBasePipe, AsyncPipeline\n\n# Define a simple transformation pipe\nclass NumberFilterPipe(AsyncBasePipe[Dict[str, int], List[int]]):\n async def __call__(self, data: Dict[str, int]) -> List[int]:\n return [v for v in data.values() if v > 100]\n\n# Create and use a pipeline\nasync def main():\n pipeline = AsyncPipeline([NumberFilterPipe()])\n \n # Process data\n data = {\"a\": 150, \"b\": 50, \"c\": 200}\n result = await pipeline(data) # [150, 200]\n print(result)\n\nif __name__ == \"__main__\":\n import asyncio\n asyncio.run(main())\n```\n\n## \ud83d\udcda Documentation\n\nVisit our [comprehensive documentation](https://rivusio.readthedocs.io) for:\n- Detailed guides and tutorials\n- API reference\n- Best practices\n- Advanced examples\n- Configuration options\n\n## \ud83d\udd27 Development Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/zbytealchemy/rivusio.git\ncd rivusio\n\n# Install dependencies\npoetry install\n\n# Run tests\npoetry run pytest\n\n# Build documentation\npoetry install --with docs\nmake docs\n```\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details on:\n- Code of Conduct\n- Development process\n- Pull request guidelines\n- Testing requirements\n\n## \ud83d\udccb Requirements\n\n- Python 3.10+\n- Poetry for dependency management\n- Git for version control\n\n## \ud83d\udcc8 Performance\n\nRivusio is designed for high performance:\n- Efficient batch processing\n- Parallel execution capabilities\n- Optimized stream processing\n- Minimal overhead\n\n## \ud83d\udd12 Security\n\n- Regular security audits\n- Type-safe operations\n- Input validation\n- Comprehensive testing\n\n## \ud83d\udcc4 License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n## \ud83d\ude4f Acknowledgments\n\nSpecial thanks to all [contributors](https://github.com/zbytealchemy/rivusio/graphs/contributors) who have helped make Rivusio better.\n\n---\n\nBuilt with \u2764\ufe0f by [ZbyteAlchemy](https://github.com/zbytealchemy)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A type-safe, async-first data processing pipeline framework",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://zbytealchemy.github.io/rivusio",
"Homepage": "https://github.com/zbytealchemy/rivusio",
"Repository": "https://github.com/zbytealchemy/rivusio"
},
"split_keywords": [
"pipeline",
" data processing",
" async",
" sync",
" type-safe",
" streaming",
" etl",
" data engineering",
" fast",
" parallel execution"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "80f99171ee6e1d8b753e2428c1d1a1728a786c0751165250994f69dcede5e0cd",
"md5": "8d2e7966c8e6093e3f3a03b22bc8f013",
"sha256": "4d4cd89da07713c2148d2d5db246b0d47d50c7adc245fd470975903d341e41c8"
},
"downloads": -1,
"filename": "rivusio-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8d2e7966c8e6093e3f3a03b22bc8f013",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 31954,
"upload_time": "2025-02-02T06:57:22",
"upload_time_iso_8601": "2025-02-02T06:57:22.089859Z",
"url": "https://files.pythonhosted.org/packages/80/f9/9171ee6e1d8b753e2428c1d1a1728a786c0751165250994f69dcede5e0cd/rivusio-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c67659884c1572d6e49f950810fe3c111661635c1de8f18a6647915198c11a3a",
"md5": "1f3aa9779866262e6f0d08d202e19252",
"sha256": "906135547b55fb42658a0104716d624b08c7b6d99cffadb160d785f48ccf56cb"
},
"downloads": -1,
"filename": "rivusio-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "1f3aa9779866262e6f0d08d202e19252",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 25654,
"upload_time": "2025-02-02T06:57:24",
"upload_time_iso_8601": "2025-02-02T06:57:24.619611Z",
"url": "https://files.pythonhosted.org/packages/c6/76/59884c1572d6e49f950810fe3c111661635c1de8f18a6647915198c11a3a/rivusio-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-02 06:57:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "zbytealchemy",
"github_project": "rivusio",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "rivusio"
}