rivusio


Namerivusio JSON
Version 0.2.0 PyPI version JSON
download
home_pagehttps://github.com/zbytealchemy/rivusio
SummaryA type-safe, async-first data processing pipeline framework
upload_time2025-02-02 06:57:24
maintainerNone
docs_urlNone
authorZbigniew Mastylo
requires_python<4.0,>=3.10
licenseMIT
keywords pipeline data processing async sync type-safe streaming etl data engineering fast parallel execution
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Rivusio

[![codecov](https://codecov.io/gh/zbytealchemy/rivusio/branch/main/graph/badge.svg)](https://codecov.io/gh/zbytealchemy/rivusio)
[![PyPI version](https://badge.fury.io/py/rivusio.svg)](https://badge.fury.io/py/rivusio)
[![Documentation Status](https://readthedocs.org/projects/rivusio/badge/?version=latest)](https://rivusio.readthedocs.io/en/latest/?badge=latest)
[![Python versions](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org)
[![License](https://img.shields.io/github/license/zbytealchemy/rivusio)](https://github.com/zbytealchemy/rivusio/blob/main/LICENSE)
[![Tests](https://github.com/zbytealchemy/rivusio/actions/workflows/test.yml/badge.svg)](https://github.com/zbytealchemy/rivusio/actions/workflows/test.yml)
[![Downloads](https://static.pepy.tech/badge/rivusio/month)](https://pepy.tech/project/rivusio)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Type Checking: mypy](https://img.shields.io/badge/type%20checking-mypy-blue)](http://mypy-lang.org/)
[![Security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit)

A blazing-fast, type-safe data processing pipeline framework in Python, designed for seamless integration of both synchronous and asynchronous operations, complete with robust parallel execution for ultimate performance and scalability.

## 🚀 Key Features

- **Type Safety**: Comprehensive type hints and runtime validation
- **Flexible Processing**: Support for both sync and async operations
- **High Performance**: Parallel execution strategies and optimized processing
- **Stream Processing**: Built-in support for various windowing strategies
- **Robust Error Handling**: Configurable retries and error recovery
- **Monitoring**: Built-in metrics collection and monitoring
- **Configuration**: Type-safe configuration using Pydantic
- **Extensible**: Plugin system for custom components
- **Composable**: Easy pipeline composition with the `>>` operator
- **Comprehensive Documentation**: Extensive guides and examples

## 🛠️ Installation

```bash
pip install rivusio
```

## 🎯 Quick Start

```python
from typing import Dict, List
from rivusio import AsyncBasePipe, SyncBasePipe, AsyncPipeline

# Define a simple transformation pipe
class NumberFilterPipe(AsyncBasePipe[Dict[str, int], List[int]]):
    async def __call__(self, data: Dict[str, int]) -> List[int]:
        return [v for v in data.values() if v > 100]

# Create and use a pipeline
async def main():
    pipeline = AsyncPipeline([NumberFilterPipe()])
    
    # Process data
    data = {"a": 150, "b": 50, "c": 200}
    result = await pipeline(data)  # [150, 200]
    print(result)

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())
```

## 📚 Documentation

Visit our [comprehensive documentation](https://rivusio.readthedocs.io) for:
- Detailed guides and tutorials
- API reference
- Best practices
- Advanced examples
- Configuration options

## 🔧 Development Setup

```bash
# Clone the repository
git clone https://github.com/zbytealchemy/rivusio.git
cd rivusio

# Install dependencies
poetry install

# Run tests
poetry run pytest

# Build documentation
poetry install --with docs
make docs
```

## 🤝 Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details on:
- Code of Conduct
- Development process
- Pull request guidelines
- Testing requirements

## 📋 Requirements

- Python 3.10+
- Poetry for dependency management
- Git for version control

## 📈 Performance

Rivusio is designed for high performance:
- Efficient batch processing
- Parallel execution capabilities
- Optimized stream processing
- Minimal overhead

## 🔒 Security

- Regular security audits
- Type-safe operations
- Input validation
- Comprehensive testing

## 📄 License

MIT License - see [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

Special thanks to all [contributors](https://github.com/zbytealchemy/rivusio/graphs/contributors) who have helped make Rivusio better.

---

Built with ❤️ by [ZbyteAlchemy](https://github.com/zbytealchemy)

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/zbytealchemy/rivusio",
    "name": "rivusio",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4.0,>=3.10",
    "maintainer_email": null,
    "keywords": "pipeline, data processing, async, sync, type-safe, streaming, etl, data engineering, fast, parallel execution",
    "author": "Zbigniew Mastylo",
    "author_email": "zmastylo@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/c6/76/59884c1572d6e49f950810fe3c111661635c1de8f18a6647915198c11a3a/rivusio-0.2.0.tar.gz",
    "platform": null,
    "description": "# Rivusio\n\n[![codecov](https://codecov.io/gh/zbytealchemy/rivusio/branch/main/graph/badge.svg)](https://codecov.io/gh/zbytealchemy/rivusio)\n[![PyPI version](https://badge.fury.io/py/rivusio.svg)](https://badge.fury.io/py/rivusio)\n[![Documentation Status](https://readthedocs.org/projects/rivusio/badge/?version=latest)](https://rivusio.readthedocs.io/en/latest/?badge=latest)\n[![Python versions](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org)\n[![License](https://img.shields.io/github/license/zbytealchemy/rivusio)](https://github.com/zbytealchemy/rivusio/blob/main/LICENSE)\n[![Tests](https://github.com/zbytealchemy/rivusio/actions/workflows/test.yml/badge.svg)](https://github.com/zbytealchemy/rivusio/actions/workflows/test.yml)\n[![Downloads](https://static.pepy.tech/badge/rivusio/month)](https://pepy.tech/project/rivusio)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Type Checking: mypy](https://img.shields.io/badge/type%20checking-mypy-blue)](http://mypy-lang.org/)\n[![Security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit)\n\nA blazing-fast, type-safe data processing pipeline framework in Python, designed for seamless integration of both synchronous and asynchronous operations, complete with robust parallel execution for ultimate performance and scalability.\n\n## \ud83d\ude80 Key Features\n\n- **Type Safety**: Comprehensive type hints and runtime validation\n- **Flexible Processing**: Support for both sync and async operations\n- **High Performance**: Parallel execution strategies and optimized processing\n- **Stream Processing**: Built-in support for various windowing strategies\n- **Robust Error Handling**: Configurable retries and error recovery\n- **Monitoring**: Built-in metrics collection and monitoring\n- **Configuration**: Type-safe configuration using Pydantic\n- **Extensible**: Plugin system for custom components\n- **Composable**: Easy pipeline composition with the `>>` operator\n- **Comprehensive Documentation**: Extensive guides and examples\n\n## \ud83d\udee0\ufe0f Installation\n\n```bash\npip install rivusio\n```\n\n## \ud83c\udfaf Quick Start\n\n```python\nfrom typing import Dict, List\nfrom rivusio import AsyncBasePipe, SyncBasePipe, AsyncPipeline\n\n# Define a simple transformation pipe\nclass NumberFilterPipe(AsyncBasePipe[Dict[str, int], List[int]]):\n    async def __call__(self, data: Dict[str, int]) -> List[int]:\n        return [v for v in data.values() if v > 100]\n\n# Create and use a pipeline\nasync def main():\n    pipeline = AsyncPipeline([NumberFilterPipe()])\n    \n    # Process data\n    data = {\"a\": 150, \"b\": 50, \"c\": 200}\n    result = await pipeline(data)  # [150, 200]\n    print(result)\n\nif __name__ == \"__main__\":\n    import asyncio\n    asyncio.run(main())\n```\n\n## \ud83d\udcda Documentation\n\nVisit our [comprehensive documentation](https://rivusio.readthedocs.io) for:\n- Detailed guides and tutorials\n- API reference\n- Best practices\n- Advanced examples\n- Configuration options\n\n## \ud83d\udd27 Development Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/zbytealchemy/rivusio.git\ncd rivusio\n\n# Install dependencies\npoetry install\n\n# Run tests\npoetry run pytest\n\n# Build documentation\npoetry install --with docs\nmake docs\n```\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details on:\n- Code of Conduct\n- Development process\n- Pull request guidelines\n- Testing requirements\n\n## \ud83d\udccb Requirements\n\n- Python 3.10+\n- Poetry for dependency management\n- Git for version control\n\n## \ud83d\udcc8 Performance\n\nRivusio is designed for high performance:\n- Efficient batch processing\n- Parallel execution capabilities\n- Optimized stream processing\n- Minimal overhead\n\n## \ud83d\udd12 Security\n\n- Regular security audits\n- Type-safe operations\n- Input validation\n- Comprehensive testing\n\n## \ud83d\udcc4 License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n## \ud83d\ude4f Acknowledgments\n\nSpecial thanks to all [contributors](https://github.com/zbytealchemy/rivusio/graphs/contributors) who have helped make Rivusio better.\n\n---\n\nBuilt with \u2764\ufe0f by [ZbyteAlchemy](https://github.com/zbytealchemy)\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "A type-safe, async-first data processing pipeline framework",
    "version": "0.2.0",
    "project_urls": {
        "Documentation": "https://zbytealchemy.github.io/rivusio",
        "Homepage": "https://github.com/zbytealchemy/rivusio",
        "Repository": "https://github.com/zbytealchemy/rivusio"
    },
    "split_keywords": [
        "pipeline",
        " data processing",
        " async",
        " sync",
        " type-safe",
        " streaming",
        " etl",
        " data engineering",
        " fast",
        " parallel execution"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "80f99171ee6e1d8b753e2428c1d1a1728a786c0751165250994f69dcede5e0cd",
                "md5": "8d2e7966c8e6093e3f3a03b22bc8f013",
                "sha256": "4d4cd89da07713c2148d2d5db246b0d47d50c7adc245fd470975903d341e41c8"
            },
            "downloads": -1,
            "filename": "rivusio-0.2.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "8d2e7966c8e6093e3f3a03b22bc8f013",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.10",
            "size": 31954,
            "upload_time": "2025-02-02T06:57:22",
            "upload_time_iso_8601": "2025-02-02T06:57:22.089859Z",
            "url": "https://files.pythonhosted.org/packages/80/f9/9171ee6e1d8b753e2428c1d1a1728a786c0751165250994f69dcede5e0cd/rivusio-0.2.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c67659884c1572d6e49f950810fe3c111661635c1de8f18a6647915198c11a3a",
                "md5": "1f3aa9779866262e6f0d08d202e19252",
                "sha256": "906135547b55fb42658a0104716d624b08c7b6d99cffadb160d785f48ccf56cb"
            },
            "downloads": -1,
            "filename": "rivusio-0.2.0.tar.gz",
            "has_sig": false,
            "md5_digest": "1f3aa9779866262e6f0d08d202e19252",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.10",
            "size": 25654,
            "upload_time": "2025-02-02T06:57:24",
            "upload_time_iso_8601": "2025-02-02T06:57:24.619611Z",
            "url": "https://files.pythonhosted.org/packages/c6/76/59884c1572d6e49f950810fe3c111661635c1de8f18a6647915198c11a3a/rivusio-0.2.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-02-02 06:57:24",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "zbytealchemy",
    "github_project": "rivusio",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "rivusio"
}
        
Elapsed time: 4.00585s