u-shape-frame


Nameu-shape-frame JSON
Version 0.0.2 PyPI version JSON
download
home_pagehttps://github.com/al6nlee/u-shape-frame
SummaryA U-shaped middleware framework for building data processing pipelines
upload_time2025-10-23 10:31:14
maintainerNone
docs_urlNone
authoralan
requires_python>=3.10
licenseNone
keywords middleware pipeline dataflow u-shape processor
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            # U-Shape Frame

A lightweight U-shaped middleware framework for building data processing pipelines with a unique bidirectional flow pattern.

## Overview

U-Shape Frame is a Python framework that implements a U-shaped processing architecture, where data flows down through a series of processors during the input phase, and then flows back up in reverse order during the output phase. This pattern is particularly useful for building middleware chains, request/response handlers, and data transformation pipelines.

## Key Features

- **U-Shaped Flow Pattern**: Data flows down through processors in forward order (input phase) and back up in reverse order (output phase)
- **Stateless Processors**: Clean, reusable processor components with separated input/output logic
- **Workflow Management**: Create and manage multiple processing workflows with different processor chains
- **Context Sharing**: Share data between processors through a dataflow context
- **Easy Registration**: Simple processor registration and workflow creation through the Engine

## Installation

```bash
pip install u-shape-frame
```

Or install from source:

```bash
git clone https://github.com/al6nlee/u-shape-frame.git
cd u-shape-frame
pip install -r requirements.txt
python setup.py install
```

## Quick Start

### 1. Define Your Processors

```python
from u_shape_frame.core.processor import Processor

class LoggingProcessor(Processor):
    """Log input and output"""
    name = "Logging"

    def input(self, dataflow):
        print(f"Request started with: {dataflow.input}")
        dataflow.set_context("start_time", time.time())

    def output(self, dataflow):
        duration = time.time() - dataflow.get_context("start_time")
        print(f"Request completed in {duration}s")

class ValidationProcessor(Processor):
    """Validate data"""
    name = "Validation"

    def input(self, dataflow):
        if "text" not in dataflow.input:
            raise ValueError("Missing 'text' field")

    def output(self, dataflow):
        result = dataflow.get_output()
        if not result:
            raise ValueError("Output cannot be empty")

class TransformProcessor(Processor):
    """Transform data"""
    name = "Transform"

    def input(self, dataflow):
        text = dataflow.input["text"].strip()
        dataflow.set_context("cleaned_text", text)

    def output(self, dataflow):
        text = dataflow.get_context("cleaned_text")
        result = f"PROCESSED: {text.upper()}"
        dataflow.set_output(result)
```

### 2. Create and Run a Workflow

```python
from u_shape_frame.core.engine import Engine

# Create engine and register processors
engine = Engine()
engine.register_processor(LoggingProcessor)
engine.register_processor(ValidationProcessor)
engine.register_processor(TransformProcessor)

# Create workflow with processor chain
workflow = engine.create_workflow(
    "RequestPipeline",
    ["Logging", "Validation", "Transform"]
)

# Create a procession and run
procession = workflow.create_procession()
result = procession.run({"text": "  hello world  "})

print(f"Result: {result.output}")
# Output: PROCESSED: HELLO WORLD
```

## How It Works

The U-shaped flow pattern processes data in two phases:

```
Input Phase (Down):
    Logging.input()
        Validation.input()
            Transform.input()

Output Phase (Up - Reverse Order):
            Transform.output()
        Validation.output()
    Logging.output()
```

This pattern is similar to middleware in web frameworks like Express.js or Django, where each layer can:
- Preprocess data on the way down (input phase)
- Postprocess data on the way up (output phase)
- Share context between phases
- Validate at appropriate layers

## Architecture

![U-Shape Frame Architecture](./docs/u-shape-frame-architecture.png)

## Core Components

### Processor
Abstract base class for creating processing units. Each processor must implement:
- `input(dataflow)`: Called during the downward phase
- `output(dataflow)`: Called during the upward phase

### Engine
Global runtime container that manages:
- Processor registration
- Workflow creation
- Processor instance lifecycle

### Workflow
Defines a processing pipeline by:
- Organizing processors in a specific order
- Creating processions for execution

### Procession
Executes a single run of a workflow:
- Runs processors in forward order (input phase)
- Runs processors in reverse order (output phase)
- Manages dataflow between processors

### Dataflow
Carries data through the processing pipeline:
- Stores input data
- Shares context between processors
- Holds final output

## Use Cases

- **Request/Response Middleware**: Web request processing with logging, authentication, validation, and transformation
- **Data Pipelines**: ETL processes with validation and transformation steps
- **Event Processing**: Event handlers with pre/post processing hooks
- **API Gateways**: Request preprocessing and response postprocessing

## Requirements

- Python >= 3.10
- Cython >= 3.0.12
- cffi >= 1.17.1

See `requirements.txt` for full dependencies.

## Development

### Running Tests

```bash
cd tests
python run.py
```

### Building from Source

```bash
# Standard installation
python setup.py install

# Build with Cython
python setup.py build_ext

# Create distribution
python setup.py bdist_wheel
```

## License

MIT License - see LICENSE file for details

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request

## Links

- **GitHub**: https://github.com/al6nlee/u-shape-frame
- **Bug Reports**: https://github.com/al6nlee/u-shape-frame/issues
- **Author**: alan (alanlee.snao@gmail.com)

## Changelog

### 0.0.1 (Initial Release)
- U-shaped flow pattern implementation
- Core processor, engine, workflow, and procession components
- Basic example processors
- Cython build support

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/al6nlee/u-shape-frame",
    "name": "u-shape-frame",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "middleware pipeline dataflow u-shape processor",
    "author": "alan",
    "author_email": "alanlee.snao@gmail.com",
    "download_url": null,
    "platform": null,
    "description": "# U-Shape Frame\n\nA lightweight U-shaped middleware framework for building data processing pipelines with a unique bidirectional flow pattern.\n\n## Overview\n\nU-Shape Frame is a Python framework that implements a U-shaped processing architecture, where data flows down through a series of processors during the input phase, and then flows back up in reverse order during the output phase. This pattern is particularly useful for building middleware chains, request/response handlers, and data transformation pipelines.\n\n## Key Features\n\n- **U-Shaped Flow Pattern**: Data flows down through processors in forward order (input phase) and back up in reverse order (output phase)\n- **Stateless Processors**: Clean, reusable processor components with separated input/output logic\n- **Workflow Management**: Create and manage multiple processing workflows with different processor chains\n- **Context Sharing**: Share data between processors through a dataflow context\n- **Easy Registration**: Simple processor registration and workflow creation through the Engine\n\n## Installation\n\n```bash\npip install u-shape-frame\n```\n\nOr install from source:\n\n```bash\ngit clone https://github.com/al6nlee/u-shape-frame.git\ncd u-shape-frame\npip install -r requirements.txt\npython setup.py install\n```\n\n## Quick Start\n\n### 1. Define Your Processors\n\n```python\nfrom u_shape_frame.core.processor import Processor\n\nclass LoggingProcessor(Processor):\n    \"\"\"Log input and output\"\"\"\n    name = \"Logging\"\n\n    def input(self, dataflow):\n        print(f\"Request started with: {dataflow.input}\")\n        dataflow.set_context(\"start_time\", time.time())\n\n    def output(self, dataflow):\n        duration = time.time() - dataflow.get_context(\"start_time\")\n        print(f\"Request completed in {duration}s\")\n\nclass ValidationProcessor(Processor):\n    \"\"\"Validate data\"\"\"\n    name = \"Validation\"\n\n    def input(self, dataflow):\n        if \"text\" not in dataflow.input:\n            raise ValueError(\"Missing 'text' field\")\n\n    def output(self, dataflow):\n        result = dataflow.get_output()\n        if not result:\n            raise ValueError(\"Output cannot be empty\")\n\nclass TransformProcessor(Processor):\n    \"\"\"Transform data\"\"\"\n    name = \"Transform\"\n\n    def input(self, dataflow):\n        text = dataflow.input[\"text\"].strip()\n        dataflow.set_context(\"cleaned_text\", text)\n\n    def output(self, dataflow):\n        text = dataflow.get_context(\"cleaned_text\")\n        result = f\"PROCESSED: {text.upper()}\"\n        dataflow.set_output(result)\n```\n\n### 2. Create and Run a Workflow\n\n```python\nfrom u_shape_frame.core.engine import Engine\n\n# Create engine and register processors\nengine = Engine()\nengine.register_processor(LoggingProcessor)\nengine.register_processor(ValidationProcessor)\nengine.register_processor(TransformProcessor)\n\n# Create workflow with processor chain\nworkflow = engine.create_workflow(\n    \"RequestPipeline\",\n    [\"Logging\", \"Validation\", \"Transform\"]\n)\n\n# Create a procession and run\nprocession = workflow.create_procession()\nresult = procession.run({\"text\": \"  hello world  \"})\n\nprint(f\"Result: {result.output}\")\n# Output: PROCESSED: HELLO WORLD\n```\n\n## How It Works\n\nThe U-shaped flow pattern processes data in two phases:\n\n```\nInput Phase (Down):\n    Logging.input()\n        Validation.input()\n            Transform.input()\n\nOutput Phase (Up - Reverse Order):\n            Transform.output()\n        Validation.output()\n    Logging.output()\n```\n\nThis pattern is similar to middleware in web frameworks like Express.js or Django, where each layer can:\n- Preprocess data on the way down (input phase)\n- Postprocess data on the way up (output phase)\n- Share context between phases\n- Validate at appropriate layers\n\n## Architecture\n\n![U-Shape Frame Architecture](./docs/u-shape-frame-architecture.png)\n\n## Core Components\n\n### Processor\nAbstract base class for creating processing units. Each processor must implement:\n- `input(dataflow)`: Called during the downward phase\n- `output(dataflow)`: Called during the upward phase\n\n### Engine\nGlobal runtime container that manages:\n- Processor registration\n- Workflow creation\n- Processor instance lifecycle\n\n### Workflow\nDefines a processing pipeline by:\n- Organizing processors in a specific order\n- Creating processions for execution\n\n### Procession\nExecutes a single run of a workflow:\n- Runs processors in forward order (input phase)\n- Runs processors in reverse order (output phase)\n- Manages dataflow between processors\n\n### Dataflow\nCarries data through the processing pipeline:\n- Stores input data\n- Shares context between processors\n- Holds final output\n\n## Use Cases\n\n- **Request/Response Middleware**: Web request processing with logging, authentication, validation, and transformation\n- **Data Pipelines**: ETL processes with validation and transformation steps\n- **Event Processing**: Event handlers with pre/post processing hooks\n- **API Gateways**: Request preprocessing and response postprocessing\n\n## Requirements\n\n- Python >= 3.10\n- Cython >= 3.0.12\n- cffi >= 1.17.1\n\nSee `requirements.txt` for full dependencies.\n\n## Development\n\n### Running Tests\n\n```bash\ncd tests\npython run.py\n```\n\n### Building from Source\n\n```bash\n# Standard installation\npython setup.py install\n\n# Build with Cython\npython setup.py build_ext\n\n# Create distribution\npython setup.py bdist_wheel\n```\n\n## License\n\nMIT License - see LICENSE file for details\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## Links\n\n- **GitHub**: https://github.com/al6nlee/u-shape-frame\n- **Bug Reports**: https://github.com/al6nlee/u-shape-frame/issues\n- **Author**: alan (alanlee.snao@gmail.com)\n\n## Changelog\n\n### 0.0.1 (Initial Release)\n- U-shaped flow pattern implementation\n- Core processor, engine, workflow, and procession components\n- Basic example processors\n- Cython build support\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "A U-shaped middleware framework for building data processing pipelines",
    "version": "0.0.2",
    "project_urls": {
        "Bug Reports": "https://github.com/al6nlee/u-shape-frame/issues",
        "Homepage": "https://github.com/al6nlee/u-shape-frame",
        "Source": "https://github.com/al6nlee/u-shape-frame"
    },
    "split_keywords": [
        "middleware",
        "pipeline",
        "dataflow",
        "u-shape",
        "processor"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c9a093432b88fc5725e4f5c07fd38b38c886cd41842ee8a36b22b74e21ffbeaf",
                "md5": "c057d7e15a3fcf0fbefc868116f09016",
                "sha256": "e9e318f31fc4a13fc90a89fd0c5d8786329fdea909347d53dea9de92d564fdcc"
            },
            "downloads": -1,
            "filename": "u_shape_frame-0.0.2-cp312-cp312-macosx_11_0_arm64.whl",
            "has_sig": false,
            "md5_digest": "c057d7e15a3fcf0fbefc868116f09016",
            "packagetype": "bdist_wheel",
            "python_version": "cp312",
            "requires_python": ">=3.10",
            "size": 103858,
            "upload_time": "2025-10-23T10:31:14",
            "upload_time_iso_8601": "2025-10-23T10:31:14.738054Z",
            "url": "https://files.pythonhosted.org/packages/c9/a0/93432b88fc5725e4f5c07fd38b38c886cd41842ee8a36b22b74e21ffbeaf/u_shape_frame-0.0.2-cp312-cp312-macosx_11_0_arm64.whl",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-10-23 10:31:14",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "al6nlee",
    "github_project": "u-shape-frame",
    "github_not_found": true,
    "lcname": "u-shape-frame"
}
alan
Elapsed time: 0.88932s