# AIMQ
[](https://pypi.org/project/aimq/)
[](https://pypi.org/project/aimq/)
[](https://github.com/bldxio/aimq/actions/workflows/ci.yml)
[](https://codecov.io/gh/bldxio/aimq)
[](https://bldxio.github.io/aimq/)
[](https://opensource.org/licenses/MIT)
AIMQ (AI Message Queue) is a robust message queue processor designed specifically for Supabase's pgmq integration. It provides a powerful framework for processing queued tasks with built-in support for AI-powered document processing and OCR capabilities.
## Features
- **Supabase pgmq Integration**: Seamlessly process messages from Supabase's message queue
- **Document OCR Processing**: Extract text from images using EasyOCR
- **Queue-based Processing**: Efficient handling of document processing tasks
- **AI-powered Analysis**: Leverage machine learning for advanced text analysis
- **Flexible Architecture**: Easy to extend with new processing tools and capabilities
## Quick Start (Zero Installation)
The fastest way to get started with AIMQ is using `uvx`, which requires no installation:
```bash
# Initialize a new AIMQ project
uvx aimq init my-project
cd my-project
# Configure your .env file with Supabase credentials
cp .env.example .env
# Edit .env with your SUPABASE_URL and SUPABASE_KEY
# Edit tasks.py to define your task queues
# (A template is already created for you)
# Start the worker
uvx aimq start
```
That's it! No `pip install`, no virtual environments, no dependency conflicts.
## Installation Options
### Option 1: Using uvx (Recommended for Quick Start)
Run AIMQ directly without installing:
```bash
# Run any command with uvx
uvx aimq init
uvx aimq start tasks.py
uvx aimq send my-queue '{"message": "hello"}'
```
### Option 2: Install as a Tool (Recommended for Regular Use)
Install AIMQ as a persistent tool using uv:
```bash
# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install aimq as a tool
uv tool install aimq
# Now you can use aimq directly
aimq init my-project
aimq start
```
### Option 3: Traditional pip Install
```bash
pip install aimq
aimq start
```
### Option 4: Development Setup
For contributing to AIMQ or building from source:
```bash
# Clone the repository
git clone https://github.com/bldxio/aimq.git
cd aimq
# Install dependencies from lockfile (production)
uv sync
# For development (includes test/dev tools)
uv sync --group dev
# Run from source
uv run aimq start
```
### Key uv Commands for Development
```bash
# Add a new dependency
uv add requests
# Add to dev dependency group
uv add --group dev pytest
# Remove a dependency
uv remove requests
# Update dependencies
uv lock --upgrade
# Run commands in the uv environment
uv run python -m aimq.worker
uv run pytest
```
## Configuration
### Environment Variables
AIMQ uses environment variables for configuration. Create a `.env` file in your project root:
```env
# Supabase Configuration (Required)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_KEY=your-anon-key
# Worker Configuration
WORKER_NAME=aimq-worker
WORKER_PATH=./tasks.py
WORKER_LOG_LEVEL=info
WORKER_IDLE_WAIT=10.0
# AI Provider API Keys (Optional)
OPENAI_API_KEY=sk-...
MISTRAL_API_KEY=...
# LangChain Tracing (Optional - for debugging)
LANGCHAIN_TRACING_V2=false
LANGCHAIN_API_KEY=...
```
When you run `aimq init`, an `.env.example` file is created with all available options documented.
### Supabase Setup
Configure your Supabase project with pgmq:
1. Go to your Supabase project dashboard
2. Navigate to Database → Extensions
3. Enable the `pgmq` extension
4. Run the AIMQ migration (created by `aimq init --supabase`)
For more details, see the [Supabase pgmq documentation](https://supabase.com/docs/guides/database/extensions/pgmq).
## Usage
### Defining Tasks
Create a `tasks.py` file that defines your task queues and processors:
```python
from typing import Any, Dict
from aimq.worker import Worker
# Create a worker instance
worker = Worker()
@worker.task(queue="document-processing", timeout=300)
def process_document(data: Dict[str, Any]) -> Dict[str, Any]:
"""Process a document using AI tools."""
document_url = data.get("document_url")
# Use built-in AIMQ tools for OCR, PDF extraction, etc.
# Your processing logic here...
return {"status": "processed", "text": extracted_text}
@worker.task(queue="image-analysis")
def analyze_image(data: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze an image using AI models."""
image_url = data.get("image_url")
# Your analysis logic here...
return {"analysis": results}
```
### Starting the Worker
```bash
# Start with default tasks.py
aimq start
# Start with a specific tasks file
aimq start my_tasks.py
# Start with debug logging
aimq start --debug
# Using uvx (no installation)
uvx aimq start
```
### Sending Messages to Queues
```bash
# Send a message to a queue
aimq send document-processing '{"document_url": "https://example.com/doc.pdf"}'
# Enable/disable queues
aimq enable document-processing
aimq disable document-processing
```
Or programmatically from Python:
```python
from supabase import create_client
import os
supabase = create_client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_KEY")
)
# Send a message to the queue
supabase.rpc("pgmq_send", {
"queue_name": "document-processing",
"msg": {"document_url": "https://example.com/doc.pdf"}
}).execute()
```
## Docker Deployment
AIMQ provides two Docker deployment options: local development setup and using the published image.
### Option 1: Local Development (Recommended for Getting Started)
Generate Docker files in your project:
```bash
# Initialize with Docker files
aimq init --docker
# This creates:
# - Dockerfile (optimized for your project)
# - docker-compose.yml (with volume mounts)
# - .dockerignore
# Start the worker
docker-compose up -d
# View logs
docker-compose logs -f
# Stop
docker-compose down
```
### Option 2: Published Image (Recommended for Production)
Use the pre-built AIMQ image from the registry:
**With local tasks.py (development):**
```bash
docker run --rm \
-v $(pwd)/tasks.py:/app/tasks.py:ro \
-v $(pwd)/.env:/app/.env:ro \
aimq:latest
```
**With git repository (production):**
```bash
# Load tasks from a git repository
docker run --rm \
-e AIMQ_TASKS=git:mycompany/aimq-tasks@production \
-e SUPABASE_URL=https://xxx.supabase.co \
-e SUPABASE_KEY=your-key \
aimq:latest
```
**Docker Compose with git repository:**
```yaml
version: '3.8'
services:
aimq-worker:
image: aimq:latest
environment:
- AIMQ_TASKS=git:mycompany/aimq-tasks@production
- SUPABASE_URL=${SUPABASE_URL}
- SUPABASE_KEY=${SUPABASE_KEY}
- WORKER_NAME=aimq-worker
restart: unless-stopped
```
### Git URL Patterns
AIMQ supports npm-style git URLs for loading tasks from repositories:
- `git:user/repo` - Default branch from GitHub
- `git:user/repo@branch` - Specific branch or tag
- `git:user/repo#path/to/tasks` - Subdirectory in monorepo
- `git:gitlab.com/user/repo@v1.0.0` - Full URL with version
### Production Deployment Tips
- **Scaling**: Run multiple worker containers for parallel processing
- **Git Repos**: Store tasks in version-controlled repositories for GitOps workflows
- **Secrets**: Use Docker secrets or environment variable injection
- **Monitoring**: Add health checks and logging aggregation
- **Resource Limits**: Set memory/CPU limits based on AI model requirements
- **Authentication**: For private git repos, mount SSH keys or use HTTPS tokens
For detailed Docker deployment patterns and troubleshooting, see [docker/README.md](docker/README.md).
## Development
This project uses [just](https://github.com/casey/just) as a task runner. Install it with:
```bash
# macOS
brew install just
# Other platforms: https://github.com/casey/just#installation
```
### Common Tasks
```bash
# Setup development environment
just install
# Run tests
just test
just test-cov # With coverage
# Code quality
just lint # Check code style
just format # Format code
just type-check # Type checking
just ci # Run all checks (lint + type + test)
# Docker
just dev # Start dev environment
just dev-build # Build and start
just logs # View logs
# Documentation
just docs-serve # Serve docs locally
# See all available tasks
just --list
```
### Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run quality checks: `just ci`
5. Submit a pull request
## License
MIT License - see [LICENSE](LICENSE) for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "aimq",
"maintainer": null,
"docs_url": null,
"requires_python": "<=3.13,>=3.11",
"maintainer_email": null,
"keywords": "ai, document-processing, machine-learning, ocr, pgmq, queue, supabase",
"author": "AIMQ Contributors",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/bb/68/a3786aace269ce48bb115a5c253dabd28fcf1b7f6d41c7cbdaa668e62bcf/aimq-0.1.2.tar.gz",
"platform": null,
"description": "# AIMQ\n\n[](https://pypi.org/project/aimq/)\n[](https://pypi.org/project/aimq/)\n[](https://github.com/bldxio/aimq/actions/workflows/ci.yml)\n[](https://codecov.io/gh/bldxio/aimq)\n[](https://bldxio.github.io/aimq/)\n[](https://opensource.org/licenses/MIT)\n\nAIMQ (AI Message Queue) is a robust message queue processor designed specifically for Supabase's pgmq integration. It provides a powerful framework for processing queued tasks with built-in support for AI-powered document processing and OCR capabilities.\n\n## Features\n\n- **Supabase pgmq Integration**: Seamlessly process messages from Supabase's message queue\n- **Document OCR Processing**: Extract text from images using EasyOCR\n- **Queue-based Processing**: Efficient handling of document processing tasks\n- **AI-powered Analysis**: Leverage machine learning for advanced text analysis\n- **Flexible Architecture**: Easy to extend with new processing tools and capabilities\n\n## Quick Start (Zero Installation)\n\nThe fastest way to get started with AIMQ is using `uvx`, which requires no installation:\n\n```bash\n# Initialize a new AIMQ project\nuvx aimq init my-project\ncd my-project\n\n# Configure your .env file with Supabase credentials\ncp .env.example .env\n# Edit .env with your SUPABASE_URL and SUPABASE_KEY\n\n# Edit tasks.py to define your task queues\n# (A template is already created for you)\n\n# Start the worker\nuvx aimq start\n```\n\nThat's it! No `pip install`, no virtual environments, no dependency conflicts.\n\n## Installation Options\n\n### Option 1: Using uvx (Recommended for Quick Start)\n\nRun AIMQ directly without installing:\n\n```bash\n# Run any command with uvx\nuvx aimq init\nuvx aimq start tasks.py\nuvx aimq send my-queue '{\"message\": \"hello\"}'\n```\n\n### Option 2: Install as a Tool (Recommended for Regular Use)\n\nInstall AIMQ as a persistent tool using uv:\n\n```bash\n# Install uv if you haven't already\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Install aimq as a tool\nuv tool install aimq\n\n# Now you can use aimq directly\naimq init my-project\naimq start\n```\n\n### Option 3: Traditional pip Install\n\n```bash\npip install aimq\naimq start\n```\n\n### Option 4: Development Setup\n\nFor contributing to AIMQ or building from source:\n\n```bash\n# Clone the repository\ngit clone https://github.com/bldxio/aimq.git\ncd aimq\n\n# Install dependencies from lockfile (production)\nuv sync\n\n# For development (includes test/dev tools)\nuv sync --group dev\n\n# Run from source\nuv run aimq start\n```\n\n### Key uv Commands for Development\n\n```bash\n# Add a new dependency\nuv add requests\n\n# Add to dev dependency group\nuv add --group dev pytest\n\n# Remove a dependency\nuv remove requests\n\n# Update dependencies\nuv lock --upgrade\n\n# Run commands in the uv environment\nuv run python -m aimq.worker\nuv run pytest\n```\n\n## Configuration\n\n### Environment Variables\n\nAIMQ uses environment variables for configuration. Create a `.env` file in your project root:\n\n```env\n# Supabase Configuration (Required)\nSUPABASE_URL=https://your-project.supabase.co\nSUPABASE_KEY=your-anon-key\n\n# Worker Configuration\nWORKER_NAME=aimq-worker\nWORKER_PATH=./tasks.py\nWORKER_LOG_LEVEL=info\nWORKER_IDLE_WAIT=10.0\n\n# AI Provider API Keys (Optional)\nOPENAI_API_KEY=sk-...\nMISTRAL_API_KEY=...\n\n# LangChain Tracing (Optional - for debugging)\nLANGCHAIN_TRACING_V2=false\nLANGCHAIN_API_KEY=...\n```\n\nWhen you run `aimq init`, an `.env.example` file is created with all available options documented.\n\n### Supabase Setup\n\nConfigure your Supabase project with pgmq:\n\n1. Go to your Supabase project dashboard\n2. Navigate to Database \u2192 Extensions\n3. Enable the `pgmq` extension\n4. Run the AIMQ migration (created by `aimq init --supabase`)\n\nFor more details, see the [Supabase pgmq documentation](https://supabase.com/docs/guides/database/extensions/pgmq).\n\n## Usage\n\n### Defining Tasks\n\nCreate a `tasks.py` file that defines your task queues and processors:\n\n```python\nfrom typing import Any, Dict\nfrom aimq.worker import Worker\n\n# Create a worker instance\nworker = Worker()\n\n@worker.task(queue=\"document-processing\", timeout=300)\ndef process_document(data: Dict[str, Any]) -> Dict[str, Any]:\n \"\"\"Process a document using AI tools.\"\"\"\n document_url = data.get(\"document_url\")\n\n # Use built-in AIMQ tools for OCR, PDF extraction, etc.\n # Your processing logic here...\n\n return {\"status\": \"processed\", \"text\": extracted_text}\n\n@worker.task(queue=\"image-analysis\")\ndef analyze_image(data: Dict[str, Any]) -> Dict[str, Any]:\n \"\"\"Analyze an image using AI models.\"\"\"\n image_url = data.get(\"image_url\")\n\n # Your analysis logic here...\n\n return {\"analysis\": results}\n```\n\n### Starting the Worker\n\n```bash\n# Start with default tasks.py\naimq start\n\n# Start with a specific tasks file\naimq start my_tasks.py\n\n# Start with debug logging\naimq start --debug\n\n# Using uvx (no installation)\nuvx aimq start\n```\n\n### Sending Messages to Queues\n\n```bash\n# Send a message to a queue\naimq send document-processing '{\"document_url\": \"https://example.com/doc.pdf\"}'\n\n# Enable/disable queues\naimq enable document-processing\naimq disable document-processing\n```\n\nOr programmatically from Python:\n\n```python\nfrom supabase import create_client\nimport os\n\nsupabase = create_client(\n os.getenv(\"SUPABASE_URL\"),\n os.getenv(\"SUPABASE_KEY\")\n)\n\n# Send a message to the queue\nsupabase.rpc(\"pgmq_send\", {\n \"queue_name\": \"document-processing\",\n \"msg\": {\"document_url\": \"https://example.com/doc.pdf\"}\n}).execute()\n```\n\n## Docker Deployment\n\nAIMQ provides two Docker deployment options: local development setup and using the published image.\n\n### Option 1: Local Development (Recommended for Getting Started)\n\nGenerate Docker files in your project:\n\n```bash\n# Initialize with Docker files\naimq init --docker\n\n# This creates:\n# - Dockerfile (optimized for your project)\n# - docker-compose.yml (with volume mounts)\n# - .dockerignore\n\n# Start the worker\ndocker-compose up -d\n\n# View logs\ndocker-compose logs -f\n\n# Stop\ndocker-compose down\n```\n\n### Option 2: Published Image (Recommended for Production)\n\nUse the pre-built AIMQ image from the registry:\n\n**With local tasks.py (development):**\n```bash\ndocker run --rm \\\n -v $(pwd)/tasks.py:/app/tasks.py:ro \\\n -v $(pwd)/.env:/app/.env:ro \\\n aimq:latest\n```\n\n**With git repository (production):**\n```bash\n# Load tasks from a git repository\ndocker run --rm \\\n -e AIMQ_TASKS=git:mycompany/aimq-tasks@production \\\n -e SUPABASE_URL=https://xxx.supabase.co \\\n -e SUPABASE_KEY=your-key \\\n aimq:latest\n```\n\n**Docker Compose with git repository:**\n```yaml\nversion: '3.8'\n\nservices:\n aimq-worker:\n image: aimq:latest\n environment:\n - AIMQ_TASKS=git:mycompany/aimq-tasks@production\n - SUPABASE_URL=${SUPABASE_URL}\n - SUPABASE_KEY=${SUPABASE_KEY}\n - WORKER_NAME=aimq-worker\n restart: unless-stopped\n```\n\n### Git URL Patterns\n\nAIMQ supports npm-style git URLs for loading tasks from repositories:\n\n- `git:user/repo` - Default branch from GitHub\n- `git:user/repo@branch` - Specific branch or tag\n- `git:user/repo#path/to/tasks` - Subdirectory in monorepo\n- `git:gitlab.com/user/repo@v1.0.0` - Full URL with version\n\n### Production Deployment Tips\n\n- **Scaling**: Run multiple worker containers for parallel processing\n- **Git Repos**: Store tasks in version-controlled repositories for GitOps workflows\n- **Secrets**: Use Docker secrets or environment variable injection\n- **Monitoring**: Add health checks and logging aggregation\n- **Resource Limits**: Set memory/CPU limits based on AI model requirements\n- **Authentication**: For private git repos, mount SSH keys or use HTTPS tokens\n\nFor detailed Docker deployment patterns and troubleshooting, see [docker/README.md](docker/README.md).\n\n## Development\n\nThis project uses [just](https://github.com/casey/just) as a task runner. Install it with:\n\n```bash\n# macOS\nbrew install just\n\n# Other platforms: https://github.com/casey/just#installation\n```\n\n### Common Tasks\n\n```bash\n# Setup development environment\njust install\n\n# Run tests\njust test\njust test-cov # With coverage\n\n# Code quality\njust lint # Check code style\njust format # Format code\njust type-check # Type checking\njust ci # Run all checks (lint + type + test)\n\n# Docker\njust dev # Start dev environment\njust dev-build # Build and start\njust logs # View logs\n\n# Documentation\njust docs-serve # Serve docs locally\n\n# See all available tasks\njust --list\n```\n\n### Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Run quality checks: `just ci`\n5. Submit a pull request\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A robust message queue processor for Supabase pgmq with AI-powered document processing capabilities",
"version": "0.1.2",
"project_urls": {
"Documentation": "https://bldxio.github.io/aimq",
"Homepage": "https://github.com/bldxio/aimq",
"Repository": "https://github.com/bldxio/aimq"
},
"split_keywords": [
"ai",
" document-processing",
" machine-learning",
" ocr",
" pgmq",
" queue",
" supabase"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e3a0ced2de54adc5c4ddfb2be6946140b11836de0feaf94fd43c199012ed42ab",
"md5": "193a108f28d081ad0ed4358d94a7c5e1",
"sha256": "00f043f80a09d7ed8e3a6a955aa8cef0a28995fae829dd05b0e98c2aaaec550c"
},
"downloads": -1,
"filename": "aimq-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "193a108f28d081ad0ed4358d94a7c5e1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<=3.13,>=3.11",
"size": 58365,
"upload_time": "2025-10-21T11:51:48",
"upload_time_iso_8601": "2025-10-21T11:51:48.386175Z",
"url": "https://files.pythonhosted.org/packages/e3/a0/ced2de54adc5c4ddfb2be6946140b11836de0feaf94fd43c199012ed42ab/aimq-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "bb68a3786aace269ce48bb115a5c253dabd28fcf1b7f6d41c7cbdaa668e62bcf",
"md5": "d464247d8bbf1c050b98ae90252e7bd4",
"sha256": "00aaab760c6fea70a18da7c6d0f14b136b19fe56347143a9e790e5551ee99bdd"
},
"downloads": -1,
"filename": "aimq-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "d464247d8bbf1c050b98ae90252e7bd4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<=3.13,>=3.11",
"size": 60986,
"upload_time": "2025-10-21T11:51:49",
"upload_time_iso_8601": "2025-10-21T11:51:49.582553Z",
"url": "https://files.pythonhosted.org/packages/bb/68/a3786aace269ce48bb115a5c253dabd28fcf1b7f6d41c7cbdaa668e62bcf/aimq-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-21 11:51:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bldxio",
"github_project": "aimq",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "aimq"
}
JSON
