<div align="center">
# ⚛️ QuantumEngine ⚡
<p>
<strong>A powerful, multi-backend Object-Document Mapper (ODM) for Python</strong>
</p>
<p>
Unified API for both transactional and analytical databases<br>
Supporting SurrealDB (graph/document) and ClickHouse (columnar analytical) with a single, consistent interface
</p>
<p>
<a href="https://pypi.org/project/quantumengine/"><img src="https://img.shields.io/pypi/v/quantumengine.svg" alt="PyPI version"></a>
<a href="https://pypi.org/project/quantumengine/"><img src="https://img.shields.io/pypi/pyversions/quantumengine.svg" alt="Python versions"></a>
<a href="https://github.com/iristech-systems/QuantumEngine/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
<a href="https://github.com/iristech-systems/QuantumEngine/actions"><img src="https://github.com/iristech-systems/QuantumEngine/workflows/Tests/badge.svg" alt="Tests"></a>
<a href="https://pypi.org/project/quantumengine/"><img src="https://img.shields.io/pypi/dm/quantumengine.svg" alt="Downloads"></a>
</p>
</div>
---
## 📦 Installation
QuantumEngine uses a **modular installation system** - install only the backends you need:
```bash
# Core package only (lightweight)
pip install quantumengine
# With ClickHouse support
pip install quantumengine[clickhouse]
# With SurrealDB support
pip install quantumengine[surrealdb]
# With both backends
pip install quantumengine[clickhouse,surrealdb]
# Everything (all backends + dev tools)
pip install quantumengine[all]
```
See [INSTALLATION.md](INSTALLATION.md) for detailed installation options and troubleshooting.
## 🚀 Quick Start
```python
import os
from quantumengine import Document, StringField, IntField, create_connection
# Define a document model
class User(Document):
username = StringField(required=True)
email = StringField(required=True)
age = IntField(min_value=0)
class Meta:
collection = "users"
backend = "surrealdb" # or "clickhouse"
# Create connection
connection = create_connection(
url="ws://localhost:8000/rpc",
namespace="test_ns",
database="test_db",
username=os.environ.get("SURREALDB_USER"),
password=os.environ.get("SURREALDB_PASS"),
make_default=True
)
await connection.connect()
# Create table
await User.create_table()
# CRUD operations
user = User(username="alice", email="alice@example.com", age=25)
await user.save()
users = await User.objects.filter(age__gt=18).all()
await User.objects.filter(username="alice").update(age=26)
```
## 🏗️ Architecture
QuantumORM provides a unified interface for multiple database backends:
- **SurrealDB Backend**: Graph database with native support for relations, transactions, and complex queries
- **ClickHouse Backend**: High-performance columnar database optimized for analytics and time-series data
### Multi-Backend Support
```python
# SurrealDB connection for transactional data
user_connection = create_connection(
name="surrealdb_main",
url="ws://localhost:8000/rpc",
backend="surrealdb",
make_default=True
)
# ClickHouse connection for analytics
analytics_connection = create_connection(
name="clickhouse_analytics",
url="host.clickhouse.cloud",
backend="clickhouse",
port=443,
secure=True
)
class User(Document):
class Meta:
backend = "surrealdb"
class AnalyticsEvent(Document):
class Meta:
backend = "clickhouse"
```
## 📋 Features
### ✅ Core Features
- **Multi-Backend Architecture**: SurrealDB + ClickHouse support
- **Type-Safe Field System**: 15+ field types with validation
- **Query System**: Q objects, QueryExpressions, and advanced filtering
- **Relationship Management**: Graph relations and references
- **Schema Management**: Both SCHEMAFULL and SCHEMALESS table support
- **Async/Sync APIs**: Complete async/await support with sync alternatives
- **Connection Management**: Named connections and connection pooling
- **Performance Optimization**: Direct record access and bulk operations
- **Drop Table & Migrations**: Schema migration tools and table management
### 🔧 Field Types
| Field Type | Description | SurrealDB | ClickHouse |
|------------|-------------|-----------|------------|
| `StringField` | Text fields with validation | ✅ | ✅ |
| `IntField` | Integer with min/max constraints | ✅ | ✅ |
| `FloatField` | Floating point numbers | ✅ | ✅ |
| `BooleanField` | Boolean values | ✅ | ✅ |
| `DateTimeField` | Date and time with timezone | ✅ | ✅ |
| `DecimalField` | High-precision decimals | ✅ | ✅ |
| `UUIDField` | UUID generation and validation | ✅ | ✅ |
| `ListField` | Arrays/lists with typed elements | ✅ | ✅ |
| `DictField` | JSON/dictionary storage | ✅ | ✅ |
| `ReferenceField` | References to other documents | ✅ | ❌ |
| `IPAddressField` | IPv4/IPv6 address validation | ✅ | ✅ |
| `DurationField` | Time periods and durations | ✅ | ❌ |
| `RangeField` | Range values with bounds | ✅ | ❌ |
| `OptionField` | Optional field wrapper | ✅ | ❌ |
| `RecordIDField` | SurrealDB record identifiers | ✅ | ❌ |
### 🔍 Query Capabilities
#### Basic Filtering
```python
# Simple filters
users = await User.objects.filter(age__gt=18).all()
users = await User.objects.filter(username__contains="admin").all()
users = await User.objects.filter(active=True).all()
# Count
count = await User.objects.filter(age__gte=21).count()
```
#### Q Objects for Complex Queries
```python
from quantumengine import Q
# Combine conditions
complex_query = Q(age__gte=18) & Q(age__lte=65) & Q(active=True)
users = await User.objects.filter(complex_query).all()
# OR conditions
either_query = Q(role="admin") | Q(permissions__contains="admin")
users = await User.objects.filter(either_query).all()
# NOT conditions
not_query = ~Q(status="banned")
users = await User.objects.filter(not_query).all()
# Raw queries
raw_query = Q.raw("age > 25 AND string::contains(username, 'admin')")
users = await User.objects.filter(raw_query).all()
```
#### QueryExpressions with FETCH
```python
# Fetch related documents (SurrealDB)
expr = QueryExpression(where=Q(published=True)).fetch("author")
posts = await Post.objects.filter(expr).all()
# Complex expressions
complex_expr = (QueryExpression(where=Q(active=True))
.order_by("created_at", "DESC")
.limit(10)
.fetch("profile"))
users = await User.objects.filter(complex_expr).all()
```
### 🔗 Relationships (SurrealDB)
#### Document References
```python
class Post(Document):
title = StringField(required=True)
author = ReferenceField(User, required=True)
categories = ListField(field_type=ReferenceField(Category))
class Meta:
collection = "posts"
backend = "surrealdb"
```
#### Graph Relations
```python
# Create relations between documents
await author1.relate_to("collaborated_with", author2, project="Novel")
# Fetch relations
collaborators = await author1.fetch_relation("collaborated_with")
# Resolve relations (get related documents)
related_authors = await author1.resolve_relation("collaborated_with")
```
#### Relation Documents
```python
class AuthorCollaboration(RelationDocument):
project_name = StringField(required=True)
start_date = DateTimeField()
contribution_percent = FloatField()
class Meta:
collection = "collaborated_with"
# Create relation with metadata
relation = await AuthorCollaboration.create_relation(
author1, author2,
project_name="Science Fiction Novel",
start_date=datetime.now(),
contribution_percent=60.0
)
```
### 📊 Schema Management
#### Table Creation
```python
# SCHEMAFULL tables (strict schema)
await User.create_table(schemafull=True)
# SCHEMALESS tables (flexible schema)
await User.create_table(schemafull=False)
# Backend-specific table creation
await analytics_backend.create_table(
AnalyticsEvent,
engine="MergeTree",
order_by="(event_time, user_id)"
)
```
#### Drop Tables & Migration Support
```python
from quantumengine import (
generate_drop_statements, generate_migration_statements,
drop_tables_from_module
)
# Drop table functionality
await User.drop_table(if_exists=True)
User.drop_table_sync(if_exists=True)
# Generate drop statements for migration scripts
drop_statements = generate_drop_statements(User)
# ['REMOVE INDEX IF EXISTS idx_email ON users;',
# 'REMOVE FIELD IF EXISTS email ON users;', ...]
# Generate migration between document versions
migration = generate_migration_statements(UserV1, UserV2, schemafull=True)
print(migration['up']) # Forward migration statements
print(migration['down']) # Rollback migration statements
# Drop all tables in a module
await drop_tables_from_module('myapp.models')
```
#### Hybrid Schema Support
```python
class Product(Document):
# Always defined in schema
name = StringField(required=True, define_schema=True)
price = FloatField(define_schema=True)
# Only defined in SCHEMAFULL tables
description = StringField()
metadata = DictField()
class Meta:
collection = "products"
```
#### Index Management
```python
class User(Document):
username = StringField(required=True)
email = StringField(required=True)
class Meta:
collection = "users"
indexes = [
{"name": "user_username_idx", "fields": ["username"], "unique": True},
{"name": "user_email_idx", "fields": ["email"], "unique": True},
{"name": "user_age_idx", "fields": ["age"]}
]
# Create indexes
await User.create_indexes()
```
### ⚡ Performance Features
#### Direct Record Access
```python
# Optimized ID-based queries use direct record access
users = await User.objects.filter(id__in=['user:1', 'user:2']).all()
# Convenience methods for ID operations
users = await User.objects.get_many([1, 2, 3]).all()
users = await User.objects.get_range(100, 200, inclusive=True).all()
```
#### Query Analysis
```python
# Explain query execution plan
plan = await User.objects.filter(age__gt=25).explain()
# Get index suggestions
suggestions = User.objects.filter(age__lt=30).suggest_indexes()
```
#### Bulk Operations
```python
# Bulk updates
updated = await User.objects.filter(active=False).update(status="inactive")
# Bulk deletes
deleted_count = await User.objects.filter(last_login__lt=cutoff_date).delete()
```
### 🔄 Sync API Support
```python
# Create sync connection
connection = create_connection(
url="ws://localhost:8000/rpc",
namespace="test_ns",
database="test_db",
username=os.environ.get("SURREALDB_USER"),
password=os.environ.get("SURREALDB_PASS"),
async_mode=False
)
with connection:
# Synchronous operations
User.create_table_sync(schemafull=True)
user = User(username="alice", email="alice@example.com")
user.save_sync()
users = User.objects.all_sync()
user = User.objects.get_sync(id=user_id)
user.delete_sync()
```
### 📈 DataGrid Helpers
```python
from quantumengine import get_grid_data, parse_datatables_params
# Efficient grid operations for web interfaces
result = await get_grid_data(
User, # Document class
request_args, # Request parameters
search_fields=['username', 'email'],
custom_filters={'active': 'active'},
default_sort='created_at'
)
# DataTables integration
params = parse_datatables_params(request_args)
result = format_datatables_response(total, rows, draw)
```
## 📦 Installation
```bash
pip install quantumengine
# For SurrealDB support
pip install surrealdb
# For ClickHouse support
pip install clickhouse-connect
```
## 🔧 Configuration
### Environment Variables
```python
import os
from quantumengine import create_connection
# Using environment variables
connection = create_connection(
url=os.environ.get("SURREALDB_URL", "ws://localhost:8000/rpc"),
namespace=os.environ.get("SURREALDB_NS", "production"),
database=os.environ.get("SURREALDB_DB", "main"),
username=os.environ.get("SURREALDB_USER"),
password=os.environ.get("SURREALDB_PASS"),
make_default=True
)
```
### Multiple Named Connections
```python
# Main transactional database
main_db = create_connection(
name="main_db",
url="ws://localhost:8000/rpc",
backend="surrealdb",
make_default=True
)
# Analytics database
analytics_db = create_connection(
name="analytics_db",
url="https://analytics.clickhouse.cloud",
backend="clickhouse"
)
# Use specific connection
await User.create_table(connection=main_db)
await AnalyticsEvent.create_table(connection=analytics_db)
```
## 🏃♂️ Quick Examples
### Basic CRUD
```python
# Create
user = User(username="alice", email="alice@example.com", age=25)
await user.save()
# Read
user = await User.objects.get(username="alice")
users = await User.objects.filter(age__gte=18).all()
# Update
user.age = 26
await user.save()
# Delete
await user.delete()
```
### Advanced Queries
```python
from quantumengine import Q, QueryExpression
# Complex filtering
active_adults = await User.objects.filter(
Q(age__gte=18) & Q(active=True)
).all()
# Fetch relations
posts_with_authors = await Post.objects.filter(
QueryExpression(where=Q(published=True)).fetch("author")
).all()
# Pagination and sorting
recent_users = await User.objects.filter(
active=True
).order_by("-created_at").limit(10).all()
```
### Multi-Backend Usage
```python
# User data in SurrealDB (transactional)
class User(Document):
username = StringField(required=True)
email = StringField(required=True)
class Meta:
collection = "users"
backend = "surrealdb"
# Analytics events in ClickHouse (analytical)
class PageView(Document):
user_id = StringField(required=True)
page_url = StringField(required=True)
timestamp = DateTimeField(required=True)
class Meta:
collection = "page_views"
backend = "clickhouse"
# Use both seamlessly
user = await User.objects.get(username="alice")
page_views = await PageView.objects.filter(user_id=str(user.id)).all()
```
### Schema Management Examples
```python
# Generate schema statements
from quantumengine import generate_schema_statements, generate_drop_statements
# Create schema
schema_statements = generate_schema_statements(User, schemafull=True)
for stmt in schema_statements:
print(stmt)
# Drop schema
drop_statements = generate_drop_statements(User)
for stmt in drop_statements:
print(stmt)
# Generate migration between versions
from quantumengine import generate_migration_statements
class UserV1(Document):
username = StringField(required=True)
email = StringField(required=True)
class UserV2(Document):
username = StringField(required=True)
email = StringField(required=True)
active = BooleanField(default=True) # New field
migration = generate_migration_statements(UserV1, UserV2)
print("UP migration:")
for stmt in migration['up']:
print(f" {stmt}")
print("DOWN migration:")
for stmt in migration['down']:
print(f" {stmt}")
```
## 🧪 Testing
The codebase includes comprehensive tests demonstrating real database operations:
```bash
# Run working tests
python tests/working/test_multi_backend_real_connections.py
python tests/working/test_clickhouse_simple_working.py
python tests/working/test_working_surrealdb_backend.py
# Run working examples
python example_scripts/working/basic_crud_example.py
python example_scripts/working/multi_backend_example.py
python example_scripts/working/advanced_features_example.py
```
## 📚 Examples
The `example_scripts/working/` directory contains fully functional examples:
- **basic_crud_example.py**: Core CRUD operations
- **advanced_features_example.py**: Complex field types and validation
- **multi_backend_example.py**: Using SurrealDB and ClickHouse together
- **relation_example.py**: Graph relations and RelationDocuments
- **query_expressions_example.py**: Advanced querying with Q objects
- **sync_api_example.py**: Synchronous API usage
- **test_performance_optimizations.py**: Performance features and optimization
- **test_drop_and_migration.py**: Drop table and migration functionality
## 🔄 Backend Capabilities
### SurrealDB Backend Features
- ✅ Graph relations and traversal
- ✅ Transactions
- ✅ Direct record access
- ✅ Full-text search
- ✅ References between documents
- ✅ Complex data types (Duration, Range, Option)
- ✅ SCHEMAFULL and SCHEMALESS tables
### ClickHouse Backend Features
- ✅ High-performance analytical queries
- ✅ Bulk operations optimization
- ✅ Time-series data handling
- ✅ Columnar storage benefits
- ✅ Aggregation functions
- ❌ Graph relations (not applicable)
- ❌ Transactions (limited support)
### Backend Detection
```python
# Check backend capabilities
if connection.backend.supports_graph_relations():
await user.relate_to("follows", other_user)
if connection.backend.supports_bulk_operations():
await Document.objects.filter(...).bulk_update(status="processed")
# Get backend-specific optimizations
optimizations = connection.backend.get_optimized_methods()
print(optimizations)
```
## ⚡ Performance Features
### Automatic Query Optimizations
- **Direct Record Access**: ID-based queries use `SELECT * FROM user:1, user:2`
- **Range Access**: Range queries use `SELECT * FROM user:1..=100`
- **Bulk Operations**: Optimized batch processing
- **Index Utilization**: Automatic index suggestions
### Measured Performance Improvements
- **Direct Record Access**: Up to 3.4x faster than traditional WHERE clauses
- **Bulk Operations**: Significant improvement for batch processing
- **Memory Efficiency**: Reduced data transfer and memory usage
## 📚 Documentation
### Online Documentation
- **GitHub Pages**: [Full Sphinx Documentation](https://your-username.github.io/quantumengine/) - Complete API reference with examples
- **API Reference**: Detailed class and method documentation
- **Quick Start Guide**: Step-by-step getting started tutorial
- **Module Documentation**: Auto-generated from source code docstrings
### Local Documentation
- **API Reference**: `API_REFERENCE.md` - Complete class and method documentation
- **Examples**: `example_scripts/working/` - Working examples for all features
- **Tests**: `tests/working/` - Test files demonstrating functionality
- **Sphinx Docs**: `docs/` - Build locally with `cd docs && uv run make html`
### Building Documentation
```bash
# Install dependencies
uv sync
# Build Sphinx documentation
cd docs
uv run make html
# View locally
open docs/_build/html/index.html
```
## 🤝 Contributing
QuantumORM is actively developed with a focus on real-world usage and multi-backend support. See the `tests/working/` directory for examples of tested functionality.
For detailed contribution guidelines, see:
- **CONTRIBUTING.md**: Development setup, Docker instructions, and contribution workflow
- **docs/README.md**: Documentation contribution guidelines
## 🙏 Acknowledgments
QuantumEngine draws significant inspiration from [MongoEngine](https://github.com/MongoEngine/mongoengine), whose elegant document-oriented design patterns and query API have influenced our multi-backend approach. We're grateful to the MongoEngine community for pioneering many of the concepts that make QuantumEngine possible.
## 📄 License
MIT License - see LICENSE file for details.
---
**QuantumEngine**: Unified database access for modern Python applications with comprehensive multi-backend support, schema management, and performance optimizations.
Raw data
{
"_id": null,
"home_page": null,
"name": "quantumengine",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Iristech Systems <contact@iristech.systems>",
"keywords": "orm, odm, clickhouse, surrealdb, database, async, multi-backend, analytics, graph-database, quantum, engine",
"author": null,
"author_email": "Iristech Systems <contact@iristech.systems>",
"download_url": "https://files.pythonhosted.org/packages/40/c3/a49d48f15b7f631dbd393025be8e4821e8598e3964ef5e0925bc1ad62e7f/quantumengine-0.1.2.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n \n# \u269b\ufe0f QuantumEngine \u26a1\n \n <p>\n <strong>A powerful, multi-backend Object-Document Mapper (ODM) for Python</strong>\n </p>\n \n <p>\n Unified API for both transactional and analytical databases<br>\n Supporting SurrealDB (graph/document) and ClickHouse (columnar analytical) with a single, consistent interface\n </p>\n \n <p>\n <a href=\"https://pypi.org/project/quantumengine/\"><img src=\"https://img.shields.io/pypi/v/quantumengine.svg\" alt=\"PyPI version\"></a>\n <a href=\"https://pypi.org/project/quantumengine/\"><img src=\"https://img.shields.io/pypi/pyversions/quantumengine.svg\" alt=\"Python versions\"></a>\n <a href=\"https://github.com/iristech-systems/QuantumEngine/blob/main/LICENSE\"><img src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"License\"></a>\n <a href=\"https://github.com/iristech-systems/QuantumEngine/actions\"><img src=\"https://github.com/iristech-systems/QuantumEngine/workflows/Tests/badge.svg\" alt=\"Tests\"></a>\n <a href=\"https://pypi.org/project/quantumengine/\"><img src=\"https://img.shields.io/pypi/dm/quantumengine.svg\" alt=\"Downloads\"></a>\n </p>\n</div>\n\n---\n\n## \ud83d\udce6 Installation\n\nQuantumEngine uses a **modular installation system** - install only the backends you need:\n\n```bash\n# Core package only (lightweight)\npip install quantumengine\n\n# With ClickHouse support\npip install quantumengine[clickhouse]\n\n# With SurrealDB support \npip install quantumengine[surrealdb]\n\n# With both backends\npip install quantumengine[clickhouse,surrealdb]\n\n# Everything (all backends + dev tools)\npip install quantumengine[all]\n```\n\nSee [INSTALLATION.md](INSTALLATION.md) for detailed installation options and troubleshooting.\n\n## \ud83d\ude80 Quick Start\n\n```python\nimport os\nfrom quantumengine import Document, StringField, IntField, create_connection\n\n# Define a document model\nclass User(Document):\n username = StringField(required=True)\n email = StringField(required=True)\n age = IntField(min_value=0)\n \n class Meta:\n collection = \"users\"\n backend = \"surrealdb\" # or \"clickhouse\"\n\n# Create connection\nconnection = create_connection(\n url=\"ws://localhost:8000/rpc\",\n namespace=\"test_ns\",\n database=\"test_db\",\n username=os.environ.get(\"SURREALDB_USER\"),\n password=os.environ.get(\"SURREALDB_PASS\"),\n make_default=True\n)\n\nawait connection.connect()\n\n# Create table\nawait User.create_table()\n\n# CRUD operations\nuser = User(username=\"alice\", email=\"alice@example.com\", age=25)\nawait user.save()\n\nusers = await User.objects.filter(age__gt=18).all()\nawait User.objects.filter(username=\"alice\").update(age=26)\n```\n\n## \ud83c\udfd7\ufe0f Architecture\n\nQuantumORM provides a unified interface for multiple database backends:\n\n- **SurrealDB Backend**: Graph database with native support for relations, transactions, and complex queries\n- **ClickHouse Backend**: High-performance columnar database optimized for analytics and time-series data\n\n### Multi-Backend Support\n\n```python\n# SurrealDB connection for transactional data\nuser_connection = create_connection(\n name=\"surrealdb_main\",\n url=\"ws://localhost:8000/rpc\",\n backend=\"surrealdb\",\n make_default=True\n)\n\n# ClickHouse connection for analytics\nanalytics_connection = create_connection(\n name=\"clickhouse_analytics\", \n url=\"host.clickhouse.cloud\",\n backend=\"clickhouse\",\n port=443,\n secure=True\n)\n\nclass User(Document):\n class Meta:\n backend = \"surrealdb\"\n\nclass AnalyticsEvent(Document):\n class Meta:\n backend = \"clickhouse\"\n```\n\n## \ud83d\udccb Features\n\n### \u2705 Core Features\n- **Multi-Backend Architecture**: SurrealDB + ClickHouse support\n- **Type-Safe Field System**: 15+ field types with validation\n- **Query System**: Q objects, QueryExpressions, and advanced filtering\n- **Relationship Management**: Graph relations and references\n- **Schema Management**: Both SCHEMAFULL and SCHEMALESS table support\n- **Async/Sync APIs**: Complete async/await support with sync alternatives\n- **Connection Management**: Named connections and connection pooling\n- **Performance Optimization**: Direct record access and bulk operations\n- **Drop Table & Migrations**: Schema migration tools and table management\n\n### \ud83d\udd27 Field Types\n\n| Field Type | Description | SurrealDB | ClickHouse |\n|------------|-------------|-----------|------------|\n| `StringField` | Text fields with validation | \u2705 | \u2705 |\n| `IntField` | Integer with min/max constraints | \u2705 | \u2705 |\n| `FloatField` | Floating point numbers | \u2705 | \u2705 |\n| `BooleanField` | Boolean values | \u2705 | \u2705 |\n| `DateTimeField` | Date and time with timezone | \u2705 | \u2705 |\n| `DecimalField` | High-precision decimals | \u2705 | \u2705 |\n| `UUIDField` | UUID generation and validation | \u2705 | \u2705 |\n| `ListField` | Arrays/lists with typed elements | \u2705 | \u2705 |\n| `DictField` | JSON/dictionary storage | \u2705 | \u2705 |\n| `ReferenceField` | References to other documents | \u2705 | \u274c |\n| `IPAddressField` | IPv4/IPv6 address validation | \u2705 | \u2705 |\n| `DurationField` | Time periods and durations | \u2705 | \u274c |\n| `RangeField` | Range values with bounds | \u2705 | \u274c |\n| `OptionField` | Optional field wrapper | \u2705 | \u274c |\n| `RecordIDField` | SurrealDB record identifiers | \u2705 | \u274c |\n\n### \ud83d\udd0d Query Capabilities\n\n#### Basic Filtering\n```python\n# Simple filters\nusers = await User.objects.filter(age__gt=18).all()\nusers = await User.objects.filter(username__contains=\"admin\").all()\nusers = await User.objects.filter(active=True).all()\n\n# Count\ncount = await User.objects.filter(age__gte=21).count()\n```\n\n#### Q Objects for Complex Queries\n```python\nfrom quantumengine import Q\n\n# Combine conditions\ncomplex_query = Q(age__gte=18) & Q(age__lte=65) & Q(active=True)\nusers = await User.objects.filter(complex_query).all()\n\n# OR conditions\neither_query = Q(role=\"admin\") | Q(permissions__contains=\"admin\")\nusers = await User.objects.filter(either_query).all()\n\n# NOT conditions\nnot_query = ~Q(status=\"banned\")\nusers = await User.objects.filter(not_query).all()\n\n# Raw queries\nraw_query = Q.raw(\"age > 25 AND string::contains(username, 'admin')\")\nusers = await User.objects.filter(raw_query).all()\n```\n\n#### QueryExpressions with FETCH\n```python\n# Fetch related documents (SurrealDB)\nexpr = QueryExpression(where=Q(published=True)).fetch(\"author\")\nposts = await Post.objects.filter(expr).all()\n\n# Complex expressions\ncomplex_expr = (QueryExpression(where=Q(active=True))\n .order_by(\"created_at\", \"DESC\")\n .limit(10)\n .fetch(\"profile\"))\nusers = await User.objects.filter(complex_expr).all()\n```\n\n### \ud83d\udd17 Relationships (SurrealDB)\n\n#### Document References\n```python\nclass Post(Document):\n title = StringField(required=True)\n author = ReferenceField(User, required=True)\n categories = ListField(field_type=ReferenceField(Category))\n \n class Meta:\n collection = \"posts\"\n backend = \"surrealdb\"\n```\n\n#### Graph Relations\n```python\n# Create relations between documents\nawait author1.relate_to(\"collaborated_with\", author2, project=\"Novel\")\n\n# Fetch relations\ncollaborators = await author1.fetch_relation(\"collaborated_with\")\n\n# Resolve relations (get related documents)\nrelated_authors = await author1.resolve_relation(\"collaborated_with\")\n```\n\n#### Relation Documents\n```python\nclass AuthorCollaboration(RelationDocument):\n project_name = StringField(required=True)\n start_date = DateTimeField()\n contribution_percent = FloatField()\n \n class Meta:\n collection = \"collaborated_with\"\n\n# Create relation with metadata\nrelation = await AuthorCollaboration.create_relation(\n author1, author2,\n project_name=\"Science Fiction Novel\",\n start_date=datetime.now(),\n contribution_percent=60.0\n)\n```\n\n### \ud83d\udcca Schema Management\n\n#### Table Creation\n```python\n# SCHEMAFULL tables (strict schema)\nawait User.create_table(schemafull=True)\n\n# SCHEMALESS tables (flexible schema) \nawait User.create_table(schemafull=False)\n\n# Backend-specific table creation\nawait analytics_backend.create_table(\n AnalyticsEvent,\n engine=\"MergeTree\",\n order_by=\"(event_time, user_id)\"\n)\n```\n\n#### Drop Tables & Migration Support\n```python\nfrom quantumengine import (\n generate_drop_statements, generate_migration_statements,\n drop_tables_from_module\n)\n\n# Drop table functionality\nawait User.drop_table(if_exists=True)\nUser.drop_table_sync(if_exists=True)\n\n# Generate drop statements for migration scripts\ndrop_statements = generate_drop_statements(User)\n# ['REMOVE INDEX IF EXISTS idx_email ON users;', \n# 'REMOVE FIELD IF EXISTS email ON users;', ...]\n\n# Generate migration between document versions\nmigration = generate_migration_statements(UserV1, UserV2, schemafull=True)\nprint(migration['up']) # Forward migration statements\nprint(migration['down']) # Rollback migration statements\n\n# Drop all tables in a module\nawait drop_tables_from_module('myapp.models')\n```\n\n#### Hybrid Schema Support\n```python\nclass Product(Document):\n # Always defined in schema\n name = StringField(required=True, define_schema=True)\n price = FloatField(define_schema=True)\n \n # Only defined in SCHEMAFULL tables\n description = StringField()\n metadata = DictField()\n \n class Meta:\n collection = \"products\"\n```\n\n#### Index Management\n```python\nclass User(Document):\n username = StringField(required=True)\n email = StringField(required=True)\n \n class Meta:\n collection = \"users\"\n indexes = [\n {\"name\": \"user_username_idx\", \"fields\": [\"username\"], \"unique\": True},\n {\"name\": \"user_email_idx\", \"fields\": [\"email\"], \"unique\": True},\n {\"name\": \"user_age_idx\", \"fields\": [\"age\"]}\n ]\n\n# Create indexes\nawait User.create_indexes()\n```\n\n### \u26a1 Performance Features\n\n#### Direct Record Access\n```python\n# Optimized ID-based queries use direct record access\nusers = await User.objects.filter(id__in=['user:1', 'user:2']).all()\n\n# Convenience methods for ID operations\nusers = await User.objects.get_many([1, 2, 3]).all()\nusers = await User.objects.get_range(100, 200, inclusive=True).all()\n```\n\n#### Query Analysis\n```python\n# Explain query execution plan\nplan = await User.objects.filter(age__gt=25).explain()\n\n# Get index suggestions\nsuggestions = User.objects.filter(age__lt=30).suggest_indexes()\n```\n\n#### Bulk Operations\n```python\n# Bulk updates\nupdated = await User.objects.filter(active=False).update(status=\"inactive\")\n\n# Bulk deletes\ndeleted_count = await User.objects.filter(last_login__lt=cutoff_date).delete()\n```\n\n### \ud83d\udd04 Sync API Support\n\n```python\n# Create sync connection\nconnection = create_connection(\n url=\"ws://localhost:8000/rpc\",\n namespace=\"test_ns\",\n database=\"test_db\", \n username=os.environ.get(\"SURREALDB_USER\"),\n password=os.environ.get(\"SURREALDB_PASS\"),\n async_mode=False\n)\n\nwith connection:\n # Synchronous operations\n User.create_table_sync(schemafull=True)\n \n user = User(username=\"alice\", email=\"alice@example.com\")\n user.save_sync()\n \n users = User.objects.all_sync()\n user = User.objects.get_sync(id=user_id)\n user.delete_sync()\n```\n\n### \ud83d\udcc8 DataGrid Helpers\n\n```python\nfrom quantumengine import get_grid_data, parse_datatables_params\n\n# Efficient grid operations for web interfaces\nresult = await get_grid_data(\n User, # Document class\n request_args, # Request parameters\n search_fields=['username', 'email'],\n custom_filters={'active': 'active'},\n default_sort='created_at'\n)\n\n# DataTables integration\nparams = parse_datatables_params(request_args)\nresult = format_datatables_response(total, rows, draw)\n```\n\n## \ud83d\udce6 Installation\n\n```bash\npip install quantumengine\n\n# For SurrealDB support\npip install surrealdb\n\n# For ClickHouse support \npip install clickhouse-connect\n```\n\n## \ud83d\udd27 Configuration\n\n### Environment Variables\n```python\nimport os\nfrom quantumengine import create_connection\n\n# Using environment variables\nconnection = create_connection(\n url=os.environ.get(\"SURREALDB_URL\", \"ws://localhost:8000/rpc\"),\n namespace=os.environ.get(\"SURREALDB_NS\", \"production\"),\n database=os.environ.get(\"SURREALDB_DB\", \"main\"),\n username=os.environ.get(\"SURREALDB_USER\"),\n password=os.environ.get(\"SURREALDB_PASS\"),\n make_default=True\n)\n```\n\n### Multiple Named Connections\n```python\n# Main transactional database\nmain_db = create_connection(\n name=\"main_db\",\n url=\"ws://localhost:8000/rpc\",\n backend=\"surrealdb\",\n make_default=True\n)\n\n# Analytics database\nanalytics_db = create_connection(\n name=\"analytics_db\",\n url=\"https://analytics.clickhouse.cloud\",\n backend=\"clickhouse\"\n)\n\n# Use specific connection\nawait User.create_table(connection=main_db)\nawait AnalyticsEvent.create_table(connection=analytics_db)\n```\n\n## \ud83c\udfc3\u200d\u2642\ufe0f Quick Examples\n\n### Basic CRUD\n```python\n# Create\nuser = User(username=\"alice\", email=\"alice@example.com\", age=25)\nawait user.save()\n\n# Read\nuser = await User.objects.get(username=\"alice\")\nusers = await User.objects.filter(age__gte=18).all()\n\n# Update\nuser.age = 26\nawait user.save()\n\n# Delete\nawait user.delete()\n```\n\n### Advanced Queries\n```python\nfrom quantumengine import Q, QueryExpression\n\n# Complex filtering\nactive_adults = await User.objects.filter(\n Q(age__gte=18) & Q(active=True)\n).all()\n\n# Fetch relations\nposts_with_authors = await Post.objects.filter(\n QueryExpression(where=Q(published=True)).fetch(\"author\")\n).all()\n\n# Pagination and sorting\nrecent_users = await User.objects.filter(\n active=True\n).order_by(\"-created_at\").limit(10).all()\n```\n\n### Multi-Backend Usage\n```python\n# User data in SurrealDB (transactional)\nclass User(Document):\n username = StringField(required=True)\n email = StringField(required=True)\n \n class Meta:\n collection = \"users\"\n backend = \"surrealdb\"\n\n# Analytics events in ClickHouse (analytical)\nclass PageView(Document):\n user_id = StringField(required=True)\n page_url = StringField(required=True)\n timestamp = DateTimeField(required=True)\n \n class Meta:\n collection = \"page_views\"\n backend = \"clickhouse\"\n\n# Use both seamlessly\nuser = await User.objects.get(username=\"alice\")\npage_views = await PageView.objects.filter(user_id=str(user.id)).all()\n```\n\n### Schema Management Examples\n```python\n# Generate schema statements\nfrom quantumengine import generate_schema_statements, generate_drop_statements\n\n# Create schema\nschema_statements = generate_schema_statements(User, schemafull=True)\nfor stmt in schema_statements:\n print(stmt)\n\n# Drop schema\ndrop_statements = generate_drop_statements(User)\nfor stmt in drop_statements:\n print(stmt)\n\n# Generate migration between versions\nfrom quantumengine import generate_migration_statements\n\nclass UserV1(Document):\n username = StringField(required=True)\n email = StringField(required=True)\n\nclass UserV2(Document):\n username = StringField(required=True)\n email = StringField(required=True)\n active = BooleanField(default=True) # New field\n\nmigration = generate_migration_statements(UserV1, UserV2)\nprint(\"UP migration:\")\nfor stmt in migration['up']:\n print(f\" {stmt}\")\n\nprint(\"DOWN migration:\")\nfor stmt in migration['down']:\n print(f\" {stmt}\")\n```\n\n## \ud83e\uddea Testing\n\nThe codebase includes comprehensive tests demonstrating real database operations:\n\n```bash\n# Run working tests\npython tests/working/test_multi_backend_real_connections.py\npython tests/working/test_clickhouse_simple_working.py\npython tests/working/test_working_surrealdb_backend.py\n\n# Run working examples\npython example_scripts/working/basic_crud_example.py\npython example_scripts/working/multi_backend_example.py\npython example_scripts/working/advanced_features_example.py\n```\n\n## \ud83d\udcda Examples\n\nThe `example_scripts/working/` directory contains fully functional examples:\n\n- **basic_crud_example.py**: Core CRUD operations\n- **advanced_features_example.py**: Complex field types and validation\n- **multi_backend_example.py**: Using SurrealDB and ClickHouse together\n- **relation_example.py**: Graph relations and RelationDocuments\n- **query_expressions_example.py**: Advanced querying with Q objects\n- **sync_api_example.py**: Synchronous API usage\n- **test_performance_optimizations.py**: Performance features and optimization\n- **test_drop_and_migration.py**: Drop table and migration functionality\n\n## \ud83d\udd04 Backend Capabilities\n\n### SurrealDB Backend Features\n- \u2705 Graph relations and traversal\n- \u2705 Transactions \n- \u2705 Direct record access\n- \u2705 Full-text search\n- \u2705 References between documents\n- \u2705 Complex data types (Duration, Range, Option)\n- \u2705 SCHEMAFULL and SCHEMALESS tables\n\n### ClickHouse Backend Features \n- \u2705 High-performance analytical queries\n- \u2705 Bulk operations optimization\n- \u2705 Time-series data handling\n- \u2705 Columnar storage benefits\n- \u2705 Aggregation functions\n- \u274c Graph relations (not applicable)\n- \u274c Transactions (limited support)\n\n### Backend Detection\n```python\n# Check backend capabilities\nif connection.backend.supports_graph_relations():\n await user.relate_to(\"follows\", other_user)\n\nif connection.backend.supports_bulk_operations():\n await Document.objects.filter(...).bulk_update(status=\"processed\")\n\n# Get backend-specific optimizations\noptimizations = connection.backend.get_optimized_methods()\nprint(optimizations)\n```\n\n## \u26a1 Performance Features\n\n### Automatic Query Optimizations\n- **Direct Record Access**: ID-based queries use `SELECT * FROM user:1, user:2`\n- **Range Access**: Range queries use `SELECT * FROM user:1..=100`\n- **Bulk Operations**: Optimized batch processing\n- **Index Utilization**: Automatic index suggestions\n\n### Measured Performance Improvements\n- **Direct Record Access**: Up to 3.4x faster than traditional WHERE clauses\n- **Bulk Operations**: Significant improvement for batch processing\n- **Memory Efficiency**: Reduced data transfer and memory usage\n\n## \ud83d\udcda Documentation\n\n### Online Documentation\n\n- **GitHub Pages**: [Full Sphinx Documentation](https://your-username.github.io/quantumengine/) - Complete API reference with examples\n- **API Reference**: Detailed class and method documentation\n- **Quick Start Guide**: Step-by-step getting started tutorial\n- **Module Documentation**: Auto-generated from source code docstrings\n\n### Local Documentation\n\n- **API Reference**: `API_REFERENCE.md` - Complete class and method documentation \n- **Examples**: `example_scripts/working/` - Working examples for all features\n- **Tests**: `tests/working/` - Test files demonstrating functionality\n- **Sphinx Docs**: `docs/` - Build locally with `cd docs && uv run make html`\n\n### Building Documentation\n\n```bash\n# Install dependencies\nuv sync\n\n# Build Sphinx documentation\ncd docs\nuv run make html\n\n# View locally\nopen docs/_build/html/index.html\n```\n\n## \ud83e\udd1d Contributing\n\nQuantumORM is actively developed with a focus on real-world usage and multi-backend support. See the `tests/working/` directory for examples of tested functionality.\n\nFor detailed contribution guidelines, see:\n- **CONTRIBUTING.md**: Development setup, Docker instructions, and contribution workflow\n- **docs/README.md**: Documentation contribution guidelines\n\n## \ud83d\ude4f Acknowledgments\n\nQuantumEngine draws significant inspiration from [MongoEngine](https://github.com/MongoEngine/mongoengine), whose elegant document-oriented design patterns and query API have influenced our multi-backend approach. We're grateful to the MongoEngine community for pioneering many of the concepts that make QuantumEngine possible.\n\n## \ud83d\udcc4 License\n\nMIT License - see LICENSE file for details.\n\n---\n\n**QuantumEngine**: Unified database access for modern Python applications with comprehensive multi-backend support, schema management, and performance optimizations.\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Multi-Backend Object-Document Mapper (ODM) for ClickHouse, SurrealDB, and more - Quantum-Powered Database Engine",
"version": "0.1.2",
"project_urls": {
"Bug Tracker": "https://github.com/iristech-systems/QuantumEngine/issues",
"Changelog": "https://github.com/iristech-systems/QuantumEngine/releases",
"Documentation": "https://github.com/iristech-systems/QuantumEngine#readme",
"Download": "https://pypi.org/project/quantumengine/",
"Homepage": "https://github.com/iristech-systems/QuantumEngine",
"Repository": "https://github.com/iristech-systems/QuantumEngine"
},
"split_keywords": [
"orm",
" odm",
" clickhouse",
" surrealdb",
" database",
" async",
" multi-backend",
" analytics",
" graph-database",
" quantum",
" engine"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6b7221b3461119ef2c0a1941a19956d7e9a9ff5f138583b2dfb599ac8d0e46b4",
"md5": "4e87571a2437e6b3e477b54ba7bc7067",
"sha256": "3b2c52a876c95b71d5dac3f400a1bf2380a547c077326aa4d75c4c2b56551ac0"
},
"downloads": -1,
"filename": "quantumengine-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4e87571a2437e6b3e477b54ba7bc7067",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 134830,
"upload_time": "2025-07-13T11:52:04",
"upload_time_iso_8601": "2025-07-13T11:52:04.560573Z",
"url": "https://files.pythonhosted.org/packages/6b/72/21b3461119ef2c0a1941a19956d7e9a9ff5f138583b2dfb599ac8d0e46b4/quantumengine-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "40c3a49d48f15b7f631dbd393025be8e4821e8598e3964ef5e0925bc1ad62e7f",
"md5": "36d87d8eb79c592f82ee12d1c2e628c1",
"sha256": "2257997d23d4be46f738c18b3c8b17a6a21a99f13bc7cdc4203c136e6d51360f"
},
"downloads": -1,
"filename": "quantumengine-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "36d87d8eb79c592f82ee12d1c2e628c1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 131626,
"upload_time": "2025-07-13T11:52:05",
"upload_time_iso_8601": "2025-07-13T11:52:05.766013Z",
"url": "https://files.pythonhosted.org/packages/40/c3/a49d48f15b7f631dbd393025be8e4821e8598e3964ef5e0925bc1ad62e7f/quantumengine-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-13 11:52:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "iristech-systems",
"github_project": "QuantumEngine",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "quantumengine"
}
JSON
