# Sombra - A Graph Database in Rust
Sombra is a file-based graph database inspired by SQLite's single-file architecture. Built in Rust with a focus on correctness, performance, and ACID transaction support.
## Features
### Core Features
- **Property Graph Model**: Nodes, edges, and flexible properties
- **Single File Storage**: SQLite-style database files
- **ACID Transactions**: Full transactional support with rollback
- **Write-Ahead Logging**: Crash-safe operations
- **Page-Based Storage**: Efficient memory-mapped I/O
### Performance Features ✨ NEW
- **Label Index**: O(1) label-based queries (500-1000x faster)
- **LRU Node Cache**: 2000x faster repeated reads with 90% hit rate
- **B-tree Primary Index**: 25-40% memory reduction, better cache locality
- **Optimized BFS Traversal**: 2,500% faster graph traversals vs SQLite
- **Performance Metrics**: Real-time monitoring of cache, queries, and traversals
- **Scalability Testing**: Validated for 100K+ node graphs
### Language Support
- **Rust API**: Core library with full feature support
- **TypeScript/Node.js API**: Complete NAPI bindings for JavaScript/TypeScript
- **Python API**: PyO3 bindings with native performance (build with `maturin -F python`)
- **Cross-Platform**: Linux, macOS, and Windows support
### Testing & Quality
- **39 Comprehensive Tests**: Unit, integration, and stress tests
- **Production Ready**: Zero breaking changes, automatic migration
- **Benchmark Suite**: Performance regression testing
## Quick Start
### Rust API
```rust
use sombra::prelude::*;
// Open or create a database
let mut db = GraphDB::open("my_graph.db")?;
// Use transactions for safe operations
let mut tx = db.begin_transaction()?;
// Add nodes and edges
let user = tx.add_node(Node::new(0))?;
let post = tx.add_node(Node::new(1))?;
tx.add_edge(Edge::new(user, post, "AUTHORED"))?;
// Commit to make changes permanent
tx.commit()?;
// Query the graph
let neighbors = db.get_neighbors(user)?;
println!("User {} authored {} posts", user, neighbors.len());
```
### TypeScript/Node.js API
```typescript
import { SombraDB, SombraPropertyValue } from 'sombradb';
const db = new SombraDB('./my_graph.db');
const createProp = (type: 'string' | 'int' | 'float' | 'bool', value: any): SombraPropertyValue => ({
type,
value
});
const alice = db.addNode(['Person'], {
name: createProp('string', 'Alice'),
age: createProp('int', 30)
});
const bob = db.addNode(['Person'], {
name: createProp('string', 'Bob'),
age: createProp('int', 25)
});
const knows = db.addEdge(alice, bob, 'KNOWS', {
since: createProp('int', 2020)
});
const aliceNode = db.getNode(alice);
console.log('Alice:', aliceNode);
const neighbors = db.getNeighbors(alice);
console.log(`Alice has ${neighbors.length} connections`);
const bfsResults = db.bfsTraversal(alice, 3);
console.log('BFS traversal:', bfsResults);
const tx = db.beginTransaction();
try {
const charlie = tx.addNode(['Person'], {
name: createProp('string', 'Charlie')
});
tx.addEdge(alice, charlie, 'KNOWS');
tx.commit();
} catch (error) {
tx.rollback();
throw error;
}
db.flush();
db.checkpoint();
```
### Python API
```python
from sombra import SombraDB
db = SombraDB("./my_graph.db")
alice = db.add_node(["Person"], {"name": "Alice", "age": 30})
bob = db.add_node(["Person"], {"name": "Bob", "age": 25})
db.add_edge(alice, bob, "KNOWS", {"since": 2020})
node = db.get_node(alice)
print(f"Alice -> {node.labels}, properties={node.properties}")
neighbors = db.get_neighbors(alice)
print(f"Alice has {len(neighbors)} connections")
tx = db.begin_transaction()
try:
charlie = tx.add_node(["Person"], {"name": "Charlie"})
tx.add_edge(alice, charlie, "KNOWS")
tx.commit()
except Exception:
tx.rollback()
raise
```
## Installation
### Rust
```bash
cargo add sombra
```
### TypeScript/Node.js
```bash
npm install sombra
```
### Python
```bash
# Install from PyPI (coming soon)
pip install sombra
# Or build from source
pip install maturin
maturin build --release -F python
pip install target/wheels/sombra-*.whl
```
## Architecture
Sombra is built in layers:
1. **Storage Layer**: Page-based file storage with 8KB pages
2. **Pager Layer**: In-memory caching and dirty page tracking
3. **WAL Layer**: Write-ahead logging for crash safety
4. **Transaction Layer**: ACID transaction support
5. **Graph API**: High-level graph operations
6. **NAPI Bindings**: TypeScript/Node.js interface layer
## Documentation
### User Guides
- [Transactional Commit Layer](docs/transactional_commit_layer.md) - Complete user guide
- [Optimization API Guide](docs/optimization_api_guide.md) - Performance best practices
- [Performance Metrics](docs/performance_metrics.md) - Monitoring guide
- [Python Usage](docs/python_usage.md) - Building and calling the PyO3 bindings
### Technical Specifications
- [Transaction Design](docs/transactions.md) - Technical design specification
- [Data Model](docs/data_model.md) - Graph data structure details
- [B-tree Index Implementation](docs/btree_index_implementation.md) - Primary index details
- [Phase 1 Completion Report](docs/phase1_completion_report.md) - Optimization results
### Planning & Development
- [Lookup Optimization Plan](docs/lookup_optimization_plan.md) - Performance roadmap
- [Implementation Status](IMPLEMENTATION_STATUS.md) - Current progress
- [Roadmap](docs/roadmap.md) - Future development plans
- [Contributing](docs/contributing.md) - Development guidelines
## Testing
```bash
# Run all tests
cargo test
# Run transaction tests specifically
cargo test transactions
# Run smoke tests
cargo test smoke
# Run stress tests
cargo test stress
```
## Performance
### Phase 1 Optimizations ✅ COMPLETE
Sombra now includes production-ready performance optimizations:
| Optimization | Improvement | Status |
|--------------|-------------|--------|
| Label Index | 500-1000x faster queries | ✅ Complete |
| Node Cache | 2000x faster repeated reads | ✅ Complete |
| B-tree Index | 25-40% memory reduction | ✅ Complete |
| Metrics System | Real-time monitoring | ✅ Complete |
**Benchmark Results** (100K nodes):
```
Label Query: 390ms → 0.04ms (9,750x faster)
Cached Reads: 2-4µs → 45ns (2,000x faster)
BFS Traversal: 42 ops/sec → 1,092 ops/sec (2,500% faster)
Index Memory: 3.2MB → 2.4MB (25% reduction)
Cache Hit Rate: 0% → 90% (after warmup)
```
**Graph Traversal Performance** (vs SQLite):
- Medium Dataset: 7,778 ops/sec vs 452 ops/sec (18x faster)
- Large Dataset: 1,092 ops/sec vs 48 ops/sec (23x faster)
### Running Benchmarks
```bash
# Index performance comparison
cargo bench --bench index_benchmark --features benchmarks
# BFS traversal performance
cargo bench --bench small_read_benchmark --features benchmarks
# Scalability testing (50K-500K nodes)
cargo bench --bench scalability_benchmark --features benchmarks
# Performance metrics demo
cargo run --example performance_metrics_demo --features benchmarks
```
## Current Status
✅ **Phase 1 Complete** (Production Ready):
- Core graph operations (add/get nodes and edges)
- Page-based storage with B-tree indexing
- Write-ahead logging (WAL)
- ACID transactions with rollback
- Crash recovery
- Label secondary index
- LRU node cache
- Optimized BFS traversal (2,500% faster)
- Performance metrics system
- TypeScript/Node.js NAPI bindings
- Comprehensive test suite (39/39 passing)
🚧 **Phase 2 Planned** (Next 2-3 months):
- Adjacency indexing (5-10x traversal speedup)
- Property-based indexes
- Query planner with cost-based optimization
- Concurrent readers
🔮 **Phase 3 Future**:
- CSR representation for dense graphs
- Neighbor caching for hub nodes
- Path compression
- Custom B-tree implementation
## Examples
See the `tests/` directory for comprehensive examples:
- `tests/smoke.rs` - Basic usage patterns
- `tests/stress.rs` - Performance and scalability
- `tests/transactions.rs` - Transaction usage examples
## License
This project is open source. See [LICENSE](LICENSE) for details.
## Contributing
See [Contributing Guidelines](docs/contributing.md) for information on how to contribute to Sombra.
Raw data
{
"_id": null,
"home_page": null,
"name": "sombra",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "graph, database, graphdb, acid, transactions, rust, performance",
"author": null,
"author_email": "mask <maskdotdev@gmail.com>",
"download_url": null,
"platform": null,
"description": "# Sombra - A Graph Database in Rust\n\nSombra is a file-based graph database inspired by SQLite's single-file architecture. Built in Rust with a focus on correctness, performance, and ACID transaction support.\n\n## Features\n\n### Core Features\n- **Property Graph Model**: Nodes, edges, and flexible properties\n- **Single File Storage**: SQLite-style database files\n- **ACID Transactions**: Full transactional support with rollback\n- **Write-Ahead Logging**: Crash-safe operations\n- **Page-Based Storage**: Efficient memory-mapped I/O\n\n### Performance Features \u2728 NEW\n- **Label Index**: O(1) label-based queries (500-1000x faster)\n- **LRU Node Cache**: 2000x faster repeated reads with 90% hit rate\n- **B-tree Primary Index**: 25-40% memory reduction, better cache locality\n- **Optimized BFS Traversal**: 2,500% faster graph traversals vs SQLite\n- **Performance Metrics**: Real-time monitoring of cache, queries, and traversals\n- **Scalability Testing**: Validated for 100K+ node graphs\n\n### Language Support\n- **Rust API**: Core library with full feature support\n- **TypeScript/Node.js API**: Complete NAPI bindings for JavaScript/TypeScript\n- **Python API**: PyO3 bindings with native performance (build with `maturin -F python`)\n- **Cross-Platform**: Linux, macOS, and Windows support\n\n### Testing & Quality\n- **39 Comprehensive Tests**: Unit, integration, and stress tests\n- **Production Ready**: Zero breaking changes, automatic migration\n- **Benchmark Suite**: Performance regression testing\n\n## Quick Start\n\n### Rust API\n\n```rust\nuse sombra::prelude::*;\n\n// Open or create a database\nlet mut db = GraphDB::open(\"my_graph.db\")?;\n\n// Use transactions for safe operations\nlet mut tx = db.begin_transaction()?;\n\n// Add nodes and edges\nlet user = tx.add_node(Node::new(0))?;\nlet post = tx.add_node(Node::new(1))?;\ntx.add_edge(Edge::new(user, post, \"AUTHORED\"))?;\n\n// Commit to make changes permanent\ntx.commit()?;\n\n// Query the graph\nlet neighbors = db.get_neighbors(user)?;\nprintln!(\"User {} authored {} posts\", user, neighbors.len());\n```\n\n### TypeScript/Node.js API\n\n```typescript\nimport { SombraDB, SombraPropertyValue } from 'sombradb';\n\nconst db = new SombraDB('./my_graph.db');\n\nconst createProp = (type: 'string' | 'int' | 'float' | 'bool', value: any): SombraPropertyValue => ({\n type,\n value\n});\n\nconst alice = db.addNode(['Person'], {\n name: createProp('string', 'Alice'),\n age: createProp('int', 30)\n});\n\nconst bob = db.addNode(['Person'], {\n name: createProp('string', 'Bob'),\n age: createProp('int', 25)\n});\n\nconst knows = db.addEdge(alice, bob, 'KNOWS', {\n since: createProp('int', 2020)\n});\n\nconst aliceNode = db.getNode(alice);\nconsole.log('Alice:', aliceNode);\n\nconst neighbors = db.getNeighbors(alice);\nconsole.log(`Alice has ${neighbors.length} connections`);\n\nconst bfsResults = db.bfsTraversal(alice, 3);\nconsole.log('BFS traversal:', bfsResults);\n\nconst tx = db.beginTransaction();\ntry {\n const charlie = tx.addNode(['Person'], {\n name: createProp('string', 'Charlie')\n });\n tx.addEdge(alice, charlie, 'KNOWS');\n tx.commit();\n} catch (error) {\n tx.rollback();\n throw error;\n}\n\ndb.flush();\ndb.checkpoint();\n```\n\n### Python API\n\n```python\nfrom sombra import SombraDB\n\ndb = SombraDB(\"./my_graph.db\")\n\nalice = db.add_node([\"Person\"], {\"name\": \"Alice\", \"age\": 30})\nbob = db.add_node([\"Person\"], {\"name\": \"Bob\", \"age\": 25})\n\ndb.add_edge(alice, bob, \"KNOWS\", {\"since\": 2020})\n\nnode = db.get_node(alice)\nprint(f\"Alice -> {node.labels}, properties={node.properties}\")\n\nneighbors = db.get_neighbors(alice)\nprint(f\"Alice has {len(neighbors)} connections\")\n\ntx = db.begin_transaction()\ntry:\n charlie = tx.add_node([\"Person\"], {\"name\": \"Charlie\"})\n tx.add_edge(alice, charlie, \"KNOWS\")\n tx.commit()\nexcept Exception:\n tx.rollback()\n raise\n```\n\n## Installation\n\n### Rust\n```bash\ncargo add sombra\n```\n\n### TypeScript/Node.js\n```bash\nnpm install sombra\n```\n\n### Python\n```bash\n# Install from PyPI (coming soon)\npip install sombra\n\n# Or build from source\npip install maturin\nmaturin build --release -F python\npip install target/wheels/sombra-*.whl\n```\n\n## Architecture\n\nSombra is built in layers:\n\n1. **Storage Layer**: Page-based file storage with 8KB pages\n2. **Pager Layer**: In-memory caching and dirty page tracking\n3. **WAL Layer**: Write-ahead logging for crash safety\n4. **Transaction Layer**: ACID transaction support\n5. **Graph API**: High-level graph operations\n6. **NAPI Bindings**: TypeScript/Node.js interface layer\n\n## Documentation\n\n### User Guides\n- [Transactional Commit Layer](docs/transactional_commit_layer.md) - Complete user guide\n- [Optimization API Guide](docs/optimization_api_guide.md) - Performance best practices\n- [Performance Metrics](docs/performance_metrics.md) - Monitoring guide\n- [Python Usage](docs/python_usage.md) - Building and calling the PyO3 bindings\n\n### Technical Specifications\n- [Transaction Design](docs/transactions.md) - Technical design specification\n- [Data Model](docs/data_model.md) - Graph data structure details\n- [B-tree Index Implementation](docs/btree_index_implementation.md) - Primary index details\n- [Phase 1 Completion Report](docs/phase1_completion_report.md) - Optimization results\n\n### Planning & Development\n- [Lookup Optimization Plan](docs/lookup_optimization_plan.md) - Performance roadmap\n- [Implementation Status](IMPLEMENTATION_STATUS.md) - Current progress\n- [Roadmap](docs/roadmap.md) - Future development plans\n- [Contributing](docs/contributing.md) - Development guidelines\n\n## Testing\n\n```bash\n# Run all tests\ncargo test\n\n# Run transaction tests specifically\ncargo test transactions\n\n# Run smoke tests\ncargo test smoke\n\n# Run stress tests\ncargo test stress\n```\n\n## Performance\n\n### Phase 1 Optimizations \u2705 COMPLETE\n\nSombra now includes production-ready performance optimizations:\n\n| Optimization | Improvement | Status |\n|--------------|-------------|--------|\n| Label Index | 500-1000x faster queries | \u2705 Complete |\n| Node Cache | 2000x faster repeated reads | \u2705 Complete |\n| B-tree Index | 25-40% memory reduction | \u2705 Complete |\n| Metrics System | Real-time monitoring | \u2705 Complete |\n\n**Benchmark Results** (100K nodes):\n```\nLabel Query: 390ms \u2192 0.04ms (9,750x faster)\nCached Reads: 2-4\u00b5s \u2192 45ns (2,000x faster)\nBFS Traversal: 42 ops/sec \u2192 1,092 ops/sec (2,500% faster)\nIndex Memory: 3.2MB \u2192 2.4MB (25% reduction)\nCache Hit Rate: 0% \u2192 90% (after warmup)\n```\n\n**Graph Traversal Performance** (vs SQLite):\n- Medium Dataset: 7,778 ops/sec vs 452 ops/sec (18x faster)\n- Large Dataset: 1,092 ops/sec vs 48 ops/sec (23x faster)\n\n### Running Benchmarks\n\n```bash\n# Index performance comparison\ncargo bench --bench index_benchmark --features benchmarks\n\n# BFS traversal performance\ncargo bench --bench small_read_benchmark --features benchmarks\n\n# Scalability testing (50K-500K nodes)\ncargo bench --bench scalability_benchmark --features benchmarks\n\n# Performance metrics demo\ncargo run --example performance_metrics_demo --features benchmarks\n```\n\n## Current Status\n\n\u2705 **Phase 1 Complete** (Production Ready):\n- Core graph operations (add/get nodes and edges)\n- Page-based storage with B-tree indexing\n- Write-ahead logging (WAL)\n- ACID transactions with rollback\n- Crash recovery\n- Label secondary index\n- LRU node cache\n- Optimized BFS traversal (2,500% faster)\n- Performance metrics system\n- TypeScript/Node.js NAPI bindings\n- Comprehensive test suite (39/39 passing)\n\n\ud83d\udea7 **Phase 2 Planned** (Next 2-3 months):\n- Adjacency indexing (5-10x traversal speedup)\n- Property-based indexes\n- Query planner with cost-based optimization\n- Concurrent readers\n\n\ud83d\udd2e **Phase 3 Future**:\n- CSR representation for dense graphs\n- Neighbor caching for hub nodes\n- Path compression\n- Custom B-tree implementation\n\n## Examples\n\nSee the `tests/` directory for comprehensive examples:\n- `tests/smoke.rs` - Basic usage patterns\n- `tests/stress.rs` - Performance and scalability\n- `tests/transactions.rs` - Transaction usage examples\n\n## License\n\nThis project is open source. See [LICENSE](LICENSE) for details.\n\n## Contributing\n\nSee [Contributing Guidelines](docs/contributing.md) for information on how to contribute to Sombra.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A high-performance graph database with ACID transactions",
"version": "0.1.26",
"project_urls": {
"Homepage": "https://github.com/maskdotdev/sombra",
"Issues": "https://github.com/maskdotdev/sombra/issues",
"Repository": "https://github.com/maskdotdev/sombra"
},
"split_keywords": [
"graph",
" database",
" graphdb",
" acid",
" transactions",
" rust",
" performance"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "af1d361f35d103ea866b0c1de809afdc1309ffa907dca4631344310eb8608a24",
"md5": "88120ee8fbe1f6abad3a2b5f38512eee",
"sha256": "08b2619738da5d977ad92b4d41fa15cf5af2f0ebfd4fd23a6179a7f4741a671b"
},
"downloads": -1,
"filename": "sombra-0.1.26-cp311-cp311-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "88120ee8fbe1f6abad3a2b5f38512eee",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.8",
"size": 409369,
"upload_time": "2025-10-19T20:00:39",
"upload_time_iso_8601": "2025-10-19T20:00:39.114588Z",
"url": "https://files.pythonhosted.org/packages/af/1d/361f35d103ea866b0c1de809afdc1309ffa907dca4631344310eb8608a24/sombra-0.1.26-cp311-cp311-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a3218871be9cb0eec918c7014198205b576ad0c49b3387751928389ae2a7108b",
"md5": "7d6865ab00ea6041fc0bb293acd19d81",
"sha256": "095835b3d729a6f20cb0989cf4c28ccaf8a57cab5374c2202c79e76cbcc486e2"
},
"downloads": -1,
"filename": "sombra-0.1.26-cp311-cp311-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "7d6865ab00ea6041fc0bb293acd19d81",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.8",
"size": 470764,
"upload_time": "2025-10-19T20:00:40",
"upload_time_iso_8601": "2025-10-19T20:00:40.824299Z",
"url": "https://files.pythonhosted.org/packages/a3/21/8871be9cb0eec918c7014198205b576ad0c49b3387751928389ae2a7108b/sombra-0.1.26-cp311-cp311-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "aff5f0859e4ed75e6831f5e6c92090b8c3f70395c03855c4b97f9a44aff0bec6",
"md5": "52005b6facf104971e3c657388fe64da",
"sha256": "ed1b7a64340f7e2c2d0f7bf0067c39ea6cf500a20dd29279085e6a212a6f9eec"
},
"downloads": -1,
"filename": "sombra-0.1.26-cp311-cp311-win_amd64.whl",
"has_sig": false,
"md5_digest": "52005b6facf104971e3c657388fe64da",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.8",
"size": 308084,
"upload_time": "2025-10-19T20:00:41",
"upload_time_iso_8601": "2025-10-19T20:00:41.794194Z",
"url": "https://files.pythonhosted.org/packages/af/f5/f0859e4ed75e6831f5e6c92090b8c3f70395c03855c4b97f9a44aff0bec6/sombra-0.1.26-cp311-cp311-win_amd64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-19 20:00:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "maskdotdev",
"github_project": "sombra",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "sombra"
}
JSON
