# Meridian Runtime — A Minimal, Reusable Graph Runtime for Python
Owner: GhostWeasel (Lead: doubletap-dave)
[](https://github.com/GhostWeaselLabs/meridian-runtime/actions/workflows/ci.yml)
[](https://ghostweasellabs.github.io/meridian-runtime-docs/)
[](https://github.com/GhostWeaselLabs/meridian-runtime/actions/workflows/gh-pages.yml)
Meridian Runtime is a lightweight, framework-agnostic graph runtime for building real‑time dataflows in Python. Model your application as small, single‑responsibility nodes connected by typed edges with bounded queues. Meridian’s scheduler enforces backpressure, supports control‑plane priorities (e.g., kill switch), and emits rich observability signals by design.
Key features
- Nodes, edges, subgraphs, scheduler — simple, composable primitives
- Bounded edges with configurable overflow policies (block, drop, latest, coalesce)
- Control‑plane priority for critical flows (kill switch, admin, rate‑limit signals)
- First‑class observability (structured logs, metrics, trace hooks)
- Small‑file, SRP/DRY‑friendly codebase (aim for ~200 lines per file)
- uv‑native development workflow (fast, reproducible)
Use cases
- Real‑time trading systems (market data, execution, risk)
- Event processing pipelines and enrichment
- Streaming ETL and log processing
- Control planes with prioritized signals
---
## Versioning & Deprecation Policy
- This project follows Semantic Versioning starting with v1.0.0 for the public API.
- Public, stable surfaces are documented across Concepts and Reference in the docs site.
- Deprecations are announced in a MINOR release and remain available for at least one subsequent MINOR before removal.
- See:
- Versioning and policy overview: https://ghostweasellabs.github.io/meridian-runtime-docs/roadmap/release-v1.0.0/
- API Reference: https://ghostweasellabs.github.io/meridian-runtime-docs/reference/api/
- Concepts (Architecture, Patterns): https://ghostweasellabs.github.io/meridian-runtime-docs/concepts/architecture/
## Documentation
- Site: https://ghostweasellabs.github.io/meridian-runtime-docs/ — Deployed via GitHub Pages (source: GitHub Actions)
- Quickstart: https://ghostweasellabs.github.io/meridian-runtime-docs/getting-started/quickstart/
- API: https://ghostweasellabs.github.io/meridian-runtime-docs/reference/api/
- Patterns: https://ghostweasellabs.github.io/meridian-runtime-docs/concepts/patterns/
- Observability: https://ghostweasellabs.github.io/meridian-runtime-docs/concepts/observability/
- Troubleshooting: https://ghostweasellabs.github.io/meridian-runtime-docs/support/troubleshooting/
- **Interactive Notebooks**: [`notebooks/`](./notebooks/) — Jupyter notebooks for hands-on learning and experimentation
- Note: Analytics is enabled for the docs site; see mkdocs.yml for the tracking configuration.
## Quick start
Prereqs
- Python 3.11+
- uv (modern Python package manager)
Initialize environment
```bash
uv lock
uv sync
```
2) Dev loop
```
# Lint
uv run ruff check .
# Format check
uv run black --check .
# Type-check
uv run mypy src
# Tests with coverage
uv run pytest
```
3) Run an example
```
uv run python -m examples.hello_graph.main
```
4) Try interactive notebooks
```bash
# Install notebook dependencies
uv sync --extra notebooks
# Start Jupyter
uv run jupyter lab
# Navigate to notebooks/ directory and try:
# - tutorials/01-getting-started.ipynb
# - examples/hello-graph-interactive.ipynb
```
4) Project layout (M1 scaffold)
```
src/meridian/
__init__.py
core/
__init__.py
observability/
__init__.py
utils/
__init__.py
examples/
__init__.py
tests/
unit/
test_smoke.py
integration/
test_examples_smoke.py
pyproject.toml
ruff.toml
mypy.ini
.editorconfig
.github/workflows/ci.yml
```
---
## Core Concepts
Node
- Single‑responsibility processing unit with typed input/output ports
- Lifecycle hooks: on_start, on_message, on_tick, on_stop
- Emits Messages on output ports
Edge
- Typed, bounded queue connecting node ports
- Overflow policies: block (default), drop, latest, coalesce(fn)
- Exposes queue depth, throughput, and backpressure metrics
Subgraph
- Composition of nodes into a reusable unit
- Exposes its own typed inputs/outputs
- Validates internal wiring and contracts
Scheduler
- Advances nodes based on readiness, priorities, and capacity
- Drives ticks (timers/housekeeping), supports graceful shutdown
- Prioritizes control‑plane edges/messages
Message
- payload: Any (type tracked by PortSpec)
- headers: {trace_id, timestamp, schema_version, content_type, ...}
PortSpec
- name: str
- schema/type: Python types, TypedDict, or Pydantic models (pluggable)
- policy: overflow handling per edge (block/drop/latest/coalesce)
Observability
- Structured logs (JSON) with correlation IDs
- Metrics (Prometheus) for nodes/edges/scheduler
- Optional tracing (OpenTelemetry hooks)
---
## Minimal Example
producer.py
```python
from meridian.core import Node, Message, MessageType, Port, PortDirection, PortSpec
class Producer(Node):
def __init__(self, n=5):
super().__init__(
name="producer",
inputs=[],
outputs=[Port("out", PortDirection.OUTPUT, spec=PortSpec("out", int))]
)
self._n = n
self._i = 0
def on_start(self):
self._i = 0
def _handle_tick(self):
if self._i < self._n:
self.emit("out", Message(type=MessageType.DATA, payload=self._i))
self._i += 1
```
consumer.py
```python
from meridian.core import Node, Port, PortDirection, PortSpec
class Consumer(Node):
def __init__(self):
super().__init__(
name="consumer",
inputs=[Port("in", PortDirection.INPUT, spec=PortSpec("in", int))],
outputs=[]
)
def _handle_message(self, port, msg):
print(f"got: {msg.payload}")
```
main.py
```python
from meridian.core import Subgraph, Scheduler, SchedulerConfig
from producer import Producer
from consumer import Consumer
g = Subgraph(name="hello_world")
g.add_node(Producer(n=3))
g.add_node(Consumer())
g.connect(("producer", "out"), ("consumer", "in"), capacity=16)
sch = Scheduler(SchedulerConfig())
sch.register(g)
sch.run()
```
Run:
```
uv run python -m examples.hello_graph.main
```
---
## Patterns and Guidance
File size and modularity
- Target ~200 lines per file. Split responsibilities into multiple nodes or utilities.
- SRP and DRY: nodes do one thing; share common helpers in utils/.
- Prefer small subgraphs over monolith graphs for composition and reuse.
Backpressure and overflow
- Default policy: block (applies backpressure upstream).
- For sporadic spikes: latest policy can drop stale data in favor of the newest.
- For aggregations: coalesce can compress bursts (e.g., merge updates).
Priorities
- Assign higher priority to control‑plane edges (e.g., kill switch, cancel_all).
- Keep control messages small and fast; avoid heavy work in control nodes.
Message schemas
- Use precise Python types or TypedDicts for ports.
- Optionally integrate Pydantic for richer validation without coupling the runtime.
Error handling
- Prefer local handling in nodes; escalate via dead‑letter subgraph if needed.
- Use retries with jitter and circuit‑breaker patterns for external IO nodes.
---
## Observability
Logs
- JSON‑structured logs with timestamps and correlation IDs (trace_id)
- Node lifecycle events, exceptions, tick durations
Metrics (Prometheus)
- Node: tick latency, messages processed, errors
- Edge: queue depth, enqueue/dequeue rate, drops, backpressure time
- Scheduler: runnable nodes, loop latency, priority applications
Tracing (optional)
- Hook into OpenTelemetry: annotate message paths and node spans
- Keep tracing optional to avoid overhead where unnecessary
Dashboards and alerts
- Track queue depths and backpressure saturation
- Alert on sustained scheduler latency or starved nodes
- Monitor error rates per node and dropped messages per edge
---
## Roadmap
v1 (initial release)
- In‑process runtime, asyncio‑friendly
- Nodes, edges (bounded queues), subgraphs, scheduler (fair scheduling)
- Observability: logs, metrics, basic trace hooks
- Examples and scaffolding scripts
v1.x
- Schema adapters (Pydantic), richer overflow policies
- More node templates and utilities
- Improved testing harness and fixtures
v2+
- Multi‑process edges (shared memory/IPC)
- Distributed graphs (brokers + codecs)
- Visual tooling and hot‑reload for graphs
---
## Development (with uv)
Install and sync
```
uv init
uv lock
uv sync
```
Run tests
```
uv run pytest
```
Lint and type‑check (example)
```
uv run ruff check .
uv run mypy src
```
Run an example
```
uv run python -m examples.hello_graph.main
```
Contributing
- Keep files small (~200 lines) and responsibilities focused.
- Include unit tests for core changes; add integration tests for subgraph behavior.
- Update examples and docs for user‑facing features.
- Follow SemVer and add entries to CHANGELOG for notable changes.
License
- BSD 3-Clause (recommended) or your organization’s standard OSS license.
---
## FAQ
See our [FAQ page](https://ghostweasellabs.github.io/meridian-runtime/support/faq/) for answers to common questions about Meridian Runtime.
---
Happy building with Meridian Runtime.
Raw data
{
"_id": null,
"home_page": null,
"name": "meridian-runtime",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "graph, runtime, scheduler, asyncio, observability, python",
"author": null,
"author_email": "GhostWeasel <devnull@ghostweasel.net>",
"download_url": "https://files.pythonhosted.org/packages/42/f0/e2a51860c25f377cd40d3a8bfcaadfe83d54aa2787cbe347922138b27323/meridian_runtime-1.2.0.tar.gz",
"platform": null,
"description": "# Meridian Runtime \u2014 A Minimal, Reusable Graph Runtime for Python\n\nOwner: GhostWeasel (Lead: doubletap-dave)\n\n[](https://github.com/GhostWeaselLabs/meridian-runtime/actions/workflows/ci.yml)\n[](https://ghostweasellabs.github.io/meridian-runtime-docs/)\n[](https://github.com/GhostWeaselLabs/meridian-runtime/actions/workflows/gh-pages.yml)\n\nMeridian Runtime is a lightweight, framework-agnostic graph runtime for building real\u2011time dataflows in Python. Model your application as small, single\u2011responsibility nodes connected by typed edges with bounded queues. Meridian\u2019s scheduler enforces backpressure, supports control\u2011plane priorities (e.g., kill switch), and emits rich observability signals by design.\n\nKey features\n- Nodes, edges, subgraphs, scheduler \u2014 simple, composable primitives\n- Bounded edges with configurable overflow policies (block, drop, latest, coalesce)\n- Control\u2011plane priority for critical flows (kill switch, admin, rate\u2011limit signals)\n- First\u2011class observability (structured logs, metrics, trace hooks)\n- Small\u2011file, SRP/DRY\u2011friendly codebase (aim for ~200 lines per file)\n- uv\u2011native development workflow (fast, reproducible)\n\nUse cases\n- Real\u2011time trading systems (market data, execution, risk)\n- Event processing pipelines and enrichment\n- Streaming ETL and log processing\n- Control planes with prioritized signals\n\n---\n\n## Versioning & Deprecation Policy\n\n- This project follows Semantic Versioning starting with v1.0.0 for the public API.\n- Public, stable surfaces are documented across Concepts and Reference in the docs site.\n- Deprecations are announced in a MINOR release and remain available for at least one subsequent MINOR before removal.\n- See:\n - Versioning and policy overview: https://ghostweasellabs.github.io/meridian-runtime-docs/roadmap/release-v1.0.0/\n - API Reference: https://ghostweasellabs.github.io/meridian-runtime-docs/reference/api/\n - Concepts (Architecture, Patterns): https://ghostweasellabs.github.io/meridian-runtime-docs/concepts/architecture/\n\n## Documentation\n\n- Site: https://ghostweasellabs.github.io/meridian-runtime-docs/ \u2014 Deployed via GitHub Pages (source: GitHub Actions)\n- Quickstart: https://ghostweasellabs.github.io/meridian-runtime-docs/getting-started/quickstart/\n- API: https://ghostweasellabs.github.io/meridian-runtime-docs/reference/api/\n- Patterns: https://ghostweasellabs.github.io/meridian-runtime-docs/concepts/patterns/\n- Observability: https://ghostweasellabs.github.io/meridian-runtime-docs/concepts/observability/\n- Troubleshooting: https://ghostweasellabs.github.io/meridian-runtime-docs/support/troubleshooting/\n- **Interactive Notebooks**: [`notebooks/`](./notebooks/) \u2014 Jupyter notebooks for hands-on learning and experimentation\n- Note: Analytics is enabled for the docs site; see mkdocs.yml for the tracking configuration.\n\n\n\n\n## Quick start\n\nPrereqs\n- Python 3.11+\n- uv (modern Python package manager)\n\nInitialize environment\n```bash\nuv lock\nuv sync\n```\n\n2) Dev loop\n```\n# Lint\nuv run ruff check .\n\n# Format check\nuv run black --check .\n\n# Type-check\nuv run mypy src\n\n# Tests with coverage\nuv run pytest\n```\n\n3) Run an example\n```\nuv run python -m examples.hello_graph.main\n```\n\n4) Try interactive notebooks\n```bash\n# Install notebook dependencies\nuv sync --extra notebooks\n\n# Start Jupyter\nuv run jupyter lab\n\n# Navigate to notebooks/ directory and try:\n# - tutorials/01-getting-started.ipynb\n# - examples/hello-graph-interactive.ipynb\n```\n\n4) Project layout (M1 scaffold)\n```\nsrc/meridian/\n __init__.py\n core/\n __init__.py\n observability/\n __init__.py\n utils/\n __init__.py\nexamples/\n __init__.py\ntests/\n unit/\n test_smoke.py\n integration/\n test_examples_smoke.py\npyproject.toml\nruff.toml\nmypy.ini\n.editorconfig\n.github/workflows/ci.yml\n```\n\n---\n\n## Core Concepts\n\nNode\n- Single\u2011responsibility processing unit with typed input/output ports\n- Lifecycle hooks: on_start, on_message, on_tick, on_stop\n- Emits Messages on output ports\n\nEdge\n- Typed, bounded queue connecting node ports\n- Overflow policies: block (default), drop, latest, coalesce(fn)\n- Exposes queue depth, throughput, and backpressure metrics\n\nSubgraph\n- Composition of nodes into a reusable unit\n- Exposes its own typed inputs/outputs\n- Validates internal wiring and contracts\n\nScheduler\n- Advances nodes based on readiness, priorities, and capacity\n- Drives ticks (timers/housekeeping), supports graceful shutdown\n- Prioritizes control\u2011plane edges/messages\n\nMessage\n- payload: Any (type tracked by PortSpec)\n- headers: {trace_id, timestamp, schema_version, content_type, ...}\n\nPortSpec\n- name: str\n- schema/type: Python types, TypedDict, or Pydantic models (pluggable)\n- policy: overflow handling per edge (block/drop/latest/coalesce)\n\nObservability\n- Structured logs (JSON) with correlation IDs\n- Metrics (Prometheus) for nodes/edges/scheduler\n- Optional tracing (OpenTelemetry hooks)\n\n---\n\n## Minimal Example\n\nproducer.py\n```python\nfrom meridian.core import Node, Message, MessageType, Port, PortDirection, PortSpec\n\nclass Producer(Node):\n def __init__(self, n=5):\n super().__init__(\n name=\"producer\",\n inputs=[],\n outputs=[Port(\"out\", PortDirection.OUTPUT, spec=PortSpec(\"out\", int))]\n )\n self._n = n\n self._i = 0\n\n def on_start(self):\n self._i = 0\n\n def _handle_tick(self):\n if self._i < self._n:\n self.emit(\"out\", Message(type=MessageType.DATA, payload=self._i))\n self._i += 1\n```\n\nconsumer.py\n```python\nfrom meridian.core import Node, Port, PortDirection, PortSpec\n\nclass Consumer(Node):\n def __init__(self):\n super().__init__(\n name=\"consumer\",\n inputs=[Port(\"in\", PortDirection.INPUT, spec=PortSpec(\"in\", int))],\n outputs=[]\n )\n\n def _handle_message(self, port, msg):\n print(f\"got: {msg.payload}\")\n```\n\nmain.py\n```python\nfrom meridian.core import Subgraph, Scheduler, SchedulerConfig\nfrom producer import Producer\nfrom consumer import Consumer\n\ng = Subgraph(name=\"hello_world\")\ng.add_node(Producer(n=3))\ng.add_node(Consumer())\ng.connect((\"producer\", \"out\"), (\"consumer\", \"in\"), capacity=16)\n\nsch = Scheduler(SchedulerConfig())\nsch.register(g)\nsch.run()\n```\n\nRun:\n```\nuv run python -m examples.hello_graph.main\n```\n\n---\n\n## Patterns and Guidance\n\nFile size and modularity\n- Target ~200 lines per file. Split responsibilities into multiple nodes or utilities.\n- SRP and DRY: nodes do one thing; share common helpers in utils/.\n- Prefer small subgraphs over monolith graphs for composition and reuse.\n\nBackpressure and overflow\n- Default policy: block (applies backpressure upstream).\n- For sporadic spikes: latest policy can drop stale data in favor of the newest.\n- For aggregations: coalesce can compress bursts (e.g., merge updates).\n\nPriorities\n- Assign higher priority to control\u2011plane edges (e.g., kill switch, cancel_all).\n- Keep control messages small and fast; avoid heavy work in control nodes.\n\nMessage schemas\n- Use precise Python types or TypedDicts for ports.\n- Optionally integrate Pydantic for richer validation without coupling the runtime.\n\nError handling\n- Prefer local handling in nodes; escalate via dead\u2011letter subgraph if needed.\n- Use retries with jitter and circuit\u2011breaker patterns for external IO nodes.\n\n---\n\n## Observability\n\nLogs\n- JSON\u2011structured logs with timestamps and correlation IDs (trace_id)\n- Node lifecycle events, exceptions, tick durations\n\nMetrics (Prometheus)\n- Node: tick latency, messages processed, errors\n- Edge: queue depth, enqueue/dequeue rate, drops, backpressure time\n- Scheduler: runnable nodes, loop latency, priority applications\n\nTracing (optional)\n- Hook into OpenTelemetry: annotate message paths and node spans\n- Keep tracing optional to avoid overhead where unnecessary\n\nDashboards and alerts\n- Track queue depths and backpressure saturation\n- Alert on sustained scheduler latency or starved nodes\n- Monitor error rates per node and dropped messages per edge\n\n---\n\n## Roadmap\n\nv1 (initial release)\n- In\u2011process runtime, asyncio\u2011friendly\n- Nodes, edges (bounded queues), subgraphs, scheduler (fair scheduling)\n- Observability: logs, metrics, basic trace hooks\n- Examples and scaffolding scripts\n\nv1.x\n- Schema adapters (Pydantic), richer overflow policies\n- More node templates and utilities\n- Improved testing harness and fixtures\n\nv2+\n- Multi\u2011process edges (shared memory/IPC)\n- Distributed graphs (brokers + codecs)\n- Visual tooling and hot\u2011reload for graphs\n\n---\n\n## Development (with uv)\n\nInstall and sync\n```\nuv init\nuv lock\nuv sync\n```\n\nRun tests\n```\nuv run pytest\n```\n\nLint and type\u2011check (example)\n```\nuv run ruff check .\nuv run mypy src\n```\n\nRun an example\n```\nuv run python -m examples.hello_graph.main\n```\n\nContributing\n- Keep files small (~200 lines) and responsibilities focused.\n- Include unit tests for core changes; add integration tests for subgraph behavior.\n- Update examples and docs for user\u2011facing features.\n- Follow SemVer and add entries to CHANGELOG for notable changes.\n\nLicense\n- BSD 3-Clause (recommended) or your organization\u2019s standard OSS license.\n\n---\n\n## FAQ\n\nSee our [FAQ page](https://ghostweasellabs.github.io/meridian-runtime/support/faq/) for answers to common questions about Meridian Runtime.\n\n---\n\nHappy building with Meridian Runtime.\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "A minimal, reusable graph runtime for Python with bounded edges, fairness, and observability.",
"version": "1.2.0",
"project_urls": {
"Changelog": "https://github.com/ghostweasellabs/meridian-runtime/blob/main/CHANGELOG.md",
"Documentation": "https://ghostweasellabs.github.io/meridian-runtime-docs",
"Homepage": "https://ghostweasellabs.github.io/meridian-runtime-docs",
"Repository": "https://github.com/ghostweasellabs/meridian-runtime.git"
},
"split_keywords": [
"graph",
" runtime",
" scheduler",
" asyncio",
" observability",
" python"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "24cb5f12127871eb3d73ab37442296806b2b5aa03a55ae5b1e27a05be849b9ec",
"md5": "6f2800939bcb851091215d6688abce1d",
"sha256": "d0932c118d5efe67b0a8884c4531feda190f5f7da6e761ec3ec5d109c91a3fea"
},
"downloads": -1,
"filename": "meridian_runtime-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6f2800939bcb851091215d6688abce1d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 87654,
"upload_time": "2025-08-08T17:43:51",
"upload_time_iso_8601": "2025-08-08T17:43:51.236159Z",
"url": "https://files.pythonhosted.org/packages/24/cb/5f12127871eb3d73ab37442296806b2b5aa03a55ae5b1e27a05be849b9ec/meridian_runtime-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "42f0e2a51860c25f377cd40d3a8bfcaadfe83d54aa2787cbe347922138b27323",
"md5": "80656396584c80011967ae3734af135a",
"sha256": "d1ea4ea89beebbe530a81d4fa1f2b122f47cbcb5975519994aa0689a6c153e59"
},
"downloads": -1,
"filename": "meridian_runtime-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "80656396584c80011967ae3734af135a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 63349,
"upload_time": "2025-08-08T17:43:52",
"upload_time_iso_8601": "2025-08-08T17:43:52.534344Z",
"url": "https://files.pythonhosted.org/packages/42/f0/e2a51860c25f377cd40d3a8bfcaadfe83d54aa2787cbe347922138b27323/meridian_runtime-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-08 17:43:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ghostweasellabs",
"github_project": "meridian-runtime",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "meridian-runtime"
}
JSON
