# tracelet
[](https://img.shields.io/github/v/release/prassanna-ravishankar/tracelet)
[](https://github.com/prassanna-ravishankar/tracelet/actions/workflows/main.yml?query=branch%3Amain)
[](https://codecov.io/gh/prassanna-ravishankar/tracelet)
[](https://github.com/prassanna-ravishankar/tracelet/commit-activity/m/prassanna-ravishankar/tracelet)
[](https://img.shields.io/github/license/prassanna-ravishankar/tracelet)
<p align="center">
<img src="https://github.com/prassanna-ravishankar/tracelet/raw/main/docs/tracelet.webp" alt="Tracelet Logo" width="120" height="120">
</p>
Tracelet is an intelligent experiment tracking library that automatically captures PyTorch and PyTorch Lightning metrics, seamlessly integrating with popular experiment tracking platforms through a modular plugin system. With **automagic instrumentation**, Tracelet can automatically detect and log hyperparameters from your code with zero configuration.
## Key Features
### 🔌 Modular Plugin System
- Dynamic plugin discovery and lifecycle management
- Easy to extend with custom backends and collectors
- Thread-safe metric routing with configurable workers
- Dependency resolution for complex plugin hierarchies
### 🚀 Automatic Metric Capture
- 🔮 **Automagic Instrumentation** - Zero-config hyperparameter detection and logging
- 🔄 PyTorch TensorBoard integration - automatically captures `writer.add_scalar()` calls
- ⚡ PyTorch Lightning support - seamlessly tracks trainer metrics
- 📊 System metrics monitoring (CPU, Memory, GPU support planned)
- 📝 Automatic Git repository and environment tracking
### 🎯 Production-Ready Backends
- **MLflow** - Local and remote server support with full experiment tracking
- **ClearML** - Enterprise-grade experiment management with artifact storage
- **Weights & Biases** - Cloud-based tracking with rich visualizations
- **AIM** - Open-source experiment tracking with powerful UI
### 🛡️ Robust Architecture
- Thread-safe data flow orchestration
- Backpressure handling for high-frequency metrics
- Configurable metric routing and filtering
- Comprehensive error handling and logging
## Installation
Install the base package (includes PyTorch, TensorBoard, and W&B):
```bash
pip install tracelet
```
### Additional Backend Dependencies
Install specific backends as needed:
```bash
# Additional backend integrations
pip install tracelet[mlflow] # MLflow backend
pip install tracelet[clearml] # ClearML backend
pip install tracelet[aim] # AIM backend (Python <3.13)
# Framework integrations
pip install tracelet[lightning] # PyTorch Lightning support
pip install tracelet[automagic] # Automagic instrumentation
# Install multiple extras
pip install tracelet[mlflow,clearml] # Multiple backends
pip install tracelet[backends] # All backends
pip install tracelet[all] # Everything
```
**Base dependencies included**: PyTorch, TorchVision, TensorBoard, Weights & Biases, GitPython, Psutil
**Supported Python versions**: 3.9, 3.10, 3.11, 3.12, 3.13
**Note**: The AIM backend currently requires Python <3.13 due to dependency constraints.
## Demo
<p align="center">
<a href="https://github.com/prassanna-ravishankar/tracelet/raw/main/docs/video.mp4">
<img src="https://img.shields.io/badge/🎥_Watch_Demo_Video-4.7MB_MP4-red?style=for-the-badge&logo=youtube&logoColor=white" alt="Watch Demo Video" />
</a>
</p>
**📺 See Tracelet in action!** Click the button above to download and watch our demo video showing how easy it is to get started with automatic experiment tracking.
> **Note**: GitHub doesn't support embedded video playback in README files. The link above will download the MP4 file directly.
## Quick Start
### Basic Usage
```python
import tracelet
import torch
from torch.utils.tensorboard import SummaryWriter
# Start experiment tracking with your preferred backend
tracelet.start_logging(
exp_name="my_experiment",
project="my_project",
backend="mlflow" # or "clearml", "wandb", "aim"
)
# Use TensorBoard as usual - metrics are automatically captured
writer = SummaryWriter()
for epoch in range(100):
loss = train_one_epoch() # Your training logic
writer.add_scalar('Loss/train', loss, epoch)
# Metrics are automatically sent to MLflow!
# Stop tracking when done
tracelet.stop_logging()
```
### PyTorch Lightning Integration
```python
import tracelet
import pytorch_lightning as pl
# Start Tracelet before training
tracelet.start_logging("lightning_experiment", backend="clearml")
# Train your model - all Lightning metrics are captured
trainer = pl.Trainer(max_epochs=10)
trainer.fit(model, datamodule)
# Experiment data is automatically tracked!
tracelet.stop_logging()
```
### 🔮 Automagic Instrumentation
Tracelet's most powerful feature is **automagic instrumentation** - zero-configuration automatic detection and logging of machine learning hyperparameters. Just enable automagic and Tracelet intelligently captures your experiment parameters:
```python
import tracelet
from tracelet import Experiment
# Enable automagic mode - that's it!
experiment = Experiment(
name="automagic_experiment",
backend=["mlflow"],
automagic=True # ✨ The magic happens here!
)
experiment.start()
# Define your hyperparameters normally
learning_rate = 0.001
batch_size = 64
epochs = 100
dropout = 0.3
hidden_layers = [256, 128, 64]
optimizer = "adam"
# Your training code here...
# All hyperparameters are automatically captured and logged!
experiment.end()
```
#### How Automagic Works
Automagic uses intelligent heuristics to detect ML-relevant parameters:
- **📝 Name patterns**: `learning_rate`, `batch_size`, `num_layers`
- **🔢 Value ranges**: 0.001-0.1 for learning rates, 16-512 for batch sizes
- **📊 Data types**: floats in (0,1) for rates, ints for counts
- **🏷️ Keywords**: `rate`, `size`, `dim`, `num`, `alpha`, `beta`
- **✅ Boolean flags**: `use_*`, `enable_*`, `has_*`
- **📝 String configs**: optimizer names, activation functions
#### Framework Integration
Automagic automatically hooks into popular ML frameworks:
```python
# PyTorch - automatic capture of model parameters, loss, gradients
model = torch.nn.Sequential(...)
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
criterion = torch.nn.CrossEntropyLoss()
# Training loop - metrics captured automatically via hooks
for epoch in range(epochs):
loss = criterion(model(data), targets)
optimizer.step() # Learning rate automatically logged
# Loss and gradient norms captured via framework hooks
```
#### Comparison: Manual vs Automagic
**Manual Tracking** (traditional approach):
```python
# 🔧 MANUAL: Explicit logging required
experiment.log_params({
"learning_rate": 0.001,
"batch_size": 64,
"epochs": 100,
"dropout": 0.3,
"hidden_layers": [256, 128, 64],
"optimizer": "adam",
# ... 20+ more parameters
})
```
**Automagic Tracking** (zero-config):
```python
# 🔮 AUTOMAGIC: Just define variables normally
learning_rate = 0.001
batch_size = 64
epochs = 100
dropout = 0.3
hidden_layers = [256, 128, 64]
optimizer = "adam"
# All parameters automatically captured! ✨
```
**Benefits:**
- 🚀 **95% fewer logging calls** compared to manual tracking
- 🧠 **Intelligent parameter detection** with ML-specific heuristics
- 🔧 **Framework hooks** automatically capture training metrics
- ⚡ **Real-time monitoring** with minimal overhead
- 🎯 **Focus on research**, not logging boilerplate
### Advanced Configuration
```python
import tracelet
from tracelet import get_active_experiment
# Start with custom configuration
experiment = tracelet.start_logging(
exp_name="advanced_example",
project="my_project",
backend="mlflow",
config={
"track_system": True, # System monitoring
"metrics_interval": 5.0, # Log every 5 seconds
"track_git": True, # Git info tracking
"track_env": True, # Environment capture
"track_tensorboard": True, # Auto-capture TB metrics
"track_lightning": True, # PyTorch Lightning support
},
automagic=True # Enable automagic instrumentation
)
# Log custom parameters
experiment.log_params({
"model": "resnet50",
"batch_size": 32,
"learning_rate": 0.001
})
# Log custom metrics programmatically
for epoch in range(10):
metrics = train_epoch()
experiment.log_metric("accuracy", metrics["acc"], epoch)
experiment.log_metric("loss", metrics["loss"], epoch)
```
## Configuration
Tracelet can be configured via environment variables or through the settings interface:
```python
from tracelet.settings import TraceletSettings
settings = TraceletSettings(
project="my_project", # or project_name (alias)
backend=["mlflow"], # List of backends
track_system=True, # System metrics tracking
metrics_interval=10.0, # Collection interval
track_tensorboard=True, # TensorBoard integration
track_lightning=True, # PyTorch Lightning support
track_git=True, # Git repository info
track_env=True, # Environment capture
enable_automagic=True, # Enable automagic instrumentation
automagic_frameworks={"pytorch", "sklearn", "xgboost"} # Frameworks to instrument
)
```
Key environment variables:
- `TRACELET_PROJECT`: Project name
- `TRACELET_BACKEND`: Comma-separated backends ("mlflow,wandb")
- `TRACELET_BACKEND_URL`: Backend server URL
- `TRACELET_API_KEY`: API key for backend service
- `TRACELET_TRACK_SYSTEM`: Enable system metrics tracking
- `TRACELET_METRICS_INTERVAL`: System metrics collection interval
- `TRACELET_TRACK_TENSORBOARD`: Enable TensorBoard integration
- `TRACELET_TRACK_LIGHTNING`: Enable PyTorch Lightning support
- `TRACELET_TRACK_GIT`: Enable Git repository tracking
- `TRACELET_TRACK_ENV`: Enable environment capture
- `TRACELET_ENABLE_AUTOMAGIC`: Enable automagic instrumentation
- `TRACELET_AUTOMAGIC_FRAMEWORKS`: Comma-separated frameworks ("pytorch,sklearn")
## Plugin Development
Tracelet's plugin system makes it easy to add new backends or metric collectors:
```python
from tracelet.core.plugins import BackendPlugin, PluginMetadata, PluginType
class MyCustomBackend(BackendPlugin):
@classmethod
def get_metadata(cls) -> PluginMetadata:
return PluginMetadata(
name="my_backend",
version="1.0.0",
type=PluginType.BACKEND,
description="My custom experiment tracking backend"
)
def initialize(self, config: dict):
# Set up your backend connection
self.client = MyBackendClient(config["api_key"])
def log_metric(self, name: str, value: float, iteration: int):
# Send metrics to your backend
self.client.log(name, value, iteration)
```
Plugins are automatically discovered from:
- Built-in: `tracelet/plugins/` directory
- User: `~/.tracelet/plugins/` directory
- Custom: Set `TRACELET_PLUGIN_PATH` environment variable
## Documentation
For more detailed documentation, visit:
- [Documentation](https://prassanna-ravishankar.github.io/tracelet/)
- [GitHub Repository](https://github.com/prassanna-ravishankar/tracelet/)
- [Examples](https://github.com/prassanna-ravishankar/tracelet/tree/main/examples)
## Architecture
Tracelet uses a sophisticated multi-threaded architecture:
```
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ Framework │────▶│ Orchestrator │────▶│ Backend │
│ (PyTorch) │ │ (Router) │ │ (MLflow) │
└─────────────┘ └──────────────┘ └─────────────┘
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ Collector │────▶│ Queue │────▶│ Plugin │
│ (System) │ │ (Threaded) │ │ (ClearML) │
└─────────────┘ └──────────────┘ └─────────────┘
```
- **Metric Sources**: Frameworks and collectors that generate metrics
- **Orchestrator**: Routes metrics to appropriate backends based on rules
- **Backends**: Plugins that handle experiment tracking and storage
## Performance
Tracelet is designed for minimal overhead:
- Non-blocking metric collection using thread-safe queues
- Configurable worker threads for parallel processing
- Automatic backpressure handling to prevent memory issues
- Efficient metric batching for reduced network calls
## Troubleshooting
### Common Issues
**Import errors for backends**: Make sure you've installed the appropriate extras:
```bash
# If you see: ImportError: MLflow is not installed
pip install tracelet[mlflow]
```
**ClearML offline mode**: For testing or CI environments without ClearML credentials:
```python
import os
os.environ["CLEARML_WEB_HOST"] = ""
os.environ["CLEARML_API_HOST"] = ""
os.environ["CLEARML_FILES_HOST"] = ""
```
**High memory usage**: Disable unnecessary tracking features:
```python
experiment = tracelet.start_logging(
config={
"track_system": False, # Disable system metrics
"track_git": False, # Disable git tracking
"metrics_interval": 30.0, # Reduce collection frequency
}
)
```
## Roadmap
- [x] 🔮 **Automagic Instrumentation** - Zero-config hyperparameter detection
- [ ] AWS SageMaker integration
- [ ] Prometheus metrics export
- [ ] Real-time metric streaming
- [ ] Web UI for local experiments
- [ ] Distributed training support
- [ ] Enhanced automagic model architecture capture
- [ ] Automagic dataset profiling and statistics
## Contributing
We welcome contributions! Please see our [Contributing Guidelines](https://github.com/prassanna-ravishankar/tracelet/blob/main/CONTRIBUTING.md) for details.
### Development Setup
```bash
# Clone the repository
git clone https://github.com/prassanna-ravishankar/tracelet.git
cd tracelet
# Install with development dependencies
pip install -e ".[dev]"
# Run tests
pytest
# Run linting
make check
```
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/prassanna-ravishankar/tracelet/blob/main/LICENSE) file for details.
## Acknowledgments
- Built with the excellent [uv](https://github.com/astral-sh/uv) package manager
- Repository initiated with [fpgmaas/cookiecutter-uv](https://github.com/fpgmaas/cookiecutter-uv)
- Thanks to all contributors and the open-source community!
Raw data
{
"_id": null,
"home_page": null,
"name": "tracelet",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "python, pytorch, machine-learning, experiment-tracking, mlops",
"author": null,
"author_email": "Prassanna Ravishankar <me@prassanna.io>",
"download_url": "https://files.pythonhosted.org/packages/5b/39/dac238129e2ec36fa5ad8c01bcf71b12b7db5e7101a05c939b707e27ce18/tracelet-0.0.7.tar.gz",
"platform": null,
"description": "# tracelet\n\n[](https://img.shields.io/github/v/release/prassanna-ravishankar/tracelet)\n[](https://github.com/prassanna-ravishankar/tracelet/actions/workflows/main.yml?query=branch%3Amain)\n[](https://codecov.io/gh/prassanna-ravishankar/tracelet)\n[](https://github.com/prassanna-ravishankar/tracelet/commit-activity/m/prassanna-ravishankar/tracelet)\n[](https://img.shields.io/github/license/prassanna-ravishankar/tracelet)\n\n<p align=\"center\">\n <img src=\"https://github.com/prassanna-ravishankar/tracelet/raw/main/docs/tracelet.webp\" alt=\"Tracelet Logo\" width=\"120\" height=\"120\">\n</p>\n\nTracelet is an intelligent experiment tracking library that automatically captures PyTorch and PyTorch Lightning metrics, seamlessly integrating with popular experiment tracking platforms through a modular plugin system. With **automagic instrumentation**, Tracelet can automatically detect and log hyperparameters from your code with zero configuration.\n\n## Key Features\n\n### \ud83d\udd0c Modular Plugin System\n\n- Dynamic plugin discovery and lifecycle management\n- Easy to extend with custom backends and collectors\n- Thread-safe metric routing with configurable workers\n- Dependency resolution for complex plugin hierarchies\n\n### \ud83d\ude80 Automatic Metric Capture\n\n- \ud83d\udd2e **Automagic Instrumentation** - Zero-config hyperparameter detection and logging\n- \ud83d\udd04 PyTorch TensorBoard integration - automatically captures `writer.add_scalar()` calls\n- \u26a1 PyTorch Lightning support - seamlessly tracks trainer metrics\n- \ud83d\udcca System metrics monitoring (CPU, Memory, GPU support planned)\n- \ud83d\udcdd Automatic Git repository and environment tracking\n\n### \ud83c\udfaf Production-Ready Backends\n\n- **MLflow** - Local and remote server support with full experiment tracking\n- **ClearML** - Enterprise-grade experiment management with artifact storage\n- **Weights & Biases** - Cloud-based tracking with rich visualizations\n- **AIM** - Open-source experiment tracking with powerful UI\n\n### \ud83d\udee1\ufe0f Robust Architecture\n\n- Thread-safe data flow orchestration\n- Backpressure handling for high-frequency metrics\n- Configurable metric routing and filtering\n- Comprehensive error handling and logging\n\n## Installation\n\nInstall the base package (includes PyTorch, TensorBoard, and W&B):\n\n```bash\npip install tracelet\n```\n\n### Additional Backend Dependencies\n\nInstall specific backends as needed:\n\n```bash\n# Additional backend integrations\npip install tracelet[mlflow] # MLflow backend\npip install tracelet[clearml] # ClearML backend\npip install tracelet[aim] # AIM backend (Python <3.13)\n\n# Framework integrations\npip install tracelet[lightning] # PyTorch Lightning support\npip install tracelet[automagic] # Automagic instrumentation\n\n# Install multiple extras\npip install tracelet[mlflow,clearml] # Multiple backends\npip install tracelet[backends] # All backends\npip install tracelet[all] # Everything\n```\n\n**Base dependencies included**: PyTorch, TorchVision, TensorBoard, Weights & Biases, GitPython, Psutil\n\n**Supported Python versions**: 3.9, 3.10, 3.11, 3.12, 3.13\n\n**Note**: The AIM backend currently requires Python <3.13 due to dependency constraints.\n\n## Demo\n\n<p align=\"center\">\n <a href=\"https://github.com/prassanna-ravishankar/tracelet/raw/main/docs/video.mp4\">\n <img src=\"https://img.shields.io/badge/\ud83c\udfa5_Watch_Demo_Video-4.7MB_MP4-red?style=for-the-badge&logo=youtube&logoColor=white\" alt=\"Watch Demo Video\" />\n </a>\n</p>\n\n**\ud83d\udcfa See Tracelet in action!** Click the button above to download and watch our demo video showing how easy it is to get started with automatic experiment tracking.\n\n> **Note**: GitHub doesn't support embedded video playback in README files. The link above will download the MP4 file directly.\n\n## Quick Start\n\n### Basic Usage\n\n```python\nimport tracelet\nimport torch\nfrom torch.utils.tensorboard import SummaryWriter\n\n# Start experiment tracking with your preferred backend\ntracelet.start_logging(\n exp_name=\"my_experiment\",\n project=\"my_project\",\n backend=\"mlflow\" # or \"clearml\", \"wandb\", \"aim\"\n)\n\n# Use TensorBoard as usual - metrics are automatically captured\nwriter = SummaryWriter()\nfor epoch in range(100):\n loss = train_one_epoch() # Your training logic\n writer.add_scalar('Loss/train', loss, epoch)\n # Metrics are automatically sent to MLflow!\n\n# Stop tracking when done\ntracelet.stop_logging()\n```\n\n### PyTorch Lightning Integration\n\n```python\nimport tracelet\nimport pytorch_lightning as pl\n\n# Start Tracelet before training\ntracelet.start_logging(\"lightning_experiment\", backend=\"clearml\")\n\n# Train your model - all Lightning metrics are captured\ntrainer = pl.Trainer(max_epochs=10)\ntrainer.fit(model, datamodule)\n\n# Experiment data is automatically tracked!\ntracelet.stop_logging()\n```\n\n### \ud83d\udd2e Automagic Instrumentation\n\nTracelet's most powerful feature is **automagic instrumentation** - zero-configuration automatic detection and logging of machine learning hyperparameters. Just enable automagic and Tracelet intelligently captures your experiment parameters:\n\n```python\nimport tracelet\nfrom tracelet import Experiment\n\n# Enable automagic mode - that's it!\nexperiment = Experiment(\n name=\"automagic_experiment\",\n backend=[\"mlflow\"],\n automagic=True # \u2728 The magic happens here!\n)\nexperiment.start()\n\n# Define your hyperparameters normally\nlearning_rate = 0.001\nbatch_size = 64\nepochs = 100\ndropout = 0.3\nhidden_layers = [256, 128, 64]\noptimizer = \"adam\"\n\n# Your training code here...\n# All hyperparameters are automatically captured and logged!\n\nexperiment.end()\n```\n\n#### How Automagic Works\n\nAutomagic uses intelligent heuristics to detect ML-relevant parameters:\n\n- **\ud83d\udcdd Name patterns**: `learning_rate`, `batch_size`, `num_layers`\n- **\ud83d\udd22 Value ranges**: 0.001-0.1 for learning rates, 16-512 for batch sizes\n- **\ud83d\udcca Data types**: floats in (0,1) for rates, ints for counts\n- **\ud83c\udff7\ufe0f Keywords**: `rate`, `size`, `dim`, `num`, `alpha`, `beta`\n- **\u2705 Boolean flags**: `use_*`, `enable_*`, `has_*`\n- **\ud83d\udcdd String configs**: optimizer names, activation functions\n\n#### Framework Integration\n\nAutomagic automatically hooks into popular ML frameworks:\n\n```python\n# PyTorch - automatic capture of model parameters, loss, gradients\nmodel = torch.nn.Sequential(...)\noptimizer = torch.optim.Adam(model.parameters(), lr=0.001)\ncriterion = torch.nn.CrossEntropyLoss()\n\n# Training loop - metrics captured automatically via hooks\nfor epoch in range(epochs):\n loss = criterion(model(data), targets)\n optimizer.step() # Learning rate automatically logged\n # Loss and gradient norms captured via framework hooks\n```\n\n#### Comparison: Manual vs Automagic\n\n**Manual Tracking** (traditional approach):\n\n```python\n# \ud83d\udd27 MANUAL: Explicit logging required\nexperiment.log_params({\n \"learning_rate\": 0.001,\n \"batch_size\": 64,\n \"epochs\": 100,\n \"dropout\": 0.3,\n \"hidden_layers\": [256, 128, 64],\n \"optimizer\": \"adam\",\n # ... 20+ more parameters\n})\n```\n\n**Automagic Tracking** (zero-config):\n\n```python\n# \ud83d\udd2e AUTOMAGIC: Just define variables normally\nlearning_rate = 0.001\nbatch_size = 64\nepochs = 100\ndropout = 0.3\nhidden_layers = [256, 128, 64]\noptimizer = \"adam\"\n# All parameters automatically captured! \u2728\n```\n\n**Benefits:**\n\n- \ud83d\ude80 **95% fewer logging calls** compared to manual tracking\n- \ud83e\udde0 **Intelligent parameter detection** with ML-specific heuristics\n- \ud83d\udd27 **Framework hooks** automatically capture training metrics\n- \u26a1 **Real-time monitoring** with minimal overhead\n- \ud83c\udfaf **Focus on research**, not logging boilerplate\n\n### Advanced Configuration\n\n```python\nimport tracelet\nfrom tracelet import get_active_experiment\n\n# Start with custom configuration\nexperiment = tracelet.start_logging(\n exp_name=\"advanced_example\",\n project=\"my_project\",\n backend=\"mlflow\",\n config={\n \"track_system\": True, # System monitoring\n \"metrics_interval\": 5.0, # Log every 5 seconds\n \"track_git\": True, # Git info tracking\n \"track_env\": True, # Environment capture\n \"track_tensorboard\": True, # Auto-capture TB metrics\n \"track_lightning\": True, # PyTorch Lightning support\n },\n automagic=True # Enable automagic instrumentation\n)\n\n# Log custom parameters\nexperiment.log_params({\n \"model\": \"resnet50\",\n \"batch_size\": 32,\n \"learning_rate\": 0.001\n})\n\n# Log custom metrics programmatically\nfor epoch in range(10):\n metrics = train_epoch()\n experiment.log_metric(\"accuracy\", metrics[\"acc\"], epoch)\n experiment.log_metric(\"loss\", metrics[\"loss\"], epoch)\n```\n\n## Configuration\n\nTracelet can be configured via environment variables or through the settings interface:\n\n```python\nfrom tracelet.settings import TraceletSettings\n\nsettings = TraceletSettings(\n project=\"my_project\", # or project_name (alias)\n backend=[\"mlflow\"], # List of backends\n track_system=True, # System metrics tracking\n metrics_interval=10.0, # Collection interval\n track_tensorboard=True, # TensorBoard integration\n track_lightning=True, # PyTorch Lightning support\n track_git=True, # Git repository info\n track_env=True, # Environment capture\n enable_automagic=True, # Enable automagic instrumentation\n automagic_frameworks={\"pytorch\", \"sklearn\", \"xgboost\"} # Frameworks to instrument\n)\n```\n\nKey environment variables:\n\n- `TRACELET_PROJECT`: Project name\n- `TRACELET_BACKEND`: Comma-separated backends (\"mlflow,wandb\")\n- `TRACELET_BACKEND_URL`: Backend server URL\n- `TRACELET_API_KEY`: API key for backend service\n- `TRACELET_TRACK_SYSTEM`: Enable system metrics tracking\n- `TRACELET_METRICS_INTERVAL`: System metrics collection interval\n- `TRACELET_TRACK_TENSORBOARD`: Enable TensorBoard integration\n- `TRACELET_TRACK_LIGHTNING`: Enable PyTorch Lightning support\n- `TRACELET_TRACK_GIT`: Enable Git repository tracking\n- `TRACELET_TRACK_ENV`: Enable environment capture\n- `TRACELET_ENABLE_AUTOMAGIC`: Enable automagic instrumentation\n- `TRACELET_AUTOMAGIC_FRAMEWORKS`: Comma-separated frameworks (\"pytorch,sklearn\")\n\n## Plugin Development\n\nTracelet's plugin system makes it easy to add new backends or metric collectors:\n\n```python\nfrom tracelet.core.plugins import BackendPlugin, PluginMetadata, PluginType\n\nclass MyCustomBackend(BackendPlugin):\n @classmethod\n def get_metadata(cls) -> PluginMetadata:\n return PluginMetadata(\n name=\"my_backend\",\n version=\"1.0.0\",\n type=PluginType.BACKEND,\n description=\"My custom experiment tracking backend\"\n )\n\n def initialize(self, config: dict):\n # Set up your backend connection\n self.client = MyBackendClient(config[\"api_key\"])\n\n def log_metric(self, name: str, value: float, iteration: int):\n # Send metrics to your backend\n self.client.log(name, value, iteration)\n```\n\nPlugins are automatically discovered from:\n\n- Built-in: `tracelet/plugins/` directory\n- User: `~/.tracelet/plugins/` directory\n- Custom: Set `TRACELET_PLUGIN_PATH` environment variable\n\n## Documentation\n\nFor more detailed documentation, visit:\n\n- [Documentation](https://prassanna-ravishankar.github.io/tracelet/)\n- [GitHub Repository](https://github.com/prassanna-ravishankar/tracelet/)\n- [Examples](https://github.com/prassanna-ravishankar/tracelet/tree/main/examples)\n\n## Architecture\n\nTracelet uses a sophisticated multi-threaded architecture:\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Framework \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 Orchestrator \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 Backend \u2502\n\u2502 (PyTorch) \u2502 \u2502 (Router) \u2502 \u2502 (MLflow) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502 \u2502\n \u25bc \u25bc \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Collector \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 Queue \u2502\u2500\u2500\u2500\u2500\u25b6\u2502 Plugin \u2502\n\u2502 (System) \u2502 \u2502 (Threaded) \u2502 \u2502 (ClearML) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n- **Metric Sources**: Frameworks and collectors that generate metrics\n- **Orchestrator**: Routes metrics to appropriate backends based on rules\n- **Backends**: Plugins that handle experiment tracking and storage\n\n## Performance\n\nTracelet is designed for minimal overhead:\n\n- Non-blocking metric collection using thread-safe queues\n- Configurable worker threads for parallel processing\n- Automatic backpressure handling to prevent memory issues\n- Efficient metric batching for reduced network calls\n\n## Troubleshooting\n\n### Common Issues\n\n**Import errors for backends**: Make sure you've installed the appropriate extras:\n\n```bash\n# If you see: ImportError: MLflow is not installed\npip install tracelet[mlflow]\n```\n\n**ClearML offline mode**: For testing or CI environments without ClearML credentials:\n\n```python\nimport os\nos.environ[\"CLEARML_WEB_HOST\"] = \"\"\nos.environ[\"CLEARML_API_HOST\"] = \"\"\nos.environ[\"CLEARML_FILES_HOST\"] = \"\"\n```\n\n**High memory usage**: Disable unnecessary tracking features:\n\n```python\nexperiment = tracelet.start_logging(\n config={\n \"track_system\": False, # Disable system metrics\n \"track_git\": False, # Disable git tracking\n \"metrics_interval\": 30.0, # Reduce collection frequency\n }\n)\n```\n\n## Roadmap\n\n- [x] \ud83d\udd2e **Automagic Instrumentation** - Zero-config hyperparameter detection\n- [ ] AWS SageMaker integration\n- [ ] Prometheus metrics export\n- [ ] Real-time metric streaming\n- [ ] Web UI for local experiments\n- [ ] Distributed training support\n- [ ] Enhanced automagic model architecture capture\n- [ ] Automagic dataset profiling and statistics\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guidelines](https://github.com/prassanna-ravishankar/tracelet/blob/main/CONTRIBUTING.md) for details.\n\n### Development Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/prassanna-ravishankar/tracelet.git\ncd tracelet\n\n# Install with development dependencies\npip install -e \".[dev]\"\n\n# Run tests\npytest\n\n# Run linting\nmake check\n```\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/prassanna-ravishankar/tracelet/blob/main/LICENSE) file for details.\n\n## Acknowledgments\n\n- Built with the excellent [uv](https://github.com/astral-sh/uv) package manager\n- Repository initiated with [fpgmaas/cookiecutter-uv](https://github.com/fpgmaas/cookiecutter-uv)\n- Thanks to all contributors and the open-source community!\n",
"bugtrack_url": null,
"license": null,
"summary": "Tracelet is an automagic pytorch metric exporter",
"version": "0.0.7",
"project_urls": {
"Changelog": "https://github.com/prassanna-ravishankar/tracelet/releases",
"Documentation": "https://prassanna.io/tracelet/",
"Homepage": "https://prassanna.io/tracelet/",
"Repository": "https://github.com/prassanna-ravishankar/tracelet"
},
"split_keywords": [
"python",
" pytorch",
" machine-learning",
" experiment-tracking",
" mlops"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1da40b5bfecc5313fd1ac633e92373af9efc4adc0a5552cf94b82a8dc2bd3e53",
"md5": "a28e31356e317a21f840eefb9381fd33",
"sha256": "ddb5489b8a032424a38f4d41fceeb5c4e95818ab5c6e0f3d398696a41b68ac38"
},
"downloads": -1,
"filename": "tracelet-0.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a28e31356e317a21f840eefb9381fd33",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 7536,
"upload_time": "2025-07-24T21:14:03",
"upload_time_iso_8601": "2025-07-24T21:14:03.446751Z",
"url": "https://files.pythonhosted.org/packages/1d/a4/0b5bfecc5313fd1ac633e92373af9efc4adc0a5552cf94b82a8dc2bd3e53/tracelet-0.0.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "5b39dac238129e2ec36fa5ad8c01bcf71b12b7db5e7101a05c939b707e27ce18",
"md5": "c3da1805ba2643f80d2430fe65f166c9",
"sha256": "8f00557baa1db2f723d24595cf6ff14c6f957c61b10cd2b5f9f76e551667a705"
},
"downloads": -1,
"filename": "tracelet-0.0.7.tar.gz",
"has_sig": false,
"md5_digest": "c3da1805ba2643f80d2430fe65f166c9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 9629,
"upload_time": "2025-07-24T21:14:04",
"upload_time_iso_8601": "2025-07-24T21:14:04.334305Z",
"url": "https://files.pythonhosted.org/packages/5b/39/dac238129e2ec36fa5ad8c01bcf71b12b7db5e7101a05c939b707e27ce18/tracelet-0.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-24 21:14:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "prassanna-ravishankar",
"github_project": "tracelet",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "tracelet"
}
JSON
