| Name | crashlens JSON |
| Version |
2.9.15
JSON |
| download |
| home_page | None |
| Summary | CLI to detect GPT token waste from Langfuse logs with automated CI/CD setup |
| upload_time | 2025-09-03 18:09:51 |
| maintainer | None |
| docs_url | None |
| author | Aditya Singh |
| requires_python | <4.0,>=3.12 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# π§ CrashLens: AI Token Waste Detective
<div align="center">
[](https://pypi.org/project/crashlens/)
[](https://python.org)
[](LICENSE)
[]()
**The Ultimate AI Cost Optimization Tool**
*Detect hidden token waste β’ Prevent budget overruns β’ Optimize LLM usage*
[Quick Start](#-quick-start) β’ [Features](#-features) β’ [Documentation](#-documentation) β’ [Examples](#-examples)
</div>
---
## π― What is CrashLens?
CrashLens is a **developer-first CLI tool** that analyzes your AI API logs to uncover **hidden token waste**, retry loops, model overkill, and inefficient patterns. It helps you **optimize OpenAI, Anthropic, and Langfuse usage** by providing actionable cost-saving insights.
### π **Why CrashLens?**
> *"You can't optimize what you can't see."*
- **π° Cost Savings**: Identify 40-60% potential savings in AI spending
- **π Privacy First**: 100% local analysis, no data leaves your machine
- **β‘ Production Ready**: Battle-tested policy engine with CI/CD integration
- **π― Actionable**: Specific recommendations, not just analytics
---
## π Quick Start
### Prerequisites
- **Python 3.12+** ([Download here](https://python.org/downloads/))
- AI usage logs in JSONL format (OpenAI, Anthropic, Langfuse)
### Installation & Setup
```bash
# Install CrashLens
pip install crashlens
# Interactive setup wizard
crashlens init
# Quick scan with demo data
crashlens scan --demo
# Analyze your logs
crashlens scan your-logs.jsonl
# Optional: Set up Slack notifications
export CRASHLENS_SLACK_WEBHOOK="your-webhook-url"
crashlens scan your-logs.jsonl # Auto-posts to Slack
```
### Non-Interactive Setup (CI/CD)
**macOS/Linux:**
```bash
export CRASHLENS_TEMPLATES="all"
export CRASHLENS_SEVERITY="medium"
crashlens init --non-interactive
```
**Windows (PowerShell):**
```powershell
$env:CRASHLENS_TEMPLATES = "all"
$env:CRASHLENS_SEVERITY = "medium"
crashlens init --non-interactive
```
---
## οΏ½ Core Features
### π΅οΈ **Waste Detection Engine**
- **π Retry Loop Detection**: Exponential backoff failures, redundant retries
- **β Model Overkill**: GPT-4 used for simple 3-token completions
- **π’ Fallback Storms**: Cascading model failures wasting tokens
- **β‘ Prompt Inefficiency**: Long prompts generating tiny responses
- **πΈ Budget Violations**: Expensive calls exceeding thresholds
### π‘οΈ **Policy Enforcement**
- **Production-Grade Rules**: 10+ built-in policy templates
- **Custom Policies**: YAML-based rule definitions
- **CI/CD Integration**: Fail builds on policy violations
- **Severity Levels**: Critical, High, Medium, Low classifications
- **Smart Suppression**: Prevents alert fatigue
### π **Analysis & Reporting**
- **Multi-Format Reports**: Markdown, JSON, Slack-ready notifications
- **Auto Slack Integration**: Real-time team notifications with webhook setup
- **Cost Breakdown**: Per-model, per-trace, per-pattern analysis
- **Token Accounting**: Detailed waste calculations
- **Trend Analysis**: Historical cost patterns
- **ROI Calculator**: Quantified savings recommendations
---
## π» Commands Overview
### π **Scan & Analyze**
```bash
# Basic scan with smart reporting
crashlens scan logs.jsonl
# Generate detailed JSON reports
crashlens scan logs.jsonl --detailed
# Demo mode with sample data
crashlens scan --demo
# Custom output formats
crashlens scan logs.jsonl --format slack # Slack-ready format
crashlens scan logs.jsonl --format json # Machine-readable format
# Auto-post to Slack (requires CRASHLENS_SLACK_WEBHOOK env var)
export CRASHLENS_SLACK_WEBHOOK="your-webhook-url"
crashlens scan logs.jsonl # Auto-sends to Slack when configured
```
### π‘οΈ **Policy Enforcement**
```bash
# Check logs against all policies (generates report.md)
crashlens policy-check logs.jsonl --policy-template all
# Generate detailed JSON report for CI/CD integration
crashlens policy-check logs.jsonl --policy-template all --detailed
# Custom output paths and quiet mode
crashlens policy-check logs.jsonl --policy-template all --detailed \
--out-report violations.md --out-detailed violations.json --quiet
# Use specific policy templates
crashlens policy-check logs.jsonl --policy-template model-overkill-detection,retry-loop-prevention
# Custom policy file
crashlens policy-check logs.jsonl --policy-file my-policy.yaml
# Fail on violations (CI/CD mode)
crashlens policy-check logs.jsonl --policy-template all --fail-on-violations
# Privacy-safe reports (strip PII, exclude content)
crashlens policy-check logs.jsonl --policy-template all --strip-pii --no-content
```
**π Output Organization:**
Policy violation reports are automatically organized in the `policy-violations/` folder:
- `policy-violations/reports/` - Concise Markdown summaries
- `policy-violations/traces/` - Detailed JSON analysis files
- See `policy-violations/README.md` for complete documentation
### π οΈ **Management & Simulation**
**Cross-platform:**
```bash
# List available policy templates
crashlens list-policy-templates
# Simulate different usage patterns
crashlens simulate --pattern retry-loop --count 100
# Setup project with policies
crashlens init
```
---
## π Example Report Output
### π¨ **Cost Analysis Report**
```markdown
π¨ CrashLens Token Waste Report π¨
π Analysis Date: 2025-08-17
π Report Summary:
β’ π° Total AI Spend: $859.52
β’ π₯ Potential Savings: $859.52 (100%)
β’ π― Wasted Tokens: 38,213,010
β’ β οΈ Issues Found: 53,185
β’ π Traces Analyzed: 156
π Retry Loop β’ 187 traces β’ $859.52 wasted
π‘ Fix: exponential backoff
π― Wasted tokens: 24,555,498
π Traces: trace_retry_loop_01, trace_retry_loop_02, +185 more
β Overkill Model β’ 52,998 traces β’ $560.24 wasted
π‘ Fix: optimize usage
π― Wasted tokens: 13,657,512
π Traces: trace_overkill_01, trace_overkill_02, +52,996 more
π Top Expensive Traces:
β’ #1 β trace_norm_76 β gpt-4 β $65.78
β’ #2 β trace_norm_65 β gpt-4 β $52.60
β’ #3 β trace_norm_38 β gpt-4 β $44.10
π€ Cost by Model:
β’ gpt-4 β $845.65 (98%)
β’ gpt-3.5-turbo β $13.87 (2%)
```
### π‘οΈ **Policy Violations Report**
```markdown
β οΈ Found 431,970 policy violations:
π¨ CRITICAL SEVERITY (6,534 violations):
1. high_cost_per_token (line 62437)
Reason: cost=0.06075 (rule: >0.05)
Action: fail
Suggestion: Very expensive API call detected (over $0.05).
Immediate actions:
- Review if this cost is justified
- Check for prompt optimization opportunities
- Consider model downgrading
β οΈ HIGH SEVERITY (227,238 violations):
1. gpt4_for_simple_tasks (line 727)
Reason: model=gpt-4 AND prompt_tokens=5 (rule: <50)
Action: fail
Suggestion: GPT-4 used for simple task.
Cost optimization opportunities:
- Use gpt-4o-mini (90% cheaper, similar quality)
- Use gpt-3.5-turbo for classification <500 tokens
- Reserve GPT-4 for complex reasoning tasks
```
---
## ποΈ Policy Templates
CrashLens includes production-ready policy templates:
| Template | Purpose | Estimated Savings |
|----------|---------|-------------------|
| `model-overkill-detection` | Prevent expensive models for simple tasks | 30-50% |
| `retry-loop-prevention` | Block inefficient retry patterns | 20-40% |
| `prompt-optimization` | Optimize prompt efficiency | 15-30% |
| `budget-protection` | Enforce spending limits | Varies |
| `fallback-storm-detection` | Prevent cascading failures | 10-35% |
| `context-window-optimization` | Efficient context usage | 10-25% |
| `production-ready` | Combined rules for production | 40-60% |
### Custom Policy Example
```yaml
# my-policy.yaml
metadata:
name: "Custom Cost Control"
description: "Strict cost controls for production"
rules:
- id: expensive_single_call
description: "Block very expensive calls"
match:
cost: ">0.10"
action: fail
severity: critical
suggestion: |
Call exceeds $0.10 threshold.
- Review prompt optimization
- Consider model downgrading
- Break into smaller requests
```
---
## π§ Advanced Usage
### CI/CD Integration
#### GitHub Actions
```yaml
name: AI Cost Control
on: [push, pull_request]
jobs:
cost-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.12'
- name: Install CrashLens
run: pip install crashlens
- name: Policy Check
run: |
crashlens policy-check logs/*.jsonl \
--policy-template all \
--fail-on-violations
```
#### Docker Integration
```dockerfile
FROM python:3.12-slim
RUN pip install crashlens
WORKDIR /app
COPY logs/ ./logs/
CMD ["crashlens", "policy-check", "logs/*.jsonl", "--policy-template", "all"]
```
### Programmatic Usage
```python
from crashlens.policy.engine import PolicyEngine
from crashlens.parsers.langfuse import LangfuseParser
# Load and analyze logs
parser = LangfuseParser()
traces = parser.parse_file("logs.jsonl")
# Apply policies
engine = PolicyEngine("policy.yaml")
violations = engine.check_logs(traces)
print(f"Found {len(violations)} violations")
```
---
## π Supported Log Formats
### OpenAI API Logs
```jsonl
{"model": "gpt-4", "usage": {"prompt_tokens": 10, "completion_tokens": 5}, "cost": 0.0003}
```
### Langfuse Logs
```jsonl
{"input": {"model": "gpt-4"}, "usage": {"promptTokens": 10, "completionTokens": 5}, "calculatedTotalCost": 0.0003}
```
### Anthropic Logs
```jsonl
{"model": "claude-3-opus", "usage": {"input_tokens": 10, "output_tokens": 5}, "cost": 0.0003}
```
---
## π οΈ Configuration
### Environment Variables
**macOS/Linux:**
```bash
export CRASHLENS_TEMPLATES="all" # Default policy templates
export CRASHLENS_SEVERITY="medium" # Minimum severity level
export CRASHLENS_OUTPUT_FORMAT="slack" # Report format (slack/markdown/json)
export CRASHLENS_SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
```
**Windows (PowerShell):**
```powershell
$env:CRASHLENS_TEMPLATES = "all" # Default policy templates
$env:CRASHLENS_SEVERITY = "medium" # Minimum severity level
$env:CRASHLENS_OUTPUT_FORMAT = "slack" # Report format (slack/markdown/json)
$env:CRASHLENS_SLACK_WEBHOOK = "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
```
### Config File (`.crashlens.yaml`)
```yaml
default_templates: ["model-overkill-detection", "retry-loop-prevention"]
severity_threshold: "medium"
output_format: "slack" # Auto-formats for Slack when webhook is configured
fail_on_violations: true
slack_webhook_url: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
```
---
## π Slack Webhook Integration
CrashLens supports **automatic Slack notifications** for team collaboration and alerting. Get instant notifications when policy violations are detected or cost thresholds are exceeded.
### π Quick Setup
#### 1. **Get Your Slack Webhook URL**
1. Go to [Slack Apps](https://api.slack.com/apps) β **Create New App**
2. Choose **From scratch** β Name your app β Select workspace
3. Go to **Incoming Webhooks** β Toggle **On** β **Add New Webhook to Workspace**
4. Select your channel β **Allow** β Copy the webhook URL
#### 2. **Configure CrashLens**
**Environment Variable (Recommended):**
```bash
# macOS/Linux
export CRASHLENS_SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
# Windows PowerShell
$env:CRASHLENS_SLACK_WEBHOOK = "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
```
**Config File (`.crashlens.yaml`):**
```yaml
slack_webhook_url: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
output_format: "slack" # Auto-format for Slack
```
#### 3. **Test Integration**
```bash
# Scan and auto-send to Slack
crashlens scan logs.jsonl --format slack
# CI/CD integration (uses env variable)
crashlens scan logs.jsonl # Automatically posts if webhook configured
```
### π **What Gets Sent**
- π¨ **Policy violations** with severity levels
- π° **Cost analysis** and potential savings
- π **Key metrics** (spend, tokens, traces)
- π **Direct links** to detailed reports
- π― **Actionable recommendations**
### π§ **Advanced Setup**
For CI/CD pipelines, GitHub Actions integration, and custom notification formats, see the complete [Slack Integration Guide](docs/SLACK_INTEGRATION.md).
---
## π― Use Cases
### π’ **Enterprise**
- **Cost Center Analysis**: Track AI spending by team/project
- **Compliance Auditing**: Policy enforcement across organizations
- **Budget Controls**: Prevent runaway AI costs
- **Performance Optimization**: Identify inefficient patterns
### π¨βπ» **Development Teams**
- **Debug LLM Integrations**: Find retry loops and fallback issues
- **Code Review**: Automated cost analysis in PRs
- **Local Testing**: Analyze logs during development
- **Performance Tuning**: Optimize prompt efficiency
### π¬ **Research & Analysis**
- **Token Usage Studies**: Understand consumption patterns
- **Model Comparison**: Cost/performance analysis
- **Efficiency Research**: LLM optimization studies
- **Trend Analysis**: Historical usage patterns
---
## π Documentation
### Quick References
- [Installation Guide](docs/INSTALLATION.md)
- [Policy Writing Guide](docs/POLICY_GUIDE.md)
- [CLI Reference](docs/CLI_REFERENCE.md)
- [API Documentation](docs/API_REFERENCE.md)
### Integrations
- [Slack Integration Guide](docs/SLACK_INTEGRATION.md)
### Troubleshooting
- [Common Issues](docs/TROUBLESHOOTING.md)
- [Windows PATH Setup](docs/WINDOWS_SETUP.md)
- [Log Format Guide](docs/LOG_FORMATS.md)
- [Performance Tuning](docs/PERFORMANCE.md)
---
## π€ Contributing
We welcome contributions! See our [Contributing Guide](CONTRIBUTING.md) for details.
### Development Setup
**macOS/Linux:**
```bash
git clone https://github.com/Crashlens/crashlens.git
cd crashlens
poetry install
poetry shell
# Run tests
pytest
# Run locally
python -m crashlens scan --demo
```
**Windows (PowerShell):**
```powershell
git clone https://github.com/Crashlens/crashlens.git
cd crashlens
poetry install
poetry shell
# Run tests
pytest
# Run locally
python -m crashlens scan --demo
```
---
## π License
CrashLens is released under the [MIT License](LICENSE).
---
## π Links
- **π¦ PyPI**: [pypi.org/project/crashlens](https://pypi.org/project/crashlens/)
- **π Documentation**: [crashlens.dev/docs](https://crashlens.dev/docs)
- **π Issues**: [github.com/Crashlens/crashlens/issues](https://github.com/Crashlens/crashlens/issues)
- **π¬ Discussions**: [github.com/Crashlens/crashlens/discussions](https://github.com/Crashlens/crashlens/discussions)
---
<div align="center">
**Made with β€οΈ by the CrashLens Team**
*Save money β’ Optimize AI β’ Ship faster*
</div>
### π **Reporting & Output**
- **Multiple output formats**: Slack, Markdown, JSON
- **Detailed trace reports**: Per-trace JSON files with issue breakdown
- **Cost summaries**: With and without trace IDs
- **Professional Markdown reports**: Generated as `report.md` after every scan
### βοΈ **Configuration & Flexibility**
- **Custom pricing config**: Configure model costs and detection thresholds
- **Input methods**: File, stdin, clipboard, demo data
- **Flexible output directories**: Customize where reports are saved
- **Robust error handling**: Works with malformed or incomplete logs
- **Non-interactive setup**: Environment variable-based configuration for CI/CD and automation
### π **Privacy & Security**
- **100% local processing**: No data leaves your machine
- **No external dependencies**: Works offline
- **CLI-first design**: Integrate into any workflow or CI/CD pipeline
### π€ **Automation & CI/CD**
- **Non-interactive init**: Zero-prompt setup for CI/CD pipelines
- **Environment variable configuration**: CRASHLENS_TEMPLATES, CRASHLENS_SEVERITY, etc.
- **GitHub Actions workflow generation**: Automated CI integration
- **Cross-platform support**: PowerShell, Bash, and shell-agnostic commands
---
## 1. Clone the Repository
Replace `<repo-link>` with the actual GitHub URL:
**Cross-platform:**
```bash
git clone <repo-link>
cd crashlens
```
---
## 2. Install Python & Poetry
CrashLens requires **Python 3.12+** and [Poetry](https://python-poetry.org/) for dependency management.
### MacOS
- Install Python (if not already):
```bash
brew install python@3.12
```
- Install Poetry (stable version):
```bash
curl -sSL https://install.python-poetry.org | python3 - --version 1.8.2
# Or with Homebrew:
brew install poetry
```
- Add Poetry to your PATH if needed:
```bash
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zprofile
source ~/.zprofile
```
- Verify installation:
```bash
poetry --version
# Should show: Poetry (version 1.8.2)
```
### Windows
β οΈ **Use PowerShell, not CMD, for these commands.**
- Install Python from [python.org](https://www.python.org/downloads/)
- Install Poetry (stable version):
```powershell
(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python - --version 1.8.2
```
- Add Poetry to your PATH if `poetry --version` returns "not found":
```powershell
$userPoetryBin = "$HOME\AppData\Roaming\Python\Scripts"
if (Test-Path $userPoetryBin -and -not ($env:Path -like "*$userPoetryBin*")) {
$env:Path += ";$userPoetryBin"
[Environment]::SetEnvironmentVariable("Path", $env:Path, "User")
Write-Output "β
Poetry path added. Restart your terminal."
} else {
Write-Output "β οΈ Poetry path not found or already added. You may need to locate poetry.exe manually."
}
```
**β οΈ Restart your terminal/PowerShell after adding to PATH.**
- Verify installation:
```powershell
poetry --version
# Should show: Poetry (version 1.8.2)
```
---
## 3. Set Up the Environment
**Cross-platform:**
```bash
# From the project root:
poetry install
```
This will create a virtual environment and install all dependencies.
To activate the environment:
```bash
poetry shell
```
---
## π€ Non-Interactive Setup & Automation
CrashLens supports fully automated, environment variable-driven setup for CI/CD pipelines and headless environments.
### Quick Start (Non-Interactive)
Set environment variables and run init without prompts:
**PowerShell:**
```powershell
$env:CRASHLENS_TEMPLATES = "all"
$env:CRASHLENS_SEVERITY = "medium"
$env:CRASHLENS_FAIL_ON_VIOLATIONS = "true"
crashlens init --non-interactive
```
**Bash/Linux:**
```bash
export CRASHLENS_TEMPLATES="all"
export CRASHLENS_SEVERITY="medium"
export CRASHLENS_FAIL_ON_VIOLATIONS="true"
crashlens init --non-interactive
```
This creates:
- `.crashlens/config.yaml` - Configuration file
- `.github/workflows/crashlens.yml` - GitHub Actions workflow (if applicable)
### Environment Variables
| Variable | Description | Default | Example |
|----------|-------------|---------|---------|
| `CRASHLENS_TEMPLATES` | Policy templates to use | `"retry-loop-prevention"` | `"all"`, `"retry-loop-prevention,budget-control"` |
| `CRASHLENS_SEVERITY` | Minimum severity threshold | `"medium"` | `"low"`, `"high"`, `"critical"` |
| `CRASHLENS_FAIL_ON_VIOLATIONS` | Exit with error on violations | `"false"` | `"true"`, `"false"` |
| `CRASHLENS_LOGS_SOURCE` | Default log source path | `"logs/"` | `"logs/"`, `".llm_logs/"`, `"traces.jsonl"` |
| `CRASHLENS_OUTPUT_FORMAT` | Report output format | `"markdown"` | `"markdown"`, `"slack"`, `"json"` |
| `CRASHLENS_CREATE_WORKFLOW` | Generate GitHub Actions workflow | `"true"` | `"true"`, `"false"` |
### CI/CD Integration
The generated `.github/workflows/crashlens.yml` provides automated log analysis on every commit:
```yaml
name: CrashLens Policy Check
on: [push, pull_request]
jobs:
crashlens-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: '3.12'
- run: pip install crashlens
- run: crashlens policy-check logs.jsonl --policy-template all --severity-threshold medium --fail-on-violations
```
### Advanced Examples
**macOS/Linux:**
```bash
# Strict monitoring with custom templates
export CRASHLENS_TEMPLATES="retry-loop-prevention,budget-control"
export CRASHLENS_SEVERITY="high"
export CRASHLENS_FAIL_ON_VIOLATIONS="true"
export CRASHLENS_LOGS_SOURCE=".llm_logs/production.jsonl"
crashlens init --non-interactive
```
**Windows (PowerShell):**
```powershell
# Strict monitoring with custom templates
$env:CRASHLENS_TEMPLATES = "retry-loop-prevention,budget-control"
$env:CRASHLENS_SEVERITY = "high"
$env:CRASHLENS_FAIL_ON_VIOLATIONS = "true"
$env:CRASHLENS_LOGS_SOURCE = ".llm_logs/production.jsonl"
crashlens init --non-interactive
```
**Docker/Container Setup:**
```dockerfile
ENV CRASHLENS_TEMPLATES="all"
ENV CRASHLENS_SEVERITY="medium"
ENV CRASHLENS_FAIL_ON_VIOLATIONS="true"
RUN crashlens init --non-interactive
```
For complete documentation, see [docs/NON-INTERACTIVE-GUIDE.md](docs/NON-INTERACTIVE-GUIDE.md) and [docs/NON-INTERACTIVE-QUICK-REFERENCE.md](docs/NON-INTERACTIVE-QUICK-REFERENCE.md).
---
## 4. Running CrashLens
You can run CrashLens via Poetry or as a Python module:
### Basic Scan (from file)
**Cross-platform:**
```bash
crashlens scan examples/retry-test.jsonl
```
### Demo Mode (built-in sample data)
**Cross-platform:**
```bash
crashlens scan --demo
```
**Sample output:**
```
π¨ **CrashLens Token Waste Report** π¨
π Analysis Date: 2025-07-31 15:22:08
| Metric | Value |
|--------|-------|
| Total AI Spend | $0.09 |
| Total Potential Savings | $0.07 |
| Wasted Tokens | 1,414 |
| Issues Found | 8 |
| Traces Analyzed | 12 |
π’ **Fallback Failure** | 5 traces | $0.07 wasted | Fix: remove redundant fallbacks
π― **Wasted tokens**: 1,275
π **Traces** (5): demo_fallback_01, demo_fallback_02, demo_fallback_03, demo_fallback_04, demo_fallback_05
β **Overkill Model** | 2 traces | $0.0007 wasted | Fix: optimize usage
π― **Wasted tokens**: 31
π **Traces** (2): demo_overkill_01, demo_overkill_02
π **Retry Loop** | 1 traces | $0.0002 wasted | Fix: exponential backoff
π― **Wasted tokens**: 108
π **Traces** (1): demo_retry_01
## Top Expensive Traces
| Rank | Trace ID | Model | Cost |
|------|----------|-------|------|
| 1 | demo_norm_03 | gpt-4 | $0.03 |
| 2 | demo_norm_04 | gpt-4 | $0.02 |
| 3 | demo_fallback_05 | gpt-3.5-turbo | $0.02 |
## Cost by Model
| Model | Cost | Percentage |
|-------|------|------------|
| gpt-4 | $0.09 | 99% |
| gpt-3.5-turbo | $0.0012 | 1% |
---
## Why CrashLens? (vs. grep + Excel, LangSmith, or basic logging)
- π **grep + spreadsheet**: Too manual, error-prone, no cost context
- πΈ **LangSmith**: Powerful but complex, requires full tracing/observability stack
- π **Logging without cost visibility**: You miss $ waste and optimization opportunities
- π **CrashLens runs 100% locallyβno data leaves your machine.**
---
## Features (Ultra-Specific)
- β
Detects retry-loop storms across trace IDs
- β
Flags gpt-4, Claude, Gemini, and other expensive model usage where a cheaper model (e.g., gpt-3.5, Claude Instant) would suffice
- β
Scans stdin logs from LangChain, LlamaIndex, custom logging
- β
Generates Markdown cost reports with per-trace waste
---
## What Makes CrashLens Different?
- π΅ **Model pricing fallback** (auto-detects/corrects missing cost info)
- π **Security-by-design** (runs 100% locally, no API calls, no data leaves your machine)
- π¦ **Coming soon**: Policy enforcement, live CLI firewall, more integrations
## π Log File Structure
**Your logs must be in JSONL format (one JSON object per line) and follow this structure:**
```json
{"traceId": "trace_9", "startTime": "2025-07-19T10:36:13Z", "input": {"model": "gpt-3.5-turbo", "prompt": "How do solar panels work?"}, "usage": {"prompt_tokens": 25, "completion_tokens": 110, "total_tokens": 135}, "cost": 0.000178}
```
- Each line is a separate API call (no commas or blank lines between objects).
- Fields must be nested as shown: `input.model`, `input.prompt`, `usage.completion_tokens`, etc.
**Required fields:**
- `traceId` (string): Unique identifier for a group of related API calls
- `input.model` (string): Model name (e.g., `gpt-4`, `gpt-3.5-turbo`)
- `input.prompt` (string): The prompt sent to the model
- `usage.completion_tokens` (int): Number of completion tokens used
**Optional fields:**
- `cost` (float): Cost of the API call
- `name`, `startTime`, etc.: Any other metadata
π‘ CrashLens expects JSONL with per-call metrics (model, tokens, cost). Works with LangChain logs, OpenAI api.log, Claude, Gemini, and more.
---
## π Usage: Command Line Examples
After installation, use the `crashlens` command in your terminal (or `python -m crashlens` if running from source).
### 1. **Scan a log file**
**Cross-platform:**
```bash
crashlens scan path/to/your-logs.jsonl
```
- Scans the specified log file and generates a `report.md` in your current directory.
### 2. **Demo mode (built-in sample data)**
**Cross-platform:**
```bash
crashlens scan --demo
```
- Runs analysis on built-in example logs (requires `examples-logs/demo-logs.jsonl` file).
- **Note**: If installing from PyPI, you'll need to create sample logs or use your own data.
- **From source**: Demo data is included in the repository.
### 3. **Scan from stdin (pipe)**
**macOS/Linux:**
```bash
cat path/to/your-logs.jsonl | crashlens scan --stdin
```
**Windows (PowerShell):**
```powershell
Get-Content path/to/your-logs.jsonl | crashlens scan --stdin
```
- Reads logs from standard input (useful for pipelines or quick tests).
### 4. **Paste logs interactively**
**Cross-platform:**
```bash
crashlens scan --paste
```
- Reads JSONL data from clipboard (paste and press Enter to finish).
### 5. **Output format options**
**Cross-platform:**
```bash
crashlens scan logs.jsonl --format slack # Slack-friendly format (default)
crashlens scan logs.jsonl --format markdown # Markdown format
crashlens scan logs.jsonl --format json # JSON output
```
- Choose the format that best fits your workflow or team communication.
### 6. **Detailed reporting**
**Cross-platform:**
```bash
crashlens scan logs.jsonl --detailed
crashlens scan logs.jsonl --detailed --detailed-dir custom_reports/
```
- Creates detailed JSON files in `detailed_output/` (or custom directory) by issue type.
- Generates separate files: `fallback_failure.json`, `retry_loop.json`, etc.
### 7. **Summary options**
**Cross-platform:**
```bash
crashlens scan logs.jsonl --summary # Cost summary with breakdown
crashlens scan logs.jsonl --summary-only # Summary without trace IDs
```
- Shows cost analysis with or without detailed trace information.
### 8. **Custom pricing configuration**
**Cross-platform:**
```bash
crashlens scan logs.jsonl --config custom-pricing.yaml
```
- Use custom model pricing and detection thresholds.
- Default config is located in `crashlens/config/pricing.yaml`.
### 14. **Combined options**
**Cross-platform:**
```bash
# Multiple scan options can be combined
crashlens scan logs.jsonl --format json --detailed --summary --config custom.yaml
# Policy checking with custom settings
crashlens policy-check logs.jsonl --policy-template all --severity-threshold high --fail-on-violations
```
**macOS/Linux:**
```bash
# Non-interactive setup with custom environment
CRASHLENS_TEMPLATES="retry-loop-prevention,budget-control" crashlens init --non-interactive
```
**Windows (PowerShell):**
```powershell
# Non-interactive setup with custom environment
$env:CRASHLENS_TEMPLATES = "retry-loop-prevention,budget-control"; crashlens init --non-interactive
```
- Mix and match options for your specific analysis needs.
- Environment variables can be combined with any command.
### 11. **Project setup and configuration**
**Cross-platform:**
```bash
crashlens init # Interactive setup wizard
crashlens init --non-interactive # Automated setup (uses environment variables)
crashlens list-policy-templates # List available policy templates
```
- Set up CrashLens configuration and GitHub Actions workflow.
- Non-interactive mode uses environment variables for CI/CD integration.
### 12. **Policy checking**
**Cross-platform:**
```bash
crashlens policy-check logs.jsonl --policy-template all # Check all policies
crashlens policy-check logs.jsonl --policy-template retry-loop-prevention # Specific policy
crashlens policy-check logs.jsonl --fail-on-violations # Exit with error code
crashlens policy-check logs.jsonl --severity-threshold high # Filter by severity
```
- Validate logs against policy rules without running full waste detection.
- Useful for CI/CD gate checks and compliance validation.
### 13. **Get help**
**Cross-platform:**
```bash
crashlens --help # Main help
crashlens scan --help # Scan command help
crashlens init --help # Init command help
crashlens policy-check --help # Policy check help
```
- Shows all available options and usage details for each command.
---
## π Quick Command Reference
**Cross-platform commands:**
```bash
# Basic Usage
crashlens scan <logfile> # Basic log analysis
crashlens scan --demo # Test with demo data
# Input Methods
crashlens scan --stdin # Read from pipe/stdin
crashlens scan --paste # Read from clipboard
crashlens scan logs.jsonl # Read from file
# Output Formats
crashlens scan logs.jsonl -f slack # Slack format (default)
crashlens scan logs.jsonl -f markdown # Markdown format
crashlens scan logs.jsonl -f json # JSON format
# Reporting Options
crashlens scan logs.jsonl --summary # Show cost summary
crashlens scan logs.jsonl --summary-only # Summary without trace IDs
crashlens scan logs.jsonl --detailed # Generate detailed JSON reports
# Policy Checking
crashlens policy-check logs.jsonl --policy-template all # Check against all policies
crashlens policy-check logs.jsonl --policy-template retry-loop-prevention # Specific policy
crashlens policy-check logs.jsonl --fail-on-violations # Exit with error on violations
# Setup & Configuration
crashlens init # Interactive setup wizard
crashlens init --non-interactive # Automated setup using environment variables
crashlens list-policy-templates # List available policy templates
# Advanced Options
crashlens scan logs.jsonl -c custom.yaml # Custom pricing config
crashlens scan logs.jsonl --detailed-dir reports/ # Custom output directory
# Version Info
crashlens --version # Show current version
```
---
## 𧩠Example Workflow
1. **Install CrashLens:**
**Cross-platform:**
```bash
pip install crashlens
# OR clone and install from source as above
```
2. **Set up CrashLens configuration:**
**Interactive setup:**
**Cross-platform:**
```bash
crashlens init
# Follow the prompts to configure policies, severity, etc.
```
**Non-interactive setup (for CI/CD):**
**macOS/Linux:**
```bash
export CRASHLENS_TEMPLATES="all"
export CRASHLENS_SEVERITY="medium"
export CRASHLENS_FAIL_ON_VIOLATIONS="true"
crashlens init --non-interactive
```
**Windows (PowerShell):**
```powershell
$env:CRASHLENS_TEMPLATES = "all"
$env:CRASHLENS_SEVERITY = "medium"
$env:CRASHLENS_FAIL_ON_VIOLATIONS = "true"
crashlens init --non-interactive
```
3. **Prepare your log files:**
**Required**: CrashLens needs LLM usage logs in JSONL format. Place them in:
- `.llm_logs/` directory (recommended)
- `logs/` directory
- Or specify any `*.jsonl` file path
**Getting logs from LangFuse:**
**macOS/Linux:**
```bash
mkdir -p .llm_logs
# Export your traces from LangFuse dashboard or API
```
**Windows (PowerShell):**
```powershell
New-Item -ItemType Directory -Force -Path .llm_logs
# Export your traces from LangFuse dashboard or API
```
**Getting logs from OpenAI/custom usage:**
```python
# Example: Log API calls to .llm_logs/usage.jsonl
import json
log_entry = {
"model": "gpt-4",
"usage": {"total_tokens": 1500},
"cost": 0.03,
"timestamp": "2025-01-15T10:30:00Z"
}
with open('.llm_logs/usage.jsonl', 'a') as f:
f.write(json.dumps(log_entry) + '\n')
```
**No logs yet?** Generate test data:
**Cross-platform:**
```bash
crashlens scan --demo
```
4. **Analyze your logs:**
**Cross-platform:**
```bash
crashlens policy-check .llm_logs/*.jsonl --policy-template all
# OR for a specific file
crashlens policy-check path/to/your-logs.jsonl
# OR for waste pattern analysis
crashlens scan .llm_logs/*.jsonl --format markdown --detailed
```
5. **Review the results:** Open the generated markdown report to review findings and optimization suggestions.
6. **CI/CD Integration:** If you used `crashlens init`, a GitHub Actions workflow was created in `.github/workflows/crashlens.yml` for automated analysis.
---
## π Logging Helper
To make log analysis seamless, you can use our [`crashlens-logger`](https://github.com/Crashlens/logger) package to emit logs in the correct structure for CrashLens. This ensures compatibility and reduces manual formatting.
**Example usage:**
```sh
pip install --upgrade crashlens_logger
```
```python
from crashlens_logger import CrashLensLogger
logger = CrashLensLogger()
logger.log_event(
traceId=trace_id,
startTime=start_time,
endTime=end_time,
input={"model": model, "prompt": prompt},
usage=usage
# Optionally add: type, level, metadata, name, etc.
)
```
- The logger writes each call as a JSONL line in the required format.
- See the [`crashlens-logger` repo](https://github.com/Crashlens/logger) for full docs and advanced usage.
---
## π Troubleshooting & Tips
- **File not found:** Make sure the path to your log file is correct.
- **No traces found:** Your log file may be empty or not in the expected format.
- **Cost is $0.00:** Check that your logβs model names match those in the pricing config.
- **Virtual environment issues:** Make sure youβre using the right Python environment.
- **Need help?** Use `crashlens --help` for all options.
---
## π οΈ Full Installation (Advanced/Dev)
### **Alternative: Install from Source (GitHub)**
If you want the latest development version or want to contribute, you can install CrashLens from source:
1. **Clone the repository:**
**Cross-platform:**
```bash
git clone <repo-link>
cd crashlens
```
2. **(Optional but recommended) Create a virtual environment:**
- **On Mac/Linux:**
```bash
python3 -m venv .venv
source .venv/bin/activate
```
- **On Windows (PowerShell):**
```powershell
python -m venv .venv
.venv\Scripts\Activate.ps1
```
3. **Install dependencies:**
**Cross-platform:**
```bash
pip install -r requirements.txt
# Or, if using Poetry:
poetry install
```
4. **Run CrashLens:**
**Cross-platform:**
```bash
python -m crashlens scan path/to/your-logs.jsonl
# Or, if using Poetry:
poetry run crashlens scan path/to/your-logs.jsonl
```
---
## π¬ Support
For questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.
---
## π License
MIT License - see LICENSE file for details.
---
**CrashLens: Find your wasted tokens. Save money. Optimize your AI usage.**
### Scan from stdin (pipe or paste)
**macOS/Linux:**
```bash
cat examples/retry-test.jsonl | poetry run crashlens scan --stdin
```
**Windows (PowerShell):**
```powershell
Get-Content examples/retry-test.jsonl | poetry run crashlens scan --stdin
```
---
## 5. Output: The Markdown Report
After every scan, CrashLens creates or updates `report.md` in your current directory.
### Example Structure
```
# CrashLens Token Waste Report
π§Ύ **Total AI Spend**: $0.123456
π° **Total Potential Savings**: $0.045678
| Trace ID | Model | Prompt | Completion Length | Cost | Waste Type |
|----------|-------|--------|------------------|------|------------|
| trace_001 | gpt-4 | ... | 3 | $0.00033 | Overkill |
| ... | ... | ... | ... | ... | ... |
## Overkill Model Usage (5 issues)
- ...
## Retry Loops (3 issues)
- ...
## Fallback Failures (2 issues)
- ...
```
---
## 6. Troubleshooting
- **File not found:** Ensure the path to your log file is correct.
- **No traces found:** Your log file may be empty or malformed.
- **Cost is $0.00:** Check that your `pricing.yaml` matches the model names in your logs.
- **Virtual environment issues:** Use `poetry run` to ensure dependencies are available.
---
## 7. Example Commands
```sh
# Scan a log file
poetry run crashlens scan examples/demo-logs.jsonl
# Use demo data
poetry run crashlens scan --demo
# Scan from stdin
cat examples/demo-logs.jsonl | poetry run crashlens scan --stdin
```
---
## π Complete Command Reference
### Basic Usage
**Cross-platform:**
```bash
crashlens scan [OPTIONS] [LOGFILE]
```
### π― Examples
**Cross-platform:**
```bash
# Scan a specific log file
crashlens scan logs.jsonl
# Run on built-in sample logs
crashlens scan --demo
# Read logs from clipboard
crashlens scan --paste
# Generate detailed category JSON reports
crashlens scan --detailed
# Cost summary with categories
crashlens scan --summary
# Show summary only (no trace details)
crashlens scan --summary-only
```
**Platform-specific pipe commands:**
**macOS/Linux:**
```bash
# Pipe logs via stdin
cat logs.jsonl | crashlens scan --stdin
```
**Windows (PowerShell):**
```powershell
# Pipe logs via stdin
Get-Content logs.jsonl | crashlens scan --stdin
```
### π§ All Options
| Option | Description | Example |
|--------|-------------|---------|
| `-f, --format` | Output format: `slack`, `markdown`, `json` | `--format json` |
| `-c, --config` | Custom pricing config file path | `--config my-pricing.yaml` |
| `--demo` | Use built-in demo data (requires examples-logs/demo-logs.jsonl) | `crashlens scan --demo` |
| `--stdin` | Read from standard input | `cat logs.jsonl \| crashlens scan --stdin` (Unix) / `Get-Content logs.jsonl \| crashlens scan --stdin` (Windows) |
| `--paste` | Read JSONL data from clipboard | `crashlens scan --paste` |
| `--summary` | Show cost summary with breakdown | `crashlens scan --summary` |
| `--summary-only` | Summary without trace IDs | `crashlens scan --summary-only` |
| `--detailed` | Generate detailed category JSON reports | `crashlens scan --detailed` |
| `--detailed-dir` | Directory for detailed reports (default: detailed_output) | `--detailed-dir my_reports` |
| `--help` | Show help message | `crashlens scan --help` |
### π Detailed Reports
When using `--detailed`, CrashLens generates grouped category files:
- `detailed_output/fallback_failure.json` - All fallback failure issues
- `detailed_output/retry_loop.json` - All retry loop issues
- `detailed_output/fallback_storm.json` - All fallback storm issues
- `detailed_output/overkill_model.json` - All overkill model issues
Each file contains:
- Summary with total issues, affected traces, costs
- All issues of that type with trace IDs and details
- Specific suggestions for that category
### π Input Sources
CrashLens supports multiple input methods:
1. **File input**: `crashlens scan path/to/logs.jsonl`
2. **Demo mode**: `crashlens scan --demo` (requires examples-logs/demo-logs.jsonl file)
3. **Standard input**:
- **macOS/Linux**: `cat logs.jsonl | crashlens scan --stdin`
- **Windows**: `Get-Content logs.jsonl | crashlens scan --stdin`
4. **Clipboard**: `crashlens scan --paste` (paste logs interactively)
### π Output Formats
- **slack** (default): Slack-formatted report for team sharing
- **markdown**: Clean Markdown for documentation
- **json**: Machine-readable JSON for automation
### π‘ Pro Tips
- Use `--demo` to test CrashLens without your own logs
- Use `--detailed` to get actionable JSON reports for each issue category
- Use `--summary-only` for executive summaries without trace details
- Combine `--stdin` with shell pipelines for automation
---
## 8. Support
For questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.
---
Enjoy using CrashLens! π―
Raw data
{
"_id": null,
"home_page": null,
"name": "crashlens",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": null,
"author": "Aditya Singh",
"author_email": "coding103856@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d1/19/faf9ff57e0bf86b77ccd180898484ade728377bded76350e2b66f7702ded/crashlens-2.9.15.tar.gz",
"platform": null,
"description": "# \ud83e\udde0 CrashLens: AI Token Waste Detective\n\n<div align=\"center\">\n\n[](https://pypi.org/project/crashlens/)\n[](https://python.org)\n[](LICENSE)\n[]()\n\n**The Ultimate AI Cost Optimization Tool**\n\n*Detect hidden token waste \u2022 Prevent budget overruns \u2022 Optimize LLM usage*\n\n[Quick Start](#-quick-start) \u2022 [Features](#-features) \u2022 [Documentation](#-documentation) \u2022 [Examples](#-examples)\n\n</div>\n\n---\n\n## \ud83c\udfaf What is CrashLens?\n\nCrashLens is a **developer-first CLI tool** that analyzes your AI API logs to uncover **hidden token waste**, retry loops, model overkill, and inefficient patterns. It helps you **optimize OpenAI, Anthropic, and Langfuse usage** by providing actionable cost-saving insights.\n\n### \ud83d\udd0d **Why CrashLens?**\n\n> *\"You can't optimize what you can't see.\"*\n\n- **\ud83d\udcb0 Cost Savings**: Identify 40-60% potential savings in AI spending\n- **\ud83d\udd12 Privacy First**: 100% local analysis, no data leaves your machine \n- **\u26a1 Production Ready**: Battle-tested policy engine with CI/CD integration\n- **\ud83c\udfaf Actionable**: Specific recommendations, not just analytics\n\n---\n\n## \ud83d\ude80 Quick Start\n\n### Prerequisites\n- **Python 3.12+** ([Download here](https://python.org/downloads/))\n- AI usage logs in JSONL format (OpenAI, Anthropic, Langfuse)\n\n### Installation & Setup\n\n```bash\n# Install CrashLens\npip install crashlens\n\n# Interactive setup wizard\ncrashlens init\n\n# Quick scan with demo data\ncrashlens scan --demo\n\n# Analyze your logs\ncrashlens scan your-logs.jsonl\n\n# Optional: Set up Slack notifications\nexport CRASHLENS_SLACK_WEBHOOK=\"your-webhook-url\"\ncrashlens scan your-logs.jsonl # Auto-posts to Slack\n```\n\n### Non-Interactive Setup (CI/CD)\n\n**macOS/Linux:**\n```bash\nexport CRASHLENS_TEMPLATES=\"all\"\nexport CRASHLENS_SEVERITY=\"medium\"\ncrashlens init --non-interactive\n```\n\n**Windows (PowerShell):**\n```powershell\n$env:CRASHLENS_TEMPLATES = \"all\"\n$env:CRASHLENS_SEVERITY = \"medium\"\ncrashlens init --non-interactive\n```\n\n---\n\n## \ufffd Core Features\n\n### \ud83d\udd75\ufe0f **Waste Detection Engine**\n- **\ud83d\udd04 Retry Loop Detection**: Exponential backoff failures, redundant retries\n- **\u2753 Model Overkill**: GPT-4 used for simple 3-token completions \n- **\ud83d\udce2 Fallback Storms**: Cascading model failures wasting tokens\n- **\u26a1 Prompt Inefficiency**: Long prompts generating tiny responses\n- **\ud83d\udcb8 Budget Violations**: Expensive calls exceeding thresholds\n\n### \ud83d\udee1\ufe0f **Policy Enforcement**\n- **Production-Grade Rules**: 10+ built-in policy templates\n- **Custom Policies**: YAML-based rule definitions\n- **CI/CD Integration**: Fail builds on policy violations\n- **Severity Levels**: Critical, High, Medium, Low classifications\n- **Smart Suppression**: Prevents alert fatigue\n\n### \ud83d\udcca **Analysis & Reporting**\n- **Multi-Format Reports**: Markdown, JSON, Slack-ready notifications\n- **Auto Slack Integration**: Real-time team notifications with webhook setup\n- **Cost Breakdown**: Per-model, per-trace, per-pattern analysis\n- **Token Accounting**: Detailed waste calculations\n- **Trend Analysis**: Historical cost patterns\n- **ROI Calculator**: Quantified savings recommendations\n\n---\n\n## \ud83d\udcbb Commands Overview\n\n### \ud83d\udd0d **Scan & Analyze**\n```bash\n# Basic scan with smart reporting\ncrashlens scan logs.jsonl\n\n# Generate detailed JSON reports\ncrashlens scan logs.jsonl --detailed\n\n# Demo mode with sample data\ncrashlens scan --demo\n\n# Custom output formats\ncrashlens scan logs.jsonl --format slack # Slack-ready format\ncrashlens scan logs.jsonl --format json # Machine-readable format\n\n# Auto-post to Slack (requires CRASHLENS_SLACK_WEBHOOK env var)\nexport CRASHLENS_SLACK_WEBHOOK=\"your-webhook-url\"\ncrashlens scan logs.jsonl # Auto-sends to Slack when configured\n```\n\n### \ud83d\udee1\ufe0f **Policy Enforcement**\n```bash\n# Check logs against all policies (generates report.md)\ncrashlens policy-check logs.jsonl --policy-template all\n\n# Generate detailed JSON report for CI/CD integration\ncrashlens policy-check logs.jsonl --policy-template all --detailed\n\n# Custom output paths and quiet mode\ncrashlens policy-check logs.jsonl --policy-template all --detailed \\\n --out-report violations.md --out-detailed violations.json --quiet\n\n# Use specific policy templates\ncrashlens policy-check logs.jsonl --policy-template model-overkill-detection,retry-loop-prevention\n\n# Custom policy file\ncrashlens policy-check logs.jsonl --policy-file my-policy.yaml\n\n# Fail on violations (CI/CD mode)\ncrashlens policy-check logs.jsonl --policy-template all --fail-on-violations\n\n# Privacy-safe reports (strip PII, exclude content)\ncrashlens policy-check logs.jsonl --policy-template all --strip-pii --no-content\n```\n\n**\ud83d\udcc1 Output Organization:**\nPolicy violation reports are automatically organized in the `policy-violations/` folder:\n- `policy-violations/reports/` - Concise Markdown summaries\n- `policy-violations/traces/` - Detailed JSON analysis files\n- See `policy-violations/README.md` for complete documentation\n\n### \ud83d\udee0\ufe0f **Management & Simulation**\n\n**Cross-platform:**\n```bash\n# List available policy templates\ncrashlens list-policy-templates\n\n# Simulate different usage patterns\ncrashlens simulate --pattern retry-loop --count 100\n\n# Setup project with policies\ncrashlens init\n```\n\n---\n\n## \ud83d\udcc8 Example Report Output\n\n### \ud83d\udea8 **Cost Analysis Report**\n```markdown\n\ud83d\udea8 CrashLens Token Waste Report \ud83d\udea8\n\ud83d\udcca Analysis Date: 2025-08-17\n\n\ud83d\udccb Report Summary:\n\u2022 \ud83d\udcb0 Total AI Spend: $859.52\n\u2022 \ud83d\udd25 Potential Savings: $859.52 (100%)\n\u2022 \ud83c\udfaf Wasted Tokens: 38,213,010\n\u2022 \u26a0\ufe0f Issues Found: 53,185\n\u2022 \ud83d\udcc8 Traces Analyzed: 156\n\n\ud83d\udd04 Retry Loop \u2022 187 traces \u2022 $859.52 wasted\n \ud83d\udca1 Fix: exponential backoff\n \ud83c\udfaf Wasted tokens: 24,555,498\n \ud83d\udd17 Traces: trace_retry_loop_01, trace_retry_loop_02, +185 more\n\n\u2753 Overkill Model \u2022 52,998 traces \u2022 $560.24 wasted\n \ud83d\udca1 Fix: optimize usage\n \ud83c\udfaf Wasted tokens: 13,657,512\n \ud83d\udd17 Traces: trace_overkill_01, trace_overkill_02, +52,996 more\n\n\ud83c\udfc6 Top Expensive Traces:\n\u2022 #1 \u2192 trace_norm_76 \u2192 gpt-4 \u2192 $65.78\n\u2022 #2 \u2192 trace_norm_65 \u2192 gpt-4 \u2192 $52.60\n\u2022 #3 \u2192 trace_norm_38 \u2192 gpt-4 \u2192 $44.10\n\n\ud83e\udd16 Cost by Model:\n\u2022 gpt-4 \u2192 $845.65 (98%)\n\u2022 gpt-3.5-turbo \u2192 $13.87 (2%)\n```\n\n### \ud83d\udee1\ufe0f **Policy Violations Report**\n```markdown\n\u26a0\ufe0f Found 431,970 policy violations:\n\n\ud83d\udea8 CRITICAL SEVERITY (6,534 violations):\n 1. high_cost_per_token (line 62437)\n Reason: cost=0.06075 (rule: >0.05)\n Action: fail\n Suggestion: Very expensive API call detected (over $0.05).\n Immediate actions:\n - Review if this cost is justified\n - Check for prompt optimization opportunities\n - Consider model downgrading\n\n\u26a0\ufe0f HIGH SEVERITY (227,238 violations):\n 1. gpt4_for_simple_tasks (line 727)\n Reason: model=gpt-4 AND prompt_tokens=5 (rule: <50)\n Action: fail\n Suggestion: GPT-4 used for simple task.\n Cost optimization opportunities:\n - Use gpt-4o-mini (90% cheaper, similar quality)\n - Use gpt-3.5-turbo for classification <500 tokens\n - Reserve GPT-4 for complex reasoning tasks\n```\n\n---\n\n## \ud83c\udfd7\ufe0f Policy Templates\n\nCrashLens includes production-ready policy templates:\n\n| Template | Purpose | Estimated Savings |\n|----------|---------|-------------------|\n| `model-overkill-detection` | Prevent expensive models for simple tasks | 30-50% |\n| `retry-loop-prevention` | Block inefficient retry patterns | 20-40% |\n| `prompt-optimization` | Optimize prompt efficiency | 15-30% |\n| `budget-protection` | Enforce spending limits | Varies |\n| `fallback-storm-detection` | Prevent cascading failures | 10-35% |\n| `context-window-optimization` | Efficient context usage | 10-25% |\n| `production-ready` | Combined rules for production | 40-60% |\n\n### Custom Policy Example\n```yaml\n# my-policy.yaml\nmetadata:\n name: \"Custom Cost Control\"\n description: \"Strict cost controls for production\"\n \nrules:\n - id: expensive_single_call\n description: \"Block very expensive calls\"\n match:\n cost: \">0.10\"\n action: fail\n severity: critical\n suggestion: |\n Call exceeds $0.10 threshold.\n - Review prompt optimization\n - Consider model downgrading\n - Break into smaller requests\n```\n\n---\n\n## \ud83d\udd27 Advanced Usage\n\n### CI/CD Integration\n\n#### GitHub Actions\n```yaml\nname: AI Cost Control\non: [push, pull_request]\n\njobs:\n cost-check:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - uses: actions/setup-python@v4\n with:\n python-version: '3.12'\n \n - name: Install CrashLens\n run: pip install crashlens\n \n - name: Policy Check\n run: |\n crashlens policy-check logs/*.jsonl \\\n --policy-template all \\\n --fail-on-violations\n```\n\n#### Docker Integration\n```dockerfile\nFROM python:3.12-slim\n\nRUN pip install crashlens\n\nWORKDIR /app\nCOPY logs/ ./logs/\n\nCMD [\"crashlens\", \"policy-check\", \"logs/*.jsonl\", \"--policy-template\", \"all\"]\n```\n\n### Programmatic Usage\n```python\nfrom crashlens.policy.engine import PolicyEngine\nfrom crashlens.parsers.langfuse import LangfuseParser\n\n# Load and analyze logs\nparser = LangfuseParser()\ntraces = parser.parse_file(\"logs.jsonl\")\n\n# Apply policies\nengine = PolicyEngine(\"policy.yaml\")\nviolations = engine.check_logs(traces)\n\nprint(f\"Found {len(violations)} violations\")\n```\n\n---\n\n## \ud83d\udcc1 Supported Log Formats\n\n### OpenAI API Logs\n```jsonl\n{\"model\": \"gpt-4\", \"usage\": {\"prompt_tokens\": 10, \"completion_tokens\": 5}, \"cost\": 0.0003}\n```\n\n### Langfuse Logs\n```jsonl\n{\"input\": {\"model\": \"gpt-4\"}, \"usage\": {\"promptTokens\": 10, \"completionTokens\": 5}, \"calculatedTotalCost\": 0.0003}\n```\n\n### Anthropic Logs\n```jsonl\n{\"model\": \"claude-3-opus\", \"usage\": {\"input_tokens\": 10, \"output_tokens\": 5}, \"cost\": 0.0003}\n```\n\n---\n\n## \ud83d\udee0\ufe0f Configuration\n\n### Environment Variables\n\n**macOS/Linux:**\n```bash\nexport CRASHLENS_TEMPLATES=\"all\" # Default policy templates\nexport CRASHLENS_SEVERITY=\"medium\" # Minimum severity level\nexport CRASHLENS_OUTPUT_FORMAT=\"slack\" # Report format (slack/markdown/json)\nexport CRASHLENS_SLACK_WEBHOOK=\"https://hooks.slack.com/services/YOUR/WEBHOOK/URL\"\n```\n\n**Windows (PowerShell):**\n```powershell\n$env:CRASHLENS_TEMPLATES = \"all\" # Default policy templates\n$env:CRASHLENS_SEVERITY = \"medium\" # Minimum severity level\n$env:CRASHLENS_OUTPUT_FORMAT = \"slack\" # Report format (slack/markdown/json)\n$env:CRASHLENS_SLACK_WEBHOOK = \"https://hooks.slack.com/services/YOUR/WEBHOOK/URL\"\n```\n\n### Config File (`.crashlens.yaml`)\n```yaml\ndefault_templates: [\"model-overkill-detection\", \"retry-loop-prevention\"]\nseverity_threshold: \"medium\"\noutput_format: \"slack\" # Auto-formats for Slack when webhook is configured\nfail_on_violations: true\nslack_webhook_url: \"https://hooks.slack.com/services/YOUR/WEBHOOK/URL\"\n```\n\n---\n\n## \ud83d\udd14 Slack Webhook Integration\n\nCrashLens supports **automatic Slack notifications** for team collaboration and alerting. Get instant notifications when policy violations are detected or cost thresholds are exceeded.\n\n### \ud83d\ude80 Quick Setup\n\n#### 1. **Get Your Slack Webhook URL**\n1. Go to [Slack Apps](https://api.slack.com/apps) \u2192 **Create New App**\n2. Choose **From scratch** \u2192 Name your app \u2192 Select workspace\n3. Go to **Incoming Webhooks** \u2192 Toggle **On** \u2192 **Add New Webhook to Workspace**\n4. Select your channel \u2192 **Allow** \u2192 Copy the webhook URL\n\n#### 2. **Configure CrashLens**\n\n**Environment Variable (Recommended):**\n```bash\n# macOS/Linux\nexport CRASHLENS_SLACK_WEBHOOK=\"https://hooks.slack.com/services/YOUR/WEBHOOK/URL\"\n\n# Windows PowerShell\n$env:CRASHLENS_SLACK_WEBHOOK = \"https://hooks.slack.com/services/YOUR/WEBHOOK/URL\"\n```\n\n**Config File (`.crashlens.yaml`):**\n```yaml\nslack_webhook_url: \"https://hooks.slack.com/services/YOUR/WEBHOOK/URL\"\noutput_format: \"slack\" # Auto-format for Slack\n```\n\n#### 3. **Test Integration**\n```bash\n# Scan and auto-send to Slack\ncrashlens scan logs.jsonl --format slack\n\n# CI/CD integration (uses env variable)\ncrashlens scan logs.jsonl # Automatically posts if webhook configured\n```\n\n### \ud83d\udccb **What Gets Sent**\n- \ud83d\udea8 **Policy violations** with severity levels\n- \ud83d\udcb0 **Cost analysis** and potential savings\n- \ud83d\udcca **Key metrics** (spend, tokens, traces)\n- \ud83d\udd17 **Direct links** to detailed reports\n- \ud83c\udfaf **Actionable recommendations**\n\n### \ud83d\udd27 **Advanced Setup**\nFor CI/CD pipelines, GitHub Actions integration, and custom notification formats, see the complete [Slack Integration Guide](docs/SLACK_INTEGRATION.md).\n\n---\n\n## \ud83c\udfaf Use Cases\n\n### \ud83c\udfe2 **Enterprise**\n- **Cost Center Analysis**: Track AI spending by team/project\n- **Compliance Auditing**: Policy enforcement across organizations\n- **Budget Controls**: Prevent runaway AI costs\n- **Performance Optimization**: Identify inefficient patterns\n\n### \ud83d\udc68\u200d\ud83d\udcbb **Development Teams**\n- **Debug LLM Integrations**: Find retry loops and fallback issues\n- **Code Review**: Automated cost analysis in PRs\n- **Local Testing**: Analyze logs during development\n- **Performance Tuning**: Optimize prompt efficiency\n\n### \ud83d\udd2c **Research & Analysis**\n- **Token Usage Studies**: Understand consumption patterns\n- **Model Comparison**: Cost/performance analysis\n- **Efficiency Research**: LLM optimization studies\n- **Trend Analysis**: Historical usage patterns\n\n---\n\n## \ud83d\udcda Documentation\n\n### Quick References\n- [Installation Guide](docs/INSTALLATION.md)\n- [Policy Writing Guide](docs/POLICY_GUIDE.md)\n- [CLI Reference](docs/CLI_REFERENCE.md)\n- [API Documentation](docs/API_REFERENCE.md)\n\n### Integrations\n- [Slack Integration Guide](docs/SLACK_INTEGRATION.md)\n\n### Troubleshooting\n- [Common Issues](docs/TROUBLESHOOTING.md)\n- [Windows PATH Setup](docs/WINDOWS_SETUP.md)\n- [Log Format Guide](docs/LOG_FORMATS.md)\n- [Performance Tuning](docs/PERFORMANCE.md)\n\n---\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! See our [Contributing Guide](CONTRIBUTING.md) for details.\n\n### Development Setup\n\n**macOS/Linux:**\n```bash\ngit clone https://github.com/Crashlens/crashlens.git\ncd crashlens\npoetry install\npoetry shell\n\n# Run tests\npytest\n\n# Run locally\npython -m crashlens scan --demo\n```\n\n**Windows (PowerShell):**\n```powershell\ngit clone https://github.com/Crashlens/crashlens.git\ncd crashlens\npoetry install\npoetry shell\n\n# Run tests\npytest\n\n# Run locally\npython -m crashlens scan --demo\n```\n\n---\n\n## \ud83d\udcc4 License\n\nCrashLens is released under the [MIT License](LICENSE).\n\n---\n\n## \ud83d\udd17 Links\n\n- **\ud83d\udce6 PyPI**: [pypi.org/project/crashlens](https://pypi.org/project/crashlens/)\n- **\ud83d\udcd6 Documentation**: [crashlens.dev/docs](https://crashlens.dev/docs)\n- **\ud83d\udc1b Issues**: [github.com/Crashlens/crashlens/issues](https://github.com/Crashlens/crashlens/issues)\n- **\ud83d\udcac Discussions**: [github.com/Crashlens/crashlens/discussions](https://github.com/Crashlens/crashlens/discussions)\n\n---\n\n<div align=\"center\">\n\n**Made with \u2764\ufe0f by the CrashLens Team**\n\n*Save money \u2022 Optimize AI \u2022 Ship faster*\n\n</div>\n\n### \ud83d\udcca **Reporting & Output**\n- **Multiple output formats**: Slack, Markdown, JSON\n- **Detailed trace reports**: Per-trace JSON files with issue breakdown\n- **Cost summaries**: With and without trace IDs\n- **Professional Markdown reports**: Generated as `report.md` after every scan\n\n### \u2699\ufe0f **Configuration & Flexibility**\n- **Custom pricing config**: Configure model costs and detection thresholds\n- **Input methods**: File, stdin, clipboard, demo data\n- **Flexible output directories**: Customize where reports are saved\n- **Robust error handling**: Works with malformed or incomplete logs\n- **Non-interactive setup**: Environment variable-based configuration for CI/CD and automation\n\n### \ud83d\udd12 **Privacy & Security**\n- **100% local processing**: No data leaves your machine\n- **No external dependencies**: Works offline\n- **CLI-first design**: Integrate into any workflow or CI/CD pipeline\n\n### \ud83e\udd16 **Automation & CI/CD**\n- **Non-interactive init**: Zero-prompt setup for CI/CD pipelines\n- **Environment variable configuration**: CRASHLENS_TEMPLATES, CRASHLENS_SEVERITY, etc.\n- **GitHub Actions workflow generation**: Automated CI integration\n- **Cross-platform support**: PowerShell, Bash, and shell-agnostic commands\n\n---\n\n## 1. Clone the Repository\n\nReplace `<repo-link>` with the actual GitHub URL:\n\n**Cross-platform:**\n```bash\ngit clone <repo-link>\ncd crashlens\n```\n\n---\n\n## 2. Install Python & Poetry\n\nCrashLens requires **Python 3.12+** and [Poetry](https://python-poetry.org/) for dependency management.\n\n### MacOS\n- Install Python (if not already):\n ```bash\n brew install python@3.12\n ```\n- Install Poetry (stable version):\n ```bash\n curl -sSL https://install.python-poetry.org | python3 - --version 1.8.2\n # Or with Homebrew:\n brew install poetry\n ```\n- Add Poetry to your PATH if needed:\n ```bash\n echo 'export PATH=\"$HOME/.local/bin:$PATH\"' >> ~/.zprofile\n source ~/.zprofile\n ```\n- Verify installation:\n ```bash\n poetry --version\n # Should show: Poetry (version 1.8.2)\n ```\n\n### Windows\n\u26a0\ufe0f **Use PowerShell, not CMD, for these commands.**\n\n- Install Python from [python.org](https://www.python.org/downloads/)\n- Install Poetry (stable version):\n ```powershell\n (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python - --version 1.8.2\n ```\n- Add Poetry to your PATH if `poetry --version` returns \"not found\":\n ```powershell\n $userPoetryBin = \"$HOME\\AppData\\Roaming\\Python\\Scripts\"\n \n if (Test-Path $userPoetryBin -and -not ($env:Path -like \"*$userPoetryBin*\")) {\n $env:Path += \";$userPoetryBin\"\n [Environment]::SetEnvironmentVariable(\"Path\", $env:Path, \"User\")\n Write-Output \"\u2705 Poetry path added. Restart your terminal.\"\n } else {\n Write-Output \"\u26a0\ufe0f Poetry path not found or already added. You may need to locate poetry.exe manually.\"\n }\n ```\n **\u26a0\ufe0f Restart your terminal/PowerShell after adding to PATH.**\n- Verify installation:\n ```powershell\n poetry --version\n # Should show: Poetry (version 1.8.2)\n ```\n\n---\n\n## 3. Set Up the Environment\n\n**Cross-platform:**\n```bash\n# From the project root:\npoetry install\n```\n\nThis will create a virtual environment and install all dependencies.\n\nTo activate the environment:\n```bash\npoetry shell\n```\n\n---\n\n## \ud83e\udd16 Non-Interactive Setup & Automation\n\nCrashLens supports fully automated, environment variable-driven setup for CI/CD pipelines and headless environments.\n\n### Quick Start (Non-Interactive)\n\nSet environment variables and run init without prompts:\n\n**PowerShell:**\n```powershell\n$env:CRASHLENS_TEMPLATES = \"all\"\n$env:CRASHLENS_SEVERITY = \"medium\"\n$env:CRASHLENS_FAIL_ON_VIOLATIONS = \"true\"\ncrashlens init --non-interactive\n```\n\n**Bash/Linux:**\n```bash\nexport CRASHLENS_TEMPLATES=\"all\"\nexport CRASHLENS_SEVERITY=\"medium\"\nexport CRASHLENS_FAIL_ON_VIOLATIONS=\"true\"\ncrashlens init --non-interactive\n```\n\nThis creates:\n- `.crashlens/config.yaml` - Configuration file\n- `.github/workflows/crashlens.yml` - GitHub Actions workflow (if applicable)\n\n### Environment Variables\n\n| Variable | Description | Default | Example |\n|----------|-------------|---------|---------|\n| `CRASHLENS_TEMPLATES` | Policy templates to use | `\"retry-loop-prevention\"` | `\"all\"`, `\"retry-loop-prevention,budget-control\"` |\n| `CRASHLENS_SEVERITY` | Minimum severity threshold | `\"medium\"` | `\"low\"`, `\"high\"`, `\"critical\"` |\n| `CRASHLENS_FAIL_ON_VIOLATIONS` | Exit with error on violations | `\"false\"` | `\"true\"`, `\"false\"` |\n| `CRASHLENS_LOGS_SOURCE` | Default log source path | `\"logs/\"` | `\"logs/\"`, `\".llm_logs/\"`, `\"traces.jsonl\"` |\n| `CRASHLENS_OUTPUT_FORMAT` | Report output format | `\"markdown\"` | `\"markdown\"`, `\"slack\"`, `\"json\"` |\n| `CRASHLENS_CREATE_WORKFLOW` | Generate GitHub Actions workflow | `\"true\"` | `\"true\"`, `\"false\"` |\n\n### CI/CD Integration\n\nThe generated `.github/workflows/crashlens.yml` provides automated log analysis on every commit:\n\n```yaml\nname: CrashLens Policy Check\non: [push, pull_request]\njobs:\n crashlens-check:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-python@v4\n with:\n python-version: '3.12'\n - run: pip install crashlens\n - run: crashlens policy-check logs.jsonl --policy-template all --severity-threshold medium --fail-on-violations\n```\n\n### Advanced Examples\n\n**macOS/Linux:**\n```bash\n# Strict monitoring with custom templates\nexport CRASHLENS_TEMPLATES=\"retry-loop-prevention,budget-control\"\nexport CRASHLENS_SEVERITY=\"high\"\nexport CRASHLENS_FAIL_ON_VIOLATIONS=\"true\"\nexport CRASHLENS_LOGS_SOURCE=\".llm_logs/production.jsonl\"\ncrashlens init --non-interactive\n```\n\n**Windows (PowerShell):**\n```powershell\n# Strict monitoring with custom templates\n$env:CRASHLENS_TEMPLATES = \"retry-loop-prevention,budget-control\"\n$env:CRASHLENS_SEVERITY = \"high\"\n$env:CRASHLENS_FAIL_ON_VIOLATIONS = \"true\"\n$env:CRASHLENS_LOGS_SOURCE = \".llm_logs/production.jsonl\"\ncrashlens init --non-interactive\n```\n\n**Docker/Container Setup:**\n```dockerfile\nENV CRASHLENS_TEMPLATES=\"all\"\nENV CRASHLENS_SEVERITY=\"medium\"\nENV CRASHLENS_FAIL_ON_VIOLATIONS=\"true\"\nRUN crashlens init --non-interactive\n```\n\nFor complete documentation, see [docs/NON-INTERACTIVE-GUIDE.md](docs/NON-INTERACTIVE-GUIDE.md) and [docs/NON-INTERACTIVE-QUICK-REFERENCE.md](docs/NON-INTERACTIVE-QUICK-REFERENCE.md).\n\n---\n\n## 4. Running CrashLens\n\nYou can run CrashLens via Poetry or as a Python module:\n\n### Basic Scan (from file)\n**Cross-platform:**\n```bash\ncrashlens scan examples/retry-test.jsonl\n```\n\n### Demo Mode (built-in sample data)\n**Cross-platform:**\n```bash\ncrashlens scan --demo\n```\n\n**Sample output:**\n```\n\ud83d\udea8 **CrashLens Token Waste Report** \ud83d\udea8\n\ud83d\udcca Analysis Date: 2025-07-31 15:22:08\n\n| Metric | Value |\n|--------|-------|\n| Total AI Spend | $0.09 |\n| Total Potential Savings | $0.07 |\n| Wasted Tokens | 1,414 |\n| Issues Found | 8 |\n| Traces Analyzed | 12 |\n\n\ud83d\udce2 **Fallback Failure** | 5 traces | $0.07 wasted | Fix: remove redundant fallbacks\n \ud83c\udfaf **Wasted tokens**: 1,275\n \ud83d\udd17 **Traces** (5): demo_fallback_01, demo_fallback_02, demo_fallback_03, demo_fallback_04, demo_fallback_05\n\n\u2753 **Overkill Model** | 2 traces | $0.0007 wasted | Fix: optimize usage\n \ud83c\udfaf **Wasted tokens**: 31\n \ud83d\udd17 **Traces** (2): demo_overkill_01, demo_overkill_02\n\n\ud83d\udd04 **Retry Loop** | 1 traces | $0.0002 wasted | Fix: exponential backoff\n \ud83c\udfaf **Wasted tokens**: 108\n \ud83d\udd17 **Traces** (1): demo_retry_01\n\n\n## Top Expensive Traces\n\n| Rank | Trace ID | Model | Cost |\n|------|----------|-------|------|\n| 1 | demo_norm_03 | gpt-4 | $0.03 |\n| 2 | demo_norm_04 | gpt-4 | $0.02 |\n| 3 | demo_fallback_05 | gpt-3.5-turbo | $0.02 |\n\n## Cost by Model\n\n| Model | Cost | Percentage |\n|-------|------|------------|\n| gpt-4 | $0.09 | 99% |\n| gpt-3.5-turbo | $0.0012 | 1% |\n\n\n\n---\n\n## Why CrashLens? (vs. grep + Excel, LangSmith, or basic logging)\n\n- \ud83d\udd01 **grep + spreadsheet**: Too manual, error-prone, no cost context\n- \ud83d\udcb8 **LangSmith**: Powerful but complex, requires full tracing/observability stack\n- \ud83d\udd0d **Logging without cost visibility**: You miss $ waste and optimization opportunities\n- \ud83d\udd12 **CrashLens runs 100% locally\u2014no data leaves your machine.**\n\n---\n\n## Features (Ultra-Specific)\n\n- \u2705 Detects retry-loop storms across trace IDs\n- \u2705 Flags gpt-4, Claude, Gemini, and other expensive model usage where a cheaper model (e.g., gpt-3.5, Claude Instant) would suffice\n- \u2705 Scans stdin logs from LangChain, LlamaIndex, custom logging\n- \u2705 Generates Markdown cost reports with per-trace waste\n\n---\n\n## What Makes CrashLens Different?\n\n- \ud83d\udcb5 **Model pricing fallback** (auto-detects/corrects missing cost info)\n- \ud83d\udd12 **Security-by-design** (runs 100% locally, no API calls, no data leaves your machine)\n- \ud83d\udea6 **Coming soon**: Policy enforcement, live CLI firewall, more integrations\n\n\n\n## \ud83d\udcc4 Log File Structure\n\n**Your logs must be in JSONL format (one JSON object per line) and follow this structure:**\n\n```json\n{\"traceId\": \"trace_9\", \"startTime\": \"2025-07-19T10:36:13Z\", \"input\": {\"model\": \"gpt-3.5-turbo\", \"prompt\": \"How do solar panels work?\"}, \"usage\": {\"prompt_tokens\": 25, \"completion_tokens\": 110, \"total_tokens\": 135}, \"cost\": 0.000178}\n```\n\n- Each line is a separate API call (no commas or blank lines between objects).\n- Fields must be nested as shown: `input.model`, `input.prompt`, `usage.completion_tokens`, etc.\n\n**Required fields:**\n- `traceId` (string): Unique identifier for a group of related API calls\n- `input.model` (string): Model name (e.g., `gpt-4`, `gpt-3.5-turbo`)\n- `input.prompt` (string): The prompt sent to the model\n- `usage.completion_tokens` (int): Number of completion tokens used\n\n**Optional fields:**\n- `cost` (float): Cost of the API call\n- `name`, `startTime`, etc.: Any other metadata\n\n\ud83d\udca1 CrashLens expects JSONL with per-call metrics (model, tokens, cost). Works with LangChain logs, OpenAI api.log, Claude, Gemini, and more.\n\n---\n\n## \ud83d\ude80 Usage: Command Line Examples\n\nAfter installation, use the `crashlens` command in your terminal (or `python -m crashlens` if running from source).\n\n### 1. **Scan a log file**\n**Cross-platform:**\n```bash\ncrashlens scan path/to/your-logs.jsonl\n```\n- Scans the specified log file and generates a `report.md` in your current directory.\n\n### 2. **Demo mode (built-in sample data)**\n**Cross-platform:**\n```bash\ncrashlens scan --demo\n```\n- Runs analysis on built-in example logs (requires `examples-logs/demo-logs.jsonl` file).\n- **Note**: If installing from PyPI, you'll need to create sample logs or use your own data.\n- **From source**: Demo data is included in the repository.\n\n### 3. **Scan from stdin (pipe)**\n**macOS/Linux:**\n```bash\ncat path/to/your-logs.jsonl | crashlens scan --stdin\n```\n\n**Windows (PowerShell):**\n```powershell\nGet-Content path/to/your-logs.jsonl | crashlens scan --stdin\n```\n- Reads logs from standard input (useful for pipelines or quick tests).\n\n### 4. **Paste logs interactively**\n**Cross-platform:**\n```bash\ncrashlens scan --paste\n```\n- Reads JSONL data from clipboard (paste and press Enter to finish).\n\n### 5. **Output format options**\n**Cross-platform:**\n```bash\ncrashlens scan logs.jsonl --format slack # Slack-friendly format (default)\ncrashlens scan logs.jsonl --format markdown # Markdown format\ncrashlens scan logs.jsonl --format json # JSON output\n```\n- Choose the format that best fits your workflow or team communication.\n\n### 6. **Detailed reporting**\n**Cross-platform:**\n```bash\ncrashlens scan logs.jsonl --detailed\ncrashlens scan logs.jsonl --detailed --detailed-dir custom_reports/\n```\n- Creates detailed JSON files in `detailed_output/` (or custom directory) by issue type.\n- Generates separate files: `fallback_failure.json`, `retry_loop.json`, etc.\n\n### 7. **Summary options**\n**Cross-platform:**\n```bash\ncrashlens scan logs.jsonl --summary # Cost summary with breakdown\ncrashlens scan logs.jsonl --summary-only # Summary without trace IDs\n```\n- Shows cost analysis with or without detailed trace information.\n\n### 8. **Custom pricing configuration**\n**Cross-platform:**\n```bash\ncrashlens scan logs.jsonl --config custom-pricing.yaml\n```\n- Use custom model pricing and detection thresholds.\n- Default config is located in `crashlens/config/pricing.yaml`.\n\n### 14. **Combined options**\n**Cross-platform:**\n```bash\n# Multiple scan options can be combined\ncrashlens scan logs.jsonl --format json --detailed --summary --config custom.yaml\n\n# Policy checking with custom settings\ncrashlens policy-check logs.jsonl --policy-template all --severity-threshold high --fail-on-violations\n```\n\n**macOS/Linux:**\n```bash\n# Non-interactive setup with custom environment\nCRASHLENS_TEMPLATES=\"retry-loop-prevention,budget-control\" crashlens init --non-interactive\n```\n\n**Windows (PowerShell):**\n```powershell\n# Non-interactive setup with custom environment\n$env:CRASHLENS_TEMPLATES = \"retry-loop-prevention,budget-control\"; crashlens init --non-interactive\n```\n- Mix and match options for your specific analysis needs.\n- Environment variables can be combined with any command.\n\n### 11. **Project setup and configuration**\n**Cross-platform:**\n```bash\ncrashlens init # Interactive setup wizard\ncrashlens init --non-interactive # Automated setup (uses environment variables)\ncrashlens list-policy-templates # List available policy templates\n```\n- Set up CrashLens configuration and GitHub Actions workflow.\n- Non-interactive mode uses environment variables for CI/CD integration.\n\n### 12. **Policy checking**\n**Cross-platform:**\n```bash\ncrashlens policy-check logs.jsonl --policy-template all # Check all policies\ncrashlens policy-check logs.jsonl --policy-template retry-loop-prevention # Specific policy\ncrashlens policy-check logs.jsonl --fail-on-violations # Exit with error code\ncrashlens policy-check logs.jsonl --severity-threshold high # Filter by severity\n```\n- Validate logs against policy rules without running full waste detection.\n- Useful for CI/CD gate checks and compliance validation.\n\n### 13. **Get help**\n**Cross-platform:**\n```bash\ncrashlens --help # Main help\ncrashlens scan --help # Scan command help\ncrashlens init --help # Init command help\ncrashlens policy-check --help # Policy check help\n```\n- Shows all available options and usage details for each command.\n\n---\n\n## \ud83d\udcd6 Quick Command Reference\n\n**Cross-platform commands:**\n```bash\n# Basic Usage\ncrashlens scan <logfile> # Basic log analysis\ncrashlens scan --demo # Test with demo data\n\n# Input Methods \ncrashlens scan --stdin # Read from pipe/stdin\ncrashlens scan --paste # Read from clipboard\ncrashlens scan logs.jsonl # Read from file\n\n# Output Formats\ncrashlens scan logs.jsonl -f slack # Slack format (default)\ncrashlens scan logs.jsonl -f markdown # Markdown format \ncrashlens scan logs.jsonl -f json # JSON format\n\n# Reporting Options\ncrashlens scan logs.jsonl --summary # Show cost summary\ncrashlens scan logs.jsonl --summary-only # Summary without trace IDs\ncrashlens scan logs.jsonl --detailed # Generate detailed JSON reports\n\n# Policy Checking\ncrashlens policy-check logs.jsonl --policy-template all # Check against all policies\ncrashlens policy-check logs.jsonl --policy-template retry-loop-prevention # Specific policy\ncrashlens policy-check logs.jsonl --fail-on-violations # Exit with error on violations\n\n# Setup & Configuration\ncrashlens init # Interactive setup wizard\ncrashlens init --non-interactive # Automated setup using environment variables\ncrashlens list-policy-templates # List available policy templates\n\n# Advanced Options\ncrashlens scan logs.jsonl -c custom.yaml # Custom pricing config\ncrashlens scan logs.jsonl --detailed-dir reports/ # Custom output directory\n\n# Version Info\ncrashlens --version # Show current version\n```\n\n---\n\n## \ud83e\udde9 Example Workflow\n\n1. **Install CrashLens:**\n **Cross-platform:**\n ```bash\n pip install crashlens\n # OR clone and install from source as above\n ```\n\n2. **Set up CrashLens configuration:**\n \n **Interactive setup:**\n **Cross-platform:**\n ```bash\n crashlens init\n # Follow the prompts to configure policies, severity, etc.\n ```\n \n **Non-interactive setup (for CI/CD):**\n \n **macOS/Linux:**\n ```bash\n export CRASHLENS_TEMPLATES=\"all\"\n export CRASHLENS_SEVERITY=\"medium\" \n export CRASHLENS_FAIL_ON_VIOLATIONS=\"true\"\n crashlens init --non-interactive\n ```\n \n **Windows (PowerShell):**\n ```powershell\n $env:CRASHLENS_TEMPLATES = \"all\"\n $env:CRASHLENS_SEVERITY = \"medium\" \n $env:CRASHLENS_FAIL_ON_VIOLATIONS = \"true\"\n crashlens init --non-interactive\n ```\n\n3. **Prepare your log files:**\n \n **Required**: CrashLens needs LLM usage logs in JSONL format. Place them in:\n - `.llm_logs/` directory (recommended)\n - `logs/` directory \n - Or specify any `*.jsonl` file path\n\n **Getting logs from LangFuse:**\n \n **macOS/Linux:**\n ```bash\n mkdir -p .llm_logs\n # Export your traces from LangFuse dashboard or API\n ```\n \n **Windows (PowerShell):**\n ```powershell\n New-Item -ItemType Directory -Force -Path .llm_logs\n # Export your traces from LangFuse dashboard or API\n ```\n\n **Getting logs from OpenAI/custom usage:**\n ```python\n # Example: Log API calls to .llm_logs/usage.jsonl\n import json\n log_entry = {\n \"model\": \"gpt-4\",\n \"usage\": {\"total_tokens\": 1500},\n \"cost\": 0.03,\n \"timestamp\": \"2025-01-15T10:30:00Z\"\n }\n with open('.llm_logs/usage.jsonl', 'a') as f:\n f.write(json.dumps(log_entry) + '\\n')\n ```\n\n **No logs yet?** Generate test data:\n **Cross-platform:**\n ```bash\n crashlens scan --demo\n ```\n\n4. **Analyze your logs:**\n **Cross-platform:**\n ```bash\n crashlens policy-check .llm_logs/*.jsonl --policy-template all\n # OR for a specific file\n crashlens policy-check path/to/your-logs.jsonl\n # OR for waste pattern analysis\n crashlens scan .llm_logs/*.jsonl --format markdown --detailed\n ```\n5. **Review the results:** Open the generated markdown report to review findings and optimization suggestions.\n\n6. **CI/CD Integration:** If you used `crashlens init`, a GitHub Actions workflow was created in `.github/workflows/crashlens.yml` for automated analysis.\n\n---\n\n## \ud83d\udcdd Logging Helper\n\nTo make log analysis seamless, you can use our [`crashlens-logger`](https://github.com/Crashlens/logger) package to emit logs in the correct structure for CrashLens. This ensures compatibility and reduces manual formatting.\n\n**Example usage:**\n```sh\npip install --upgrade crashlens_logger\n```\n```python\nfrom crashlens_logger import CrashLensLogger\n\nlogger = CrashLensLogger()\nlogger.log_event(\n traceId=trace_id,\n startTime=start_time,\n endTime=end_time,\n input={\"model\": model, \"prompt\": prompt},\n usage=usage\n # Optionally add: type, level, metadata, name, etc.\n)\n```\n\n- The logger writes each call as a JSONL line in the required format.\n- See the [`crashlens-logger` repo](https://github.com/Crashlens/logger) for full docs and advanced usage.\n\n---\n\n## \ud83c\udd98 Troubleshooting & Tips\n\n- **File not found:** Make sure the path to your log file is correct.\n- **No traces found:** Your log file may be empty or not in the expected format.\n- **Cost is $0.00:** Check that your log\u2019s model names match those in the pricing config.\n- **Virtual environment issues:** Make sure you\u2019re using the right Python environment.\n- **Need help?** Use `crashlens --help` for all options.\n\n---\n\n## \ud83d\udee0\ufe0f Full Installation (Advanced/Dev)\n\n### **Alternative: Install from Source (GitHub)**\n\nIf you want the latest development version or want to contribute, you can install CrashLens from source:\n\n1. **Clone the repository:**\n **Cross-platform:**\n ```bash\n git clone <repo-link>\n cd crashlens\n ```\n2. **(Optional but recommended) Create a virtual environment:**\n - **On Mac/Linux:**\n ```bash\n python3 -m venv .venv\n source .venv/bin/activate\n ```\n - **On Windows (PowerShell):**\n ```powershell\n python -m venv .venv\n .venv\\Scripts\\Activate.ps1\n ```\n3. **Install dependencies:**\n **Cross-platform:**\n ```bash\n pip install -r requirements.txt\n # Or, if using Poetry:\n poetry install\n ```\n4. **Run CrashLens:**\n **Cross-platform:**\n ```bash\n python -m crashlens scan path/to/your-logs.jsonl\n # Or, if using Poetry:\n poetry run crashlens scan path/to/your-logs.jsonl\n ```\n\n---\n\n## \ud83d\udcec Support\nFor questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.\n\n---\n\n## \ud83d\udcc4 License\nMIT License - see LICENSE file for details.\n\n---\n\n**CrashLens: Find your wasted tokens. Save money. Optimize your AI usage.** \n\n### Scan from stdin (pipe or paste)\n\n**macOS/Linux:**\n```bash\ncat examples/retry-test.jsonl | poetry run crashlens scan --stdin\n```\n\n**Windows (PowerShell):**\n```powershell\nGet-Content examples/retry-test.jsonl | poetry run crashlens scan --stdin\n```\n\n---\n\n## 5. Output: The Markdown Report\n\nAfter every scan, CrashLens creates or updates `report.md` in your current directory.\n\n### Example Structure\n```\n# CrashLens Token Waste Report\n\n\ud83e\uddfe **Total AI Spend**: $0.123456\n\ud83d\udcb0 **Total Potential Savings**: $0.045678\n\n| Trace ID | Model | Prompt | Completion Length | Cost | Waste Type |\n|----------|-------|--------|------------------|------|------------|\n| trace_001 | gpt-4 | ... | 3 | $0.00033 | Overkill |\n| ... | ... | ... | ... | ... | ... |\n\n## Overkill Model Usage (5 issues)\n- ...\n\n## Retry Loops (3 issues)\n- ...\n\n## Fallback Failures (2 issues)\n- ...\n```\n\n---\n\n## 6. Troubleshooting\n- **File not found:** Ensure the path to your log file is correct.\n- **No traces found:** Your log file may be empty or malformed.\n- **Cost is $0.00:** Check that your `pricing.yaml` matches the model names in your logs.\n- **Virtual environment issues:** Use `poetry run` to ensure dependencies are available.\n\n---\n\n## 7. Example Commands\n\n```sh\n# Scan a log file\npoetry run crashlens scan examples/demo-logs.jsonl\n\n# Use demo data\npoetry run crashlens scan --demo\n\n# Scan from stdin\ncat examples/demo-logs.jsonl | poetry run crashlens scan --stdin\n```\n\n---\n\n## \ud83d\udcda Complete Command Reference\n\n### Basic Usage\n**Cross-platform:**\n```bash\ncrashlens scan [OPTIONS] [LOGFILE]\n```\n\n### \ud83c\udfaf Examples\n\n**Cross-platform:**\n```bash\n# Scan a specific log file\ncrashlens scan logs.jsonl\n\n# Run on built-in sample logs\ncrashlens scan --demo\n\n# Read logs from clipboard\ncrashlens scan --paste\n\n# Generate detailed category JSON reports\ncrashlens scan --detailed\n\n# Cost summary with categories\ncrashlens scan --summary\n\n# Show summary only (no trace details)\ncrashlens scan --summary-only\n```\n\n**Platform-specific pipe commands:**\n\n**macOS/Linux:**\n```bash\n# Pipe logs via stdin\ncat logs.jsonl | crashlens scan --stdin\n```\n\n**Windows (PowerShell):**\n```powershell\n# Pipe logs via stdin\nGet-Content logs.jsonl | crashlens scan --stdin\n```\n\n### \ud83d\udd27 All Options\n\n| Option | Description | Example |\n|--------|-------------|---------|\n| `-f, --format` | Output format: `slack`, `markdown`, `json` | `--format json` |\n| `-c, --config` | Custom pricing config file path | `--config my-pricing.yaml` |\n| `--demo` | Use built-in demo data (requires examples-logs/demo-logs.jsonl) | `crashlens scan --demo` |\n| `--stdin` | Read from standard input | `cat logs.jsonl \\| crashlens scan --stdin` (Unix) / `Get-Content logs.jsonl \\| crashlens scan --stdin` (Windows) |\n| `--paste` | Read JSONL data from clipboard | `crashlens scan --paste` |\n| `--summary` | Show cost summary with breakdown | `crashlens scan --summary` |\n| `--summary-only` | Summary without trace IDs | `crashlens scan --summary-only` |\n| `--detailed` | Generate detailed category JSON reports | `crashlens scan --detailed` |\n| `--detailed-dir` | Directory for detailed reports (default: detailed_output) | `--detailed-dir my_reports` |\n| `--help` | Show help message | `crashlens scan --help` |\n\n### \ud83d\udcc2 Detailed Reports\nWhen using `--detailed`, CrashLens generates grouped category files:\n- `detailed_output/fallback_failure.json` - All fallback failure issues\n- `detailed_output/retry_loop.json` - All retry loop issues \n- `detailed_output/fallback_storm.json` - All fallback storm issues\n- `detailed_output/overkill_model.json` - All overkill model issues\n\nEach file contains:\n- Summary with total issues, affected traces, costs\n- All issues of that type with trace IDs and details\n- Specific suggestions for that category\n\n### \ud83d\udd0d Input Sources\nCrashLens supports multiple input methods:\n\n1. **File input**: `crashlens scan path/to/logs.jsonl`\n2. **Demo mode**: `crashlens scan --demo` (requires examples-logs/demo-logs.jsonl file)\n3. **Standard input**: \n - **macOS/Linux**: `cat logs.jsonl | crashlens scan --stdin`\n - **Windows**: `Get-Content logs.jsonl | crashlens scan --stdin`\n4. **Clipboard**: `crashlens scan --paste` (paste logs interactively)\n\n### \ud83d\udcca Output Formats\n- **slack** (default): Slack-formatted report for team sharing\n- **markdown**: Clean Markdown for documentation\n- **json**: Machine-readable JSON for automation\n\n### \ud83d\udca1 Pro Tips\n- Use `--demo` to test CrashLens without your own logs\n- Use `--detailed` to get actionable JSON reports for each issue category\n- Use `--summary-only` for executive summaries without trace details\n- Combine `--stdin` with shell pipelines for automation\n\n---\n\n## 8. Support\nFor questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.\n\n---\n\nEnjoy using CrashLens! \ud83c\udfaf \n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "CLI to detect GPT token waste from Langfuse logs with automated CI/CD setup",
"version": "2.9.15",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8f015283e5645dd54e5f8c78eed7ea120e4a32349da60fc1de4f8a340f5a9558",
"md5": "301a9b8da2ed8c5353c93f6804523f2b",
"sha256": "cb5e8c1a4d75bb4c3f7a418626caa56425bb21527355c2aee4da517ef08e794e"
},
"downloads": -1,
"filename": "crashlens-2.9.15-py3-none-any.whl",
"has_sig": false,
"md5_digest": "301a9b8da2ed8c5353c93f6804523f2b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 131239,
"upload_time": "2025-09-03T18:09:48",
"upload_time_iso_8601": "2025-09-03T18:09:48.939069Z",
"url": "https://files.pythonhosted.org/packages/8f/01/5283e5645dd54e5f8c78eed7ea120e4a32349da60fc1de4f8a340f5a9558/crashlens-2.9.15-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d119faf9ff57e0bf86b77ccd180898484ade728377bded76350e2b66f7702ded",
"md5": "e41d75f0330d0803343ffe4482f7a14c",
"sha256": "143811e07cf07713a2b5fbced3142fc797df8b93b62b6311e06902fe26c7c245"
},
"downloads": -1,
"filename": "crashlens-2.9.15.tar.gz",
"has_sig": false,
"md5_digest": "e41d75f0330d0803343ffe4482f7a14c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 112943,
"upload_time": "2025-09-03T18:09:51",
"upload_time_iso_8601": "2025-09-03T18:09:51.266870Z",
"url": "https://files.pythonhosted.org/packages/d1/19/faf9ff57e0bf86b77ccd180898484ade728377bded76350e2b66f7702ded/crashlens-2.9.15.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-03 18:09:51",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "crashlens"
}
