| Name | crashlens-detector JSON |
| Version |
1.2.0
JSON |
| download |
| home_page | None |
| Summary | CLI to detect GPT token waste from Langfuse logs |
| upload_time | 2025-08-04 19:36:32 |
| maintainer | None |
| docs_url | None |
| author | Aditya Singh |
| requires_python | <4.0,>=3.12 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
## 🧠 What is CrashLens?
CrashLens is a developer tool to **analyze GPT API logs** and uncover hidden **token waste**, retry loops, and overkill model usage. It helps you **optimize your OpenAI, Anthropic, or Langfuse API usage** by generating a cost breakdown and **suggesting cost-saving actions**.
#### 🔍 Use it when you want to:
- Understand how your GPT API budget is being spent
- Reduce unnecessary model calls or retries
- Audit logs for fallback logic inefficiencies
- Analyze Langfuse/OpenAI JSONL logs locally, with full privacy
🧾 Supports: OpenAI, Anthropic, Langfuse JSONL logs
💻 Platform: 100% CLI, 100% local
---
### 💡 Why use CrashLens?
> "You can't optimize what you can't see."
> CrashLens gives you visibility into how you're *actually* using LLMs — and how much it's costing you.
---
## 👨💻 Use Cases
- Track and reduce monthly OpenAI bills
- Debug retry loops and fallback logic in LangChain or custom agents
- Detect inefficient prompt-to-model usage (e.g., using GPT-4 for 3-token completions)
- Generate token audit logs for compliance or team analysis
- CLI tool to audit GPT usage and optimize OpenAI API costs
- Analyze GPT token usage and efficiency in LLM logs
- Reduce LLM spending with actionable insights
---
## TL;DR
```sh
pip install crashlens
crashlens scan path/to/your-logs.jsonl
# Generates report.md with per-trace waste, cost, and suggestions
```
---
## ⚠️ Python Requirement
CrashLens requires **Python 3.12 or higher**. [Download Python 3.12+ here.](https://www.python.org/downloads/)
---
## ⚠️ Windows PATH Warning
If you see a warning like:
```
WARNING: The script crashlens.exe is installed in 'C:\Users\<user>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
```
This means the `crashlens` command may not work from any folder until you add the above Scripts directory to your system PATH.
**How to fix:**
1. Copy the path shown in the warning (ending with `\Scripts`).
2. Open the Windows Start menu, search for "Environment Variables", and open "Edit the system environment variables".
3. Click "Environment Variables...".
4. Under "User variables" or "System variables", select `Path` and click "Edit".
5. Click "New" and paste the Scripts path.
6. Click OK to save. Restart your terminal/command prompt.
Now you can run `crashlens` from any folder.
---
**CrashLens** analyzes your logs for patterns like fallback failures, retry loops, and overkill model usage, and generates a detailed Markdown report (`report.md`) with cost breakdowns and actionable insights.
## 📝 Example CrashLens Report
Below is a sample of what the actual `report.md` looks like after running CrashLens:
🚨 **CrashLens Token Waste Report** 🚨
📊 Analysis Date: 2025-07-31 15:24:48
| Metric | Value |
|--------|-------|
| Total AI Spend | $1.18 |
| Total Potential Savings | $0.82 |
| Wasted Tokens | 19,831 |
| Issues Found | 73 |
| Traces Analyzed | 156 |
❓ **Overkill Model** | 59 traces | $0.68 wasted | Fix: optimize usage
🎯 **Wasted tokens**: 16,496
🔗 **Traces** (57): trace_overkill_01, trace_norm_02, trace_overkill_02, trace_overkill_03, trace_norm_06, +52 more
📢 **Fallback Failure** | 7 traces | $0.08 wasted | Fix: remove redundant fallbacks
🎯 **Wasted tokens**: 1,330
🔗 **Traces** (7): trace_fallback_success_01, trace_fallback_success_02, trace_fallback_success_03, trace_fallback_success_04, trace_fallback_success_05, +2 more
⚡ **Fallback Storm** | 5 traces | $0.07 wasted | Fix: optimize model selection
🎯 **Wasted tokens**: 1,877
🔗 **Traces** (5): trace_fallback_failure_01, trace_fallback_failure_02, trace_fallback_failure_03, trace_fallback_failure_04, trace_fallback_failure_05
🔄 **Retry Loop** | 2 traces | $0.0001 wasted | Fix: exponential backoff
🎯 **Wasted tokens**: 128
🔗 **Traces** (2): trace_retry_loop_07, trace_retry_loop_10
## Top Expensive Traces
| Rank | Trace ID | Model | Cost |
|------|----------|-------|------|
| 1 | trace_norm_76 | gpt-4 | $0.09 |
| 2 | trace_norm_65 | gpt-4 | $0.07 |
| 3 | trace_norm_38 | gpt-4 | $0.06 |
## Cost by Model
| Model | Cost | Percentage |
|-------|------|------------|
| gpt-4 | $1.16 | 98% |
| gpt-3.5-turbo | $0.02 | 2% |
---
## 🚀 Features
- Detects token waste patterns: fallback failures, retry loops, overkill/short completions
- Supports OpenAI, Anthropic, and Langfuse-style logs (JSONL)
- Robust error handling for malformed or incomplete logs
- Configurable model pricing and thresholds via `pricing.yaml`
- Generates a professional Markdown report (`report.md`) after every scan
- 100% local: No data leaves your machine
---
## 1. Clone the Repository
Replace `<repo-link>` with the actual GitHub URL:
```sh
git clone <repo-link>
cd crashlens
```
---
## 2. Install Python & Poetry
CrashLens requires **Python 3.8+** and [Poetry](https://python-poetry.org/) for dependency management.
### MacOS
- Install Python (if not already):
```sh
brew install python@3.12
```
- Install Poetry (stable version):
```sh
curl -sSL https://install.python-poetry.org | python3 - --version 1.8.2
# Or with Homebrew:
brew install poetry
```
- Add Poetry to your PATH if needed:
```sh
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zprofile
source ~/.zprofile
```
- Verify installation:
```sh
poetry --version
# Should show: Poetry (version 1.8.2)
```
### Windows
⚠️ **Use PowerShell, not CMD, for these commands.**
- Install Python from [python.org](https://www.python.org/downloads/)
- Install Poetry (stable version):
```powershell
(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python - --version 1.8.2
```
- Add Poetry to your PATH if `poetry --version` returns "not found":
```powershell
$userPoetryBin = "$HOME\AppData\Roaming\Python\Scripts"
if (Test-Path $userPoetryBin -and -not ($env:Path -like "*$userPoetryBin*")) {
$env:Path += ";$userPoetryBin"
[Environment]::SetEnvironmentVariable("Path", $env:Path, "User")
Write-Output "✅ Poetry path added. Restart your terminal."
} else {
Write-Output "⚠️ Poetry path not found or already added. You may need to locate poetry.exe manually."
}
```
**⚠️ Restart your terminal/PowerShell after adding to PATH.**
- Verify installation:
```powershell
poetry --version
# Should show: Poetry (version 1.8.2)
```
---
## 3. Set Up the Environment
```sh
# From the project root:
poetry install
```
This will create a virtual environment and install all dependencies.
To activate the environment :
```sh
poetry shell
```
---
## 4. Running CrashLens
You can run CrashLens via Poetry or as a Python module:
### Basic Scan (from file)
```sh
crashlens scan examples/retry-test.jsonl
```
### Demo Mode (built-in sample data)
```sh
crashlens scan --demo
```
🚨 **CrashLens Token Waste Report** 🚨
📊 Analysis Date: 2025-07-31 15:22:08
| Metric | Value |
|--------|-------|
| Total AI Spend | $0.09 |
| Total Potential Savings | $0.07 |
| Wasted Tokens | 1,414 |
| Issues Found | 8 |
| Traces Analyzed | 12 |
📢 **Fallback Failure** | 5 traces | $0.07 wasted | Fix: remove redundant fallbacks
🎯 **Wasted tokens**: 1,275
🔗 **Traces** (5): demo_fallback_01, demo_fallback_02, demo_fallback_03, demo_fallback_04, demo_fallback_05
❓ **Overkill Model** | 2 traces | $0.0007 wasted | Fix: optimize usage
🎯 **Wasted tokens**: 31
🔗 **Traces** (2): demo_overkill_01, demo_overkill_02
🔄 **Retry Loop** | 1 traces | $0.0002 wasted | Fix: exponential backoff
🎯 **Wasted tokens**: 108
🔗 **Traces** (1): demo_retry_01
## Top Expensive Traces
| Rank | Trace ID | Model | Cost |
|------|----------|-------|------|
| 1 | demo_norm_03 | gpt-4 | $0.03 |
| 2 | demo_norm_04 | gpt-4 | $0.02 |
| 3 | demo_fallback_05 | gpt-3.5-turbo | $0.02 |
## Cost by Model
| Model | Cost | Percentage |
|-------|------|------------|
| gpt-4 | $0.09 | 99% |
| gpt-3.5-turbo | $0.0012 | 1% |
---
## Why CrashLens? (vs. grep + Excel, LangSmith, or basic logging)
- 🔁 **grep + spreadsheet**: Too manual, error-prone, no cost context
- 💸 **LangSmith**: Powerful but complex, requires full tracing/observability stack
- 🔍 **Logging without cost visibility**: You miss $ waste and optimization opportunities
- 🔒 **CrashLens runs 100% locally—no data leaves your machine.**
---
## Features (Ultra-Specific)
- ✅ Detects retry-loop storms across trace IDs
- ✅ Flags gpt-4, Claude, Gemini, and other expensive model usage where a cheaper model (e.g., gpt-3.5, Claude Instant) would suffice
- ✅ Scans stdin logs from LangChain, LlamaIndex, custom logging
- ✅ Generates Markdown cost reports with per-trace waste
---
## What Makes CrashLens Different?
- 💵 **Model pricing fallback** (auto-detects/corrects missing cost info)
- 🔒 **Security-by-design** (runs 100% locally, no API calls, no data leaves your machine)
- 🚦 **Coming soon**: Policy enforcement, live CLI firewall, more integrations
## 📄 Log File Structure
**Your logs must be in JSONL format (one JSON object per line) and follow this structure:**
```json
{"traceId": "trace_9", "startTime": "2025-07-19T10:36:13Z", "input": {"model": "gpt-3.5-turbo", "prompt": "How do solar panels work?"}, "usage": {"prompt_tokens": 25, "completion_tokens": 110, "total_tokens": 135}, "cost": 0.000178}
```
- Each line is a separate API call (no commas or blank lines between objects).
- Fields must be nested as shown: `input.model`, `input.prompt`, `usage.completion_tokens`, etc.
**Required fields:**
- `traceId` (string): Unique identifier for a group of related API calls
- `input.model` (string): Model name (e.g., `gpt-4`, `gpt-3.5-turbo`)
- `input.prompt` (string): The prompt sent to the model
- `usage.completion_tokens` (int): Number of completion tokens used
**Optional fields:**
- `cost` (float): Cost of the API call
- `name`, `startTime`, etc.: Any other metadata
💡 CrashLens expects JSONL with per-call metrics (model, tokens, cost). Works with LangChain logs, OpenAI api.log, Claude, Gemini, and more.
---
## 🚀 Usage: Command Line Examples
After installation, use the `crashlens` command in your terminal (or `python -m crashlens` if running from source).
### 1. **Scan a log file**
```sh
crashlens scan path/to/your-logs.jsonl
```
- Scans the specified log file and generates a `report.md` in your current directory.
### 2. **Demo mode (built-in sample data)**
```sh
crashlens scan --demo
```
- Runs analysis on built-in example logs (requires `examples-logs/demo-logs.jsonl` file).
- **Note**: If installing from PyPI, you'll need to create sample logs or use your own data.
- **From source**: Demo data is included in the repository.
### 3. **Scan from stdin (pipe)**
```sh
cat path/to/your-logs.jsonl | crashlens scan --stdin
```
- Reads logs from standard input (useful for pipelines or quick tests).
### 4. **Paste logs interactively**
```sh
crashlens scan --paste
```
- Reads JSONL data from clipboard (paste and press Enter to finish).
### 5. **Generate detailed category reports**
```sh
crashlens scan --detailed
```
- Creates grouped JSON files in `detailed_output/` by issue type (fallback_failure.json, retry_loop.json, etc.).
### 6. **Get cost summaries**
```sh
crashlens scan --summary # Cost summary with breakdown
crashlens scan --summary-only # Summary without trace IDs
```
- Shows cost analysis with or without detailed trace information.
### 7. **Change output format**
```sh
crashlens scan --format json # JSON output
crashlens scan --format markdown # Markdown format
```
- Default format is `slack` for team sharing.
### 8. **Get help**
```sh
crashlens --help
crashlens scan --help
```
- Shows all available options and usage details.
---
## 🧩 Example Workflow
1. **Install CrashLens:**
```sh
pip install crashlens
# OR clone and install from source as above
```
2. **Scan your logs:**
```sh
crashlens scan path/to/your-logs.jsonl
# OR
python -m crashlens scan path/to/your-logs.jsonl
```
3. **Open `report.md`** in your favorite Markdown viewer or editor to review the findings and suggestions.
---
## 📝 Logging Helper
To make log analysis seamless, you can use our [`crashlens-logger`](https://github.com/Crashlens/logger) package to emit logs in the correct structure for CrashLens. This ensures compatibility and reduces manual formatting.
**Example usage:**
```sh
pip install --upgrade crashlens_logger
```
```python
from crashlens_logger import CrashLensLogger
logger = CrashLensLogger()
logger.log_event(
traceId=trace_id,
startTime=start_time,
endTime=end_time,
input={"model": model, "prompt": prompt},
usage=usage
# Optionally add: type, level, metadata, name, etc.
)
```
- The logger writes each call as a JSONL line in the required format.
- See the [`crashlens-logger` repo](https://github.com/Crashlens/logger) for full docs and advanced usage.
---
## 🆘 Troubleshooting & Tips
- **File not found:** Make sure the path to your log file is correct.
- **No traces found:** Your log file may be empty or not in the expected format.
- **Cost is $0.00:** Check that your log’s model names match those in the pricing config.
- **Virtual environment issues:** Make sure you’re using the right Python environment.
- **Need help?** Use `crashlens --help` for all options.
---
## 🛠️ Full Installation (Advanced/Dev)
### **Alternative: Install from Source (GitHub)**
If you want the latest development version or want to contribute, you can install CrashLens from source:
1. **Clone the repository:**
```sh
git clone <repo-link>
cd crashlens
```
2. **(Optional but recommended) Create a virtual environment:**
- **On Mac/Linux:**
```sh
python3 -m venv .venv
source .venv/bin/activate
```
- **On Windows:**
```sh
python -m venv .venv
.venv\Scripts\activate
```
3. **Install dependencies:**
```sh
pip install -r requirements.txt
# Or, if using Poetry:
poetry install
```
4. **Run CrashLens:**
```sh
python -m crashlens scan path/to/your-logs.jsonl
# Or, if using Poetry:
poetry run crashlens scan path/to/your-logs.jsonl
```
---
## 📬 Support
For questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.
---
## 📄 License
MIT License - see LICENSE file for details.
---
**CrashLens: Find your wasted tokens. Save money. Optimize your AI usage.**
### Scan from stdin (pipe or paste)
```sh
cat examples/retry-test.jsonl | poetry run crashlens scan --stdin
```
---
## 5. Output: The Markdown Report
After every scan, CrashLens creates or updates `report.md` in your current directory.
### Example Structure
```
# CrashLens Token Waste Report
🧾 **Total AI Spend**: $0.123456
💰 **Total Potential Savings**: $0.045678
| Trace ID | Model | Prompt | Completion Length | Cost | Waste Type |
|----------|-------|--------|------------------|------|------------|
| trace_001 | gpt-4 | ... | 3 | $0.00033 | Overkill |
| ... | ... | ... | ... | ... | ... |
## Overkill Model Usage (5 issues)
- ...
## Retry Loops (3 issues)
- ...
## Fallback Failures (2 issues)
- ...
```
---
## 6. Troubleshooting
- **File not found:** Ensure the path to your log file is correct.
- **No traces found:** Your log file may be empty or malformed.
- **Cost is $0.00:** Check that your `pricing.yaml` matches the model names in your logs.
- **Virtual environment issues:** Use `poetry run` to ensure dependencies are available.
---
## 7. Example Commands
```sh
# Scan a log file
poetry run crashlens scan examples/demo-logs.jsonl
# Use demo data
poetry run crashlens scan --demo
# Scan from stdin
cat examples/demo-logs.jsonl | poetry run crashlens scan --stdin
```
---
## 📚 Complete Command Reference
### Basic Usage
```sh
crashlens scan [OPTIONS] [LOGFILE]
```
### 🎯 Examples
```sh
# Scan a specific log file
crashlens scan logs.jsonl
# Run on built-in sample logs
crashlens scan --demo
# Pipe logs via stdin
cat logs.jsonl | crashlens scan --stdin
# Read logs from clipboard
crashlens scan --paste
# Generate detailed category JSON reports
crashlens scan --detailed
# Cost summary with categories
crashlens scan --summary
# Show summary only (no trace details)
crashlens scan --summary-only
```
### 🔧 All Options
| Option | Description | Example |
|--------|-------------|---------|
| `-f, --format` | Output format: `slack`, `markdown`, `json` | `--format json` |
| `-c, --config` | Custom pricing config file path | `--config my-pricing.yaml` |
| `--demo` | Use built-in demo data (requires examples-logs/demo-logs.jsonl) | `crashlens scan --demo` |
| `--stdin` | Read from standard input | `cat logs.jsonl \| crashlens scan --stdin` |
| `--paste` | Read JSONL data from clipboard | `crashlens scan --paste` |
| `--summary` | Show cost summary with breakdown | `crashlens scan --summary` |
| `--summary-only` | Summary without trace IDs | `crashlens scan --summary-only` |
| `--detailed` | Generate detailed category JSON reports | `crashlens scan --detailed` |
| `--detailed-dir` | Directory for detailed reports (default: detailed_output) | `--detailed-dir my_reports` |
| `--help` | Show help message | `crashlens scan --help` |
### 📂 Detailed Reports
When using `--detailed`, CrashLens generates grouped category files:
- `detailed_output/fallback_failure.json` - All fallback failure issues
- `detailed_output/retry_loop.json` - All retry loop issues
- `detailed_output/fallback_storm.json` - All fallback storm issues
- `detailed_output/overkill_model.json` - All overkill model issues
Each file contains:
- Summary with total issues, affected traces, costs
- All issues of that type with trace IDs and details
- Specific suggestions for that category
### 🔍 Input Sources
CrashLens supports multiple input methods:
1. **File input**: `crashlens scan path/to/logs.jsonl`
2. **Demo mode**: `crashlens scan --demo` (requires examples-logs/demo-logs.jsonl file)
3. **Standard input**: `cat logs.jsonl | crashlens scan --stdin`
4. **Clipboard**: `crashlens scan --paste` (paste logs interactively)
### 📊 Output Formats
- **slack** (default): Slack-formatted report for team sharing
- **markdown**: Clean Markdown for documentation
- **json**: Machine-readable JSON for automation
### 💡 Pro Tips
- Use `--demo` to test CrashLens without your own logs
- Use `--detailed` to get actionable JSON reports for each issue category
- Use `--summary-only` for executive summaries without trace details
- Combine `--stdin` with shell pipelines for automation
---
## 8. Support
For questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.
---
Enjoy using CrashLens! 🎯
Raw data
{
"_id": null,
"home_page": null,
"name": "crashlens-detector",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": null,
"author": "Aditya Singh",
"author_email": "coding103856@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/8b/55/4f869b3332eedd5187390fdf9342be4108594aec50b7c5cb7a4d328a9da3/crashlens_detector-1.2.0.tar.gz",
"platform": null,
"description": "## \ud83e\udde0 What is CrashLens?\n\n\nCrashLens is a developer tool to **analyze GPT API logs** and uncover hidden **token waste**, retry loops, and overkill model usage. It helps you **optimize your OpenAI, Anthropic, or Langfuse API usage** by generating a cost breakdown and **suggesting cost-saving actions**.\n\n#### \ud83d\udd0d Use it when you want to:\n\n- Understand how your GPT API budget is being spent\n- Reduce unnecessary model calls or retries\n- Audit logs for fallback logic inefficiencies\n- Analyze Langfuse/OpenAI JSONL logs locally, with full privacy\n\n\ud83e\uddfe Supports: OpenAI, Anthropic, Langfuse JSONL logs \n\ud83d\udcbb Platform: 100% CLI, 100% local\n\n---\n\n### \ud83d\udca1 Why use CrashLens?\n\n> \"You can't optimize what you can't see.\"\n> CrashLens gives you visibility into how you're *actually* using LLMs \u2014 and how much it's costing you.\n\n---\n\n## \ud83d\udc68\u200d\ud83d\udcbb Use Cases\n\n- Track and reduce monthly OpenAI bills\n- Debug retry loops and fallback logic in LangChain or custom agents\n- Detect inefficient prompt-to-model usage (e.g., using GPT-4 for 3-token completions)\n- Generate token audit logs for compliance or team analysis\n- CLI tool to audit GPT usage and optimize OpenAI API costs\n- Analyze GPT token usage and efficiency in LLM logs\n- Reduce LLM spending with actionable insights\n\n---\n\n## TL;DR\n\n```sh\npip install crashlens\ncrashlens scan path/to/your-logs.jsonl\n# Generates report.md with per-trace waste, cost, and suggestions\n```\n\n---\n\n## \u26a0\ufe0f Python Requirement\n\nCrashLens requires **Python 3.12 or higher**. [Download Python 3.12+ here.](https://www.python.org/downloads/)\n\n---\n\n## \u26a0\ufe0f Windows PATH Warning\n\nIf you see a warning like:\n\n```\nWARNING: The script crashlens.exe is installed in 'C:\\Users\\<user>\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python312\\Scripts' which is not on PATH.\nConsider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\n```\n\nThis means the `crashlens` command may not work from any folder until you add the above Scripts directory to your system PATH.\n\n**How to fix:**\n1. Copy the path shown in the warning (ending with `\\Scripts`).\n2. Open the Windows Start menu, search for \"Environment Variables\", and open \"Edit the system environment variables\".\n3. Click \"Environment Variables...\".\n4. Under \"User variables\" or \"System variables\", select `Path` and click \"Edit\".\n5. Click \"New\" and paste the Scripts path.\n6. Click OK to save. Restart your terminal/command prompt.\n\nNow you can run `crashlens` from any folder.\n\n---\n\n**CrashLens** analyzes your logs for patterns like fallback failures, retry loops, and overkill model usage, and generates a detailed Markdown report (`report.md`) with cost breakdowns and actionable insights.\n\n## \ud83d\udcdd Example CrashLens Report\n\nBelow is a sample of what the actual `report.md` looks like after running CrashLens:\n\n\ud83d\udea8 **CrashLens Token Waste Report** \ud83d\udea8\n\ud83d\udcca Analysis Date: 2025-07-31 15:24:48\n\n| Metric | Value |\n|--------|-------|\n| Total AI Spend | $1.18 |\n| Total Potential Savings | $0.82 |\n| Wasted Tokens | 19,831 |\n| Issues Found | 73 |\n| Traces Analyzed | 156 |\n\n\u2753 **Overkill Model** | 59 traces | $0.68 wasted | Fix: optimize usage\n \ud83c\udfaf **Wasted tokens**: 16,496\n \ud83d\udd17 **Traces** (57): trace_overkill_01, trace_norm_02, trace_overkill_02, trace_overkill_03, trace_norm_06, +52 more\n\n\ud83d\udce2 **Fallback Failure** | 7 traces | $0.08 wasted | Fix: remove redundant fallbacks\n \ud83c\udfaf **Wasted tokens**: 1,330\n \ud83d\udd17 **Traces** (7): trace_fallback_success_01, trace_fallback_success_02, trace_fallback_success_03, trace_fallback_success_04, trace_fallback_success_05, +2 more\n\n\u26a1 **Fallback Storm** | 5 traces | $0.07 wasted | Fix: optimize model selection\n \ud83c\udfaf **Wasted tokens**: 1,877\n \ud83d\udd17 **Traces** (5): trace_fallback_failure_01, trace_fallback_failure_02, trace_fallback_failure_03, trace_fallback_failure_04, trace_fallback_failure_05\n\n\ud83d\udd04 **Retry Loop** | 2 traces | $0.0001 wasted | Fix: exponential backoff\n \ud83c\udfaf **Wasted tokens**: 128\n \ud83d\udd17 **Traces** (2): trace_retry_loop_07, trace_retry_loop_10\n\n\n## Top Expensive Traces\n\n| Rank | Trace ID | Model | Cost |\n|------|----------|-------|------|\n| 1 | trace_norm_76 | gpt-4 | $0.09 |\n| 2 | trace_norm_65 | gpt-4 | $0.07 |\n| 3 | trace_norm_38 | gpt-4 | $0.06 |\n\n## Cost by Model\n\n| Model | Cost | Percentage |\n|-------|------|------------|\n| gpt-4 | $1.16 | 98% |\n| gpt-3.5-turbo | $0.02 | 2% |\n\n\n\n---\n\n## \ud83d\ude80 Features\n- Detects token waste patterns: fallback failures, retry loops, overkill/short completions\n- Supports OpenAI, Anthropic, and Langfuse-style logs (JSONL)\n- Robust error handling for malformed or incomplete logs\n- Configurable model pricing and thresholds via `pricing.yaml`\n- Generates a professional Markdown report (`report.md`) after every scan\n- 100% local: No data leaves your machine\n\n---\n\n## 1. Clone the Repository\n\nReplace `<repo-link>` with the actual GitHub URL:\n\n```sh\ngit clone <repo-link>\ncd crashlens\n```\n\n---\n\n## 2. Install Python & Poetry\n\nCrashLens requires **Python 3.8+** and [Poetry](https://python-poetry.org/) for dependency management.\n\n### MacOS\n- Install Python (if not already):\n ```sh\n brew install python@3.12\n ```\n- Install Poetry (stable version):\n ```sh\n curl -sSL https://install.python-poetry.org | python3 - --version 1.8.2\n # Or with Homebrew:\n brew install poetry\n ```\n- Add Poetry to your PATH if needed:\n ```sh\n echo 'export PATH=\"$HOME/.local/bin:$PATH\"' >> ~/.zprofile\n source ~/.zprofile\n ```\n- Verify installation:\n ```sh\n poetry --version\n # Should show: Poetry (version 1.8.2)\n ```\n\n### Windows\n\u26a0\ufe0f **Use PowerShell, not CMD, for these commands.**\n\n- Install Python from [python.org](https://www.python.org/downloads/)\n- Install Poetry (stable version):\n ```powershell\n (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python - --version 1.8.2\n ```\n- Add Poetry to your PATH if `poetry --version` returns \"not found\":\n ```powershell\n $userPoetryBin = \"$HOME\\AppData\\Roaming\\Python\\Scripts\"\n \n if (Test-Path $userPoetryBin -and -not ($env:Path -like \"*$userPoetryBin*\")) {\n $env:Path += \";$userPoetryBin\"\n [Environment]::SetEnvironmentVariable(\"Path\", $env:Path, \"User\")\n Write-Output \"\u2705 Poetry path added. Restart your terminal.\"\n } else {\n Write-Output \"\u26a0\ufe0f Poetry path not found or already added. You may need to locate poetry.exe manually.\"\n }\n ```\n **\u26a0\ufe0f Restart your terminal/PowerShell after adding to PATH.**\n- Verify installation:\n ```powershell\n poetry --version\n # Should show: Poetry (version 1.8.2)\n ```\n\n---\n\n## 3. Set Up the Environment\n\n```sh\n# From the project root:\npoetry install\n```\n\nThis will create a virtual environment and install all dependencies.\n\nTo activate the environment :\n ```sh\n poetry shell\n ```\n\n---\n\n## 4. Running CrashLens\n\nYou can run CrashLens via Poetry or as a Python module:\n\n### Basic Scan (from file)\n```sh\ncrashlens scan examples/retry-test.jsonl\n```\n\n### Demo Mode (built-in sample data)\n```sh\ncrashlens scan --demo\n\n```\n\ud83d\udea8 **CrashLens Token Waste Report** \ud83d\udea8\n\ud83d\udcca Analysis Date: 2025-07-31 15:22:08\n\n| Metric | Value |\n|--------|-------|\n| Total AI Spend | $0.09 |\n| Total Potential Savings | $0.07 |\n| Wasted Tokens | 1,414 |\n| Issues Found | 8 |\n| Traces Analyzed | 12 |\n\n\ud83d\udce2 **Fallback Failure** | 5 traces | $0.07 wasted | Fix: remove redundant fallbacks\n \ud83c\udfaf **Wasted tokens**: 1,275\n \ud83d\udd17 **Traces** (5): demo_fallback_01, demo_fallback_02, demo_fallback_03, demo_fallback_04, demo_fallback_05\n\n\u2753 **Overkill Model** | 2 traces | $0.0007 wasted | Fix: optimize usage\n \ud83c\udfaf **Wasted tokens**: 31\n \ud83d\udd17 **Traces** (2): demo_overkill_01, demo_overkill_02\n\n\ud83d\udd04 **Retry Loop** | 1 traces | $0.0002 wasted | Fix: exponential backoff\n \ud83c\udfaf **Wasted tokens**: 108\n \ud83d\udd17 **Traces** (1): demo_retry_01\n\n\n## Top Expensive Traces\n\n| Rank | Trace ID | Model | Cost |\n|------|----------|-------|------|\n| 1 | demo_norm_03 | gpt-4 | $0.03 |\n| 2 | demo_norm_04 | gpt-4 | $0.02 |\n| 3 | demo_fallback_05 | gpt-3.5-turbo | $0.02 |\n\n## Cost by Model\n\n| Model | Cost | Percentage |\n|-------|------|------------|\n| gpt-4 | $0.09 | 99% |\n| gpt-3.5-turbo | $0.0012 | 1% |\n\n\n\n---\n\n## Why CrashLens? (vs. grep + Excel, LangSmith, or basic logging)\n\n- \ud83d\udd01 **grep + spreadsheet**: Too manual, error-prone, no cost context\n- \ud83d\udcb8 **LangSmith**: Powerful but complex, requires full tracing/observability stack\n- \ud83d\udd0d **Logging without cost visibility**: You miss $ waste and optimization opportunities\n- \ud83d\udd12 **CrashLens runs 100% locally\u2014no data leaves your machine.**\n\n---\n\n## Features (Ultra-Specific)\n\n- \u2705 Detects retry-loop storms across trace IDs\n- \u2705 Flags gpt-4, Claude, Gemini, and other expensive model usage where a cheaper model (e.g., gpt-3.5, Claude Instant) would suffice\n- \u2705 Scans stdin logs from LangChain, LlamaIndex, custom logging\n- \u2705 Generates Markdown cost reports with per-trace waste\n\n---\n\n## What Makes CrashLens Different?\n\n- \ud83d\udcb5 **Model pricing fallback** (auto-detects/corrects missing cost info)\n- \ud83d\udd12 **Security-by-design** (runs 100% locally, no API calls, no data leaves your machine)\n- \ud83d\udea6 **Coming soon**: Policy enforcement, live CLI firewall, more integrations\n\n\n\n## \ud83d\udcc4 Log File Structure\n\n**Your logs must be in JSONL format (one JSON object per line) and follow this structure:**\n\n```json\n{\"traceId\": \"trace_9\", \"startTime\": \"2025-07-19T10:36:13Z\", \"input\": {\"model\": \"gpt-3.5-turbo\", \"prompt\": \"How do solar panels work?\"}, \"usage\": {\"prompt_tokens\": 25, \"completion_tokens\": 110, \"total_tokens\": 135}, \"cost\": 0.000178}\n```\n\n- Each line is a separate API call (no commas or blank lines between objects).\n- Fields must be nested as shown: `input.model`, `input.prompt`, `usage.completion_tokens`, etc.\n\n**Required fields:**\n- `traceId` (string): Unique identifier for a group of related API calls\n- `input.model` (string): Model name (e.g., `gpt-4`, `gpt-3.5-turbo`)\n- `input.prompt` (string): The prompt sent to the model\n- `usage.completion_tokens` (int): Number of completion tokens used\n\n**Optional fields:**\n- `cost` (float): Cost of the API call\n- `name`, `startTime`, etc.: Any other metadata\n\n\ud83d\udca1 CrashLens expects JSONL with per-call metrics (model, tokens, cost). Works with LangChain logs, OpenAI api.log, Claude, Gemini, and more.\n\n---\n\n## \ud83d\ude80 Usage: Command Line Examples\n\nAfter installation, use the `crashlens` command in your terminal (or `python -m crashlens` if running from source).\n\n### 1. **Scan a log file**\n```sh\ncrashlens scan path/to/your-logs.jsonl\n```\n- Scans the specified log file and generates a `report.md` in your current directory.\n\n### 2. **Demo mode (built-in sample data)**\n```sh\ncrashlens scan --demo\n```\n- Runs analysis on built-in example logs (requires `examples-logs/demo-logs.jsonl` file).\n- **Note**: If installing from PyPI, you'll need to create sample logs or use your own data.\n- **From source**: Demo data is included in the repository.\n\n### 3. **Scan from stdin (pipe)**\n```sh\ncat path/to/your-logs.jsonl | crashlens scan --stdin\n```\n- Reads logs from standard input (useful for pipelines or quick tests).\n\n### 4. **Paste logs interactively**\n```sh\ncrashlens scan --paste\n```\n- Reads JSONL data from clipboard (paste and press Enter to finish).\n\n### 5. **Generate detailed category reports**\n```sh\ncrashlens scan --detailed\n```\n- Creates grouped JSON files in `detailed_output/` by issue type (fallback_failure.json, retry_loop.json, etc.).\n\n### 6. **Get cost summaries**\n```sh\ncrashlens scan --summary # Cost summary with breakdown\ncrashlens scan --summary-only # Summary without trace IDs\n```\n- Shows cost analysis with or without detailed trace information.\n\n### 7. **Change output format**\n```sh\ncrashlens scan --format json # JSON output\ncrashlens scan --format markdown # Markdown format \n```\n- Default format is `slack` for team sharing.\n\n### 8. **Get help**\n```sh\ncrashlens --help\ncrashlens scan --help\n```\n- Shows all available options and usage details.\n\n---\n\n## \ud83e\udde9 Example Workflow\n\n1. **Install CrashLens:**\n ```sh\n pip install crashlens\n # OR clone and install from source as above\n ```\n2. **Scan your logs:**\n ```sh\n crashlens scan path/to/your-logs.jsonl\n # OR\n python -m crashlens scan path/to/your-logs.jsonl\n ```\n3. **Open `report.md`** in your favorite Markdown viewer or editor to review the findings and suggestions.\n\n---\n\n## \ud83d\udcdd Logging Helper\n\nTo make log analysis seamless, you can use our [`crashlens-logger`](https://github.com/Crashlens/logger) package to emit logs in the correct structure for CrashLens. This ensures compatibility and reduces manual formatting.\n\n**Example usage:**\n```sh\npip install --upgrade crashlens_logger\n```\n```python\nfrom crashlens_logger import CrashLensLogger\n\nlogger = CrashLensLogger()\nlogger.log_event(\n traceId=trace_id,\n startTime=start_time,\n endTime=end_time,\n input={\"model\": model, \"prompt\": prompt},\n usage=usage\n # Optionally add: type, level, metadata, name, etc.\n)\n```\n\n- The logger writes each call as a JSONL line in the required format.\n- See the [`crashlens-logger` repo](https://github.com/Crashlens/logger) for full docs and advanced usage.\n\n---\n\n## \ud83c\udd98 Troubleshooting & Tips\n\n- **File not found:** Make sure the path to your log file is correct.\n- **No traces found:** Your log file may be empty or not in the expected format.\n- **Cost is $0.00:** Check that your log\u2019s model names match those in the pricing config.\n- **Virtual environment issues:** Make sure you\u2019re using the right Python environment.\n- **Need help?** Use `crashlens --help` for all options.\n\n---\n\n## \ud83d\udee0\ufe0f Full Installation (Advanced/Dev)\n\n### **Alternative: Install from Source (GitHub)**\n\nIf you want the latest development version or want to contribute, you can install CrashLens from source:\n\n1. **Clone the repository:**\n ```sh\n git clone <repo-link>\n cd crashlens\n ```\n2. **(Optional but recommended) Create a virtual environment:**\n - **On Mac/Linux:**\n ```sh\n python3 -m venv .venv\n source .venv/bin/activate\n ```\n - **On Windows:**\n ```sh\n python -m venv .venv\n .venv\\Scripts\\activate\n ```\n3. **Install dependencies:**\n ```sh\n pip install -r requirements.txt\n # Or, if using Poetry:\n poetry install\n ```\n4. **Run CrashLens:**\n ```sh\n python -m crashlens scan path/to/your-logs.jsonl\n # Or, if using Poetry:\n poetry run crashlens scan path/to/your-logs.jsonl\n ```\n\n---\n\n## \ud83d\udcec Support\nFor questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.\n\n---\n\n## \ud83d\udcc4 License\nMIT License - see LICENSE file for details.\n\n---\n\n**CrashLens: Find your wasted tokens. Save money. Optimize your AI usage.** \n\n### Scan from stdin (pipe or paste)\n```sh\ncat examples/retry-test.jsonl | poetry run crashlens scan --stdin\n```\n\n---\n\n## 5. Output: The Markdown Report\n\nAfter every scan, CrashLens creates or updates `report.md` in your current directory.\n\n### Example Structure\n```\n# CrashLens Token Waste Report\n\n\ud83e\uddfe **Total AI Spend**: $0.123456\n\ud83d\udcb0 **Total Potential Savings**: $0.045678\n\n| Trace ID | Model | Prompt | Completion Length | Cost | Waste Type |\n|----------|-------|--------|------------------|------|------------|\n| trace_001 | gpt-4 | ... | 3 | $0.00033 | Overkill |\n| ... | ... | ... | ... | ... | ... |\n\n## Overkill Model Usage (5 issues)\n- ...\n\n## Retry Loops (3 issues)\n- ...\n\n## Fallback Failures (2 issues)\n- ...\n```\n\n---\n\n## 6. Troubleshooting\n- **File not found:** Ensure the path to your log file is correct.\n- **No traces found:** Your log file may be empty or malformed.\n- **Cost is $0.00:** Check that your `pricing.yaml` matches the model names in your logs.\n- **Virtual environment issues:** Use `poetry run` to ensure dependencies are available.\n\n---\n\n## 7. Example Commands\n\n```sh\n# Scan a log file\npoetry run crashlens scan examples/demo-logs.jsonl\n\n# Use demo data\npoetry run crashlens scan --demo\n\n# Scan from stdin\ncat examples/demo-logs.jsonl | poetry run crashlens scan --stdin\n```\n\n---\n\n## \ud83d\udcda Complete Command Reference\n\n### Basic Usage\n```sh\ncrashlens scan [OPTIONS] [LOGFILE]\n```\n\n### \ud83c\udfaf Examples\n```sh\n# Scan a specific log file\ncrashlens scan logs.jsonl\n\n# Run on built-in sample logs\ncrashlens scan --demo\n\n# Pipe logs via stdin\ncat logs.jsonl | crashlens scan --stdin\n\n# Read logs from clipboard\ncrashlens scan --paste\n\n# Generate detailed category JSON reports\ncrashlens scan --detailed\n\n# Cost summary with categories\ncrashlens scan --summary\n\n# Show summary only (no trace details)\ncrashlens scan --summary-only\n```\n\n### \ud83d\udd27 All Options\n\n| Option | Description | Example |\n|--------|-------------|---------|\n| `-f, --format` | Output format: `slack`, `markdown`, `json` | `--format json` |\n| `-c, --config` | Custom pricing config file path | `--config my-pricing.yaml` |\n| `--demo` | Use built-in demo data (requires examples-logs/demo-logs.jsonl) | `crashlens scan --demo` |\n| `--stdin` | Read from standard input | `cat logs.jsonl \\| crashlens scan --stdin` |\n| `--paste` | Read JSONL data from clipboard | `crashlens scan --paste` |\n| `--summary` | Show cost summary with breakdown | `crashlens scan --summary` |\n| `--summary-only` | Summary without trace IDs | `crashlens scan --summary-only` |\n| `--detailed` | Generate detailed category JSON reports | `crashlens scan --detailed` |\n| `--detailed-dir` | Directory for detailed reports (default: detailed_output) | `--detailed-dir my_reports` |\n| `--help` | Show help message | `crashlens scan --help` |\n\n### \ud83d\udcc2 Detailed Reports\nWhen using `--detailed`, CrashLens generates grouped category files:\n- `detailed_output/fallback_failure.json` - All fallback failure issues\n- `detailed_output/retry_loop.json` - All retry loop issues \n- `detailed_output/fallback_storm.json` - All fallback storm issues\n- `detailed_output/overkill_model.json` - All overkill model issues\n\nEach file contains:\n- Summary with total issues, affected traces, costs\n- All issues of that type with trace IDs and details\n- Specific suggestions for that category\n\n### \ud83d\udd0d Input Sources\nCrashLens supports multiple input methods:\n\n1. **File input**: `crashlens scan path/to/logs.jsonl`\n2. **Demo mode**: `crashlens scan --demo` (requires examples-logs/demo-logs.jsonl file)\n3. **Standard input**: `cat logs.jsonl | crashlens scan --stdin`\n4. **Clipboard**: `crashlens scan --paste` (paste logs interactively)\n\n### \ud83d\udcca Output Formats\n- **slack** (default): Slack-formatted report for team sharing\n- **markdown**: Clean Markdown for documentation\n- **json**: Machine-readable JSON for automation\n\n### \ud83d\udca1 Pro Tips\n- Use `--demo` to test CrashLens without your own logs\n- Use `--detailed` to get actionable JSON reports for each issue category\n- Use `--summary-only` for executive summaries without trace details\n- Combine `--stdin` with shell pipelines for automation\n\n---\n\n## 8. Support\nFor questions, issues, or feature requests, open an issue on GitHub or contact the maintainer.\n\n---\n\nEnjoy using CrashLens! \ud83c\udfaf \n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "CLI to detect GPT token waste from Langfuse logs",
"version": "1.2.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3fa2de9db5ba6fd7083146ceef9fc04e979fa49aa42f1696c00c0765a14184e8",
"md5": "31a81b4136a8cae929e7bea4589504eb",
"sha256": "5e60991a2a69e5163034d270073d7447babfcc24bb48e8f1a19cb265643021b6"
},
"downloads": -1,
"filename": "crashlens_detector-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "31a81b4136a8cae929e7bea4589504eb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 51890,
"upload_time": "2025-08-04T19:36:30",
"upload_time_iso_8601": "2025-08-04T19:36:30.704307Z",
"url": "https://files.pythonhosted.org/packages/3f/a2/de9db5ba6fd7083146ceef9fc04e979fa49aa42f1696c00c0765a14184e8/crashlens_detector-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8b554f869b3332eedd5187390fdf9342be4108594aec50b7c5cb7a4d328a9da3",
"md5": "3ac5e0aae0f6a684cdc0628e07cde7a6",
"sha256": "ce90adea12ee38713267fa843da59b3181eacb8b64a99d90a34924e889778e2b"
},
"downloads": -1,
"filename": "crashlens_detector-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "3ac5e0aae0f6a684cdc0628e07cde7a6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 41278,
"upload_time": "2025-08-04T19:36:32",
"upload_time_iso_8601": "2025-08-04T19:36:32.619469Z",
"url": "https://files.pythonhosted.org/packages/8b/55/4f869b3332eedd5187390fdf9342be4108594aec50b7c5cb7a4d328a9da3/crashlens_detector-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-04 19:36:32",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "crashlens-detector"
}
