# AI-First Toolkit: LLM-Powered Automation
[](https://badge.fury.io/py/llm-ci-runner) [](https://github.com/Nantero1/ai-first-devops-toolkit/actions/workflows/ci.yml) [](https://github.com/Nantero1/ai-first-devops-toolkit/actions/workflows/unit-tests.yml) [](https://htmlpreview.github.io/?https://github.com/Nantero1/ai-first-devops-toolkit/blob/python-coverage-comment-action-data/htmlcov/index.html) [](https://github.com/Nantero1/ai-first-devops-toolkit/actions/workflows/github-code-scanning/codeql) [](https://opensource.org/licenses/MIT) [](http://mypy-lang.org/) [](https://github.com/astral-sh/ruff) [](https://www.bestpractices.dev/projects/10922) [](https://www.pepy.tech/projects/llm-ci-runner)
> **🚀 The Future of DevOps is AI-First**
> This toolkit represents a step
> toward [AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html) - where
> intelligent automation handles the entire development lifecycle. Built for teams ready to embrace the exponential
> productivity gains of AI-powered development. Please
> read [the blog post](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html) for more details on
> the motivation.
## TLDR: What This Tool Does
**Purpose**: Transform any unstructured business knowledge into reliable, structured data that powers intelligent
automation across your entire organization.
**Perfect For**:
- 🏦 **Financial Operations**: Convert loan applications, audits, and regulatory docs into structured compliance data
- 🏥 **Healthcare Systems**: Transform patient records, clinical notes, and research data into medical formats
- ⚖️ **Legal & Compliance**: Process contracts, court docs, and regulatory texts into actionable risk assessments
- 🏭 **Supply Chain**: Turn logistics reports, supplier communications, and forecasts into optimization insights
- 👥 **Human Resources**: Convert resumes, performance reviews, and feedback into structured talent analytics
- 🛡️ **Security Operations**: Transform threat reports, incident logs, and assessments into standard frameworks
- 🚀 **DevOps & Engineering**: Use commit logs, deployment reports, and system logs for automated AI actions
- 🔗 **Enterprise Integration**: Connect any business process to downstream systems with guaranteed consistency
---
### Simple structured output example
```bash
# Install and use immediately
pip install llm-ci-runner
llm-ci-runner --input-file examples/02-devops/pr-description/input.json --schema-file examples/02-devops/pr-description/schema.json
```
## The AI-First Development Revolution
This toolkit embodies the principles outlined
in [Building AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html):
| Traditional DevOps | AI-First DevOps (This Tool) |
|-----------------------------|-----------------------------------------------------|
| Manual code reviews | 🤖 AI-powered reviews with structured findings |
| Human-written documentation | 📝 AI-generated docs with guaranteed consistency |
| Reactive security scanning | 🔍 Proactive AI security analysis |
| Manual quality gates | 🎯 AI-driven validation with schema enforcement |
| Linear productivity | 📈 Exponential gains through intelligent automation |
## Features
- 🎯 **100% Schema Enforcement**: Your pipeline never gets invalid data. Token-level schema enforcement with guaranteed
compliance
- 🔄 **Resilient execution**: Retries with exponential back-off and jitter plus a clear exception hierarchy keep
transient cloud faults from breaking your CI.
- 🚀 **Zero-Friction CLI**: Single script, minimal configuration for pipeline integration and automation
- 🔐 **Enterprise Security**: Azure RBAC via DefaultAzureCredential with fallback to API Key
- 📦 **CI-friendly CLI**: Stateless command that reads JSON/YAML, writes JSON/YAML, and exits with proper codes
- 🎨 **Beautiful Logging**: Rich console output with timestamps and colors
- 📁 **File-based I/O**: CI/CD friendly with JSON/YAML input/output
- 📋 **Template-Driven Workflows**: Handlebars and Jinja2 templates with YAML variables for dynamic prompt generation
- 📄 **YAML Support**: Use YAML for schemas, input files, and output files - more readable than JSON
- 🔧 **Simple & Extensible**: Easy to understand and modify for your specific needs
- 🤖 **Semantic Kernel foundation**: async, service-oriented design ready for skills, memories, orchestration, and future
model upgrades
- 📚 **Documentation**: Comprehensive documentation for all features and usage examples. Use your semantic kernel skills
to extend the functionality.
- 🧑⚖️ **Acceptance Tests**: pytest framework with the LLM-as-Judge pattern for quality gates. Test your scripts before
you run them in production.
- 💰 **Coming soon**: token usage and cost estimation appended to each result for budgeting and optimisation
## 🚀 The Only Enterprise AI DevOps Tool That Delivers RBAC Security, Robustness and Simplicity
**LLM-CI-Runner stands alone in the market** as the only tool combining **100% schema enforcement**, **enterprise RBAC
authentication**, and robust **Semantic Kernel integration with templates** in a single CLI solution. **No other tool
delivers all three critical enterprise requirements together**.
## Installation
```bash
pip install llm-ci-runner
```
That's it! No complex setup, no dependency management - just install and use. Perfect for CI/CD pipelines and local
development.
## Quick Start
### 1. Install from PyPI
```bash
pip install llm-ci-runner
```
### 2. Set Environment Variables
**Azure OpenAI (Priority 1):**
```bash
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_MODEL="gpt-4.1-nano" # or any other GPT deployment name
export AZURE_OPENAI_API_VERSION="2024-12-01-preview" # Optional
```
**OpenAI (Fallback):**
```bash
export OPENAI_API_KEY="your-very-secret-api-key"
export OPENAI_CHAT_MODEL_ID="gpt-4.1-nano" # or any OpenAI model
export OPENAI_ORG_ID="org-your-org-id" # Optional
```
**Authentication Options:**
- **Azure RBAC (Recommended)**: Uses `DefaultAzureCredential` for Azure RBAC authentication - no API key needed!
See [Microsoft Docs](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python)
for setup.
- **Azure API Key**: Set `AZURE_OPENAI_API_KEY` environment variable if not using RBAC.
- **OpenAI API Key**: Required for OpenAI fallback when Azure is not configured.
**Priority**: Azure OpenAI takes priority when both Azure and OpenAI environment variables are present.
### 3a. Basic Usage
```bash
# Simple chat example
llm-ci-runner --input-file examples/01-basic/simple-chat/input.json
# With structured output schema
llm-ci-runner \
--input-file examples/01-basic/sentiment-analysis/input.json \
--schema-file examples/01-basic/sentiment-analysis/schema.json
# Custom output file
llm-ci-runner \
--input-file examples/02-devops/pr-description/input.json \
--schema-file examples/02-devops/pr-description/schema.json \
--output-file pr-analysis.json
# YAML input files (alternative to JSON)
llm-ci-runner \
--input-file config.yaml \
--schema-file schema.yaml \
--output-file result.yaml
```
### 3b. Template-Based Workflows
**Dynamic prompt generation with YAML, Handlebars or Jinja2 templates:**
```bash
# Handlebars template example
llm-ci-runner \
--template-file examples/05-templates/handlebars-template/template.hbs \
--template-vars examples/05-templates/handlebars-template/template-vars.yaml \
--schema-file examples/05-templates/handlebars-template/schema.yaml \
--output-file handlebars-result.yaml
# Or using Jinja2 templates
llm-ci-runner \
--template-file examples/05-templates/jinja2-template/template.j2 \
--template-vars examples/05-templates/jinja2-template/template-vars.yaml \
--schema-file examples/05-templates/jinja2-template/schema.yaml \
--output-file jinja2-result.yaml
```
For more examples see the [examples directory](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples).
**Benefits of Template Approach:**
- 🎯 **Reusable Templates**: Create once, use across multiple scenarios
- 📝 **YAML Configuration**: More readable than JSON for complex setups
- 🔄 **Dynamic Content**: Variables and conditional rendering
- 🚀 **CI/CD Ready**: Perfect for parameterized pipeline workflows
### 4. Python Library Usage
**You can use LLM CI Runner directly from Python with both file-based and string-based templates, and with either dict
or file-based variables and schemas. The main entrypoint is:**
```python
from llm_ci_runner.core import run_llm_task # Adjust import as needed for your package layout
```
#### Basic Usage: File-Based Input
```python
import asyncio
from llm_ci_runner.core import run_llm_task
async def main():
# Run with a traditional JSON input file (messages, etc)
response = await run_llm_task(_input_file="examples/01-basic/simple-chat/input.json")
print(response)
asyncio.run(main())
```
#### File-Based Template Usage
```python
import asyncio
from llm_ci_runner.core import run_llm_task
async def main():
# Handlebars, Jinja2, or Semantic Kernel YAML template via file
response = await run_llm_task(
template_file="examples/05-templates/pr-review-template/template.hbs",
template_vars_file="examples/05-templates/pr-review-template/template-vars.yaml",
schema_file="examples/05-templates/pr-review-template/schema.yaml",
output_file="analysis.json"
)
print(response)
asyncio.run(main())
```
#### String-Based Template Usage
```python
import asyncio
from llm_ci_runner.core import run_llm_task
async def main():
# String template (Handlebars example)
response = await run_llm_task(
template_content="Hello {{name}}!",
template_format="handlebars",
template_vars={"name": "World"},
)
print(response)
asyncio.run(main())
```
#### Semantic Kernel YAML Template with Embedded Schema
Please refer to the full example in [examples/05-templates/sem-ker-structured-analysis/README.md](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/examples/05-templates/sem-ker-structured-analysis/README.md).
```python
import asyncio
from llm_ci_runner.core import run_llm_task
async def main():
template_content = """
template: "Analyze: {{input_text}}"
input_variables:
- name: input_text
execution_settings:
azure_openai:
temperature: 0.1
"""
response = await run_llm_task(
template_content=template_content,
template_format="semantic-kernel",
template_vars={"input_text": "Sample data"}
)
print(response)
asyncio.run(main())
```
#### Advanced: Dict-based Schema and Variables
```python
import asyncio
from llm_ci_runner.core import run_llm_task
async def main():
schema = {
"type": "object",
"properties": {
"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]},
"confidence": {"type": "number", "minimum": 0, "maximum": 1}
},
"required": ["sentiment", "confidence"]
}
template = "Analyze this review: {{review}}"
variables = {"review": "I love the new update!"}
response = await run_llm_task(
template_content=template,
template_format="handlebars",
template_vars=variables,
schema=schema
)
print(response)
asyncio.run(main())
```
#### Notes & Tips
- **Only one of** `_input_file`, `template_file`, or `template_content` **may be specified** at a time.
- **Template variables**: Use `template_vars` (Python dict or YAML file path), or `template_vars_file` (YAML file path).
- **Schema**: Use `schema` (dict or JSON/YAML file path), or `schema_file` (file path).
- **template_format** is required with `template_content`. Allowed: `"handlebars"`, `"jinja2"`, `"semantic-kernel"`.
- **output_file**: If specified, writes response to file.
**Returns:** String (for text output) or dict (for structured JSON output).
**Errors:** Raises `InputValidationError` or `LLMRunnerError` on invalid input or execution failure.
### 5. Development Setup (Optional)
For contributors or advanced users who want to modify the source:
```bash
# Install UV if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# Clone and install for development
git clone https://github.com/Nantero1/ai-first-devops-toolkit.git
cd ai-first-devops-toolkit
uv sync
# Run from source
uv run llm-ci-runner --input-file examples/01-basic/simple-chat/input.json
```
## The AI-First Transformation: Why Unstructured → Structured Matters
LLMs excel at extracting meaning from messy text, logs, documents, and mixed-format data, then emitting *
*schema-compliant JSON/YAML** that downstream systems can trust. This unlocks:
- **🔄 Straight-Through Processing**: Structured payloads feed BI dashboards, RPA robots, and CI/CD gates without human
parsing
- **🎯 Context-Aware Decisions**: LLMs fuse domain knowledge with live telemetry to prioritize incidents, forecast
demand, and spot security drift
- **📋 Auditable Compliance**: Formal outputs make it easy to track decisions for regulators and ISO/NIST audits
- **⚡ Rapid Workflow Automation**: Enable automation across customer service, supply-chain planning, HR case handling,
and security triage
- **🔗 Safe Pipeline Composition**: Structured contracts let AI-first pipelines remain observable and composable while
capitalizing on unstructured enterprise data
## Input Formats
### Traditional JSON Input
```json
{
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Your task description here"
}
],
"context": {
"session_id": "optional-session-id",
"metadata": {
"any": "additional context"
}
}
}
```
### YAML Input
```yaml
messages:
- role: system
content: "You are a helpful assistant."
- role: user
content: "Your task description here"
context:
session_id: "optional-session-id"
metadata:
any: "additional context"
```
### Template-Based Input
**Handlebars Template** (`template.hbs`):
```handlebars
<message role="system">
You are an expert {{expertise.domain}} engineer.
Focus on {{expertise.focus_areas}}.
</message>
<message role="user">
Analyze this {{task.type}}:
{{#each task.items}}
- {{this}}
{{/each}}
Requirements: {{task.requirements}}
</message>
```
**Jinja2 Template** (`template.j2`):
```jinja2
<message role="system">
You are an expert {{expertise.domain}} engineer.
Focus on {{expertise.focus_areas}}.
</message>
<message role="user">
Analyze this {{task.type}}:
{% for item in task.items %}
- {{item}}
{% endfor %}
Requirements: {{task.requirements}}
</message>
```
**Template Variables** (`vars.yaml`):
```yaml
expertise:
domain: "DevOps"
focus_areas: "security, performance, maintainability"
task:
type: "pull request"
items:
- "Changed authentication logic"
- "Updated database queries"
- "Added input validation"
requirements: "Focus on security vulnerabilities"
```
## Structured Outputs with 100% Schema Enforcement
When you provide a `--schema-file`, the runner guarantees perfect schema compliance:
```bash
llm-ci-runner \
--input-file examples/01-basic/sentiment-analysis/input.json \
--schema-file examples/01-basic/sentiment-analysis/schema.json
```
**Note**: Output defaults to `result.json`. Use `--output-file custom-name.json` for custom output files.
**Supported Schema Features**:
✅ String constraints (enum, minLength, maxLength, pattern)
✅ Numeric constraints (minimum, maximum, multipleOf)
✅ Array constraints (minItems, maxItems, items type)
✅ Required fields enforced at generation time
✅ Type validation (string, number, integer, boolean, array)
## CI/CD Integration
### GitHub Actions Example
```yaml
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install LLM CI Runner
run: pip install llm-ci-runner
- name: Generate PR Review with Templates
run: |
llm-ci-runner \
--template-file .github/templates/pr-review.j2 \
--template-vars pr-context.yaml \
--schema-file .github/schemas/pr-review.yaml \
--output-file pr-analysis.yaml
env:
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
AZURE_OPENAI_MODEL: ${{ secrets.AZURE_OPENAI_MODEL }}
```
For complete CI/CD examples, see *
*[examples/uv-usage-example.md](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/examples/uv-usage-example.md)
**. This repo is also using itself for release note generation, **check it
out [here](https://github.com/Nantero1/ai-first-devops-toolkit/blob/c4066d347ae14d37cb674e36007a678f38b36439/.github/workflows/release.yml#L145-L149)
**.
## Authentication
**Azure OpenAI**: Uses Azure's `DefaultAzureCredential` supporting:
- Environment variables (local development)
- Managed Identity (recommended for Azure CI/CD)
- Azure CLI (local development)
- Service Principal (non-Azure CI/CD)
**OpenAI**: Uses API key authentication with optional organization ID.
## Testing
We maintain comprehensive test coverage with **100% success rate**:
```bash
# For package users - install test dependencies
pip install llm-ci-runner[dev]
# For development - install from source with test dependencies
uv sync --group dev
# Run specific test categories
pytest tests/unit/ -v # 70 unit tests
pytest tests/integration/ -v # End-to-end examples
pytest acceptance/ -v # LLM-as-judge evaluation
# Or with uv for development
uv run pytest tests/unit/ -v
uv run pytest tests/integration/ -v
uv run pytest acceptance/ -v
```
## Architecture
Built on **Microsoft Semantic Kernel** for:
- Enterprise-ready Azure OpenAI and OpenAI integration
- Future-proof model compatibility
- **100% Schema Enforcement**: KernelBaseModel integration with token-level constraints
- **Dynamic Model Creation**: Runtime JSON schema → Pydantic model conversion
- **Azure RBAC**: Azure RBAC via DefaultAzureCredential
- **Automatic Fallback**: Azure-first priority with OpenAI fallback
## The AI-First Development Journey
This toolkit is your first step
toward [AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html). As you
integrate AI into your development workflows, you'll experience:
1. **🚀 Exponential Productivity**: AI handles routine tasks while you focus on architecture
2. **🎯 Guaranteed Quality**: Schema enforcement eliminates validation errors
3. **🤖 Autonomous Operations**: AI agents make decisions in your pipelines
4. **📈 Continuous Improvement**: Every interaction improves your AI system
**The future belongs to teams that master AI-first principles.** This toolkit gives you the foundation to start that
journey today.
## Real-World Examples
You can explore the **[examples directory](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples)** for
a complete collection of self-contained examples organized by category.
For comprehensive real-world CI/CD scenarios, see *
*[examples/uv-usage-example.md](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/examples/uv-usage-example.md)
**.
### 100 AI Automation Use Cases for AI-First Automation
**DevOps & Engineering** 🔧
1. 🤖 AI-generated PR review – automated pull request analysis with structured review findings
2. 📝 Release note composer – map commits to semantic-version bump rules and structured changelogs
3. 🔍 Vulnerability scanner – map code vulnerabilities to OWASP standards with actionable remediation
4. ☸️ Kubernetes manifest optimizer – produce risk-scored diffs and security hardening recommendations
5. 📊 Log anomaly triager – convert system logs into OTEL-formatted events for SIEM ingestion
6. 💰 Cloud cost explainer – output tagged spend by team in FinOps schema for budget optimization
7. 🔄 API diff analyzer – produce backward-compatibility scorecards from specification changes
8. 🛡️ IaC drift detector – turn Terraform plans into CVE-linked security findings
9. 📋 Dependency license auditor – emit SPDX-compatible reports for compliance tracking
10. 🎯 SLA breach summarizer – file structured JIRA tickets with SMART action items
**Governance, Risk & Compliance** 🏛️
11. 📊 Regulatory delta analyzer – emit change-impact matrices from new compliance requirements
12. 🌱 ESG report synthesizer – map CSR prose to GRI indicators and sustainability metrics
13. 📋 SOX-404 narrative converter – transform controls descriptions into testable audit checklists
14. 🏦 Basel III stress-test interpreter – output capital risk buckets from regulatory scenarios
15. 🕵️ AML SAR formatter – convert investigator notes into Suspicious Activity Report structures
16. 🔒 Privacy policy parser – generate GDPR data-processing-activity logs from legal text
17. 🔍 Internal audit evidence linker – export control traceability graphs for compliance tracking
18. 📊 Carbon emission disclosure normalizer – structure sustainability data into XBRL taxonomy
19. ⚖️ Regulatory update tracker – generate structured compliance action items from guideline changes
20. 🛡️ Safety inspection checker – transform narratives into OSHA citation checklists
**Financial Services** 🏦
21. 🏦 Loan application analyzer – transform free-text applications into Basel-III risk-model inputs
22. 📊 Earnings call sentiment quantifier – output KPI deltas and investor sentiment scores
23. 💹 Budget variance explainer – produce drill-down pivot JSON for financial analysis
24. 📈 Portfolio risk dashboard builder – feed VaR models with structured investment analysis
25. 💳 Fraud alert generator – map investigation notes to CVSS-scored security metrics
26. 💰 Treasury cash-flow predictor – ingest email forecasts into structured planning models
27. 📊 Financial forecaster – summarize reports into structured cash-flow and projection objects
28. 🧾 Invoice processor – convert receipts into double-entry ledger posts with GAAP tags
29. 📋 Stress test scenario packager – structure regulatory submission data for banking compliance
30. 🏦 Insurance claim assessor – return structured claim-decision objects with risk scores
**Healthcare & Life Sciences** 🏥
31. 🏥 Patient intake processor – build HL7/FHIR-compliant patient records from free-form intake forms
32. 🧠 Mental health triage assistant – structure referral notes with priority classifications and care pathways
33. 📊 Radiology report coder – output SNOMED-coded JSON from diagnostic imaging narratives
34. 💊 Clinical trial note packager – create FDA eCTD modules from research documentation
35. 📋 Prescription parser – turn text prescriptions into structured e-Rx objects with dosage validation
36. ⚡ Vital sign anomaly summarizer – generate alert reports with clinical priority rankings
37. 🧪 Lab result organizer – output LOINC-coded tables from diagnostic test narratives
38. 🏥 Medical device log summarizer – generate UDI incident files for regulatory reporting
39. 📈 Patient feedback sentiment analyzer – feed quality-of-care KPIs from satisfaction surveys
40. 👩⚕️ Clinical observation compiler – convert research notes into structured data for trials
**Legal & Compliance** ⚖️
41. 🏛️ Legal contract parser – extract clauses and compute risk scores from contract documents
42. 📝 Court opinion digest – summarize judicial opinions into structured precedent and citation graphs
43. 🏛️ Legal discovery summarizer – extract key issues and risks from large document sets
44. 💼 Contract review summarizer – extract risk factors and key dates from legal contracts
45. 🏛️ Policy impact assessor – convert policy proposals into stakeholder impact matrices
46. 📜 Patent novelty comparator – produce claim-overlap matrices from prior art analysis
47. 🏛️ Legal bill auditor – transform billing details into itemized expense and compliance reports
48. 📋 Case strategy brainstormer – summarize likely arguments from litigation documentation
49. 💼 Legal email analyzer – extract key issues and deadlines from email threads for review
50. ⚖️ Expert witness report normalizer – create citation-linked outlines from testimony records
**Customer Experience & Sales** 🛒
51. 🎧 Tier-1 support chatbot – convert customer queries into tickets with reproducible troubleshooting steps
52. ⭐ Review sentiment miner – produce product-feature tallies from customer feedback analysis
53. 📉 Churn risk email summarizer – export CRM risk scores from customer communication patterns
54. 🗺️ Omnichannel conversation unifier – generate customer journey maps from multi-platform interactions
55. ❓ Dynamic FAQ builder – structure knowledge base content from community forum discussions
56. 📋 Proposal auto-grader – output RFP compliance matrices with scoring rubrics
57. 📈 Upsell opportunity extractor – create lead-scoring JSON from customer interaction analysis
58. 📱 Social media crisis detector – feed escalation playbooks with brand sentiment monitoring
59. 🌐 Multilingual intent router – tag customer chats to appropriate support queues by language/topic
60. 🎯 Marketing copy generator – create brand-compliant content with tone and messaging constraints
**HR & People Operations** 👥
61. 📄 CV-to-JD matcher – rank candidates with explainable competency scores and fit analysis
62. 🎤 Interview transcript summarizer – export structured competency rubrics with evaluation criteria
63. ✅ Onboarding policy compliance checker – produce new-hire checklist completion tracking
64. 📊 Performance review sentiment analyzer – create growth-plan JSON with development recommendations
65. 💰 Payroll inquiry classifier – map employee emails to structured case codes for HR processing
66. 🏥 Benefits Q&A automation – generate eligibility responses from policy documentation
67. 🚪 Exit interview insight extractor – feed retention dashboards with structured departure analytics
68. 📚 Training content gap mapper – align job roles to skill taxonomies for learning programs
69. 🛡️ Workplace incident processor – convert safety reports into OSHA 301 compliance records
70. 📊 Diversity metric synthesizer – summarize inclusion survey data into actionable insights
**Supply Chain & Manufacturing** 🏭
71. 📊 Demand forecast summarizer – output SKU-level predictions from market analysis and sales data
72. 📋 Purchase order processor – convert supplier communications into structured ERP line-items
73. 🌱 Supplier risk scanner – generate ESG compliance scores from vendor assessment reports
74. 🔧 Predictive maintenance log analyst – produce work orders from equipment telemetry narratives
75. 🚛 Logistics delay explainer – return route-change suggestions from transportation disruption reports
76. ♻️ Circular economy return classifier – create refurbishment tags from product return descriptions
77. 🌍 Carbon footprint calculator – map transport legs to CO₂e emissions for sustainability reporting
78. 📦 Safety stock alert generator – output inventory triggers with lead-time assumptions
79. 📜 Regulatory import/export harmonizer – produce HS-code sheets from trade documentation
80. 🏭 Production yield analyzer – generate efficiency reports from manufacturing floor logs
**Security & Risk Management** 🔒
81. 🛡️ MITRE ATT&CK mapper – translate IDS alerts into tactic-technique JSON for threat intelligence
82. 🎣 Phishing email extractor – produce IOC STIX bundles from security incident reports
83. 🔐 Zero-trust policy generator – convert narrative access requests into structured policy rules
84. 🚨 SOC alert deduplicator – cluster security tickets by kill-chain stage for efficient triage
85. 🏴☠️ Red team debrief summarizer – export OWASP Top-10 gaps from penetration test reports
86. 📋 Data breach notifier – craft GDPR-compliant disclosure packets with timeline and impact data
87. 🧠 Threat intel feed normalizer – convert mixed security PDFs into MISP threat objects
88. 🔍 Secret leak scanner – output GitHub code-owner mentions from repository security scans
89. 📊 Vendor risk questionnaire scorer – generate SIG Lite security assessment answers
90. 🏗️ Security audit tracker – link ISO-27001 controls to evidence artifacts for compliance
**Knowledge & Productivity** 📚
91. 🎙️ Meeting transcript processor – extract action items with owners and deadlines into project tracking JSON
92. 📚 Research paper summarizer – export citation graphs and key findings for literature review databases
93. 📋 SOP generator – convert process narratives into step-by-step validation checklists
94. 🔄 Code diff summarizer – generate reviewer hints and impact analysis from version control changes
95. 📊 API changelog analyzer – produce backward-compatibility scorecards for development teams
96. 🧠 Mind map creator – structure brainstorming sessions into hierarchical knowledge trees
97. 📖 Knowledge base gap detector – suggest article stubs from frequently asked questions analysis
98. 🎯 Personal OKR journal parser – output progress dashboards with milestone tracking
99. 💼 White paper composer – transform technical discussions into structured thought leadership content
100. 🧩 Universal transformer – convert any unstructured domain knowledge into your custom schema-validated JSON
## License
MIT License - See [LICENSE](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/LICENSE) file for details.
Copyright (c) 2025, Benjamin Linnik.
## Support
**🐛 Found a bug? 💡 Have a question? 📚 Need help?**
**GitHub is your primary destination for all support:**
- **📋 Issues & Bug Reports**: [Create an issue](https://github.com/Nantero1/ai-first-devops-toolkit/issues)
- **📖 Documentation**: [Browse examples](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples)
- **🔧 Source Code**: [View source](https://github.com/Nantero1/ai-first-devops-toolkit)
**Before opening an issue, please:**
1. ✅ Check the [examples directory](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples) for
solutions
2. ✅ Review the error logs (beautiful output with Rich!)
3. ✅ Validate your Azure authentication and permissions
4. ✅ Ensure your input JSON follows the required format
5. ✅ Search existing [issues](https://github.com/Nantero1/ai-first-devops-toolkit/issues) for similar problems
**Quick Links:**
- 🚀 [Getting Started Guide](https://github.com/Nantero1/ai-first-devops-toolkit#quick-start)
- 📚 [Complete Examples](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples)
- 🔧 [CI/CD Integration](https://github.com/Nantero1/ai-first-devops-toolkit#cicd-integration)
- 🎯 [Use Cases](https://github.com/Nantero1/ai-first-devops-toolkit#use-cases)
---
*Ready to embrace the AI-First future? Start with this toolkit and build your path to exponential productivity. Learn
more about the AI-First DevOps revolution
in [Building AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html).*
Raw data
{
"_id": null,
"home_page": null,
"name": "llm-ci-runner",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "ai, automation, azure-openai, ci-cd, devops, llm, semantic-kernel",
"author": null,
"author_email": "Benjamin Linnik <Benjamin@Linnik.IT>",
"download_url": "https://files.pythonhosted.org/packages/0f/27/c0fbe38f414846b405b27c4eac4a19c1c1da173ff8fcb14809dbbc4b4989/llm_ci_runner-1.5.1.tar.gz",
"platform": null,
"description": "# AI-First Toolkit: LLM-Powered Automation\n\n[](https://badge.fury.io/py/llm-ci-runner) [](https://github.com/Nantero1/ai-first-devops-toolkit/actions/workflows/ci.yml) [](https://github.com/Nantero1/ai-first-devops-toolkit/actions/workflows/unit-tests.yml) [](https://htmlpreview.github.io/?https://github.com/Nantero1/ai-first-devops-toolkit/blob/python-coverage-comment-action-data/htmlcov/index.html) [](https://github.com/Nantero1/ai-first-devops-toolkit/actions/workflows/github-code-scanning/codeql) [](https://opensource.org/licenses/MIT) [](http://mypy-lang.org/) [](https://github.com/astral-sh/ruff) [](https://www.bestpractices.dev/projects/10922) [](https://www.pepy.tech/projects/llm-ci-runner)\n\n> **\ud83d\ude80 The Future of DevOps is AI-First** \n> This toolkit represents a step\n> toward [AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html) - where\n> intelligent automation handles the entire development lifecycle. Built for teams ready to embrace the exponential\n> productivity gains of AI-powered development. Please\n> read [the blog post](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html) for more details on\n> the motivation.\n\n## TLDR: What This Tool Does\n\n**Purpose**: Transform any unstructured business knowledge into reliable, structured data that powers intelligent\nautomation across your entire organization.\n\n**Perfect For**:\n\n- \ud83c\udfe6 **Financial Operations**: Convert loan applications, audits, and regulatory docs into structured compliance data\n- \ud83c\udfe5 **Healthcare Systems**: Transform patient records, clinical notes, and research data into medical formats\n- \u2696\ufe0f **Legal & Compliance**: Process contracts, court docs, and regulatory texts into actionable risk assessments\n- \ud83c\udfed **Supply Chain**: Turn logistics reports, supplier communications, and forecasts into optimization insights\n- \ud83d\udc65 **Human Resources**: Convert resumes, performance reviews, and feedback into structured talent analytics\n- \ud83d\udee1\ufe0f **Security Operations**: Transform threat reports, incident logs, and assessments into standard frameworks\n- \ud83d\ude80 **DevOps & Engineering**: Use commit logs, deployment reports, and system logs for automated AI actions\n- \ud83d\udd17 **Enterprise Integration**: Connect any business process to downstream systems with guaranteed consistency\n\n---\n\n### Simple structured output example\n\n```bash\n# Install and use immediately\npip install llm-ci-runner\nllm-ci-runner --input-file examples/02-devops/pr-description/input.json --schema-file examples/02-devops/pr-description/schema.json\n```\n\n\n\n## The AI-First Development Revolution\n\nThis toolkit embodies the principles outlined\nin [Building AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html):\n\n| Traditional DevOps | AI-First DevOps (This Tool) |\n|-----------------------------|-----------------------------------------------------|\n| Manual code reviews | \ud83e\udd16 AI-powered reviews with structured findings |\n| Human-written documentation | \ud83d\udcdd AI-generated docs with guaranteed consistency |\n| Reactive security scanning | \ud83d\udd0d Proactive AI security analysis |\n| Manual quality gates | \ud83c\udfaf AI-driven validation with schema enforcement |\n| Linear productivity | \ud83d\udcc8 Exponential gains through intelligent automation |\n\n## Features\n\n- \ud83c\udfaf **100% Schema Enforcement**: Your pipeline never gets invalid data. Token-level schema enforcement with guaranteed\n compliance\n- \ud83d\udd04 **Resilient execution**: Retries with exponential back-off and jitter plus a clear exception hierarchy keep\n transient cloud faults from breaking your CI.\n- \ud83d\ude80 **Zero-Friction CLI**: Single script, minimal configuration for pipeline integration and automation\n- \ud83d\udd10 **Enterprise Security**: Azure RBAC via DefaultAzureCredential with fallback to API Key\n- \ud83d\udce6 **CI-friendly CLI**: Stateless command that reads JSON/YAML, writes JSON/YAML, and exits with proper codes\n- \ud83c\udfa8 **Beautiful Logging**: Rich console output with timestamps and colors\n- \ud83d\udcc1 **File-based I/O**: CI/CD friendly with JSON/YAML input/output\n- \ud83d\udccb **Template-Driven Workflows**: Handlebars and Jinja2 templates with YAML variables for dynamic prompt generation\n- \ud83d\udcc4 **YAML Support**: Use YAML for schemas, input files, and output files - more readable than JSON\n- \ud83d\udd27 **Simple & Extensible**: Easy to understand and modify for your specific needs\n- \ud83e\udd16 **Semantic Kernel foundation**: async, service-oriented design ready for skills, memories, orchestration, and future\n model upgrades\n- \ud83d\udcda **Documentation**: Comprehensive documentation for all features and usage examples. Use your semantic kernel skills\n to extend the functionality.\n- \ud83e\uddd1\u200d\u2696\ufe0f **Acceptance Tests**: pytest framework with the LLM-as-Judge pattern for quality gates. Test your scripts before\n you run them in production.\n- \ud83d\udcb0 **Coming soon**: token usage and cost estimation appended to each result for budgeting and optimisation\n\n## \ud83d\ude80 The Only Enterprise AI DevOps Tool That Delivers RBAC Security, Robustness and Simplicity\n\n**LLM-CI-Runner stands alone in the market** as the only tool combining **100% schema enforcement**, **enterprise RBAC\nauthentication**, and robust **Semantic Kernel integration with templates** in a single CLI solution. **No other tool\ndelivers all three critical enterprise requirements together**.\n\n## Installation\n\n```bash\npip install llm-ci-runner\n```\n\nThat's it! No complex setup, no dependency management - just install and use. Perfect for CI/CD pipelines and local\ndevelopment.\n\n## Quick Start\n\n### 1. Install from PyPI\n\n```bash\npip install llm-ci-runner\n```\n\n### 2. Set Environment Variables\n\n**Azure OpenAI (Priority 1):**\n\n```bash\nexport AZURE_OPENAI_ENDPOINT=\"https://your-resource.openai.azure.com/\"\nexport AZURE_OPENAI_MODEL=\"gpt-4.1-nano\" # or any other GPT deployment name\nexport AZURE_OPENAI_API_VERSION=\"2024-12-01-preview\" # Optional\n```\n\n**OpenAI (Fallback):**\n\n```bash\nexport OPENAI_API_KEY=\"your-very-secret-api-key\"\nexport OPENAI_CHAT_MODEL_ID=\"gpt-4.1-nano\" # or any OpenAI model\nexport OPENAI_ORG_ID=\"org-your-org-id\" # Optional\n```\n\n**Authentication Options:**\n\n- **Azure RBAC (Recommended)**: Uses `DefaultAzureCredential` for Azure RBAC authentication - no API key needed!\n See [Microsoft Docs](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python)\n for setup.\n- **Azure API Key**: Set `AZURE_OPENAI_API_KEY` environment variable if not using RBAC.\n- **OpenAI API Key**: Required for OpenAI fallback when Azure is not configured.\n\n**Priority**: Azure OpenAI takes priority when both Azure and OpenAI environment variables are present.\n\n### 3a. Basic Usage\n\n```bash\n# Simple chat example\nllm-ci-runner --input-file examples/01-basic/simple-chat/input.json\n\n# With structured output schema\nllm-ci-runner \\\n --input-file examples/01-basic/sentiment-analysis/input.json \\\n --schema-file examples/01-basic/sentiment-analysis/schema.json\n\n# Custom output file\nllm-ci-runner \\\n --input-file examples/02-devops/pr-description/input.json \\\n --schema-file examples/02-devops/pr-description/schema.json \\\n --output-file pr-analysis.json\n\n# YAML input files (alternative to JSON)\nllm-ci-runner \\\n --input-file config.yaml \\\n --schema-file schema.yaml \\\n --output-file result.yaml\n```\n\n### 3b. Template-Based Workflows\n\n**Dynamic prompt generation with YAML, Handlebars or Jinja2 templates:**\n\n```bash\n# Handlebars template example\nllm-ci-runner \\\n --template-file examples/05-templates/handlebars-template/template.hbs \\\n --template-vars examples/05-templates/handlebars-template/template-vars.yaml \\\n --schema-file examples/05-templates/handlebars-template/schema.yaml \\\n --output-file handlebars-result.yaml\n \n# Or using Jinja2 templates\nllm-ci-runner \\\n --template-file examples/05-templates/jinja2-template/template.j2 \\\n --template-vars examples/05-templates/jinja2-template/template-vars.yaml \\\n --schema-file examples/05-templates/jinja2-template/schema.yaml \\\n --output-file jinja2-result.yaml\n```\n\nFor more examples see the [examples directory](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples).\n\n**Benefits of Template Approach:**\n\n- \ud83c\udfaf **Reusable Templates**: Create once, use across multiple scenarios\n- \ud83d\udcdd **YAML Configuration**: More readable than JSON for complex setups\n- \ud83d\udd04 **Dynamic Content**: Variables and conditional rendering\n- \ud83d\ude80 **CI/CD Ready**: Perfect for parameterized pipeline workflows\n\n### 4. Python Library Usage\n\n**You can use LLM CI Runner directly from Python with both file-based and string-based templates, and with either dict\nor file-based variables and schemas. The main entrypoint is:**\n\n```python \nfrom llm_ci_runner.core import run_llm_task # Adjust import as needed for your package layout \n``` \n\n#### Basic Usage: File-Based Input\n\n```python \nimport asyncio \nfrom llm_ci_runner.core import run_llm_task \n \nasync def main(): \n # Run with a traditional JSON input file (messages, etc) \n response = await run_llm_task(_input_file=\"examples/01-basic/simple-chat/input.json\") \n print(response) \n \nasyncio.run(main()) \n``` \n\n#### File-Based Template Usage\n\n```python \nimport asyncio \nfrom llm_ci_runner.core import run_llm_task \n \nasync def main(): \n # Handlebars, Jinja2, or Semantic Kernel YAML template via file \n response = await run_llm_task( \n template_file=\"examples/05-templates/pr-review-template/template.hbs\", \n template_vars_file=\"examples/05-templates/pr-review-template/template-vars.yaml\", \n schema_file=\"examples/05-templates/pr-review-template/schema.yaml\", \n output_file=\"analysis.json\" \n ) \n print(response) \n \nasyncio.run(main()) \n``` \n\n#### String-Based Template Usage\n\n```python \nimport asyncio \nfrom llm_ci_runner.core import run_llm_task \n \nasync def main(): \n # String template (Handlebars example) \n response = await run_llm_task( \n template_content=\"Hello {{name}}!\", \n template_format=\"handlebars\", \n template_vars={\"name\": \"World\"}, \n ) \n print(response) \n \nasyncio.run(main()) \n``` \n\n#### Semantic Kernel YAML Template with Embedded Schema\n\nPlease refer to the full example in [examples/05-templates/sem-ker-structured-analysis/README.md](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/examples/05-templates/sem-ker-structured-analysis/README.md).\n\n```python \nimport asyncio \nfrom llm_ci_runner.core import run_llm_task \n \nasync def main(): \n template_content = \"\"\" \ntemplate: \"Analyze: {{input_text}}\" \ninput_variables: \n - name: input_text \nexecution_settings: \n azure_openai: \n temperature: 0.1 \n\"\"\" \n response = await run_llm_task( \n template_content=template_content, \n template_format=\"semantic-kernel\", \n template_vars={\"input_text\": \"Sample data\"} \n ) \n print(response) \n \nasyncio.run(main()) \n``` \n\n#### Advanced: Dict-based Schema and Variables\n\n```python \nimport asyncio \nfrom llm_ci_runner.core import run_llm_task \n \nasync def main(): \n schema = { \n \"type\": \"object\", \n \"properties\": { \n \"sentiment\": {\"type\": \"string\", \"enum\": [\"positive\", \"negative\", \"neutral\"]}, \n \"confidence\": {\"type\": \"number\", \"minimum\": 0, \"maximum\": 1} \n }, \n \"required\": [\"sentiment\", \"confidence\"] \n } \n template = \"Analyze this review: {{review}}\" \n variables = {\"review\": \"I love the new update!\"} \n \n response = await run_llm_task( \n template_content=template, \n template_format=\"handlebars\", \n template_vars=variables, \n schema=schema \n ) \n print(response) \n \nasyncio.run(main()) \n``` \n\n#### Notes & Tips\n\n- **Only one of** `_input_file`, `template_file`, or `template_content` **may be specified** at a time.\n- **Template variables**: Use `template_vars` (Python dict or YAML file path), or `template_vars_file` (YAML file path).\n- **Schema**: Use `schema` (dict or JSON/YAML file path), or `schema_file` (file path).\n- **template_format** is required with `template_content`. Allowed: `\"handlebars\"`, `\"jinja2\"`, `\"semantic-kernel\"`.\n- **output_file**: If specified, writes response to file.\n\n**Returns:** String (for text output) or dict (for structured JSON output).\n\n**Errors:** Raises `InputValidationError` or `LLMRunnerError` on invalid input or execution failure.\n\n### 5. Development Setup (Optional)\n\nFor contributors or advanced users who want to modify the source:\n\n```bash\n# Install UV if you haven't already\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Clone and install for development\ngit clone https://github.com/Nantero1/ai-first-devops-toolkit.git\ncd ai-first-devops-toolkit\nuv sync\n\n# Run from source\nuv run llm-ci-runner --input-file examples/01-basic/simple-chat/input.json\n```\n\n## The AI-First Transformation: Why Unstructured \u2192 Structured Matters\n\nLLMs excel at extracting meaning from messy text, logs, documents, and mixed-format data, then emitting *\n*schema-compliant JSON/YAML** that downstream systems can trust. This unlocks:\n\n- **\ud83d\udd04 Straight-Through Processing**: Structured payloads feed BI dashboards, RPA robots, and CI/CD gates without human\n parsing\n- **\ud83c\udfaf Context-Aware Decisions**: LLMs fuse domain knowledge with live telemetry to prioritize incidents, forecast\n demand, and spot security drift\n- **\ud83d\udccb Auditable Compliance**: Formal outputs make it easy to track decisions for regulators and ISO/NIST audits\n- **\u26a1 Rapid Workflow Automation**: Enable automation across customer service, supply-chain planning, HR case handling,\n and security triage\n- **\ud83d\udd17 Safe Pipeline Composition**: Structured contracts let AI-first pipelines remain observable and composable while\n capitalizing on unstructured enterprise data\n\n## Input Formats\n\n### Traditional JSON Input\n\n```json\n{\n \"messages\": [\n {\n \"role\": \"system\",\n \"content\": \"You are a helpful assistant.\"\n },\n {\n \"role\": \"user\",\n \"content\": \"Your task description here\"\n }\n ],\n \"context\": {\n \"session_id\": \"optional-session-id\",\n \"metadata\": {\n \"any\": \"additional context\"\n }\n }\n}\n```\n\n### YAML Input\n\n```yaml\nmessages:\n - role: system\n content: \"You are a helpful assistant.\"\n - role: user\n content: \"Your task description here\"\ncontext:\n session_id: \"optional-session-id\"\n metadata:\n any: \"additional context\"\n```\n\n### Template-Based Input\n\n**Handlebars Template** (`template.hbs`):\n\n```handlebars\n\n<message role=\"system\">\n You are an expert {{expertise.domain}} engineer.\n Focus on {{expertise.focus_areas}}.\n</message>\n\n<message role=\"user\">\n Analyze this {{task.type}}:\n\n {{#each task.items}}\n - {{this}}\n {{/each}}\n\n Requirements: {{task.requirements}}\n</message>\n```\n\n**Jinja2 Template** (`template.j2`):\n\n```jinja2\n<message role=\"system\">\nYou are an expert {{expertise.domain}} engineer.\nFocus on {{expertise.focus_areas}}.\n</message>\n\n<message role=\"user\">\nAnalyze this {{task.type}}:\n\n{% for item in task.items %}\n- {{item}}\n{% endfor %}\n\nRequirements: {{task.requirements}}\n</message>\n```\n\n**Template Variables** (`vars.yaml`):\n\n```yaml\nexpertise:\n domain: \"DevOps\"\n focus_areas: \"security, performance, maintainability\"\ntask:\n type: \"pull request\"\n items:\n - \"Changed authentication logic\"\n - \"Updated database queries\"\n - \"Added input validation\"\n requirements: \"Focus on security vulnerabilities\"\n```\n\n## Structured Outputs with 100% Schema Enforcement\n\nWhen you provide a `--schema-file`, the runner guarantees perfect schema compliance:\n\n```bash\nllm-ci-runner \\\n --input-file examples/01-basic/sentiment-analysis/input.json \\\n --schema-file examples/01-basic/sentiment-analysis/schema.json\n```\n\n**Note**: Output defaults to `result.json`. Use `--output-file custom-name.json` for custom output files.\n\n**Supported Schema Features**:\n\u2705 String constraints (enum, minLength, maxLength, pattern) \n\u2705 Numeric constraints (minimum, maximum, multipleOf) \n\u2705 Array constraints (minItems, maxItems, items type) \n\u2705 Required fields enforced at generation time \n\u2705 Type validation (string, number, integer, boolean, array)\n\n## CI/CD Integration\n\n### GitHub Actions Example\n\n```yaml\n- name: Setup Python\n uses: actions/setup-python@v5\n with:\n python-version: '3.12'\n\n- name: Install LLM CI Runner\n run: pip install llm-ci-runner\n\n- name: Generate PR Review with Templates\n run: |\n llm-ci-runner \\\n --template-file .github/templates/pr-review.j2 \\\n --template-vars pr-context.yaml \\\n --schema-file .github/schemas/pr-review.yaml \\\n --output-file pr-analysis.yaml\n env:\n AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}\n AZURE_OPENAI_MODEL: ${{ secrets.AZURE_OPENAI_MODEL }}\n```\n\nFor complete CI/CD examples, see *\n*[examples/uv-usage-example.md](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/examples/uv-usage-example.md)\n**. This repo is also using itself for release note generation, **check it\nout [here](https://github.com/Nantero1/ai-first-devops-toolkit/blob/c4066d347ae14d37cb674e36007a678f38b36439/.github/workflows/release.yml#L145-L149)\n**.\n\n## Authentication\n\n**Azure OpenAI**: Uses Azure's `DefaultAzureCredential` supporting:\n\n- Environment variables (local development)\n- Managed Identity (recommended for Azure CI/CD)\n- Azure CLI (local development)\n- Service Principal (non-Azure CI/CD)\n\n**OpenAI**: Uses API key authentication with optional organization ID.\n\n## Testing\n\nWe maintain comprehensive test coverage with **100% success rate**:\n\n```bash\n# For package users - install test dependencies\npip install llm-ci-runner[dev]\n\n# For development - install from source with test dependencies\nuv sync --group dev\n\n# Run specific test categories\npytest tests/unit/ -v # 70 unit tests\npytest tests/integration/ -v # End-to-end examples\npytest acceptance/ -v # LLM-as-judge evaluation\n\n# Or with uv for development\nuv run pytest tests/unit/ -v\nuv run pytest tests/integration/ -v\nuv run pytest acceptance/ -v\n```\n\n## Architecture\n\nBuilt on **Microsoft Semantic Kernel** for:\n\n- Enterprise-ready Azure OpenAI and OpenAI integration\n- Future-proof model compatibility\n- **100% Schema Enforcement**: KernelBaseModel integration with token-level constraints\n- **Dynamic Model Creation**: Runtime JSON schema \u2192 Pydantic model conversion\n- **Azure RBAC**: Azure RBAC via DefaultAzureCredential\n- **Automatic Fallback**: Azure-first priority with OpenAI fallback\n\n## The AI-First Development Journey\n\nThis toolkit is your first step\ntoward [AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html). As you\nintegrate AI into your development workflows, you'll experience:\n\n1. **\ud83d\ude80 Exponential Productivity**: AI handles routine tasks while you focus on architecture\n2. **\ud83c\udfaf Guaranteed Quality**: Schema enforcement eliminates validation errors\n3. **\ud83e\udd16 Autonomous Operations**: AI agents make decisions in your pipelines\n4. **\ud83d\udcc8 Continuous Improvement**: Every interaction improves your AI system\n\n**The future belongs to teams that master AI-first principles.** This toolkit gives you the foundation to start that\njourney today.\n\n## Real-World Examples\n\nYou can explore the **[examples directory](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples)** for\na complete collection of self-contained examples organized by category.\n\nFor comprehensive real-world CI/CD scenarios, see *\n*[examples/uv-usage-example.md](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/examples/uv-usage-example.md)\n**.\n\n### 100 AI Automation Use Cases for AI-First Automation\n\n**DevOps & Engineering** \ud83d\udd27\n\n1. \ud83e\udd16 AI-generated PR review \u2013 automated pull request analysis with structured review findings\n2. \ud83d\udcdd Release note composer \u2013 map commits to semantic-version bump rules and structured changelogs\n3. \ud83d\udd0d Vulnerability scanner \u2013 map code vulnerabilities to OWASP standards with actionable remediation\n4. \u2638\ufe0f Kubernetes manifest optimizer \u2013 produce risk-scored diffs and security hardening recommendations\n5. \ud83d\udcca Log anomaly triager \u2013 convert system logs into OTEL-formatted events for SIEM ingestion\n6. \ud83d\udcb0 Cloud cost explainer \u2013 output tagged spend by team in FinOps schema for budget optimization\n7. \ud83d\udd04 API diff analyzer \u2013 produce backward-compatibility scorecards from specification changes\n8. \ud83d\udee1\ufe0f IaC drift detector \u2013 turn Terraform plans into CVE-linked security findings\n9. \ud83d\udccb Dependency license auditor \u2013 emit SPDX-compatible reports for compliance tracking\n10. \ud83c\udfaf SLA breach summarizer \u2013 file structured JIRA tickets with SMART action items\n\n**Governance, Risk & Compliance** \ud83c\udfdb\ufe0f\n\n11. \ud83d\udcca Regulatory delta analyzer \u2013 emit change-impact matrices from new compliance requirements\n12. \ud83c\udf31 ESG report synthesizer \u2013 map CSR prose to GRI indicators and sustainability metrics\n13. \ud83d\udccb SOX-404 narrative converter \u2013 transform controls descriptions into testable audit checklists\n14. \ud83c\udfe6 Basel III stress-test interpreter \u2013 output capital risk buckets from regulatory scenarios\n15. \ud83d\udd75\ufe0f AML SAR formatter \u2013 convert investigator notes into Suspicious Activity Report structures\n16. \ud83d\udd12 Privacy policy parser \u2013 generate GDPR data-processing-activity logs from legal text\n17. \ud83d\udd0d Internal audit evidence linker \u2013 export control traceability graphs for compliance tracking\n18. \ud83d\udcca Carbon emission disclosure normalizer \u2013 structure sustainability data into XBRL taxonomy\n19. \u2696\ufe0f Regulatory update tracker \u2013 generate structured compliance action items from guideline changes\n20. \ud83d\udee1\ufe0f Safety inspection checker \u2013 transform narratives into OSHA citation checklists\n\n**Financial Services** \ud83c\udfe6\n\n21. \ud83c\udfe6 Loan application analyzer \u2013 transform free-text applications into Basel-III risk-model inputs\n22. \ud83d\udcca Earnings call sentiment quantifier \u2013 output KPI deltas and investor sentiment scores\n23. \ud83d\udcb9 Budget variance explainer \u2013 produce drill-down pivot JSON for financial analysis\n24. \ud83d\udcc8 Portfolio risk dashboard builder \u2013 feed VaR models with structured investment analysis\n25. \ud83d\udcb3 Fraud alert generator \u2013 map investigation notes to CVSS-scored security metrics\n26. \ud83d\udcb0 Treasury cash-flow predictor \u2013 ingest email forecasts into structured planning models\n27. \ud83d\udcca Financial forecaster \u2013 summarize reports into structured cash-flow and projection objects\n28. \ud83e\uddfe Invoice processor \u2013 convert receipts into double-entry ledger posts with GAAP tags\n29. \ud83d\udccb Stress test scenario packager \u2013 structure regulatory submission data for banking compliance\n30. \ud83c\udfe6 Insurance claim assessor \u2013 return structured claim-decision objects with risk scores\n\n**Healthcare & Life Sciences** \ud83c\udfe5\n\n31. \ud83c\udfe5 Patient intake processor \u2013 build HL7/FHIR-compliant patient records from free-form intake forms\n32. \ud83e\udde0 Mental health triage assistant \u2013 structure referral notes with priority classifications and care pathways\n33. \ud83d\udcca Radiology report coder \u2013 output SNOMED-coded JSON from diagnostic imaging narratives\n34. \ud83d\udc8a Clinical trial note packager \u2013 create FDA eCTD modules from research documentation\n35. \ud83d\udccb Prescription parser \u2013 turn text prescriptions into structured e-Rx objects with dosage validation\n36. \u26a1 Vital sign anomaly summarizer \u2013 generate alert reports with clinical priority rankings\n37. \ud83e\uddea Lab result organizer \u2013 output LOINC-coded tables from diagnostic test narratives\n38. \ud83c\udfe5 Medical device log summarizer \u2013 generate UDI incident files for regulatory reporting\n39. \ud83d\udcc8 Patient feedback sentiment analyzer \u2013 feed quality-of-care KPIs from satisfaction surveys\n40. \ud83d\udc69\u200d\u2695\ufe0f Clinical observation compiler \u2013 convert research notes into structured data for trials\n\n**Legal & Compliance** \u2696\ufe0f\n\n41. \ud83c\udfdb\ufe0f Legal contract parser \u2013 extract clauses and compute risk scores from contract documents\n42. \ud83d\udcdd Court opinion digest \u2013 summarize judicial opinions into structured precedent and citation graphs\n43. \ud83c\udfdb\ufe0f Legal discovery summarizer \u2013 extract key issues and risks from large document sets\n44. \ud83d\udcbc Contract review summarizer \u2013 extract risk factors and key dates from legal contracts\n45. \ud83c\udfdb\ufe0f Policy impact assessor \u2013 convert policy proposals into stakeholder impact matrices\n46. \ud83d\udcdc Patent novelty comparator \u2013 produce claim-overlap matrices from prior art analysis\n47. \ud83c\udfdb\ufe0f Legal bill auditor \u2013 transform billing details into itemized expense and compliance reports\n48. \ud83d\udccb Case strategy brainstormer \u2013 summarize likely arguments from litigation documentation\n49. \ud83d\udcbc Legal email analyzer \u2013 extract key issues and deadlines from email threads for review\n50. \u2696\ufe0f Expert witness report normalizer \u2013 create citation-linked outlines from testimony records\n\n**Customer Experience & Sales** \ud83d\uded2\n\n51. \ud83c\udfa7 Tier-1 support chatbot \u2013 convert customer queries into tickets with reproducible troubleshooting steps\n52. \u2b50 Review sentiment miner \u2013 produce product-feature tallies from customer feedback analysis\n53. \ud83d\udcc9 Churn risk email summarizer \u2013 export CRM risk scores from customer communication patterns\n54. \ud83d\uddfa\ufe0f Omnichannel conversation unifier \u2013 generate customer journey maps from multi-platform interactions\n55. \u2753 Dynamic FAQ builder \u2013 structure knowledge base content from community forum discussions\n56. \ud83d\udccb Proposal auto-grader \u2013 output RFP compliance matrices with scoring rubrics\n57. \ud83d\udcc8 Upsell opportunity extractor \u2013 create lead-scoring JSON from customer interaction analysis\n58. \ud83d\udcf1 Social media crisis detector \u2013 feed escalation playbooks with brand sentiment monitoring\n59. \ud83c\udf10 Multilingual intent router \u2013 tag customer chats to appropriate support queues by language/topic\n60. \ud83c\udfaf Marketing copy generator \u2013 create brand-compliant content with tone and messaging constraints\n\n**HR & People Operations** \ud83d\udc65\n\n61. \ud83d\udcc4 CV-to-JD matcher \u2013 rank candidates with explainable competency scores and fit analysis\n62. \ud83c\udfa4 Interview transcript summarizer \u2013 export structured competency rubrics with evaluation criteria\n63. \u2705 Onboarding policy compliance checker \u2013 produce new-hire checklist completion tracking\n64. \ud83d\udcca Performance review sentiment analyzer \u2013 create growth-plan JSON with development recommendations\n65. \ud83d\udcb0 Payroll inquiry classifier \u2013 map employee emails to structured case codes for HR processing\n66. \ud83c\udfe5 Benefits Q&A automation \u2013 generate eligibility responses from policy documentation\n67. \ud83d\udeaa Exit interview insight extractor \u2013 feed retention dashboards with structured departure analytics\n68. \ud83d\udcda Training content gap mapper \u2013 align job roles to skill taxonomies for learning programs\n69. \ud83d\udee1\ufe0f Workplace incident processor \u2013 convert safety reports into OSHA 301 compliance records\n70. \ud83d\udcca Diversity metric synthesizer \u2013 summarize inclusion survey data into actionable insights\n\n**Supply Chain & Manufacturing** \ud83c\udfed\n\n71. \ud83d\udcca Demand forecast summarizer \u2013 output SKU-level predictions from market analysis and sales data\n72. \ud83d\udccb Purchase order processor \u2013 convert supplier communications into structured ERP line-items\n73. \ud83c\udf31 Supplier risk scanner \u2013 generate ESG compliance scores from vendor assessment reports\n74. \ud83d\udd27 Predictive maintenance log analyst \u2013 produce work orders from equipment telemetry narratives\n75. \ud83d\ude9b Logistics delay explainer \u2013 return route-change suggestions from transportation disruption reports\n76. \u267b\ufe0f Circular economy return classifier \u2013 create refurbishment tags from product return descriptions\n77. \ud83c\udf0d Carbon footprint calculator \u2013 map transport legs to CO\u2082e emissions for sustainability reporting\n78. \ud83d\udce6 Safety stock alert generator \u2013 output inventory triggers with lead-time assumptions\n79. \ud83d\udcdc Regulatory import/export harmonizer \u2013 produce HS-code sheets from trade documentation\n80. \ud83c\udfed Production yield analyzer \u2013 generate efficiency reports from manufacturing floor logs\n\n**Security & Risk Management** \ud83d\udd12\n\n81. \ud83d\udee1\ufe0f MITRE ATT&CK mapper \u2013 translate IDS alerts into tactic-technique JSON for threat intelligence\n82. \ud83c\udfa3 Phishing email extractor \u2013 produce IOC STIX bundles from security incident reports\n83. \ud83d\udd10 Zero-trust policy generator \u2013 convert narrative access requests into structured policy rules\n84. \ud83d\udea8 SOC alert deduplicator \u2013 cluster security tickets by kill-chain stage for efficient triage\n85. \ud83c\udff4\u200d\u2620\ufe0f Red team debrief summarizer \u2013 export OWASP Top-10 gaps from penetration test reports\n86. \ud83d\udccb Data breach notifier \u2013 craft GDPR-compliant disclosure packets with timeline and impact data\n87. \ud83e\udde0 Threat intel feed normalizer \u2013 convert mixed security PDFs into MISP threat objects\n88. \ud83d\udd0d Secret leak scanner \u2013 output GitHub code-owner mentions from repository security scans\n89. \ud83d\udcca Vendor risk questionnaire scorer \u2013 generate SIG Lite security assessment answers\n90. \ud83c\udfd7\ufe0f Security audit tracker \u2013 link ISO-27001 controls to evidence artifacts for compliance\n\n**Knowledge & Productivity** \ud83d\udcda\n\n91. \ud83c\udf99\ufe0f Meeting transcript processor \u2013 extract action items with owners and deadlines into project tracking JSON\n92. \ud83d\udcda Research paper summarizer \u2013 export citation graphs and key findings for literature review databases\n93. \ud83d\udccb SOP generator \u2013 convert process narratives into step-by-step validation checklists\n94. \ud83d\udd04 Code diff summarizer \u2013 generate reviewer hints and impact analysis from version control changes\n95. \ud83d\udcca API changelog analyzer \u2013 produce backward-compatibility scorecards for development teams\n96. \ud83e\udde0 Mind map creator \u2013 structure brainstorming sessions into hierarchical knowledge trees\n97. \ud83d\udcd6 Knowledge base gap detector \u2013 suggest article stubs from frequently asked questions analysis\n98. \ud83c\udfaf Personal OKR journal parser \u2013 output progress dashboards with milestone tracking\n99. \ud83d\udcbc White paper composer \u2013 transform technical discussions into structured thought leadership content\n100. \ud83e\udde9 Universal transformer \u2013 convert any unstructured domain knowledge into your custom schema-validated JSON\n\n## License\n\nMIT License - See [LICENSE](https://github.com/Nantero1/ai-first-devops-toolkit/blob/main/LICENSE) file for details.\nCopyright (c) 2025, Benjamin Linnik.\n\n## Support\n\n**\ud83d\udc1b Found a bug? \ud83d\udca1 Have a question? \ud83d\udcda Need help?**\n\n**GitHub is your primary destination for all support:**\n\n- **\ud83d\udccb Issues & Bug Reports**: [Create an issue](https://github.com/Nantero1/ai-first-devops-toolkit/issues)\n- **\ud83d\udcd6 Documentation**: [Browse examples](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples)\n- **\ud83d\udd27 Source Code**: [View source](https://github.com/Nantero1/ai-first-devops-toolkit)\n\n**Before opening an issue, please:**\n\n1. \u2705 Check the [examples directory](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples) for\n solutions\n2. \u2705 Review the error logs (beautiful output with Rich!)\n3. \u2705 Validate your Azure authentication and permissions\n4. \u2705 Ensure your input JSON follows the required format\n5. \u2705 Search existing [issues](https://github.com/Nantero1/ai-first-devops-toolkit/issues) for similar problems\n\n**Quick Links:**\n\n- \ud83d\ude80 [Getting Started Guide](https://github.com/Nantero1/ai-first-devops-toolkit#quick-start)\n- \ud83d\udcda [Complete Examples](https://github.com/Nantero1/ai-first-devops-toolkit/tree/main/examples)\n- \ud83d\udd27 [CI/CD Integration](https://github.com/Nantero1/ai-first-devops-toolkit#cicd-integration)\n- \ud83c\udfaf [Use Cases](https://github.com/Nantero1/ai-first-devops-toolkit#use-cases)\n\n---\n\n*Ready to embrace the AI-First future? Start with this toolkit and build your path to exponential productivity. Learn\nmore about the AI-First DevOps revolution\nin [Building AI-First DevOps](https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html).*\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A simple CI/CD utility for running LLM tasks with Semantic Kernel",
"version": "1.5.1",
"project_urls": {
"Bug Tracker": "https://github.com/Nantero1/ai-first-devops-toolkit/issues",
"Changelog": "https://github.com/Nantero1/ai-first-devops-toolkit/releases",
"Documentation": "https://github.com/Nantero1/ai-first-devops-toolkit",
"Homepage": "https://technologyworkroom.blogspot.com/2025/06/building-ai-first-devops.html",
"Repository": "https://github.com/Nantero1/ai-first-devops-toolkit",
"Source": "https://github.com/Nantero1/ai-first-devops-toolkit"
},
"split_keywords": [
"ai",
" automation",
" azure-openai",
" ci-cd",
" devops",
" llm",
" semantic-kernel"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "bbadd19ddf246c289b398f9d1d5e009dc48b1477becb8d4d2448a0ac3795bb34",
"md5": "39d4bd191d4f1f15d058032ac265bea3",
"sha256": "4d1860e7f2ff34547a0e4d3293b4aca865715e98474d2f2e05fdfa77cf39c7a4"
},
"downloads": -1,
"filename": "llm_ci_runner-1.5.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "39d4bd191d4f1f15d058032ac265bea3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 50116,
"upload_time": "2025-07-27T17:58:06",
"upload_time_iso_8601": "2025-07-27T17:58:06.326425Z",
"url": "https://files.pythonhosted.org/packages/bb/ad/d19ddf246c289b398f9d1d5e009dc48b1477becb8d4d2448a0ac3795bb34/llm_ci_runner-1.5.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "0f27c0fbe38f414846b405b27c4eac4a19c1c1da173ff8fcb14809dbbc4b4989",
"md5": "b2784d915f8d52cc8ac7cb4e32ce4fd3",
"sha256": "27f5525ca87e017fb0e87bd314b5f0abeaeeaa9965ba54af0cac4dfd09f66ca8"
},
"downloads": -1,
"filename": "llm_ci_runner-1.5.1.tar.gz",
"has_sig": false,
"md5_digest": "b2784d915f8d52cc8ac7cb4e32ce4fd3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 822262,
"upload_time": "2025-07-27T17:58:08",
"upload_time_iso_8601": "2025-07-27T17:58:08.030856Z",
"url": "https://files.pythonhosted.org/packages/0f/27/c0fbe38f414846b405b27c4eac4a19c1c1da173ff8fcb14809dbbc4b4989/llm_ci_runner-1.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-27 17:58:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Nantero1",
"github_project": "ai-first-devops-toolkit",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "llm-ci-runner"
}
JSON
