| Name | airflow-mcp-hipposys JSON |
| Version |
0.1.0a12
JSON |
| download |
| home_page | None |
| Summary | An MCP server for Apache Airflow that enables non-technical users to interact with Apache Airflow workflows through natural language. |
| upload_time | 2025-08-03 15:47:07 |
| maintainer | None |
| docs_url | None |
| author | Hipposys |
| requires_python | <4.0,>=3.10 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Airflow MCP
This project implements an MCP server for Apache Airflow, enabling users to interact with their orchestration platform using natural language.
With a few minutes of setup, you should be able to use Claude Desktop or any MCP-enabled LLM to ask questions like:
- "What DAGs do we have in our Airflow cluster?"
- "What is our latest failed DAG?"
And more!
## About MCP and Airflow MCP
The Model Context Protocol (MCP) is an open standard creating secure connections between data sources and AI applications. This repository provides a custom MCP server for Apache Airflow that transforms how teams interact with their orchestration platform through natural language.
## 🚀 Features
- Query pipeline statuses through natural language
- Troubleshoot DAG failures efficiently
- Retrieve comprehensive DAG information
- Trigger DAGs based on their status
- Monitor execution results
- Analyze DAG components and configurations
## 🛠️ Getting Started
### Prerequisites
If you already have an Airflow instance and want to use our prebuilt Docker image, you only need:
- Docker
- Access to your Apache Airflow instance
- LLM access (Claude, ChatGPT, or AWS Bedrock)
This repository also provides a local setup for Apache Airflow, which you can use for demo purposes.
You can also build the MCP server from source, detailed below.
### Quick Start - Using the Prebuilt Docker Image
If you have an Airflow instance and want to use our prebuilt Docker image, simply follow these steps:
You'll need to configure Claude Desktop to connect to your Airflow instance. If you haven't configured Claude Desktop for use with MCP before, we recommend following the [Claude Desktop documentation](https://modelcontextprotocol.io/quickstart/user).
Here are the steps to configure Claude Desktop to connect to your Airflow instance, using our prebuilt Docker image:
- Open Claude Desktop
- Go to Settings → Developer tab
- Edit the MCP config with:
```json
{
"mcpServers": {
"airflow_mcp": {
"command": "docker",
"args": ["run", "-i", "--rm", "-e", "airflow_api_url", "-e", "airflow", "-e", "airflow", "hipposysai/airflow-mcp:latest"],
"env": {
"airflow_api_url": "http://host.docker.internal:8088/api/v1",
"airflow_username": "airflow",
"airflow_password": "airflow"
}
}
}
}
```
### Running MCP Locally with Claude Desktop
1. Clone this repository:
```
git clone https://github.com/hipposys-ltd/airflow-mcp
```
2. If you don't have a running Airflow environment, start one with:
```
just airflow
```
This will start an Airflow instance on port 8088, with username `airflow` and password `airflow`.
You can access Airflow at http://localhost:8088/ and see multiple DAGs configured:
.
These DAGs have complex dependencies, some running on a schedule and some using Airflow's Dataset functionality.
3. Configure Claude Desktop:
You'll need to configure Claude Desktop to connect to your Airflow instance. If you haven't configured Claude Desktop for use with MCP before, we recommend following the [Claude Desktop documentation](https://modelcontextprotocol.io/quickstart/user).
Here are the steps to configure Claude Desktop to connect to your Airflow instance:
- Open Claude Desktop
- Go to Settings → Developer tab
- Edit the MCP config with:
```json
{
"mcpServers": {
"airflow_mcp": {
"command": "docker",
"args": ["run", "-i", "--rm", "-e", "airflow_api_url", "-e", "airflow", "-e", "airflow", "hipposysai/airflow-mcp:latest"],
"env": {
"airflow_api_url": "http://host.docker.internal:8088/api/v1",
"airflow_username": "airflow",
"airflow_password": "airflow"
}
}
}
}
```
4. Test your setup by asking Claude: "What DAGs do we have in our Airflow cluster?"
### Integrating with LangChain
1. Set up environment:
```
cp template.env .env
```
2. Configure your LLM model in `.env`:
- For AWS Bedrock: `LLM_MODEL_ID=bedrock:...`
- For Anthropic: `LLM_MODEL_ID=anthropic:...`
- For OpenAI: `LLM_MODEL_ID=openai:...`
3. Add your API credentials to `.env`:
- AWS credentials for Bedrock
- `ANTHROPIC_API_KEY` for Claude
- `OPENAI_API_KEY` for ChatGPT
4. (Optional) Connect to your own Airflow:
```
airflow_api_url=your_airflow_api_url
airflow_username=your_airflow_username
airflow_password=your_airflow_password
```
5. Start the project:
- With bundled Airflow: `just project`
- With existing Airflow: `just project_no_airflow`
6. Open web interfaces:
```
just open_web_tabs
```
7. Try it out by asking "How many DAGs failed today?" in the Chat UI
## 📝 Example Usage
- "What DAGs do we have in our Airflow cluster?"
- "Identify all DAGs with failed status in their most recent execution and trigger a new run for each one"
- "What operators are used by the transform_forecast_attendance DAG?"
- "Has the transform_forecast_attendance DAG ever completed successfully?"
## Running MCP with LangChain
You can use `from langchain_mcp_adapters.client import MultiServerMCPClient` in order to add our Airflow MCP as one of your tools in your LangChain app.
### Option 1: Separate Container (SSE Transport)
If you run our Airflow MCP as a separate container, use SSE transport:
```python
mcp_host = os.environ.get('mcp_host', 'mcp_sse_server:8000')
mcps = {
"AirflowMCP": {
"url": f"http://{mcp_host}/sse",
"transport": "sse",
"headers": {"Authorization": f"""Bearer {
os.environ.get('MCP_TOKEN')}"""}
}
}
```
### Option 2: Embedded Server (STDIO Transport)
If you want to run our MCP server as part of the LangChain code, without any outside code, use STDIO and make sure you install our library first (`airflow-mcp-hipposys = "0.1.0a11"`):
```python
mcps = {
"AirflowMCP":
{
'command': "python",
'args': ["-m", "airflow_mcp_hipposys.mcp_airflow"],
"transport": "stdio",
'env': {k: v for k, v in {
'AIRFLOW_ASSISTENT_AI_CONN': os.getenv(
'AIRFLOW_ASSISTENT_AI_CONN'),
'airflow_api_url': os.getenv('airflow_api_url'),
'airflow_username': os.getenv('airflow_username'),
'airflow_password': os.getenv('airflow_password'),
'AIRFLOW_INSIGHTS_MODE':
os.getenv('AIRFLOW_INSIGHTS_MODE'),
'POST_MODE': os.getenv('POST_MODE'),
'TRANSPORT_TYPE': 'stdio',
'_AIRFLOW_WWW_USER_USERNAME':
os.getenv('_AIRFLOW_WWW_USER_USERNAME'),
'_AIRFLOW_WWW_USER_PASSWORD':
os.getenv('_AIRFLOW_WWW_USER_PASSWORD')
}.items() if v is not None}
}
}
```
### Using the Tools
Then in both cases, pass it to the tools:
```python
client = MultiServerMCPClient(mcps)
tools = await client.get_tools()
```
## 🤝 Contributing
We enthusiastically invite the community to contribute to this open-source initiative! Whether you're interested in:
- Adding new features
- Improving documentation
- Enhancing compatibility with different LLM providers
- Reporting bugs
- Suggesting improvements
Please feel free to submit pull requests or open issues on our GitHub repository.
## 🔗 Links
- [GitHub Repository](https://github.com/hipposys-ltd/airflow-mcp)
- [Docker Repository](https://hub.docker.com/repository/docker/hipposysai/airflow-mcp/general)
---
Raw data
{
"_id": null,
"home_page": null,
"name": "airflow-mcp-hipposys",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Hipposys",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/23/6d/06e837330ed7ebdbc291307ea5dbdce0cc2344a19ba4de94cdfb9c8844a9/airflow_mcp_hipposys-0.1.0a12.tar.gz",
"platform": null,
"description": "# Airflow MCP\n\nThis project implements an MCP server for Apache Airflow, enabling users to interact with their orchestration platform using natural language.\n\nWith a few minutes of setup, you should be able to use Claude Desktop or any MCP-enabled LLM to ask questions like:\n- \"What DAGs do we have in our Airflow cluster?\"\n- \"What is our latest failed DAG?\"\n\nAnd more!\n\n## About MCP and Airflow MCP\n\nThe Model Context Protocol (MCP) is an open standard creating secure connections between data sources and AI applications. This repository provides a custom MCP server for Apache Airflow that transforms how teams interact with their orchestration platform through natural language.\n\n## \ud83d\ude80 Features\n\n- Query pipeline statuses through natural language\n- Troubleshoot DAG failures efficiently\n- Retrieve comprehensive DAG information\n- Trigger DAGs based on their status\n- Monitor execution results\n- Analyze DAG components and configurations\n\n## \ud83d\udee0\ufe0f Getting Started\n\n### Prerequisites\n\nIf you already have an Airflow instance and want to use our prebuilt Docker image, you only need:\n- Docker\n- Access to your Apache Airflow instance\n- LLM access (Claude, ChatGPT, or AWS Bedrock)\n\nThis repository also provides a local setup for Apache Airflow, which you can use for demo purposes.\n\nYou can also build the MCP server from source, detailed below.\n\n### Quick Start - Using the Prebuilt Docker Image\n\nIf you have an Airflow instance and want to use our prebuilt Docker image, simply follow these steps:\n\n\nYou'll need to configure Claude Desktop to connect to your Airflow instance. If you haven't configured Claude Desktop for use with MCP before, we recommend following the [Claude Desktop documentation](https://modelcontextprotocol.io/quickstart/user).\n\nHere are the steps to configure Claude Desktop to connect to your Airflow instance, using our prebuilt Docker image:\n\n- Open Claude Desktop\n- Go to Settings \u2192 Developer tab\n- Edit the MCP config with:\n```json\n{\n \"mcpServers\": {\n \"airflow_mcp\": {\n \"command\": \"docker\",\n \"args\": [\"run\", \"-i\", \"--rm\", \"-e\", \"airflow_api_url\", \"-e\", \"airflow\", \"-e\", \"airflow\", \"hipposysai/airflow-mcp:latest\"],\n \"env\": {\n \"airflow_api_url\": \"http://host.docker.internal:8088/api/v1\",\n \"airflow_username\": \"airflow\",\n \"airflow_password\": \"airflow\"\n }\n }\n }\n}\n```\n\n\n\n\n\n### Running MCP Locally with Claude Desktop\n\n1. Clone this repository:\n ```\n git clone https://github.com/hipposys-ltd/airflow-mcp\n ```\n\n2. If you don't have a running Airflow environment, start one with:\n ```\n just airflow\n ```\n\n This will start an Airflow instance on port 8088, with username `airflow` and password `airflow`.\n\n You can access Airflow at http://localhost:8088/ and see multiple DAGs configured:\n\n .\n\n These DAGs have complex dependencies, some running on a schedule and some using Airflow's Dataset functionality.\n\n3. Configure Claude Desktop:\n\n You'll need to configure Claude Desktop to connect to your Airflow instance. If you haven't configured Claude Desktop for use with MCP before, we recommend following the [Claude Desktop documentation](https://modelcontextprotocol.io/quickstart/user).\n\n Here are the steps to configure Claude Desktop to connect to your Airflow instance:\n\n - Open Claude Desktop\n - Go to Settings \u2192 Developer tab\n - Edit the MCP config with:\n ```json\n {\n \"mcpServers\": {\n \"airflow_mcp\": {\n \"command\": \"docker\",\n \"args\": [\"run\", \"-i\", \"--rm\", \"-e\", \"airflow_api_url\", \"-e\", \"airflow\", \"-e\", \"airflow\", \"hipposysai/airflow-mcp:latest\"],\n \"env\": {\n \"airflow_api_url\": \"http://host.docker.internal:8088/api/v1\",\n \"airflow_username\": \"airflow\",\n \"airflow_password\": \"airflow\"\n }\n }\n }\n }\n ```\n\n4. Test your setup by asking Claude: \"What DAGs do we have in our Airflow cluster?\"\n\n### Integrating with LangChain\n\n1. Set up environment:\n ```\n cp template.env .env\n ```\n\n2. Configure your LLM model in `.env`:\n - For AWS Bedrock: `LLM_MODEL_ID=bedrock:...`\n - For Anthropic: `LLM_MODEL_ID=anthropic:...`\n - For OpenAI: `LLM_MODEL_ID=openai:...`\n\n3. Add your API credentials to `.env`:\n - AWS credentials for Bedrock\n - `ANTHROPIC_API_KEY` for Claude\n - `OPENAI_API_KEY` for ChatGPT\n\n4. (Optional) Connect to your own Airflow:\n ```\n airflow_api_url=your_airflow_api_url\n airflow_username=your_airflow_username\n airflow_password=your_airflow_password\n ```\n\n5. Start the project:\n - With bundled Airflow: `just project`\n - With existing Airflow: `just project_no_airflow`\n\n6. Open web interfaces:\n ```\n just open_web_tabs\n ```\n\n7. Try it out by asking \"How many DAGs failed today?\" in the Chat UI\n\n## \ud83d\udcdd Example Usage\n\n- \"What DAGs do we have in our Airflow cluster?\"\n- \"Identify all DAGs with failed status in their most recent execution and trigger a new run for each one\"\n- \"What operators are used by the transform_forecast_attendance DAG?\"\n- \"Has the transform_forecast_attendance DAG ever completed successfully?\"\n\n## Running MCP with LangChain\n\nYou can use `from langchain_mcp_adapters.client import MultiServerMCPClient` in order to add our Airflow MCP as one of your tools in your LangChain app.\n\n### Option 1: Separate Container (SSE Transport)\n\nIf you run our Airflow MCP as a separate container, use SSE transport:\n\n```python\nmcp_host = os.environ.get('mcp_host', 'mcp_sse_server:8000')\nmcps = {\n \"AirflowMCP\": {\n \"url\": f\"http://{mcp_host}/sse\",\n \"transport\": \"sse\",\n \"headers\": {\"Authorization\": f\"\"\"Bearer {\n os.environ.get('MCP_TOKEN')}\"\"\"}\n }\n}\n```\n\n### Option 2: Embedded Server (STDIO Transport)\n\nIf you want to run our MCP server as part of the LangChain code, without any outside code, use STDIO and make sure you install our library first (`airflow-mcp-hipposys = \"0.1.0a11\"`):\n\n```python\nmcps = {\n \"AirflowMCP\":\n {\n 'command': \"python\",\n 'args': [\"-m\", \"airflow_mcp_hipposys.mcp_airflow\"],\n \"transport\": \"stdio\",\n 'env': {k: v for k, v in {\n 'AIRFLOW_ASSISTENT_AI_CONN': os.getenv(\n 'AIRFLOW_ASSISTENT_AI_CONN'),\n 'airflow_api_url': os.getenv('airflow_api_url'),\n 'airflow_username': os.getenv('airflow_username'),\n 'airflow_password': os.getenv('airflow_password'),\n 'AIRFLOW_INSIGHTS_MODE':\n os.getenv('AIRFLOW_INSIGHTS_MODE'),\n 'POST_MODE': os.getenv('POST_MODE'),\n 'TRANSPORT_TYPE': 'stdio',\n '_AIRFLOW_WWW_USER_USERNAME':\n os.getenv('_AIRFLOW_WWW_USER_USERNAME'),\n '_AIRFLOW_WWW_USER_PASSWORD':\n os.getenv('_AIRFLOW_WWW_USER_PASSWORD')\n }.items() if v is not None}\n }\n}\n```\n\n### Using the Tools\n\nThen in both cases, pass it to the tools:\n\n```python\nclient = MultiServerMCPClient(mcps)\ntools = await client.get_tools()\n```\n\n## \ud83e\udd1d Contributing\n\nWe enthusiastically invite the community to contribute to this open-source initiative! Whether you're interested in:\n\n- Adding new features\n- Improving documentation\n- Enhancing compatibility with different LLM providers\n- Reporting bugs\n- Suggesting improvements\n\nPlease feel free to submit pull requests or open issues on our GitHub repository.\n\n## \ud83d\udd17 Links\n\n- [GitHub Repository](https://github.com/hipposys-ltd/airflow-mcp)\n- [Docker Repository](https://hub.docker.com/repository/docker/hipposysai/airflow-mcp/general)\n---\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "An MCP server for Apache Airflow that enables non-technical users to interact with Apache Airflow workflows through natural language.",
"version": "0.1.0a12",
"project_urls": {
"homepage": "https://github.com/hipposys-ltd/airflow-mcp"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e7b9150d9862757428f9ae84a4f533b52e29c8732fc843b5fb62a17e68b609de",
"md5": "503a34673a36a58ffabc6eb64e7239bd",
"sha256": "fd14d359867c1142a093f69c04ebd505192aac1ca107c2e2dca29035abc288ad"
},
"downloads": -1,
"filename": "airflow_mcp_hipposys-0.1.0a12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "503a34673a36a58ffabc6eb64e7239bd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 8471,
"upload_time": "2025-08-03T15:47:06",
"upload_time_iso_8601": "2025-08-03T15:47:06.487956Z",
"url": "https://files.pythonhosted.org/packages/e7/b9/150d9862757428f9ae84a4f533b52e29c8732fc843b5fb62a17e68b609de/airflow_mcp_hipposys-0.1.0a12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "236d06e837330ed7ebdbc291307ea5dbdce0cc2344a19ba4de94cdfb9c8844a9",
"md5": "4e02e50201df500ccfd360a9cb16213f",
"sha256": "4bac7d5970a23f8efcbf73ccea37a22b439b28aa86235983a281f54ac2a6655f"
},
"downloads": -1,
"filename": "airflow_mcp_hipposys-0.1.0a12.tar.gz",
"has_sig": false,
"md5_digest": "4e02e50201df500ccfd360a9cb16213f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 7406,
"upload_time": "2025-08-03T15:47:07",
"upload_time_iso_8601": "2025-08-03T15:47:07.525748Z",
"url": "https://files.pythonhosted.org/packages/23/6d/06e837330ed7ebdbc291307ea5dbdce0cc2344a19ba4de94cdfb9c8844a9/airflow_mcp_hipposys-0.1.0a12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-03 15:47:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hipposys-ltd",
"github_project": "airflow-mcp",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "airflow-mcp-hipposys"
}
