search-fusion-mcp


Namesearch-fusion-mcp JSON
Version 3.0.5 PyPI version JSON
download
home_pagehttps://github.com/sailaoda/search-fusion-mcp
SummaryπŸ” High-Availability Multi-Engine Search Aggregation MCP Server with Enhanced Proxy Auto-Detection
upload_time2025-08-25 19:47:02
maintainerNone
docs_urlNone
authorsailaoda
requires_python>=3.8
licenseMIT License Copyright (c) 2025 wuyesai Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
keywords mcp search aggregation ai llm jina google serper duckduckgo bing baidu exa wikipedia web-scraping high-availability failover multi-engine
VCS
bugtrack_url
requirements mcp fastmcp httpx pydantic loguru duckduckgo-search exa-py wikipedia beautifulsoup4 pytest pytest-asyncio
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            # πŸ” Search Fusion MCP Server

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![FastMCP](https://img.shields.io/badge/MCP-FastMCP-green.svg)](https://github.com/jlowin/fastmcp)
[![Version](https://img.shields.io/badge/version-3.0.0-brightgreen.svg)](https://github.com/sailaoda/search-fusion-mcp/releases)
[![Concurrency](https://img.shields.io/badge/concurrency-supported-blue.svg)](https://github.com/sailaoda/search-fusion-mcp)

**🌏 [δΈ­ζ–‡ζ–‡ζ‘£](README_zh.md)**

A **High-Availability Multi-Engine Search Aggregation MCP Server** providing intelligent failover, unified API, and LLM-optimized content processing. Search Fusion integrates multiple search engines with smart priority-based routing and automatic failover mechanisms.

> **πŸ†• What's New in v3.0.0:** Major concurrency upgrade! Enhanced multi-threading support with thread-safe operations, intelligent connection pooling, and semaphore-based request limiting. Now supports 50+ concurrent searches without race conditions or data corruption!

## ✨ Features

### πŸ”„ Multi-Engine Integration
- **Google Search** - Premium performance with API key
- **Serper Search** - Google search alternative with advanced features
- **Jina AI Search** - AI-powered search with intelligent content processing
- **DuckDuckGo** - Free search, no API key required
- **Exa Search** - AI-powered semantic search
- **Bing Search** - Microsoft search API
- **Baidu Search** - Chinese search engine

### πŸš€ Advanced Features
- **Intelligent Failover** - Automatic engine switching on failures or rate limits
- **Priority-Based Routing** - Smart engine selection based on availability and performance
- **Unified Response Format** - Consistent JSON structure across all engines
- **Rate Limiting Protection** - Built-in cooldown mechanisms
- **πŸ”„ High Concurrency Support** - Thread-safe operations with connection pooling
- **⚑ Performance Optimization** - Async operations with semaphore-based concurrency control
- **LLM-Optimized Content** - Advanced web content fetching with pagination support
- **Wikipedia Integration** - Dedicated Wikipedia search tool
- **Wayback Machine** - Historical webpage archive search
- **Environment Variable Configuration** - Pure MCP configuration without config files
- **🌐 Enhanced Proxy Auto-Detection** - Intelligent proxy detection with zero configuration

### πŸ“Š Monitoring & Analytics
- Real-time engine status monitoring
- Success rate tracking
- Error handling and recovery
- Performance metrics

### ⚑ Concurrency & Performance
- **Thread-Safe Operations** - All engine statistics and state updates are protected by async locks
- **Connection Pooling** - Shared HTTP client with configurable connection limits (max 100 connections)
- **Semaphore Control** - Concurrent request limiting (max 30 simultaneous searches)
- **Timeout Protection** - 60-second search timeout prevents request accumulation
- **Resource Management** - Efficient memory usage with automatic connection cleanup
- **Race Condition Prevention** - Double-checked locking for SearchManager initialization

## πŸ—οΈ Architecture

```
Search Fusion MCP Server
β”œβ”€β”€ πŸ”§ Configuration Manager     # MCP environment variable handling
β”œβ”€β”€ πŸ” Search Manager           # Multi-engine orchestration with concurrency control
β”œβ”€β”€ ⚑ Concurrency Layer        # Thread-safe operations & performance optimization
β”‚   β”œβ”€β”€ AsyncLock Protection    # Thread-safe state updates
β”‚   β”œβ”€β”€ HTTP Connection Pool    # Shared client with connection limits
β”‚   β”œβ”€β”€ Semaphore Control      # Concurrent request limiting (max 30)
β”‚   └── Timeout Management     # 60s timeout protection
β”œβ”€β”€ πŸš€ Engine Implementations   # Individual search engines
β”‚   β”œβ”€β”€ GoogleSearch            # Google Custom Search
β”‚   β”œβ”€β”€ SerperSearch           # Serper API
β”‚   β”œβ”€β”€ JinaSearch             # Jina AI Search
β”‚   β”œβ”€β”€ DuckDuckGoSearch       # DuckDuckGo
β”‚   β”œβ”€β”€ ExaSearch              # Exa AI
β”‚   β”œβ”€β”€ BingSearch             # Bing API
β”‚   └── BaiduSearch            # Baidu API
β”œβ”€β”€ πŸ› οΈ Advanced Fetcher         # Multi-method web scraping
└── πŸ“‘ MCP Server              # FastMCP integration
```

## πŸš€ Quick Start

### Installation

#### Option 1: Install from PyPI (Recommended)
```bash
pip install search-fusion-mcp
```

#### Option 2: Install from Source
```bash
git clone https://github.com/sailaoda/search-fusion-mcp.git
cd search-fusion-mcp
pip install -e .
```

## 🌐 Enhanced Proxy Auto-Detection (New in v2.0!)

Search Fusion now features **intelligent proxy auto-detection** inspired by [concurrent-browser-mcp](https://github.com/sailaoda/concurrent-browser-mcp), providing seamless proxy support with **zero configuration**!

### ✨ Three-Layer Detection Strategy

1. **Environment Variables** - Highest priority, checks `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`
2. **Port Scanning** - Scans common proxy ports using socket connection testing
3. **System Proxy** - Detects OS-level proxy settings (macOS supported)

### πŸ” Supported Proxy Ports (Priority Order)
- **7890** - Clash default port
- **1087** - V2Ray common port  
- **8080** - Generic HTTP proxy port
- **3128** - Squid proxy default port
- **8888** - Other proxy software port
- **10809** - V2Ray SOCKS port
- **20171** - Additional proxy port

### πŸš€ Zero Configuration Usage

**Just run directly** - proxy will be auto-detected:
```bash
search-fusion-mcp
```

**Manual override** (if needed):
```bash
env HTTP_PROXY="http://your-proxy:port" search-fusion-mcp
```

### πŸ“Š Detection Process
```
πŸ” Checking environment variables...
πŸ” Scanning proxy ports: [7890, 1087, 8080, ...]
βœ… Local proxy port detected: 7890
🌐 Auto-detected proxy: http://127.0.0.1:7890
```

### πŸ†š Comparison with concurrent-browser-mcp

| Feature | Search-Fusion | concurrent-browser-mcp |
|---------|---------------|------------------------|
| **Detection Method** | βœ… Env vars β†’ Port scan β†’ System proxy | βœ… Same strategy |
| **Port List** | βœ… 7 common ports | βœ… 7 common ports |
| **Connection Test** | βœ… Socket testing | βœ… Socket testing |
| **Timeout** | βœ… 3 seconds | βœ… 3 seconds |
| **macOS Support** | βœ… networksetup | βœ… networksetup |
| **Language** | Python | TypeScript |

### MCP Integration

#### Environment Variable Configuration

Search Fusion uses **pure MCP environment variable configuration** without requiring config files.

**MCP Client Configuration (PyPI Installation):**
```json
{
  "mcp": {
    "mcpServers": {
      "search-fusion": {
        "command": "search-fusion-mcp",
        "env": {
          "GOOGLE_API_KEY": "your_google_api_key",
          "GOOGLE_CSE_ID": "your_google_cse_id",
          "SERPER_API_KEY": "your_serper_api_key",
          "JINA_API_KEY": "your_jina_api_key",
          "EXA_API_KEY": "your_exa_api_key",
          "BING_API_KEY": "your_bing_api_key",
          "BAIDU_API_KEY": "your_baidu_api_key",
          "BAIDU_SECRET_KEY": "your_baidu_secret_key"
        }
      }
    }
  }
}
```

**MCP Client Configuration (Source Installation):**
```json
{
  "mcp": {
    "mcpServers": {
      "search-fusion": {
        "command": "python",
        "args": ["-m", "src.main"],
        "cwd": "/path/to/your/search-fusion-mcp",
        "env": {
          "GOOGLE_API_KEY": "your_google_api_key",
          "GOOGLE_CSE_ID": "your_google_cse_id",
          "SERPER_API_KEY": "your_serper_api_key",
          "JINA_API_KEY": "your_jina_api_key",
          "EXA_API_KEY": "your_exa_api_key",
          "BING_API_KEY": "your_bing_api_key",
          "BAIDU_API_KEY": "your_baidu_api_key",
          "BAIDU_SECRET_KEY": "your_baidu_secret_key"
        }
      }
    }
  }
}
```

#### Supported Environment Variables

| Search Engine | Environment Variable | Required | Description | Get API Key |
|--------------|---------------------|----------|-------------|-------------|
| Google | `GOOGLE_API_KEY`<br>`GOOGLE_CSE_ID` | Both needed | Google Custom Search API | [Get API Key](https://developers.google.com/custom-search/v1/introduction) |
| Serper | `SERPER_API_KEY` | API key | Serper Google Search API | [Get API Key](https://serper.dev/) |
| Jina AI | `JINA_API_KEY` | API key | Jina AI Search API | [Get API Key](https://jina.ai/) |
| Bing | `BING_API_KEY` | API key | Microsoft Bing Search API | [Get API Key](https://www.microsoft.com/en-us/bing/apis/bing-web-search-api) |
| Baidu | `BAIDU_API_KEY`<br>`BAIDU_SECRET_KEY` | Both needed | Baidu Search API | [Get API Key](https://ai.baidu.com/) |
| Exa | `EXA_API_KEY` | API key | Exa AI Search API | [Get API Key](https://exa.ai/) |
| DuckDuckGo | None required | - | Free search, no API key needed | - |

**Alternative Variable Names:**
```bash
# Google
GOOGLE_SEARCH_API_KEY    # Alternative to GOOGLE_API_KEY
GOOGLE_SEARCH_CSE_ID     # Alternative to GOOGLE_CSE_ID

# Serper
SERPER_SEARCH_API_KEY    # Alternative to SERPER_API_KEY

# Others follow similar pattern...
```

### Engine Priority

Search engines are prioritized automatically:
1. **Google Search** (Priority 1) - Premium performance with API key
2. **Serper Search** (Priority 1) - Google alternative with advanced features
3. **Jina AI Search** (Priority 1.5) - AI-powered search with optional API key for advanced features
4. **DuckDuckGo** (Priority 2) - Free, no API key required
5. **Exa Search** (Priority 2) - AI-powered search with API key
6. **Bing Search** (Priority 3) - Microsoft search API
7. **Baidu Search** (Priority 3) - Chinese search engine

## πŸ› οΈ MCP Tools

![Tools Overview](assets/tools.png)

### 1. `search`
Perform web searches with intelligent engine selection and failover.

**Parameters:**
- `query` (required): Search query terms
- `num_results` (default: 10): Number of results to return
- `engine` (default: "auto"): Engine preference
  - `"auto"`: Automatic engine selection (recommended)
  - `"google"`: Prefer Google Search
  - `"serper"`: Prefer Serper Search
  - `"jina"`: Prefer Jina AI Search
  - `"duckduckgo"`: Prefer DuckDuckGo
  - `"exa"`: Prefer Exa Search
  - `"bing"`: Prefer Bing Search
  - `"baidu"`: Prefer Baidu Search

### 2. `fetch_url`
Fetch and process web content with intelligent pagination and multi-method fallback.

**Parameters:**
- `url` (required): Web URL to fetch
- `use_jina` (default: true): Whether to prioritize Jina Reader for LLM-optimized content
- `with_image_alt` (default: false): Whether to generate alt text for images
- `max_length` (default: 50000): Maximum content length per page (auto-paginate if exceeded)
- `page_number` (default: 1): Retrieve specific page from previously fetched content

**Features:**
- **Intelligent Multi-Method Fallback**: Tries Jina Reader β†’ Serper Scrape β†’ Direct HTTP
- **Automatic Pagination**: Splits large content into manageable pages
- **Concurrent-Safe Caching**: Unique page IDs prevent conflicts in high-concurrency scenarios
- **LLM-Optimized Content**: Clean markdown format optimized for AI processing

### 3. `get_available_engines`
Get current status and availability of all search engines.

### 4. `search_wikipedia`
Search Wikipedia articles for entities, people, places, concepts, etc.

**Parameters:**
- `entity` (required): Entity to search for
- `first_sentences` (default: 10): Number of sentences to return (0 for full content)

### 5. `search_archived_webpage`
Search archived versions of websites using Wayback Machine.

**Parameters:**
- `url` (required): Website URL to search
- `year` (optional): Target year
- `month` (optional): Target month
- `day` (optional): Target day



## πŸ“– API Examples

### Basic Search
```python
# Automatic engine selection
result = await search("artificial intelligence trends 2024")

# Prefer specific engine
result = await search("machine learning", engine="google")
```

### Advanced Web Fetching
```python
# Fetch with intelligent pagination
result = await fetch_url("https://example.com/long-article")

# If content is paginated, get additional pages
if result.get("is_paginated"):
    page_2 = await get_page(result["page_id"], 2)
```

### Wikipedia Search
```python
# Get Wikipedia summary
result = await search_wikipedia("Python programming language")

# Get full article
result = await search_wikipedia("Quantum computing", first_sentences=0)
```

## πŸ§ͺ Development

### Development Setup
```bash
git clone https://github.com/sailaoda/search-fusion-mcp.git
cd search-fusion-mcp
pip install -r requirements.txt
pip install -e .
```

## πŸ”§ Configuration Guide

For detailed configuration instructions, see [MCP_CONFIG_GUIDE.md](MCP_CONFIG_GUIDE.md).

## πŸ“Š Performance

- **Latency**: Sub-second response times with caching
- **Availability**: 99.9% uptime with intelligent failover
- **Throughput**: Handles concurrent requests efficiently
- **Scalability**: Efficient resource utilization and concurrent processing

### πŸ“ˆ Concurrency Benchmarks

**Tested Performance (v3.0.0+):**
- βœ… **50+ concurrent searches** - No race conditions or data corruption
- βœ… **Thread-safe statistics** - Accurate request counting and error tracking
- ⚑ **Connection pooling** - Efficient HTTP resource management
- πŸ›‘οΈ **Timeout protection** - 60s per request prevents system overload
- πŸ“Š **Real-time monitoring** - Live engine status during high load

**Recommended Limits:**
- **Concurrent searches**: 10 (configurable via semaphore)
- **Connection pool**: 100 max connections, 20 keep-alive
- **Request timeout**: 60 seconds
- **Memory usage**: ~50MB baseline + ~2MB per concurrent request

## 🀝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Submit a pull request

## πŸ“„ License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🚨 Rate Limiting & Best Practices

- **Google Search**: 100 queries/day (free tier)
- **Serper API**: Varies by plan
- **Jina AI**: Rate limits apply based on subscription
- **DuckDuckGo**: No official limits, but use responsibly
- **Other engines**: Check respective API documentation

Always implement appropriate delays and respect rate limits to ensure sustainable usage.

## πŸ“ž Support

- πŸ“– [Documentation](https://github.com/sailaoda/search-fusion-mcp)
- πŸ› [Issue Tracker](https://github.com/sailaoda/search-fusion-mcp/issues)
- πŸ’¬ [Discussions](https://github.com/sailaoda/search-fusion-mcp/discussions)

---

**Made with ❀️ for the MCP community**

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/sailaoda/search-fusion-mcp",
    "name": "search-fusion-mcp",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": "Search Fusion Team <contact@searchfusion.dev>",
    "keywords": "mcp, search, aggregation, ai, llm, jina, google, serper, duckduckgo, bing, baidu, exa, wikipedia, web-scraping, high-availability, failover, multi-engine",
    "author": "sailaoda",
    "author_email": "Search Fusion Team <contact@searchfusion.dev>",
    "download_url": "https://files.pythonhosted.org/packages/2e/e4/a54d121f218d3c3bfde6e1b0a3779d439c624bd403c873763fc8eebde65e/search-fusion-mcp-3.0.5.tar.gz",
    "platform": null,
    "description": "# \ud83d\udd0d Search Fusion MCP Server\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![FastMCP](https://img.shields.io/badge/MCP-FastMCP-green.svg)](https://github.com/jlowin/fastmcp)\n[![Version](https://img.shields.io/badge/version-3.0.0-brightgreen.svg)](https://github.com/sailaoda/search-fusion-mcp/releases)\n[![Concurrency](https://img.shields.io/badge/concurrency-supported-blue.svg)](https://github.com/sailaoda/search-fusion-mcp)\n\n**\ud83c\udf0f [\u4e2d\u6587\u6587\u6863](README_zh.md)**\n\nA **High-Availability Multi-Engine Search Aggregation MCP Server** providing intelligent failover, unified API, and LLM-optimized content processing. Search Fusion integrates multiple search engines with smart priority-based routing and automatic failover mechanisms.\n\n> **\ud83c\udd95 What's New in v3.0.0:** Major concurrency upgrade! Enhanced multi-threading support with thread-safe operations, intelligent connection pooling, and semaphore-based request limiting. Now supports 50+ concurrent searches without race conditions or data corruption!\n\n## \u2728 Features\n\n### \ud83d\udd04 Multi-Engine Integration\n- **Google Search** - Premium performance with API key\n- **Serper Search** - Google search alternative with advanced features\n- **Jina AI Search** - AI-powered search with intelligent content processing\n- **DuckDuckGo** - Free search, no API key required\n- **Exa Search** - AI-powered semantic search\n- **Bing Search** - Microsoft search API\n- **Baidu Search** - Chinese search engine\n\n### \ud83d\ude80 Advanced Features\n- **Intelligent Failover** - Automatic engine switching on failures or rate limits\n- **Priority-Based Routing** - Smart engine selection based on availability and performance\n- **Unified Response Format** - Consistent JSON structure across all engines\n- **Rate Limiting Protection** - Built-in cooldown mechanisms\n- **\ud83d\udd04 High Concurrency Support** - Thread-safe operations with connection pooling\n- **\u26a1 Performance Optimization** - Async operations with semaphore-based concurrency control\n- **LLM-Optimized Content** - Advanced web content fetching with pagination support\n- **Wikipedia Integration** - Dedicated Wikipedia search tool\n- **Wayback Machine** - Historical webpage archive search\n- **Environment Variable Configuration** - Pure MCP configuration without config files\n- **\ud83c\udf10 Enhanced Proxy Auto-Detection** - Intelligent proxy detection with zero configuration\n\n### \ud83d\udcca Monitoring & Analytics\n- Real-time engine status monitoring\n- Success rate tracking\n- Error handling and recovery\n- Performance metrics\n\n### \u26a1 Concurrency & Performance\n- **Thread-Safe Operations** - All engine statistics and state updates are protected by async locks\n- **Connection Pooling** - Shared HTTP client with configurable connection limits (max 100 connections)\n- **Semaphore Control** - Concurrent request limiting (max 30 simultaneous searches)\n- **Timeout Protection** - 60-second search timeout prevents request accumulation\n- **Resource Management** - Efficient memory usage with automatic connection cleanup\n- **Race Condition Prevention** - Double-checked locking for SearchManager initialization\n\n## \ud83c\udfd7\ufe0f Architecture\n\n```\nSearch Fusion MCP Server\n\u251c\u2500\u2500 \ud83d\udd27 Configuration Manager     # MCP environment variable handling\n\u251c\u2500\u2500 \ud83d\udd0d Search Manager           # Multi-engine orchestration with concurrency control\n\u251c\u2500\u2500 \u26a1 Concurrency Layer        # Thread-safe operations & performance optimization\n\u2502   \u251c\u2500\u2500 AsyncLock Protection    # Thread-safe state updates\n\u2502   \u251c\u2500\u2500 HTTP Connection Pool    # Shared client with connection limits\n\u2502   \u251c\u2500\u2500 Semaphore Control      # Concurrent request limiting (max 30)\n\u2502   \u2514\u2500\u2500 Timeout Management     # 60s timeout protection\n\u251c\u2500\u2500 \ud83d\ude80 Engine Implementations   # Individual search engines\n\u2502   \u251c\u2500\u2500 GoogleSearch            # Google Custom Search\n\u2502   \u251c\u2500\u2500 SerperSearch           # Serper API\n\u2502   \u251c\u2500\u2500 JinaSearch             # Jina AI Search\n\u2502   \u251c\u2500\u2500 DuckDuckGoSearch       # DuckDuckGo\n\u2502   \u251c\u2500\u2500 ExaSearch              # Exa AI\n\u2502   \u251c\u2500\u2500 BingSearch             # Bing API\n\u2502   \u2514\u2500\u2500 BaiduSearch            # Baidu API\n\u251c\u2500\u2500 \ud83d\udee0\ufe0f Advanced Fetcher         # Multi-method web scraping\n\u2514\u2500\u2500 \ud83d\udce1 MCP Server              # FastMCP integration\n```\n\n## \ud83d\ude80 Quick Start\n\n### Installation\n\n#### Option 1: Install from PyPI (Recommended)\n```bash\npip install search-fusion-mcp\n```\n\n#### Option 2: Install from Source\n```bash\ngit clone https://github.com/sailaoda/search-fusion-mcp.git\ncd search-fusion-mcp\npip install -e .\n```\n\n## \ud83c\udf10 Enhanced Proxy Auto-Detection (New in v2.0!)\n\nSearch Fusion now features **intelligent proxy auto-detection** inspired by [concurrent-browser-mcp](https://github.com/sailaoda/concurrent-browser-mcp), providing seamless proxy support with **zero configuration**!\n\n### \u2728 Three-Layer Detection Strategy\n\n1. **Environment Variables** - Highest priority, checks `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`\n2. **Port Scanning** - Scans common proxy ports using socket connection testing\n3. **System Proxy** - Detects OS-level proxy settings (macOS supported)\n\n### \ud83d\udd0d Supported Proxy Ports (Priority Order)\n- **7890** - Clash default port\n- **1087** - V2Ray common port  \n- **8080** - Generic HTTP proxy port\n- **3128** - Squid proxy default port\n- **8888** - Other proxy software port\n- **10809** - V2Ray SOCKS port\n- **20171** - Additional proxy port\n\n### \ud83d\ude80 Zero Configuration Usage\n\n**Just run directly** - proxy will be auto-detected:\n```bash\nsearch-fusion-mcp\n```\n\n**Manual override** (if needed):\n```bash\nenv HTTP_PROXY=\"http://your-proxy:port\" search-fusion-mcp\n```\n\n### \ud83d\udcca Detection Process\n```\n\ud83d\udd0d Checking environment variables...\n\ud83d\udd0d Scanning proxy ports: [7890, 1087, 8080, ...]\n\u2705 Local proxy port detected: 7890\n\ud83c\udf10 Auto-detected proxy: http://127.0.0.1:7890\n```\n\n### \ud83c\udd9a Comparison with concurrent-browser-mcp\n\n| Feature | Search-Fusion | concurrent-browser-mcp |\n|---------|---------------|------------------------|\n| **Detection Method** | \u2705 Env vars \u2192 Port scan \u2192 System proxy | \u2705 Same strategy |\n| **Port List** | \u2705 7 common ports | \u2705 7 common ports |\n| **Connection Test** | \u2705 Socket testing | \u2705 Socket testing |\n| **Timeout** | \u2705 3 seconds | \u2705 3 seconds |\n| **macOS Support** | \u2705 networksetup | \u2705 networksetup |\n| **Language** | Python | TypeScript |\n\n### MCP Integration\n\n#### Environment Variable Configuration\n\nSearch Fusion uses **pure MCP environment variable configuration** without requiring config files.\n\n**MCP Client Configuration (PyPI Installation):**\n```json\n{\n  \"mcp\": {\n    \"mcpServers\": {\n      \"search-fusion\": {\n        \"command\": \"search-fusion-mcp\",\n        \"env\": {\n          \"GOOGLE_API_KEY\": \"your_google_api_key\",\n          \"GOOGLE_CSE_ID\": \"your_google_cse_id\",\n          \"SERPER_API_KEY\": \"your_serper_api_key\",\n          \"JINA_API_KEY\": \"your_jina_api_key\",\n          \"EXA_API_KEY\": \"your_exa_api_key\",\n          \"BING_API_KEY\": \"your_bing_api_key\",\n          \"BAIDU_API_KEY\": \"your_baidu_api_key\",\n          \"BAIDU_SECRET_KEY\": \"your_baidu_secret_key\"\n        }\n      }\n    }\n  }\n}\n```\n\n**MCP Client Configuration (Source Installation):**\n```json\n{\n  \"mcp\": {\n    \"mcpServers\": {\n      \"search-fusion\": {\n        \"command\": \"python\",\n        \"args\": [\"-m\", \"src.main\"],\n        \"cwd\": \"/path/to/your/search-fusion-mcp\",\n        \"env\": {\n          \"GOOGLE_API_KEY\": \"your_google_api_key\",\n          \"GOOGLE_CSE_ID\": \"your_google_cse_id\",\n          \"SERPER_API_KEY\": \"your_serper_api_key\",\n          \"JINA_API_KEY\": \"your_jina_api_key\",\n          \"EXA_API_KEY\": \"your_exa_api_key\",\n          \"BING_API_KEY\": \"your_bing_api_key\",\n          \"BAIDU_API_KEY\": \"your_baidu_api_key\",\n          \"BAIDU_SECRET_KEY\": \"your_baidu_secret_key\"\n        }\n      }\n    }\n  }\n}\n```\n\n#### Supported Environment Variables\n\n| Search Engine | Environment Variable | Required | Description | Get API Key |\n|--------------|---------------------|----------|-------------|-------------|\n| Google | `GOOGLE_API_KEY`<br>`GOOGLE_CSE_ID` | Both needed | Google Custom Search API | [Get API Key](https://developers.google.com/custom-search/v1/introduction) |\n| Serper | `SERPER_API_KEY` | API key | Serper Google Search API | [Get API Key](https://serper.dev/) |\n| Jina AI | `JINA_API_KEY` | API key | Jina AI Search API | [Get API Key](https://jina.ai/) |\n| Bing | `BING_API_KEY` | API key | Microsoft Bing Search API | [Get API Key](https://www.microsoft.com/en-us/bing/apis/bing-web-search-api) |\n| Baidu | `BAIDU_API_KEY`<br>`BAIDU_SECRET_KEY` | Both needed | Baidu Search API | [Get API Key](https://ai.baidu.com/) |\n| Exa | `EXA_API_KEY` | API key | Exa AI Search API | [Get API Key](https://exa.ai/) |\n| DuckDuckGo | None required | - | Free search, no API key needed | - |\n\n**Alternative Variable Names:**\n```bash\n# Google\nGOOGLE_SEARCH_API_KEY    # Alternative to GOOGLE_API_KEY\nGOOGLE_SEARCH_CSE_ID     # Alternative to GOOGLE_CSE_ID\n\n# Serper\nSERPER_SEARCH_API_KEY    # Alternative to SERPER_API_KEY\n\n# Others follow similar pattern...\n```\n\n### Engine Priority\n\nSearch engines are prioritized automatically:\n1. **Google Search** (Priority 1) - Premium performance with API key\n2. **Serper Search** (Priority 1) - Google alternative with advanced features\n3. **Jina AI Search** (Priority 1.5) - AI-powered search with optional API key for advanced features\n4. **DuckDuckGo** (Priority 2) - Free, no API key required\n5. **Exa Search** (Priority 2) - AI-powered search with API key\n6. **Bing Search** (Priority 3) - Microsoft search API\n7. **Baidu Search** (Priority 3) - Chinese search engine\n\n## \ud83d\udee0\ufe0f MCP Tools\n\n![Tools Overview](assets/tools.png)\n\n### 1. `search`\nPerform web searches with intelligent engine selection and failover.\n\n**Parameters:**\n- `query` (required): Search query terms\n- `num_results` (default: 10): Number of results to return\n- `engine` (default: \"auto\"): Engine preference\n  - `\"auto\"`: Automatic engine selection (recommended)\n  - `\"google\"`: Prefer Google Search\n  - `\"serper\"`: Prefer Serper Search\n  - `\"jina\"`: Prefer Jina AI Search\n  - `\"duckduckgo\"`: Prefer DuckDuckGo\n  - `\"exa\"`: Prefer Exa Search\n  - `\"bing\"`: Prefer Bing Search\n  - `\"baidu\"`: Prefer Baidu Search\n\n### 2. `fetch_url`\nFetch and process web content with intelligent pagination and multi-method fallback.\n\n**Parameters:**\n- `url` (required): Web URL to fetch\n- `use_jina` (default: true): Whether to prioritize Jina Reader for LLM-optimized content\n- `with_image_alt` (default: false): Whether to generate alt text for images\n- `max_length` (default: 50000): Maximum content length per page (auto-paginate if exceeded)\n- `page_number` (default: 1): Retrieve specific page from previously fetched content\n\n**Features:**\n- **Intelligent Multi-Method Fallback**: Tries Jina Reader \u2192 Serper Scrape \u2192 Direct HTTP\n- **Automatic Pagination**: Splits large content into manageable pages\n- **Concurrent-Safe Caching**: Unique page IDs prevent conflicts in high-concurrency scenarios\n- **LLM-Optimized Content**: Clean markdown format optimized for AI processing\n\n### 3. `get_available_engines`\nGet current status and availability of all search engines.\n\n### 4. `search_wikipedia`\nSearch Wikipedia articles for entities, people, places, concepts, etc.\n\n**Parameters:**\n- `entity` (required): Entity to search for\n- `first_sentences` (default: 10): Number of sentences to return (0 for full content)\n\n### 5. `search_archived_webpage`\nSearch archived versions of websites using Wayback Machine.\n\n**Parameters:**\n- `url` (required): Website URL to search\n- `year` (optional): Target year\n- `month` (optional): Target month\n- `day` (optional): Target day\n\n\n\n## \ud83d\udcd6 API Examples\n\n### Basic Search\n```python\n# Automatic engine selection\nresult = await search(\"artificial intelligence trends 2024\")\n\n# Prefer specific engine\nresult = await search(\"machine learning\", engine=\"google\")\n```\n\n### Advanced Web Fetching\n```python\n# Fetch with intelligent pagination\nresult = await fetch_url(\"https://example.com/long-article\")\n\n# If content is paginated, get additional pages\nif result.get(\"is_paginated\"):\n    page_2 = await get_page(result[\"page_id\"], 2)\n```\n\n### Wikipedia Search\n```python\n# Get Wikipedia summary\nresult = await search_wikipedia(\"Python programming language\")\n\n# Get full article\nresult = await search_wikipedia(\"Quantum computing\", first_sentences=0)\n```\n\n## \ud83e\uddea Development\n\n### Development Setup\n```bash\ngit clone https://github.com/sailaoda/search-fusion-mcp.git\ncd search-fusion-mcp\npip install -r requirements.txt\npip install -e .\n```\n\n## \ud83d\udd27 Configuration Guide\n\nFor detailed configuration instructions, see [MCP_CONFIG_GUIDE.md](MCP_CONFIG_GUIDE.md).\n\n## \ud83d\udcca Performance\n\n- **Latency**: Sub-second response times with caching\n- **Availability**: 99.9% uptime with intelligent failover\n- **Throughput**: Handles concurrent requests efficiently\n- **Scalability**: Efficient resource utilization and concurrent processing\n\n### \ud83d\udcc8 Concurrency Benchmarks\n\n**Tested Performance (v3.0.0+):**\n- \u2705 **50+ concurrent searches** - No race conditions or data corruption\n- \u2705 **Thread-safe statistics** - Accurate request counting and error tracking\n- \u26a1 **Connection pooling** - Efficient HTTP resource management\n- \ud83d\udee1\ufe0f **Timeout protection** - 60s per request prevents system overload\n- \ud83d\udcca **Real-time monitoring** - Live engine status during high load\n\n**Recommended Limits:**\n- **Concurrent searches**: 10 (configurable via semaphore)\n- **Connection pool**: 100 max connections, 20 keep-alive\n- **Request timeout**: 60 seconds\n- **Memory usage**: ~50MB baseline + ~2MB per concurrent request\n\n## \ud83e\udd1d Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests for new functionality\n5. Submit a pull request\n\n## \ud83d\udcc4 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## \ud83d\udea8 Rate Limiting & Best Practices\n\n- **Google Search**: 100 queries/day (free tier)\n- **Serper API**: Varies by plan\n- **Jina AI**: Rate limits apply based on subscription\n- **DuckDuckGo**: No official limits, but use responsibly\n- **Other engines**: Check respective API documentation\n\nAlways implement appropriate delays and respect rate limits to ensure sustainable usage.\n\n## \ud83d\udcde Support\n\n- \ud83d\udcd6 [Documentation](https://github.com/sailaoda/search-fusion-mcp)\n- \ud83d\udc1b [Issue Tracker](https://github.com/sailaoda/search-fusion-mcp/issues)\n- \ud83d\udcac [Discussions](https://github.com/sailaoda/search-fusion-mcp/discussions)\n\n---\n\n**Made with \u2764\ufe0f for the MCP community**\n",
    "bugtrack_url": null,
    "license": "MIT License\n        \n        Copyright (c) 2025 wuyesai\n        \n        Permission is hereby granted, free of charge, to any person obtaining a copy\n        of this software and associated documentation files (the \"Software\"), to deal\n        in the Software without restriction, including without limitation the rights\n        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n        copies of the Software, and to permit persons to whom the Software is\n        furnished to do so, subject to the following conditions:\n        \n        The above copyright notice and this permission notice shall be included in all\n        copies or substantial portions of the Software.\n        \n        THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n        SOFTWARE.\n        ",
    "summary": "\ud83d\udd0d High-Availability Multi-Engine Search Aggregation MCP Server with Enhanced Proxy Auto-Detection",
    "version": "3.0.5",
    "project_urls": {
        "Bug Tracker": "https://github.com/sailaoda/search-fusion-mcp/issues",
        "Changelog": "https://github.com/sailaoda/search-fusion-mcp/blob/main/CHANGELOG.md",
        "Documentation": "https://github.com/sailaoda/search-fusion-mcp/blob/main/README.md",
        "Homepage": "https://github.com/sailaoda/search-fusion-mcp",
        "Repository": "https://github.com/sailaoda/search-fusion-mcp"
    },
    "split_keywords": [
        "mcp",
        " search",
        " aggregation",
        " ai",
        " llm",
        " jina",
        " google",
        " serper",
        " duckduckgo",
        " bing",
        " baidu",
        " exa",
        " wikipedia",
        " web-scraping",
        " high-availability",
        " failover",
        " multi-engine"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "238fa6938a13635d9679d48f6e250a968c6c6dadf816f3023bce3c088934dca3",
                "md5": "616ad4501098d766ec2dc29b71b1f730",
                "sha256": "984d1dded90d979c36fcf03469b96c1c82642ee4a596dba463175284a2cff242"
            },
            "downloads": -1,
            "filename": "search_fusion_mcp-3.0.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "616ad4501098d766ec2dc29b71b1f730",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 36954,
            "upload_time": "2025-08-25T19:47:00",
            "upload_time_iso_8601": "2025-08-25T19:47:00.956870Z",
            "url": "https://files.pythonhosted.org/packages/23/8f/a6938a13635d9679d48f6e250a968c6c6dadf816f3023bce3c088934dca3/search_fusion_mcp-3.0.5-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "2ee4a54d121f218d3c3bfde6e1b0a3779d439c624bd403c873763fc8eebde65e",
                "md5": "3b7be6d8418daeb8cddd2b0cf75f0e1f",
                "sha256": "95dc002a59ab41917746dd9523dd5f0e0a1bb20d70faddadbf0a9af799491dca"
            },
            "downloads": -1,
            "filename": "search-fusion-mcp-3.0.5.tar.gz",
            "has_sig": false,
            "md5_digest": "3b7be6d8418daeb8cddd2b0cf75f0e1f",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 34318,
            "upload_time": "2025-08-25T19:47:02",
            "upload_time_iso_8601": "2025-08-25T19:47:02.133881Z",
            "url": "https://files.pythonhosted.org/packages/2e/e4/a54d121f218d3c3bfde6e1b0a3779d439c624bd403c873763fc8eebde65e/search-fusion-mcp-3.0.5.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-08-25 19:47:02",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "sailaoda",
    "github_project": "search-fusion-mcp",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [
        {
            "name": "mcp",
            "specs": [
                [
                    ">=",
                    "1.0.0"
                ]
            ]
        },
        {
            "name": "fastmcp",
            "specs": [
                [
                    ">=",
                    "0.1.0"
                ]
            ]
        },
        {
            "name": "httpx",
            "specs": [
                [
                    ">=",
                    "0.24.1"
                ]
            ]
        },
        {
            "name": "pydantic",
            "specs": [
                [
                    ">=",
                    "2.0.0"
                ]
            ]
        },
        {
            "name": "loguru",
            "specs": [
                [
                    ">=",
                    "0.7.0"
                ]
            ]
        },
        {
            "name": "duckduckgo-search",
            "specs": [
                [
                    ">=",
                    "6.1.7"
                ]
            ]
        },
        {
            "name": "exa-py",
            "specs": [
                [
                    ">=",
                    "1.0.0"
                ]
            ]
        },
        {
            "name": "wikipedia",
            "specs": [
                [
                    ">=",
                    "1.4.0"
                ]
            ]
        },
        {
            "name": "beautifulsoup4",
            "specs": [
                [
                    ">=",
                    "4.12.2"
                ]
            ]
        },
        {
            "name": "pytest",
            "specs": [
                [
                    ">=",
                    "7.4.3"
                ]
            ]
        },
        {
            "name": "pytest-asyncio",
            "specs": [
                [
                    ">=",
                    "0.21.1"
                ]
            ]
        }
    ],
    "lcname": "search-fusion-mcp"
}
sailaoda
Elapsed time: 0.93659s