# Mantis Web Crawler
🕷️ **Mantis** is a powerful web accessibility and UI bug detection tool that crawls websites to discover issues before they reach production.
*Originally created for Hack The North 2025*
## Features
- 🔍 **Comprehensive Web Crawling**: Deep crawl of websites with configurable depth and page limits
- ♿ **Accessibility Testing**: Automated detection of accessibility violations
- 🎨 **Visual Layout Analysis**: Detection of UI layout bugs and visual inconsistencies
- 📊 **Real-time Dashboard**: Live monitoring of crawl progress with WebSocket updates
- 📋 **Detailed Reporting**: JSON reports with bug details, evidence, and recommendations
- 🚀 **Easy CLI Interface**: Simple command-line tool for quick scans
## Installation
### From PyPI (Recommended)
```bash
pip install mantis-web-crawler
playwright install
```
### From Source (Development)
```bash
# Clone the repository
git clone https://github.com/yourusername/mantis.git
cd mantis
# Install the package
pip install -e .
# Install Playwright browsers (required)
playwright install
```
## Quick Start
After installation, you can run Mantis directly from anywhere:
```bash
# Basic crawl
mantis run https://example.com
# Advanced options
mantis run https://example.com --max-depth 2 --max-pages 25
mantis run https://example.com --output my_report.json --verbose
mantis run https://example.com --dashboard # Launch with live dashboard
```
## Usage Examples
### Basic Website Scan
```bash
mantis run https://example.com
```
### Deep Crawl with Custom Limits
```bash
mantis run https://example.com --max-depth 3 --max-pages 50
```
### Generate Custom Report
```bash
mantis run https://example.com --output accessibility_report.json --verbose
```
### Launch with Real-time Dashboard
```bash
mantis run https://example.com --dashboard
```
## Command Line Options
- `url`: Starting URL to crawl (required)
- `--max-depth`: Maximum crawl depth (default: 3)
- `--max-pages`: Maximum number of pages to crawl (default: 10)
- `--output`: Output file for JSON report (optional)
- `--verbose`: Enable verbose logging
- `--dashboard`: Launch real-time monitoring dashboard
- `--scan-type`: One of 'all', 'ui', 'accessibility' or 'performance'. Defaults to 'all'.
## Requirements
- Python 3.8+
- Playwright browsers (automatically installed with `playwright install`)
## Development
```bash
# Clone the repo
git clone https://github.com/yourusername/mantis.git
cd mantis
# Install in development mode
pip install -e .[dev]
# Install pre-commit hooks
pre-commit install
# Run tests
pytest
```
## Contributing
We welcome contributions! Please see our contributing guidelines for more details.
## License
MIT License - see LICENSE file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/yourusername/mantis",
"name": "mantis-web-crawler",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "web-crawler, accessibility, ui-testing, bug-detection, playwright",
"author": "Mantis Team",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/ad/27/c2d5ec0627c7d1170810c9bf526e703899361de9f342b1db46a037eb7599/mantis_web_crawler-1.2.0.tar.gz",
"platform": null,
"description": "# Mantis Web Crawler\n\n\ud83d\udd77\ufe0f **Mantis** is a powerful web accessibility and UI bug detection tool that crawls websites to discover issues before they reach production.\n\n*Originally created for Hack The North 2025*\n\n## Features\n\n- \ud83d\udd0d **Comprehensive Web Crawling**: Deep crawl of websites with configurable depth and page limits\n- \u267f **Accessibility Testing**: Automated detection of accessibility violations\n- \ud83c\udfa8 **Visual Layout Analysis**: Detection of UI layout bugs and visual inconsistencies \n- \ud83d\udcca **Real-time Dashboard**: Live monitoring of crawl progress with WebSocket updates\n- \ud83d\udccb **Detailed Reporting**: JSON reports with bug details, evidence, and recommendations\n- \ud83d\ude80 **Easy CLI Interface**: Simple command-line tool for quick scans\n\n## Installation\n\n### From PyPI (Recommended)\n\n```bash\npip install mantis-web-crawler\nplaywright install\n```\n\n### From Source (Development)\n\n```bash\n# Clone the repository\ngit clone https://github.com/yourusername/mantis.git\ncd mantis\n\n# Install the package\npip install -e .\n\n# Install Playwright browsers (required)\nplaywright install\n```\n\n## Quick Start\n\nAfter installation, you can run Mantis directly from anywhere:\n\n```bash\n# Basic crawl\nmantis run https://example.com\n\n# Advanced options\nmantis run https://example.com --max-depth 2 --max-pages 25\nmantis run https://example.com --output my_report.json --verbose\nmantis run https://example.com --dashboard # Launch with live dashboard\n```\n\n## Usage Examples\n\n### Basic Website Scan\n```bash\nmantis run https://example.com\n```\n\n### Deep Crawl with Custom Limits\n```bash\nmantis run https://example.com --max-depth 3 --max-pages 50\n```\n\n### Generate Custom Report\n```bash\nmantis run https://example.com --output accessibility_report.json --verbose\n```\n\n### Launch with Real-time Dashboard\n```bash\nmantis run https://example.com --dashboard\n```\n\n## Command Line Options\n\n- `url`: Starting URL to crawl (required)\n- `--max-depth`: Maximum crawl depth (default: 3)\n- `--max-pages`: Maximum number of pages to crawl (default: 10)\n- `--output`: Output file for JSON report (optional)\n- `--verbose`: Enable verbose logging\n- `--dashboard`: Launch real-time monitoring dashboard\n- `--scan-type`: One of 'all', 'ui', 'accessibility' or 'performance'. Defaults to 'all'.\n\n## Requirements\n\n- Python 3.8+\n- Playwright browsers (automatically installed with `playwright install`)\n\n## Development\n\n```bash\n# Clone the repo\ngit clone https://github.com/yourusername/mantis.git\ncd mantis\n\n# Install in development mode\npip install -e .[dev]\n\n# Install pre-commit hooks\npre-commit install\n\n# Run tests\npytest\n```\n\n## Contributing\n\nWe welcome contributions! Please see our contributing guidelines for more details.\n\n## License\n\nMIT License - see LICENSE file for details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Web accessibility and UI bug detection tool",
"version": "1.2.0",
"project_urls": {
"Bug Reports": "https://github.com/yourusername/mantis/issues",
"Homepage": "https://github.com/yourusername/mantis",
"Repository": "https://github.com/yourusername/mantis"
},
"split_keywords": [
"web-crawler",
" accessibility",
" ui-testing",
" bug-detection",
" playwright"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "81095cb0b4f5172cb3fd243d7ebd9f59350832139699c4b5768f5b48779c76c7",
"md5": "f78616935cd2e4de335183c84503890c",
"sha256": "6b61b862c4e21c36302e439af6e5e5b3d9d599fe38e5f32b5554005dc178486c"
},
"downloads": -1,
"filename": "mantis_web_crawler-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f78616935cd2e4de335183c84503890c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 97004,
"upload_time": "2025-09-14T05:51:50",
"upload_time_iso_8601": "2025-09-14T05:51:50.594626Z",
"url": "https://files.pythonhosted.org/packages/81/09/5cb0b4f5172cb3fd243d7ebd9f59350832139699c4b5768f5b48779c76c7/mantis_web_crawler-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "ad27c2d5ec0627c7d1170810c9bf526e703899361de9f342b1db46a037eb7599",
"md5": "14db00dd717767f3400d447428ecff83",
"sha256": "acce26c8d7fe7b09e274b3978419b5d78219eb5d1b575af8aed3e4d20f976040"
},
"downloads": -1,
"filename": "mantis_web_crawler-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "14db00dd717767f3400d447428ecff83",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 83726,
"upload_time": "2025-09-14T05:51:52",
"upload_time_iso_8601": "2025-09-14T05:51:52.333847Z",
"url": "https://files.pythonhosted.org/packages/ad/27/c2d5ec0627c7d1170810c9bf526e703899361de9f342b1db46a037eb7599/mantis_web_crawler-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-14 05:51:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "mantis",
"github_not_found": true,
"lcname": "mantis-web-crawler"
}
JSON
