# pytest-delta
Run only tests impacted by your code changes (delta-based selection) for pytest.
## Overview
pytest-delta is a pytest plugin that reduces test execution time by running only the tests that are potentially affected by your code changes. It creates a directional dependency graph based on Python imports and selects tests intelligently based on what files have changed since the last successful test run.
## Features
- **Smart Test Selection**: Only runs tests affected by changed files
- **Dependency Tracking**: Creates a dependency graph based on Python imports
- **Dependency Visualization**: Generate visual representations of project structure
- **Git Integration**: Compares against the last successful test run commit
- **Uncommitted Changes Support**: Includes both staged and unstaged changes
- **Force Regeneration**: Option to force running all tests and regenerate metadata
- **File-based Mapping**: Assumes test files follow standard naming conventions
## Installation
```bash
pip install pytest-delta
```
Or for development:
```bash
git clone https://github.com/CemAlpturk/pytest-delta
cd pytest-delta
pip install -e .
```
## Usage
### Basic Usage
Run tests with delta selection:
```bash
pytest --delta
```
On first run, it will execute all tests and create a `.delta.json` file with metadata.
### Command Line Options
- `--delta`: Enable delta-based test selection
- `--delta-filename NAME`: Specify filename for delta metadata file (default: `.delta`, `.json` extension added automatically)
- `--delta-dir PATH`: Specify directory for delta metadata file (default: current directory)
- `--delta-force`: Force regeneration of delta file and run all tests
- `--delta-ignore PATTERN`: Ignore file patterns during dependency analysis (can be used multiple times)
- `--delta-vis`: Generate a visual representation of the project's dependency graph
- `--delta-source-dirs PATH`: Source directories to search for Python files (default: project root and `src/`). Can be used multiple times.
- `--delta-test-dirs PATH`: Test directories to search for test files (default: `tests`). Can be used multiple times.
- `--delta-debug`: Display detailed debug information about changed files, affected files, and selected tests
### Examples
```bash
# Run only affected tests
pytest --delta
# Generate dependency visualization
pytest --delta-vis
# Combine visualization with delta testing
pytest --delta --delta-vis
# Force run all tests and regenerate metadata
pytest --delta --delta-force
# Use custom delta filename (will become custom-delta.json)
pytest --delta --delta-filename custom-delta
# Use custom directory for delta file
pytest --delta --delta-dir .metadata
# Combine custom filename and directory
pytest --delta --delta-filename my-tests --delta-dir /tmp/deltas
# Combine with other pytest options
pytest --delta -v --tb=short
# Ignore generated files during analysis
pytest --delta --delta-ignore "*generated*"
# Ignore multiple patterns
pytest --delta --delta-ignore "*generated*" --delta-ignore "vendor/*"
# Ignore test files from dependency analysis (useful for complex test hierarchies)
pytest --delta --delta-ignore "tests/integration/*"
# Use custom source directories (for non-standard project layouts)
pytest --delta --delta-source-dirs lib --delta-source-dirs modules
# Use custom test directories
pytest --delta --delta-test-dirs unit_tests --delta-test-dirs integration_tests
# Combine custom directories
pytest --delta --delta-source-dirs src --delta-source-dirs lib --delta-test-dirs tests --delta-test-dirs e2e
# Enable debug output to see detailed information
pytest --delta --delta-debug
# Debug with custom directories
pytest --delta --delta-debug --delta-source-dirs custom_src --delta-test-dirs custom_tests
```
## Dependency Visualization
The `--delta-vis` option generates visual representations of your project's dependency graph, which is useful for:
- **Understanding code relationships**: See which files depend on each other
- **Debugging the plugin**: Visualize how pytest-delta determines affected tests
- **Code review**: Get insights into project structure and coupling
- **Documentation**: Generate dependency diagrams for your project
### Visualization Output
When you run `pytest --delta-vis`, the plugin generates:
1. **Console output**: Immediate feedback showing file counts and top dependencies
2. **DOT file** (`dependency_graph.dot`): GraphViz format for rendering images
3. **Text summary** (`dependency_summary.txt`): Detailed human-readable analysis
### Example Console Output
```
📊 Dependency Graph Visualization
==================================================
Files: 7 | Dependencies: 5 | Max per file: 5
🔗 Files with most dependencies:
5 deps: tests/test_plugin.py
0 deps: src/myproject/utils.py
0 deps: src/myproject/models.py
```
### Rendering Graphical Output
If you have Graphviz installed, you can render the DOT file as an image:
```bash
# Generate visualization
pytest --delta-vis
# Render as PNG image (requires graphviz)
dot -Tpng dependency_graph.dot -o dependency_graph.png
# Render as SVG (scalable)
dot -Tsvg dependency_graph.dot -o dependency_graph.svg
# Render as PDF
dot -Tpdf dependency_graph.dot -o dependency_graph.pdf
```
### Installation of Optional Dependencies
For rendering graphical outputs, you can install Graphviz:
```bash
# On Ubuntu/Debian
sudo apt-get install graphviz
# On macOS
brew install graphviz
# On Windows (using chocolatey)
choco install graphviz
```
### Migration from Previous Versions
If you were using the old `--delta-file` option, you can migrate as follows:
```bash
# Old way (no longer supported):
# pytest --delta --delta-file /path/to/custom.json
# New way:
pytest --delta --delta-filename custom --delta-dir /path/to
# This creates: /path/to/custom.json
```
## How It Works
1. **First Run**: On the first run (or when the delta file doesn't exist), all tests are executed and a delta metadata file is created containing the current Git commit hash.
2. **Change Detection**: On subsequent runs, the plugin:
- Compares current Git state with the last successful run
- Identifies changed Python files (both committed and uncommitted)
- Builds a dependency graph based on Python imports
- Finds all files transitively affected by the changes
3. **Test Selection**: The plugin selects tests based on:
- Direct test files that were modified
- Test files that test the modified source files
- Test files that test files affected by the changes (transitive dependencies)
4. **File Mapping**: Test files are mapped to source files using naming conventions:
- `tests/test_module.py` ↔ `src/module.py`
- `tests/subdir/test_module.py` ↔ `src/subdir/module.py`
## Project Structure Assumptions
The plugin works best with projects that follow these conventions:
```
project/
├── src/ # Source code
│ ├── module1.py
│ └── package/
│ └── module2.py
├── tests/ # Test files
│ ├── test_module1.py
│ └── package/
│ └── test_module2.py
└── .delta.json # Delta metadata (auto-generated, default location)
```
## Configuration
### Configurable Directories
By default, pytest-delta searches for source files in the project root (`.`) and `src/` directories, and test files in the `tests/` directory. You can customize these locations:
```bash
# Use custom source directories
pytest --delta --delta-source-dirs lib --delta-source-dirs modules
# Use custom test directories
pytest --delta --delta-test-dirs unit_tests --delta-test-dirs integration_tests
# Combine both for complex project layouts
pytest --delta --delta-source-dirs src --delta-source-dirs lib --delta-test-dirs tests --delta-test-dirs e2e
```
This is useful for projects with non-standard layouts or when you want to exclude certain directories from analysis.
### Debug Information
Use `--delta-debug` to get detailed information about the plugin's behavior:
```bash
pytest --delta --delta-debug
```
This will show you:
- Which directories are being searched
- What files were found and analyzed
- Which files changed since the last run
- How the dependency graph was built
- Which tests were selected and why
### Ignoring Files
The `--delta-ignore` option allows you to exclude certain files from dependency analysis. This is useful for:
- **Generated files**: Auto-generated code that shouldn't trigger test runs
- **Vendor/third-party code**: External dependencies that don't need analysis
- **Temporary files**: Files that are frequently modified but don't affect tests
- **Documentation**: Markdown, text files that might be mixed with Python code
The ignore patterns support:
- **Glob patterns**: `*generated*`, `*.tmp`, `vendor/*`
- **Path matching**: Both relative and absolute paths are checked
- **Multiple patterns**: Use the option multiple times for different patterns
Examples:
```bash
# Ignore all generated files
pytest --delta --delta-ignore "*generated*"
# Ignore vendor directory and any temp files
pytest --delta --delta-ignore "vendor/*" --delta-ignore "*.tmp"
# Ignore specific test subdirectories from analysis
pytest --delta --delta-ignore "tests/integration/*" --delta-ignore "tests/e2e/*"
```
### Default Configuration
The plugin requires no configuration for basic usage. It automatically:
- Finds Python files in the current directory (`.`) and `src/` directories by default
- Searches for test files in the `tests/` directory by default
- Excludes virtual environments, `__pycache__`, and other irrelevant directories
- Creates dependency graphs based on import statements
- Maps test files to source files using naming conventions
You can override the default directories using `--delta-source-dirs` and `--delta-test-dirs` options to customize the search paths for your specific project layout.
## Error Handling
The plugin includes robust error handling:
- **No Git Repository**: Falls back to running all tests
- **Invalid Delta File**: Regenerates metadata and runs all tests
- **Git Errors**: Falls back to running all tests with warnings
- **Import Analysis Errors**: Continues with partial dependency graph
## Example Output
```bash
$ pytest --delta -v
================ test session starts ================
plugins: delta-0.1.0
[pytest-delta] Selected 3/10 tests based on code changes
[pytest-delta] Affected files: src/calculator.py, tests/test_calculator.py
tests/test_calculator.py::test_add PASSED
tests/test_calculator.py::test_multiply PASSED
tests/test_math_utils.py::test_area PASSED
[pytest-delta] Delta metadata updated successfully
================ 3 passed in 0.02s ================
```
## Development
To set up for development:
```bash
git clone https://github.com/CemAlpturk/pytest-delta
cd pytest-delta
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install -e .
pip install pytest gitpython
# Run tests
pytest tests/
# Test the plugin
pytest --delta
```
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## License
This project is licensed under the MIT License - see the LICENSE file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/CemAlpturk/pytest-delta",
"name": "pytest-delta",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": "pytest, plugin, selective, impact, test, ci, graph",
"author": "Cem Alpt\u00fcrk",
"author_email": "cem.alpturk@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c4/86/bc0481a627203456a8f4e93357bced3d958f747fa0f733d98843d135b0af/pytest_delta-0.2.3.tar.gz",
"platform": null,
"description": "# pytest-delta\n\nRun only tests impacted by your code changes (delta-based selection) for pytest.\n\n## Overview\n\npytest-delta is a pytest plugin that reduces test execution time by running only the tests that are potentially affected by your code changes. It creates a directional dependency graph based on Python imports and selects tests intelligently based on what files have changed since the last successful test run.\n\n## Features\n\n- **Smart Test Selection**: Only runs tests affected by changed files\n- **Dependency Tracking**: Creates a dependency graph based on Python imports\n- **Dependency Visualization**: Generate visual representations of project structure\n- **Git Integration**: Compares against the last successful test run commit\n- **Uncommitted Changes Support**: Includes both staged and unstaged changes\n- **Force Regeneration**: Option to force running all tests and regenerate metadata\n- **File-based Mapping**: Assumes test files follow standard naming conventions\n\n## Installation\n\n```bash\npip install pytest-delta\n```\n\nOr for development:\n\n```bash\ngit clone https://github.com/CemAlpturk/pytest-delta\ncd pytest-delta\npip install -e .\n```\n\n## Usage\n\n### Basic Usage\n\nRun tests with delta selection:\n\n```bash\npytest --delta\n```\n\nOn first run, it will execute all tests and create a `.delta.json` file with metadata.\n\n### Command Line Options\n\n- `--delta`: Enable delta-based test selection\n- `--delta-filename NAME`: Specify filename for delta metadata file (default: `.delta`, `.json` extension added automatically)\n- `--delta-dir PATH`: Specify directory for delta metadata file (default: current directory)\n- `--delta-force`: Force regeneration of delta file and run all tests\n- `--delta-ignore PATTERN`: Ignore file patterns during dependency analysis (can be used multiple times)\n- `--delta-vis`: Generate a visual representation of the project's dependency graph\n- `--delta-source-dirs PATH`: Source directories to search for Python files (default: project root and `src/`). Can be used multiple times.\n- `--delta-test-dirs PATH`: Test directories to search for test files (default: `tests`). Can be used multiple times.\n- `--delta-debug`: Display detailed debug information about changed files, affected files, and selected tests\n\n### Examples\n\n```bash\n# Run only affected tests\npytest --delta\n\n# Generate dependency visualization\npytest --delta-vis\n\n# Combine visualization with delta testing\npytest --delta --delta-vis\n\n# Force run all tests and regenerate metadata\npytest --delta --delta-force\n\n# Use custom delta filename (will become custom-delta.json)\npytest --delta --delta-filename custom-delta\n\n# Use custom directory for delta file\npytest --delta --delta-dir .metadata\n\n# Combine custom filename and directory\npytest --delta --delta-filename my-tests --delta-dir /tmp/deltas\n\n# Combine with other pytest options\npytest --delta -v --tb=short\n\n# Ignore generated files during analysis\npytest --delta --delta-ignore \"*generated*\"\n\n# Ignore multiple patterns\npytest --delta --delta-ignore \"*generated*\" --delta-ignore \"vendor/*\"\n\n# Ignore test files from dependency analysis (useful for complex test hierarchies)\npytest --delta --delta-ignore \"tests/integration/*\"\n\n# Use custom source directories (for non-standard project layouts)\npytest --delta --delta-source-dirs lib --delta-source-dirs modules\n\n# Use custom test directories\npytest --delta --delta-test-dirs unit_tests --delta-test-dirs integration_tests\n\n# Combine custom directories\npytest --delta --delta-source-dirs src --delta-source-dirs lib --delta-test-dirs tests --delta-test-dirs e2e\n\n# Enable debug output to see detailed information\npytest --delta --delta-debug\n\n# Debug with custom directories\npytest --delta --delta-debug --delta-source-dirs custom_src --delta-test-dirs custom_tests\n```\n\n## Dependency Visualization\n\nThe `--delta-vis` option generates visual representations of your project's dependency graph, which is useful for:\n\n- **Understanding code relationships**: See which files depend on each other\n- **Debugging the plugin**: Visualize how pytest-delta determines affected tests \n- **Code review**: Get insights into project structure and coupling\n- **Documentation**: Generate dependency diagrams for your project\n\n### Visualization Output\n\nWhen you run `pytest --delta-vis`, the plugin generates:\n\n1. **Console output**: Immediate feedback showing file counts and top dependencies\n2. **DOT file** (`dependency_graph.dot`): GraphViz format for rendering images\n3. **Text summary** (`dependency_summary.txt`): Detailed human-readable analysis\n\n### Example Console Output\n\n```\n\ud83d\udcca Dependency Graph Visualization\n==================================================\nFiles: 7 | Dependencies: 5 | Max per file: 5\n\n\ud83d\udd17 Files with most dependencies:\n 5 deps: tests/test_plugin.py\n 0 deps: src/myproject/utils.py\n 0 deps: src/myproject/models.py\n```\n\n### Rendering Graphical Output\n\nIf you have Graphviz installed, you can render the DOT file as an image:\n\n```bash\n# Generate visualization\npytest --delta-vis\n\n# Render as PNG image (requires graphviz)\ndot -Tpng dependency_graph.dot -o dependency_graph.png\n\n# Render as SVG (scalable) \ndot -Tsvg dependency_graph.dot -o dependency_graph.svg\n\n# Render as PDF\ndot -Tpdf dependency_graph.dot -o dependency_graph.pdf\n```\n\n### Installation of Optional Dependencies\n\nFor rendering graphical outputs, you can install Graphviz:\n\n```bash\n# On Ubuntu/Debian\nsudo apt-get install graphviz\n\n# On macOS \nbrew install graphviz\n\n# On Windows (using chocolatey)\nchoco install graphviz\n```\n\n### Migration from Previous Versions\n\nIf you were using the old `--delta-file` option, you can migrate as follows:\n\n```bash\n# Old way (no longer supported):\n# pytest --delta --delta-file /path/to/custom.json\n\n# New way:\npytest --delta --delta-filename custom --delta-dir /path/to\n# This creates: /path/to/custom.json\n```\n\n## How It Works\n\n1. **First Run**: On the first run (or when the delta file doesn't exist), all tests are executed and a delta metadata file is created containing the current Git commit hash.\n\n2. **Change Detection**: On subsequent runs, the plugin:\n - Compares current Git state with the last successful run\n - Identifies changed Python files (both committed and uncommitted)\n - Builds a dependency graph based on Python imports\n - Finds all files transitively affected by the changes\n\n3. **Test Selection**: The plugin selects tests based on:\n - Direct test files that were modified\n - Test files that test the modified source files\n - Test files that test files affected by the changes (transitive dependencies)\n\n4. **File Mapping**: Test files are mapped to source files using naming conventions:\n - `tests/test_module.py` \u2194 `src/module.py`\n - `tests/subdir/test_module.py` \u2194 `src/subdir/module.py`\n\n## Project Structure Assumptions\n\nThe plugin works best with projects that follow these conventions:\n\n```\nproject/\n\u251c\u2500\u2500 src/ # Source code\n\u2502 \u251c\u2500\u2500 module1.py\n\u2502 \u2514\u2500\u2500 package/\n\u2502 \u2514\u2500\u2500 module2.py\n\u251c\u2500\u2500 tests/ # Test files\n\u2502 \u251c\u2500\u2500 test_module1.py\n\u2502 \u2514\u2500\u2500 package/\n\u2502 \u2514\u2500\u2500 test_module2.py\n\u2514\u2500\u2500 .delta.json # Delta metadata (auto-generated, default location)\n```\n\n## Configuration\n\n### Configurable Directories\n\nBy default, pytest-delta searches for source files in the project root (`.`) and `src/` directories, and test files in the `tests/` directory. You can customize these locations:\n\n```bash\n# Use custom source directories\npytest --delta --delta-source-dirs lib --delta-source-dirs modules\n\n# Use custom test directories \npytest --delta --delta-test-dirs unit_tests --delta-test-dirs integration_tests\n\n# Combine both for complex project layouts\npytest --delta --delta-source-dirs src --delta-source-dirs lib --delta-test-dirs tests --delta-test-dirs e2e\n```\n\nThis is useful for projects with non-standard layouts or when you want to exclude certain directories from analysis.\n\n### Debug Information\n\nUse `--delta-debug` to get detailed information about the plugin's behavior:\n\n```bash\npytest --delta --delta-debug\n```\n\nThis will show you:\n- Which directories are being searched\n- What files were found and analyzed \n- Which files changed since the last run\n- How the dependency graph was built\n- Which tests were selected and why\n\n### Ignoring Files\n\nThe `--delta-ignore` option allows you to exclude certain files from dependency analysis. This is useful for:\n\n- **Generated files**: Auto-generated code that shouldn't trigger test runs\n- **Vendor/third-party code**: External dependencies that don't need analysis\n- **Temporary files**: Files that are frequently modified but don't affect tests\n- **Documentation**: Markdown, text files that might be mixed with Python code\n\nThe ignore patterns support:\n- **Glob patterns**: `*generated*`, `*.tmp`, `vendor/*`\n- **Path matching**: Both relative and absolute paths are checked\n- **Multiple patterns**: Use the option multiple times for different patterns\n\nExamples:\n```bash\n# Ignore all generated files\npytest --delta --delta-ignore \"*generated*\"\n\n# Ignore vendor directory and any temp files\npytest --delta --delta-ignore \"vendor/*\" --delta-ignore \"*.tmp\"\n\n# Ignore specific test subdirectories from analysis\npytest --delta --delta-ignore \"tests/integration/*\" --delta-ignore \"tests/e2e/*\"\n```\n\n### Default Configuration\n\nThe plugin requires no configuration for basic usage. It automatically:\n\n- Finds Python files in the current directory (`.`) and `src/` directories by default\n- Searches for test files in the `tests/` directory by default \n- Excludes virtual environments, `__pycache__`, and other irrelevant directories\n- Creates dependency graphs based on import statements\n- Maps test files to source files using naming conventions\n\nYou can override the default directories using `--delta-source-dirs` and `--delta-test-dirs` options to customize the search paths for your specific project layout.\n\n## Error Handling\n\nThe plugin includes robust error handling:\n\n- **No Git Repository**: Falls back to running all tests\n- **Invalid Delta File**: Regenerates metadata and runs all tests\n- **Git Errors**: Falls back to running all tests with warnings\n- **Import Analysis Errors**: Continues with partial dependency graph\n\n## Example Output\n\n```bash\n$ pytest --delta -v\n================ test session starts ================\nplugins: delta-0.1.0\n[pytest-delta] Selected 3/10 tests based on code changes\n[pytest-delta] Affected files: src/calculator.py, tests/test_calculator.py\n\ntests/test_calculator.py::test_add PASSED\ntests/test_calculator.py::test_multiply PASSED\ntests/test_math_utils.py::test_area PASSED\n\n[pytest-delta] Delta metadata updated successfully\n================ 3 passed in 0.02s ================\n```\n\n## Development\n\nTo set up for development:\n\n```bash\ngit clone https://github.com/CemAlpturk/pytest-delta\ncd pytest-delta\npython -m venv .venv\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activate\npip install -e .\npip install pytest gitpython\n\n# Run tests\npytest tests/\n\n# Test the plugin\npytest --delta\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Run only tests impacted by your code changes (delta-based selection) for pytest.",
"version": "0.2.3",
"project_urls": {
"Homepage": "https://github.com/CemAlpturk/pytest-delta",
"Repository": "https://github.com/CemAlpturk/pytest-delta"
},
"split_keywords": [
"pytest",
" plugin",
" selective",
" impact",
" test",
" ci",
" graph"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f4e8d3b80afc9a987b1dcb67c36c9ddb0a5b96b355830803eaa58c93a0ddea9a",
"md5": "2e377392264b457b36e2d7bef2b0d08a",
"sha256": "07b51798cb5d2ab9ee1ba7ff8782bc364f698525d2896d701982fd5a4247caed"
},
"downloads": -1,
"filename": "pytest_delta-0.2.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2e377392264b457b36e2d7bef2b0d08a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 18477,
"upload_time": "2025-09-01T08:45:17",
"upload_time_iso_8601": "2025-09-01T08:45:17.880750Z",
"url": "https://files.pythonhosted.org/packages/f4/e8/d3b80afc9a987b1dcb67c36c9ddb0a5b96b355830803eaa58c93a0ddea9a/pytest_delta-0.2.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c486bc0481a627203456a8f4e93357bced3d958f747fa0f733d98843d135b0af",
"md5": "4b8fcec80941464a16a0159eadc15c0e",
"sha256": "64fb2dfc5b1e868b37577de63b4b128e4a1f7362fcc6aa03571dd7658687b075"
},
"downloads": -1,
"filename": "pytest_delta-0.2.3.tar.gz",
"has_sig": false,
"md5_digest": "4b8fcec80941464a16a0159eadc15c0e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 18758,
"upload_time": "2025-09-01T08:45:19",
"upload_time_iso_8601": "2025-09-01T08:45:19.150509Z",
"url": "https://files.pythonhosted.org/packages/c4/86/bc0481a627203456a8f4e93357bced3d958f747fa0f733d98843d135b0af/pytest_delta-0.2.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-01 08:45:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "CemAlpturk",
"github_project": "pytest-delta",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pytest-delta"
}
JSON
