<div align="center">
<img alt="intentguard" height="200px" src="./design/logomark-256.png">
</div>
# IntentGuard
IntentGuard is a Python library for verifying code properties using natural language assertions. It integrates with testing frameworks like pytest and unittest, allowing you to express complex code expectations in plain English within your existing test suites.
## Why IntentGuard?
Traditional code testing often requires writing extensive code to verify intricate properties. IntentGuard simplifies this by enabling you to express sophisticated test cases in natural language. This is particularly useful when writing conventional test code becomes impractical or overly complex.
**Key Use Cases:**
* **Complex Property Verification:** Test intricate code behaviors that are hard to assert with standard methods.
* **Reduced Boilerplate:** Avoid writing lengthy test code for advanced checks.
* **Improved Readability:** Natural language assertions make tests easier to understand, especially for complex logic.
### Key Features
1. **Natural Language Assertions:** Write test assertions in plain English.
2. **Testing Framework Integration:** Works seamlessly with pytest and unittest.
3. **Deterministic Results:** Employs a voting mechanism and controlled sampling for consistent test outcomes.
4. **Flexible Verification:** Test properties difficult to verify using traditional techniques.
5. **Detailed Failure Explanations:** Provides clear, natural language explanations when assertions fail.
6. **Efficient Result Caching:** Caches results to speed up test execution and avoid redundant evaluations.
## When to Use IntentGuard
IntentGuard is ideal when implementing traditional tests for certain code properties is challenging or requires excessive code. Consider these scenarios:
```python
# Example 1: Error Handling Verification
def test_error_handling():
ig.assert_code(
"All methods in {module} should use the custom ErrorHandler class for exception management, and log errors before re-raising them",
{"module": my_critical_module}
)
# Example 2: Documentation Consistency Check
def test_docstring_completeness():
ig.assert_code(
"All public methods in {module} should have docstrings that include Parameters, Returns, and Examples sections",
{"module": my_api_module}
)
````
In these examples, manually writing tests to iterate through methods, parse AST, and check for specific patterns would be significantly more complex than using IntentGuard's natural language assertions.
## How It Works: Deterministic Testing
IntentGuard ensures reliable results through these mechanisms:
1. **Voting Mechanism:** Each assertion is evaluated multiple times (configurable via `num_evaluations`), and the majority result determines the outcome.
2. **Temperature Control:** Low temperature sampling in the LLM minimizes randomness.
3. **Structured Prompts:** Natural language assertions are converted into structured prompts for consistent LLM interpretation.
You can configure determinism settings:
```python
options = IntentGuardOptions(
num_evaluations=5, # Number of evaluations per assertion
)
```
## Compatibility
IntentGuard is compatible with:
* **Python:** 3.10+
* **Operating Systems:**
* Linux 2.6.18+ (most distributions since \~2007)
* Darwin (macOS) 23.1.0+ (GPU support only on ARM64)
* Windows 10+ (AMD64 only)
* FreeBSD 13+
* NetBSD 9.2+ (AMD64 only)
* OpenBSD 7+ (AMD64 only)
These OS and architecture compatibilities are inherited from [llamafile](https://github.com/Mozilla-Ocho/llamafile), which IntentGuard uses to run the model locally.
## Installation
```bash
pip install intentguard
```
## Basic Usage
### With pytest
```python
import intentguard as ig
def test_code_properties():
guard = ig.IntentGuard()
# Test code organization
guard.assert_code(
"Classes in {module} should follow the Single Responsibility Principle",
{"module": my_module}
)
# Test security practices
guard.assert_code(
"All database queries in {module} should be parameterized to prevent SQL injection",
{"module": db_module}
)
```
### With unittest
```python
import unittest
import intentguard as ig
class TestCodeQuality(unittest.TestCase):
def setUp(self):
self.guard = ig.IntentGuard()
def test_error_handling(self):
self.guard.assert_code(
"All API endpoints in {module} should have proper input validation",
{"module": api_module}
)
```
## Advanced Usage: Custom Evaluation Options
```python
import intentguard as ig
options = ig.IntentGuardOptions(
num_evaluations=7, # Increase number of evaluations
temperature=0.1, # Lower temperature for more deterministic results
)
guard = ig.IntentGuard(options)
```
## Model
IntentGuard utilizes [a custom 1B parameter model](https://huggingface.co/kdunee/IntentGuard-1), fine-tuned from Llama-3.2-1B-Instruct. This model is optimized for code analysis and verification and runs locally using [llamafile](https://github.com/Mozilla-Ocho/llamafile) for privacy and efficient inference.
## Local Development Environment Setup
To contribute to IntentGuard, set up your local environment:
1. **Prerequisites:** Python 3.10+, [Poetry](https://python-poetry.org/docs/#installation).
2. **Clone:** `git clone <repository_url> && cd intentguard`
3. **Install dev dependencies:** `make install`
4. **Run tests & checks:** `make test`
Refer to the `Makefile` for more development commands.
### Useful development commands:
* `make install`: Installs development dependencies.
* `make install-prod`: Installs production dependencies only.
* `make check`: Runs linting checks (`ruff check`).
* `make format-check`: Checks code formatting (`ruff format --check`).
* `make mypy`: Runs static type checking (`mypy`).
* `make unittest`: Runs unit tests.
* `make test`: Runs all checks and tests.
* `make clean`: Removes the virtual environment.
* `make help`: Lists available `make` commands.
## License
[MIT License](LICENSE)
-----
IntentGuard is a complementary tool for specific testing needs, not a replacement for traditional testing. It is most effective for verifying complex code properties that are difficult to test conventionally.
Raw data
{
"_id": null,
"home_page": "https://github.com/kdunee/intentguard",
"name": "intentguard",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "testing, pytest, unittest, ai-testing, llm, code-verification, natural-language, test-automation, code-quality, language-models",
"author": "Kosma Dunikowski",
"author_email": "kosmadunikowski@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f0/f2/81bc8c56c274259ece1bf61967e01d50e0ba0d1d0a4564f8a78dd94ec802/intentguard-2.0.3.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <img alt=\"intentguard\" height=\"200px\" src=\"./design/logomark-256.png\">\n</div>\n\n# IntentGuard\n\n\n\n\n\n\n\n\nIntentGuard is a Python library for verifying code properties using natural language assertions. It integrates with testing frameworks like pytest and unittest, allowing you to express complex code expectations in plain English within your existing test suites.\n\n## Why IntentGuard?\n\nTraditional code testing often requires writing extensive code to verify intricate properties. IntentGuard simplifies this by enabling you to express sophisticated test cases in natural language. This is particularly useful when writing conventional test code becomes impractical or overly complex.\n\n**Key Use Cases:**\n\n* **Complex Property Verification:** Test intricate code behaviors that are hard to assert with standard methods.\n* **Reduced Boilerplate:** Avoid writing lengthy test code for advanced checks.\n* **Improved Readability:** Natural language assertions make tests easier to understand, especially for complex logic.\n\n### Key Features\n\n1. **Natural Language Assertions:** Write test assertions in plain English.\n2. **Testing Framework Integration:** Works seamlessly with pytest and unittest.\n3. **Deterministic Results:** Employs a voting mechanism and controlled sampling for consistent test outcomes.\n4. **Flexible Verification:** Test properties difficult to verify using traditional techniques.\n5. **Detailed Failure Explanations:** Provides clear, natural language explanations when assertions fail.\n6. **Efficient Result Caching:** Caches results to speed up test execution and avoid redundant evaluations.\n\n## When to Use IntentGuard\n\nIntentGuard is ideal when implementing traditional tests for certain code properties is challenging or requires excessive code. Consider these scenarios:\n\n```python\n# Example 1: Error Handling Verification\n\ndef test_error_handling():\n ig.assert_code(\n \"All methods in {module} should use the custom ErrorHandler class for exception management, and log errors before re-raising them\",\n {\"module\": my_critical_module}\n )\n\n\n# Example 2: Documentation Consistency Check\n\ndef test_docstring_completeness():\n ig.assert_code(\n \"All public methods in {module} should have docstrings that include Parameters, Returns, and Examples sections\",\n {\"module\": my_api_module}\n )\n````\n\nIn these examples, manually writing tests to iterate through methods, parse AST, and check for specific patterns would be significantly more complex than using IntentGuard's natural language assertions.\n\n## How It Works: Deterministic Testing\n\nIntentGuard ensures reliable results through these mechanisms:\n\n1. **Voting Mechanism:** Each assertion is evaluated multiple times (configurable via `num_evaluations`), and the majority result determines the outcome.\n2. **Temperature Control:** Low temperature sampling in the LLM minimizes randomness.\n3. **Structured Prompts:** Natural language assertions are converted into structured prompts for consistent LLM interpretation.\n\nYou can configure determinism settings:\n\n```python\noptions = IntentGuardOptions(\n num_evaluations=5, # Number of evaluations per assertion\n)\n```\n\n## Compatibility\n\nIntentGuard is compatible with:\n\n * **Python:** 3.10+\n * **Operating Systems:**\n * Linux 2.6.18+ (most distributions since \\~2007)\n * Darwin (macOS) 23.1.0+ (GPU support only on ARM64)\n * Windows 10+ (AMD64 only)\n * FreeBSD 13+\n * NetBSD 9.2+ (AMD64 only)\n * OpenBSD 7+ (AMD64 only)\n\nThese OS and architecture compatibilities are inherited from [llamafile](https://github.com/Mozilla-Ocho/llamafile), which IntentGuard uses to run the model locally.\n\n## Installation\n\n```bash\npip install intentguard\n```\n\n## Basic Usage\n\n### With pytest\n\n```python\nimport intentguard as ig\n\ndef test_code_properties():\n guard = ig.IntentGuard()\n\n # Test code organization\n guard.assert_code(\n \"Classes in {module} should follow the Single Responsibility Principle\",\n {\"module\": my_module}\n )\n\n # Test security practices\n guard.assert_code(\n \"All database queries in {module} should be parameterized to prevent SQL injection\",\n {\"module\": db_module}\n )\n```\n\n### With unittest\n\n```python\nimport unittest\nimport intentguard as ig\n\nclass TestCodeQuality(unittest.TestCase):\n def setUp(self):\n self.guard = ig.IntentGuard()\n\n def test_error_handling(self):\n self.guard.assert_code(\n \"All API endpoints in {module} should have proper input validation\",\n {\"module\": api_module}\n )\n```\n\n## Advanced Usage: Custom Evaluation Options\n\n```python\nimport intentguard as ig\n\noptions = ig.IntentGuardOptions(\n num_evaluations=7, # Increase number of evaluations\n temperature=0.1, # Lower temperature for more deterministic results\n)\n\nguard = ig.IntentGuard(options)\n```\n\n## Model\n\nIntentGuard utilizes [a custom 1B parameter model](https://huggingface.co/kdunee/IntentGuard-1), fine-tuned from Llama-3.2-1B-Instruct. This model is optimized for code analysis and verification and runs locally using [llamafile](https://github.com/Mozilla-Ocho/llamafile) for privacy and efficient inference.\n\n## Local Development Environment Setup\n\nTo contribute to IntentGuard, set up your local environment:\n\n1. **Prerequisites:** Python 3.10+, [Poetry](https://python-poetry.org/docs/#installation).\n2. **Clone:** `git clone <repository_url> && cd intentguard`\n3. **Install dev dependencies:** `make install`\n4. **Run tests & checks:** `make test`\n\nRefer to the `Makefile` for more development commands.\n\n### Useful development commands:\n\n * `make install`: Installs development dependencies.\n * `make install-prod`: Installs production dependencies only.\n * `make check`: Runs linting checks (`ruff check`).\n * `make format-check`: Checks code formatting (`ruff format --check`).\n * `make mypy`: Runs static type checking (`mypy`).\n * `make unittest`: Runs unit tests.\n * `make test`: Runs all checks and tests.\n * `make clean`: Removes the virtual environment.\n * `make help`: Lists available `make` commands.\n\n## License\n\n[MIT License](LICENSE)\n\n-----\n\nIntentGuard is a complementary tool for specific testing needs, not a replacement for traditional testing. It is most effective for verifying complex code properties that are difficult to test conventionally.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python library for verifying code properties using natural language assertions.",
"version": "2.0.3",
"project_urls": {
"Homepage": "https://github.com/kdunee/intentguard"
},
"split_keywords": [
"testing",
" pytest",
" unittest",
" ai-testing",
" llm",
" code-verification",
" natural-language",
" test-automation",
" code-quality",
" language-models"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fdda6ee91d947f6a8586afc7440037bc898bfc9c56f4bf3fe4241b6af2ff730e",
"md5": "c7c468c0e28a7b9edb5edcac9800af14",
"sha256": "72ab4e4e60bfc853b8dd8c52a4bcfc65214d6cb56e922f43a7b4120154275dd4"
},
"downloads": -1,
"filename": "intentguard-2.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c7c468c0e28a7b9edb5edcac9800af14",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 23347,
"upload_time": "2025-02-16T10:34:36",
"upload_time_iso_8601": "2025-02-16T10:34:36.389181Z",
"url": "https://files.pythonhosted.org/packages/fd/da/6ee91d947f6a8586afc7440037bc898bfc9c56f4bf3fe4241b6af2ff730e/intentguard-2.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f0f281bc8c56c274259ece1bf61967e01d50e0ba0d1d0a4564f8a78dd94ec802",
"md5": "350fe454b8a59ee884af69622e3bd61e",
"sha256": "42931e83542117ce79a0c7d68713f766137c37f735ca62ff207ea1a271202c5c"
},
"downloads": -1,
"filename": "intentguard-2.0.3.tar.gz",
"has_sig": false,
"md5_digest": "350fe454b8a59ee884af69622e3bd61e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 18650,
"upload_time": "2025-02-16T10:34:38",
"upload_time_iso_8601": "2025-02-16T10:34:38.124203Z",
"url": "https://files.pythonhosted.org/packages/f0/f2/81bc8c56c274259ece1bf61967e01d50e0ba0d1d0a4564f8a78dd94ec802/intentguard-2.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-16 10:34:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kdunee",
"github_project": "intentguard",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "intentguard"
}
JSON
