<p align="center">
<h1 align="center">Agentic Security</h1>
<p align="center">
The open-source Agentic LLM Vulnerability Scanner
<br />
<br />
<p>
<img alt="GitHub Contributors" src="https://img.shields.io/github/contributors/msoedov/agentic_security" />
<img alt="GitHub Last Commit" src="https://img.shields.io/github/last-commit/msoedov/agentic_security" />
<img alt="" src="https://img.shields.io/github/repo-size/msoedov/agentic_security" />
<img alt="Downloads" src="https://static.pepy.tech/badge/agentic_security" />
<img alt="GitHub Issues" src="https://img.shields.io/github/issues/msoedov/agentic_security" />
<img alt="GitHub Pull Requests" src="https://img.shields.io/github/issues-pr/msoedov/agentic_security" />
<img alt="Github License" src="https://img.shields.io/github/license/msoedov/agentic_security" />
</p>
</p>
</p>
## Features
- Customizable Rule Sets or Agent based attacks🛠️
- Comprehensive fuzzing for any LLMs 🧪
- LLM API integration and stress testing 🛠️
- Wide range of fuzzing and attack techniques 🌀
Note: Please be aware that Agentic Security is designed as a safety scanner tool and not a foolproof solution. It cannot guarantee complete protection against all possible threats.
## 📦 Installation
To get started with Agentic Security, simply install the package using pip:
```shell
pip install agentic_security
```
## ⛓️ Quick Start
```shell
agentic_security
2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:273 - Found 1 CSV files
2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:274 - CSV files: ['prompts.csv']
INFO: Started server process [18524]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8718 (Press CTRL+C to quit)
```
```shell
python -m agentic_security
# or
agentic_security --help
agentic_security --port=PORT --host=HOST
```
## UI 🧙
<img width="100%" alt="booking-screen" src="https://res.cloudinary.com/do9qa2bqr/image/upload/v1713002396/1-ezgif.com-video-to-gif-converter_s2hsro.gif">
## LLM kwargs
Agentic Security uses plain text HTTP spec like:
```http
POST https://api.openai.com/v1/chat/completions
Authorization: Bearer sk-xxxxxxxxx
Content-Type: application/json
{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "<<PROMPT>>"}],
"temperature": 0.7
}
```
Where `<<PROMPT>>` will be replaced with the actual attack vector during the scan, insert the `Bearer XXXXX` header value with your app credentials.
### Adding LLM integration templates
TBD
```
....
```
## Adding own dataset
To add your own dataset you can place one or multiples csv files with `prompt` column, this data will be loaded on `agentic_security` startup
```
2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:273 - Found 1 CSV files
2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:274 - CSV files: ['prompts.csv']
```
## Run as CI check
ci.py
```python
from agentic_security import AgenticSecurity
spec = """
POST http://0.0.0.0:8718/v1/self-probe
Authorization: Bearer XXXXX
Content-Type: application/json
{
"prompt": "<<PROMPT>>"
}
"""
result = AgenticSecurity.scan(llmSpec=spec)
# module: failure rate
# {"Local CSV": 79.65116279069767, "llm-adaptive-attacks": 20.0}
exit(max(r.values()) > 20)
```
```
python ci.py
2024-04-27 17:15:13.545 | INFO | agentic_security.probe_data.data:load_local_csv:279 - Found 1 CSV files
2024-04-27 17:15:13.545 | INFO | agentic_security.probe_data.data:load_local_csv:280 - CSV files: ['prompts.csv']
0it [00:00, ?it/s][INFO] 2024-04-27 17:15:13.74 | data:prepare_prompts:195 | Loading Custom CSV
[INFO] 2024-04-27 17:15:13.74 | fuzzer:perform_scan:53 | Scanning Local CSV 15
18it [00:00, 176.88it/s]
+-----------+--------------+--------+
| Module | Failure Rate | Status |
+-----------+--------------+--------+
| Local CSV | 80.0% | ✘ |
+-----------+--------------+--------+
```
## Extending dataset collections
1. Add new metadata to agentic_security.probe_data.REGISTRY
```python
{
"dataset_name": "markush1/LLM-Jailbreak-Classifier",
"num_prompts": 1119,
"tokens": 19758,
"approx_cost": 0.0,
"source": "Hugging Face Datasets",
"selected": True,
"dynamic": False,
"url": "https://huggingface.co/markush1/LLM-Jailbreak-Classifier",
},
```
and implement loader into
```python
@dataclass
class ProbeDataset:
dataset_name: str
metadata: dict
prompts: list[str]
tokens: int
approx_cost: float
def metadata_summary(self):
return {
"dataset_name": self.dataset_name,
"num_prompts": len(self.prompts),
"tokens": self.tokens,
"approx_cost": self.approx_cost,
}
```
## Dynamic datasets with mutations
One of the current examples uses sampling for the existing preloaded prompt data and applying mutations yielding a new dataset
```python
class Stenography:
fn_library = {
"rot5": stenography_fn.rot5,
"rot13": stenography_fn.rot13,
"base64": stenography_fn.base64_encode,
"mirror": stenography_fn.mirror_words,
"random_case": stenography_fn.randomize_letter_case,
"scramble_words": stenography_fn.scramble_words,
"noise_characters": stenography_fn.insert_noise_characters,
"substitute_with_ascii": stenography_fn.substitute_with_ascii,
"remove_vowels": stenography_fn.remove_vowels,
"zigzag_obfuscation": stenography_fn.zigzag_obfuscation,
}
def __init__(self, prompt_groups: [ProbeDataset]):
self.prompt_groups = prompt_groups
def apply(self):
for prompt_group in self.prompt_groups:
size = len(prompt_group.prompts)
for name, fn in self.fn_library.items():
logger.info(f"Applying {name} to {prompt_group.dataset_name}")
selected_prompts = random.sample(prompt_group.prompts, int(size * 0.25))
prompts = [
f"Follow the instructions in {name}: {fn(p)}"
for p in selected_prompts
]
yield ProbeDataset(
dataset_name=f"stenography.{name}({prompt_group.dataset_name})",
metadata={},
prompts=prompts,
tokens=count_words_in_list(prompts),
approx_cost=0.0,
)
```
## Probe endpoint
In the example of custom integration, we use `/v1/self-probe` for the sake of integration testing.
```python
POST https://agentic_security-preview.vercel.app/v1/self-probe
Authorization: Bearer XXXXX
Content-Type: application/json
{
"prompt": "<<PROMPT>>"
}
```
This endpoint randomly mimics the refusal of a fake LLM.
```python
@app.post("/v1/self-probe")
def self_probe(probe: Probe):
refuse = random.random() < 0.2
message = random.choice(REFUSAL_MARKS) if refuse else "This is a test!"
message = probe.prompt + " " + message
return {
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1677858242,
"model": "gpt-3.5-turbo-0613",
"usage": {"prompt_tokens": 13, "completion_tokens": 7, "total_tokens": 20},
"choices": [
{
"message": {"role": "assistant", "content": message},
"logprobs": None,
"finish_reason": "stop",
"index": 0,
}
],
}
```
## CI/CD integration
TBD
## Documentation
For more detailed information on how to use Agentic Security, including advanced features and customization options, please refer to the official documentation.
## Roadmap and Future Goals
- \[ \] Expand dataset variety
- \[ \] Introduce two new attack vectors
- \[ \] Develop initial attacker LLM
- \[ \] Complete integration of OWASP Top 10 classification
| Tool | Source | Integrated |
|-------------------------|-------------------------------------------------------------------------------|------------|
| Garak | [leondz/garak](https://github.com/leondz/garak) | ✅ |
| InspectAI | [UKGovernmentBEIS/inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) | ✅ |
| llm-adaptive-attacks | [tml-epfl/llm-adaptive-attacks](https://github.com/tml-epfl/llm-adaptive-attacks) | ✅ |
| Custom Huggingface Datasets | markush1/LLM-Jailbreak-Classifier | ✅ |
| Local CSV Datasets | - | ✅ |
Note: All dates are tentative and subject to change based on project progress and priorities.
## 👋 Contributing
Contributions to Agentic Security are welcome! If you'd like to contribute, please follow these steps:
- Fork the repository on GitHub
- Create a new branch for your changes
- Commit your changes to the new branch
- Push your changes to the forked repository
- Open a pull request to the main Agentic Security repository
Before contributing, please read the contributing guidelines.
## License
Agentic Security is released under the Apache License v2.
## Contact us
## Repo Activity
<img width="100%" src="https://repobeats.axiom.co/api/embed/2b4b4e080d21ef9174ca69bcd801145a71f67aaf.svg" />
Raw data
{
"_id": null,
"home_page": "https://github.com/msoedov/agentic_security",
"name": "agentic_security",
"maintainer": "Alexander Miasoiedov",
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": "msoedov@gmail.com",
"keywords": "LLM vulnerability scanner, llm security, llm adversarial attacks, prompt injection, prompt leakage, prompt injection attacks, prompt leakage prevention, llm vulnerabilities, owasp-llm-top-10",
"author": "Alexander Miasoiedov",
"author_email": "msoedov@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/3e/df/ce72eb59323bf381c614df37e642709a4f14fedc5da719bf05c43ae6b7cd/agentic_security-0.2.6.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n\n<h1 align=\"center\">Agentic Security</h1>\n\n<p align=\"center\">\n The open-source Agentic LLM Vulnerability Scanner\n <br />\n <br />\n\n<p>\n<img alt=\"GitHub Contributors\" src=\"https://img.shields.io/github/contributors/msoedov/agentic_security\" />\n<img alt=\"GitHub Last Commit\" src=\"https://img.shields.io/github/last-commit/msoedov/agentic_security\" />\n<img alt=\"\" src=\"https://img.shields.io/github/repo-size/msoedov/agentic_security\" />\n<img alt=\"Downloads\" src=\"https://static.pepy.tech/badge/agentic_security\" />\n<img alt=\"GitHub Issues\" src=\"https://img.shields.io/github/issues/msoedov/agentic_security\" />\n<img alt=\"GitHub Pull Requests\" src=\"https://img.shields.io/github/issues-pr/msoedov/agentic_security\" />\n<img alt=\"Github License\" src=\"https://img.shields.io/github/license/msoedov/agentic_security\" />\n</p>\n </p>\n</p>\n\n## Features\n\n- Customizable Rule Sets or Agent based attacks\ud83d\udee0\ufe0f\n- Comprehensive fuzzing for any LLMs \ud83e\uddea\n- LLM API integration and stress testing \ud83d\udee0\ufe0f\n- Wide range of fuzzing and attack techniques \ud83c\udf00\n\nNote: Please be aware that Agentic Security is designed as a safety scanner tool and not a foolproof solution. It cannot guarantee complete protection against all possible threats.\n\n## \ud83d\udce6 Installation\n\nTo get started with Agentic Security, simply install the package using pip:\n\n```shell\npip install agentic_security\n```\n\n## \u26d3\ufe0f Quick Start\n\n```shell\nagentic_security\n\n2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:273 - Found 1 CSV files\n2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:274 - CSV files: ['prompts.csv']\nINFO: Started server process [18524]\nINFO: Waiting for application startup.\nINFO: Application startup complete.\nINFO: Uvicorn running on http://0.0.0.0:8718 (Press CTRL+C to quit)\n```\n\n```shell\npython -m agentic_security\n# or\nagentic_security --help\n\n\nagentic_security --port=PORT --host=HOST\n\n```\n\n## UI \ud83e\uddd9\n\n<img width=\"100%\" alt=\"booking-screen\" src=\"https://res.cloudinary.com/do9qa2bqr/image/upload/v1713002396/1-ezgif.com-video-to-gif-converter_s2hsro.gif\">\n\n## LLM kwargs\n\nAgentic Security uses plain text HTTP spec like:\n\n```http\nPOST https://api.openai.com/v1/chat/completions\nAuthorization: Bearer sk-xxxxxxxxx\nContent-Type: application/json\n\n{\n \"model\": \"gpt-3.5-turbo\",\n \"messages\": [{\"role\": \"user\", \"content\": \"<<PROMPT>>\"}],\n \"temperature\": 0.7\n}\n\n```\n\nWhere `<<PROMPT>>` will be replaced with the actual attack vector during the scan, insert the `Bearer XXXXX` header value with your app credentials.\n\n### Adding LLM integration templates\n\nTBD\n\n```\n....\n```\n\n## Adding own dataset\n\nTo add your own dataset you can place one or multiples csv files with `prompt` column, this data will be loaded on `agentic_security` startup\n\n```\n2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:273 - Found 1 CSV files\n2024-04-13 13:21:31.157 | INFO | agentic_security.probe_data.data:load_local_csv:274 - CSV files: ['prompts.csv']\n```\n\n## Run as CI check\n\nci.py\n\n```python\nfrom agentic_security import AgenticSecurity\n\nspec = \"\"\"\nPOST http://0.0.0.0:8718/v1/self-probe\nAuthorization: Bearer XXXXX\nContent-Type: application/json\n\n{\n \"prompt\": \"<<PROMPT>>\"\n}\n\"\"\"\nresult = AgenticSecurity.scan(llmSpec=spec)\n\n# module: failure rate\n# {\"Local CSV\": 79.65116279069767, \"llm-adaptive-attacks\": 20.0}\nexit(max(r.values()) > 20)\n```\n\n```\npython ci.py\n2024-04-27 17:15:13.545 | INFO | agentic_security.probe_data.data:load_local_csv:279 - Found 1 CSV files\n2024-04-27 17:15:13.545 | INFO | agentic_security.probe_data.data:load_local_csv:280 - CSV files: ['prompts.csv']\n0it [00:00, ?it/s][INFO] 2024-04-27 17:15:13.74 | data:prepare_prompts:195 | Loading Custom CSV\n[INFO] 2024-04-27 17:15:13.74 | fuzzer:perform_scan:53 | Scanning Local CSV 15\n18it [00:00, 176.88it/s]\n+-----------+--------------+--------+\n| Module | Failure Rate | Status |\n+-----------+--------------+--------+\n| Local CSV | 80.0% | \u2718 |\n+-----------+--------------+--------+\n```\n\n## Extending dataset collections\n\n1. Add new metadata to agentic_security.probe_data.REGISTRY\n\n```python\n {\n \"dataset_name\": \"markush1/LLM-Jailbreak-Classifier\",\n \"num_prompts\": 1119,\n \"tokens\": 19758,\n \"approx_cost\": 0.0,\n \"source\": \"Hugging Face Datasets\",\n \"selected\": True,\n \"dynamic\": False,\n \"url\": \"https://huggingface.co/markush1/LLM-Jailbreak-Classifier\",\n },\n```\n\nand implement loader into\n\n```python\n@dataclass\nclass ProbeDataset:\n dataset_name: str\n metadata: dict\n prompts: list[str]\n tokens: int\n approx_cost: float\n\n def metadata_summary(self):\n return {\n \"dataset_name\": self.dataset_name,\n \"num_prompts\": len(self.prompts),\n \"tokens\": self.tokens,\n \"approx_cost\": self.approx_cost,\n }\n\n```\n\n## Dynamic datasets with mutations\n\nOne of the current examples uses sampling for the existing preloaded prompt data and applying mutations yielding a new dataset\n\n```python\nclass Stenography:\n fn_library = {\n \"rot5\": stenography_fn.rot5,\n \"rot13\": stenography_fn.rot13,\n \"base64\": stenography_fn.base64_encode,\n \"mirror\": stenography_fn.mirror_words,\n \"random_case\": stenography_fn.randomize_letter_case,\n \"scramble_words\": stenography_fn.scramble_words,\n \"noise_characters\": stenography_fn.insert_noise_characters,\n \"substitute_with_ascii\": stenography_fn.substitute_with_ascii,\n \"remove_vowels\": stenography_fn.remove_vowels,\n \"zigzag_obfuscation\": stenography_fn.zigzag_obfuscation,\n }\n\n def __init__(self, prompt_groups: [ProbeDataset]):\n self.prompt_groups = prompt_groups\n\n def apply(self):\n for prompt_group in self.prompt_groups:\n\n size = len(prompt_group.prompts)\n for name, fn in self.fn_library.items():\n logger.info(f\"Applying {name} to {prompt_group.dataset_name}\")\n selected_prompts = random.sample(prompt_group.prompts, int(size * 0.25))\n prompts = [\n f\"Follow the instructions in {name}: {fn(p)}\"\n for p in selected_prompts\n ]\n yield ProbeDataset(\n dataset_name=f\"stenography.{name}({prompt_group.dataset_name})\",\n metadata={},\n prompts=prompts,\n tokens=count_words_in_list(prompts),\n approx_cost=0.0,\n )\n```\n\n## Probe endpoint\n\nIn the example of custom integration, we use `/v1/self-probe` for the sake of integration testing.\n\n```python\nPOST https://agentic_security-preview.vercel.app/v1/self-probe\nAuthorization: Bearer XXXXX\nContent-Type: application/json\n\n{\n \"prompt\": \"<<PROMPT>>\"\n}\n\n```\n\nThis endpoint randomly mimics the refusal of a fake LLM.\n\n```python\n@app.post(\"/v1/self-probe\")\ndef self_probe(probe: Probe):\n refuse = random.random() < 0.2\n message = random.choice(REFUSAL_MARKS) if refuse else \"This is a test!\"\n message = probe.prompt + \" \" + message\n return {\n \"id\": \"chatcmpl-abc123\",\n \"object\": \"chat.completion\",\n \"created\": 1677858242,\n \"model\": \"gpt-3.5-turbo-0613\",\n \"usage\": {\"prompt_tokens\": 13, \"completion_tokens\": 7, \"total_tokens\": 20},\n \"choices\": [\n {\n \"message\": {\"role\": \"assistant\", \"content\": message},\n \"logprobs\": None,\n \"finish_reason\": \"stop\",\n \"index\": 0,\n }\n ],\n }\n\n```\n\n## CI/CD integration\n\nTBD\n\n## Documentation\n\nFor more detailed information on how to use Agentic Security, including advanced features and customization options, please refer to the official documentation.\n\n## Roadmap and Future Goals\n\n- \\[ \\] Expand dataset variety\n- \\[ \\] Introduce two new attack vectors\n- \\[ \\] Develop initial attacker LLM\n- \\[ \\] Complete integration of OWASP Top 10 classification\n\n| Tool | Source | Integrated |\n|-------------------------|-------------------------------------------------------------------------------|------------|\n| Garak | [leondz/garak](https://github.com/leondz/garak) | \u2705 |\n| InspectAI | [UKGovernmentBEIS/inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) | \u2705 |\n| llm-adaptive-attacks | [tml-epfl/llm-adaptive-attacks](https://github.com/tml-epfl/llm-adaptive-attacks) | \u2705 |\n| Custom Huggingface Datasets | markush1/LLM-Jailbreak-Classifier | \u2705 |\n| Local CSV Datasets | - | \u2705 |\n\nNote: All dates are tentative and subject to change based on project progress and priorities.\n\n## \ud83d\udc4b Contributing\n\nContributions to Agentic Security are welcome! If you'd like to contribute, please follow these steps:\n\n- Fork the repository on GitHub\n- Create a new branch for your changes\n- Commit your changes to the new branch\n- Push your changes to the forked repository\n- Open a pull request to the main Agentic Security repository\n\nBefore contributing, please read the contributing guidelines.\n\n## License\n\nAgentic Security is released under the Apache License v2.\n\n## Contact us\n\n## Repo Activity\n\n<img width=\"100%\" src=\"https://repobeats.axiom.co/api/embed/2b4b4e080d21ef9174ca69bcd801145a71f67aaf.svg\" />\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Agentic LLM vulnerability scanner",
"version": "0.2.6",
"project_urls": {
"Homepage": "https://github.com/msoedov/agentic_security",
"Repository": "https://github.com/msoedov/agentic_security"
},
"split_keywords": [
"llm vulnerability scanner",
" llm security",
" llm adversarial attacks",
" prompt injection",
" prompt leakage",
" prompt injection attacks",
" prompt leakage prevention",
" llm vulnerabilities",
" owasp-llm-top-10"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3edfce72eb59323bf381c614df37e642709a4f14fedc5da719bf05c43ae6b7cd",
"md5": "e28957da53f1dfd06d75e2548af6fdc7",
"sha256": "cdde8542fbe2aa2971d5e1ba1d4cea99bd4920788a157890e41a4fc298167a3f"
},
"downloads": -1,
"filename": "agentic_security-0.2.6.tar.gz",
"has_sig": false,
"md5_digest": "e28957da53f1dfd06d75e2548af6fdc7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 274292,
"upload_time": "2024-10-19T13:35:51",
"upload_time_iso_8601": "2024-10-19T13:35:51.569927Z",
"url": "https://files.pythonhosted.org/packages/3e/df/ce72eb59323bf381c614df37e642709a4f14fedc5da719bf05c43ae6b7cd/agentic_security-0.2.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-19 13:35:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "msoedov",
"github_project": "agentic_security",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "fastapi",
"specs": []
},
{
"name": "httpx",
"specs": []
},
{
"name": "uvicorn",
"specs": []
},
{
"name": "tqdm",
"specs": []
},
{
"name": "httpx",
"specs": []
},
{
"name": "cache_to_disk",
"specs": []
},
{
"name": "loguru",
"specs": []
},
{
"name": "pandas",
"specs": []
}
],
"lcname": "agentic_security"
}
JSON
