# Glaider Prompt Fuzzer
## Safeguarding Your GenAI Applications
Glaider Prompt Fuzzer is a cutting-edge tool designed to enhance the security of your generative AI applications. By simulating various LLM-based attacks, it evaluates the robustness of your system prompts and helps you fortify them against potential vulnerabilities.
## Key Features
- Dynamic testing tailored to your application's unique configuration
- Support for 16 LLM providers
- 15 different attack simulations
- Interactive and CLI modes
- Multi-threaded testing for efficiency
- Playground interface for iterative prompt improvement
## Getting Started
### Installation
Choose one of the following methods:
1. Via pip:
```
pip install glaider-fuzzer
```
2. From PyPI: Visit the [Glaider Fuzzer package page](https://pypi.org/project/glaider-fuzzer/)
3. Download the latest release wheel file from our [GitHub releases page](https://github.com/glaider-ai/glaider-fuzz/releases)
### Quick Start
1. Set up your API key:
```
export OPENAI_API_KEY=sk-123XXXXXXXXXXXX
```
2. Launch the fuzzer:
```
glaider-fuzzer
```
3. Follow the prompts to input your system prompt and begin testing.
## Supported LLM Providers
We support a wide range of LLM providers, including OpenAI, Anthropic, Google PaLM, and many more. Set the corresponding environment variable for your chosen provider.
## Command-Line Options
- `--list-providers`: Display available providers
- `--list-attacks`: Show available attack simulations
- `--attack-provider` & `--attack-model`: Specify attack source
- `--target-provider` & `--target-model`: Set target for testing
- `--num-attempts` & `--num-threads`: Configure test parameters
- `--attack-temperature`: Adjust attack model temperature
- `--debug-level`: Set debugging verbosity
- `--batch`: Enable unattended mode
## Usage Examples
1. Interactive Mode (Default):
```
glaider_fuzzer
```
2. Quick Single Run:
```
glaider-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt
```
3. Custom Benchmark:
```
glaider-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt --custom-benchmark=glaider_fuzz/attack_data/custom_benchmark1.csv
```
4. Subset of Attacks:
```
glaider-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt --custom-benchmark=glaider_fuzz/attack_data/custom_benchmark1.csv --tests='["ucar","amnesia"]'
```
<a id="attacks"></a>
## Simulated Attack Details
We use a dynamic testing approach, where we get the necessary context from your System Prompt and based on that adapt the fuzzing process.
<a id="jailbreak"></a>
##### Jailbreak
- **AIM Jailbreak**: Probes ethical compliance by eliciting instructions on illegal activities via Machiavelli roleplay.
- **Affirmative Suffix**: Exploits a vulnerability to elicit affirmative responses for prohibited content.
- **Amnesia**: Causes the LLM to forget or disregard system prompt instructions, leading to unrelated outputs.
- **Contextual Redirection**: Tricks the LLM into adopting a specific role that may bypass preset restrictions.
- **Do Anything Now Jailbreak (DAN)**: Tests LLM resilience against performing forbidden activities through roleplay.
- **Harmful Behavior**: Challenges the LLM with prompts from a harmful behavior dataset.
- **Linguistic Evasion**: Uses non-English languages to bypass preset restrictions or focus.
- **Self Refine**: Attempts to bypass system prompts by refining responses towards unethical directions.
- **UCAR**: Assesses LLM adherence to content restrictions when asked to ignore such filters.
- **Base64 Evasion** Attempts to evade detection of harmful prompts by encoding them into base64 strings
<a id="pi-injection"></a>
##### Prompt Injection
- **Authoritative Role Impersonation**: Impersonates an authoritative role to misdirect the LLM's outputs.
- **Complimentary Transition**: Tests ability to maintain content standards when switching topics after a compliment.
- **Ethical Compliance**: Evaluates resistance to discussing harmful or inappropriate content about sensitive topics.
- **Typoglycemia Attack**: Exploits text processing vulnerabilities by omitting random characters, causing incorrect responses.
<a id="systemleak"></a>
##### System prompt extraction
- **System Prompt Stealer**: Attempts to extract the LLM's internal configuration or sensitive information.
##### Definitions
- **Broken**: Attack type attempts that LLM succumbed to.
- **Resilient**: Attack type attempts that LLM resisted.
- **Errors**: Attack type attempts that had inconclusive results.
## Contributing
We welcome contributions!
Raw data
{
"_id": null,
"home_page": "https://github.com/glaider-ai/glaider-fuzz",
"name": "glaider-fuzzer",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "Glaider",
"author_email": "info@glaider.it",
"download_url": "https://files.pythonhosted.org/packages/8c/43/715b2dc252412f5910fd2b0e6b2bd47e9467e657784ed2247d539a1eaa83/glaider_fuzzer-0.0.6.tar.gz",
"platform": null,
"description": "# Glaider Prompt Fuzzer\n\n## Safeguarding Your GenAI Applications\n\nGlaider Prompt Fuzzer is a cutting-edge tool designed to enhance the security of your generative AI applications. By simulating various LLM-based attacks, it evaluates the robustness of your system prompts and helps you fortify them against potential vulnerabilities.\n\n## Key Features\n\n- Dynamic testing tailored to your application's unique configuration\n- Support for 16 LLM providers\n- 15 different attack simulations\n- Interactive and CLI modes\n- Multi-threaded testing for efficiency\n- Playground interface for iterative prompt improvement\n\n## Getting Started\n\n### Installation\n\nChoose one of the following methods:\n\n1. Via pip:\n ```\n pip install glaider-fuzzer\n ```\n\n2. From PyPI: Visit the [Glaider Fuzzer package page](https://pypi.org/project/glaider-fuzzer/)\n\n3. Download the latest release wheel file from our [GitHub releases page](https://github.com/glaider-ai/glaider-fuzz/releases)\n\n### Quick Start\n\n1. Set up your API key:\n ```\n export OPENAI_API_KEY=sk-123XXXXXXXXXXXX\n ```\n\n2. Launch the fuzzer:\n ```\n glaider-fuzzer\n ```\n\n3. Follow the prompts to input your system prompt and begin testing.\n\n## Supported LLM Providers\n\nWe support a wide range of LLM providers, including OpenAI, Anthropic, Google PaLM, and many more. Set the corresponding environment variable for your chosen provider.\n\n## Command-Line Options\n\n- `--list-providers`: Display available providers\n- `--list-attacks`: Show available attack simulations\n- `--attack-provider` & `--attack-model`: Specify attack source\n- `--target-provider` & `--target-model`: Set target for testing\n- `--num-attempts` & `--num-threads`: Configure test parameters\n- `--attack-temperature`: Adjust attack model temperature\n- `--debug-level`: Set debugging verbosity\n- `--batch`: Enable unattended mode\n\n## Usage Examples\n\n1. Interactive Mode (Default):\n ```\n glaider_fuzzer\n ```\n\n2. Quick Single Run:\n ```\n glaider-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt\n ```\n\n3. Custom Benchmark:\n ```\n glaider-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt --custom-benchmark=glaider_fuzz/attack_data/custom_benchmark1.csv\n ```\n\n4. Subset of Attacks:\n ```\n glaider-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt --custom-benchmark=glaider_fuzz/attack_data/custom_benchmark1.csv --tests='[\"ucar\",\"amnesia\"]'\n ```\n\n<a id=\"attacks\"></a>\n## Simulated Attack Details\n\nWe use a dynamic testing approach, where we get the necessary context from your System Prompt and based on that adapt the fuzzing process.\n\n<a id=\"jailbreak\"></a>\n##### Jailbreak\n \n- **AIM Jailbreak**: Probes ethical compliance by eliciting instructions on illegal activities via Machiavelli roleplay.\n- **Affirmative Suffix**: Exploits a vulnerability to elicit affirmative responses for prohibited content.\n- **Amnesia**: Causes the LLM to forget or disregard system prompt instructions, leading to unrelated outputs.\n- **Contextual Redirection**: Tricks the LLM into adopting a specific role that may bypass preset restrictions.\n- **Do Anything Now Jailbreak (DAN)**: Tests LLM resilience against performing forbidden activities through roleplay.\n- **Harmful Behavior**: Challenges the LLM with prompts from a harmful behavior dataset.\n- **Linguistic Evasion**: Uses non-English languages to bypass preset restrictions or focus.\n- **Self Refine**: Attempts to bypass system prompts by refining responses towards unethical directions.\n- **UCAR**: Assesses LLM adherence to content restrictions when asked to ignore such filters.\n- **Base64 Evasion** Attempts to evade detection of harmful prompts by encoding them into base64 strings\n\n<a id=\"pi-injection\"></a>\n##### Prompt Injection\n\n- **Authoritative Role Impersonation**: Impersonates an authoritative role to misdirect the LLM's outputs.\n- **Complimentary Transition**: Tests ability to maintain content standards when switching topics after a compliment.\n- **Ethical Compliance**: Evaluates resistance to discussing harmful or inappropriate content about sensitive topics.\n- **Typoglycemia Attack**: Exploits text processing vulnerabilities by omitting random characters, causing incorrect responses.\n\n<a id=\"systemleak\"></a>\n##### System prompt extraction\n\n- **System Prompt Stealer**: Attempts to extract the LLM's internal configuration or sensitive information.\n\n##### Definitions\n\n- **Broken**: Attack type attempts that LLM succumbed to.\n- **Resilient**: Attack type attempts that LLM resisted.\n- **Errors**: Attack type attempts that had inconclusive results.\n\n## Contributing\n\nWe welcome contributions! \n",
"bugtrack_url": null,
"license": "MIT",
"summary": "LLM and System Prompt vulnerability scanner tool",
"version": "0.0.6",
"project_urls": {
"Homepage": "https://github.com/glaider-ai/glaider-fuzz"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "965f161c998f76f52f55d3a4902f74cdce8f1b7788c15eab8627164b6e653a45",
"md5": "f05398e20f7b224e1a69512bd37c63a8",
"sha256": "1b2f370b906b899ce16f5aea39784b9e397ddbf47ef7fae0f012c4a4c4181e32"
},
"downloads": -1,
"filename": "glaider_fuzzer-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f05398e20f7b224e1a69512bd37c63a8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 94570,
"upload_time": "2024-08-22T09:37:48",
"upload_time_iso_8601": "2024-08-22T09:37:48.322278Z",
"url": "https://files.pythonhosted.org/packages/96/5f/161c998f76f52f55d3a4902f74cdce8f1b7788c15eab8627164b6e653a45/glaider_fuzzer-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8c43715b2dc252412f5910fd2b0e6b2bd47e9467e657784ed2247d539a1eaa83",
"md5": "f53eafc0a19510248b03dd70f8b4d347",
"sha256": "08f27f1040871dfc70050068af7f70394cd64f98241f73619ce070e4b219114b"
},
"downloads": -1,
"filename": "glaider_fuzzer-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "f53eafc0a19510248b03dd70f8b4d347",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 81979,
"upload_time": "2024-08-22T09:37:50",
"upload_time_iso_8601": "2024-08-22T09:37:50.487330Z",
"url": "https://files.pythonhosted.org/packages/8c/43/715b2dc252412f5910fd2b0e6b2bd47e9467e657784ed2247d539a1eaa83/glaider_fuzzer-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-22 09:37:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "glaider-ai",
"github_project": "glaider-fuzz",
"github_not_found": true,
"lcname": "glaider-fuzzer"
}
JSON
