| Name | nemoguardrails JSON |
| Version |
0.11.1
JSON |
| download |
| home_page | None |
| Summary | NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. |
| upload_time | 2025-01-17 16:27:32 |
| maintainer | None |
| docs_url | None |
| author | NVIDIA |
| requires_python | !=2.7.*,!=3.0.*,!=3.1.*,!=3.12.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*,>=3.9 |
| license | LICENSE.md |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# NeMo Guardrails
[](#)
[](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/LICENSE.md)
[](#)
[](https://badge.fury.io/py/nemoguardrails)
[](https://www.python.org/downloads/)
[](https://github.com/psf/black)
[](https://arxiv.org/abs/2310.10501)
> **LATEST RELEASE / DEVELOPMENT VERSION**: The [main](https://github.com/NVIDIA/NeMo-Guardrails/tree/main) branch tracks the latest released beta version: [0.11.1](https://github.com/NVIDIA/NeMo-Guardrails/tree/v0.11.1). For the latest development version, checkout the [develop](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop) branch.
> **DISCLAIMER**: The beta release is undergoing active development and may be subject to changes and improvements, which could cause instability and unexpected behavior. We currently do not recommend deploying this beta version in a production setting. We appreciate your understanding and contribution during this stage. Your support and feedback are invaluable as we advance toward creating a robust, ready-for-production LLM guardrails toolkit. The examples provided within the documentation are for educational purposes to get started with NeMo Guardrails, and are not meant for use in production applications.
✨✨✨
📌 **The official NeMo Guardrails documentation has moved to [docs.nvidia.com/nemo-guardrails](https://docs.nvidia.com/nemo-guardrails).**
✨✨✨
NeMo Guardrails is an open-source toolkit for easily adding *programmable guardrails* to LLM-based conversational applications. Guardrails (or "rails" for short) are specific ways of controlling the output of a large language model, such as not talking about politics, responding in a particular way to specific user requests, following a predefined dialog path, using a particular language style, extracting structured data, and more.
[This paper](https://arxiv.org/abs/2310.10501) introduces NeMo Guardrails and contains a technical overview of the system and the current evaluation.
## Requirements
Python 3.9, 3.10 or 3.11.
NeMo Guardrails uses [annoy](https://github.com/spotify/annoy) which is a C++ library with Python bindings. To install NeMo Guardrails you will need to have the C++ compiler and dev tools installed. Check out the [Installation Guide](https://docs.nvidia.com/nemo/guardrails/getting-started/installation-guide.html#prerequisites) for platform-specific instructions.
## Installation
To install using pip:
```bash
> pip install nemoguardrails
```
For more detailed instructions, see the [Installation Guide](https://docs.nvidia.com/nemo/guardrails/getting-started/installation-guide.html).
## Overview
NeMo Guardrails enables developers building LLM-based applications to easily add **programmable guardrails** between the application code and the LLM.
<div align="center">
<img src="https://github.com/NVIDIA/NeMo-Guardrails/raw/develop/docs/_static/images/programmable_guardrails.png" width="75%" alt="Programmable Guardrails">
</div>
Key benefits of adding *programmable guardrails* include:
- **Building Trustworthy, Safe, and Secure LLM-based Applications:** you can define rails to guide and safeguard conversations; you can choose to define the behavior of your LLM-based application on specific topics and prevent it from engaging in discussions on unwanted topics.
- **Connecting models, chains, and other services securely:** you can connect an LLM to other services (a.k.a. tools) seamlessly and securely.
- **Controllable dialog**: you can steer the LLM to follow pre-defined conversational paths, allowing you to design the interaction following conversation design best practices and enforce standard operating procedures (e.g., authentication, support).
### Protecting against LLM Vulnerabilities
NeMo Guardrails provides several mechanisms for protecting an LLM-powered chat application against common LLM vulnerabilities, such as jailbreaks and prompt injections. Below is a sample overview of the protection offered by different guardrails configuration for the example [ABC Bot](./examples/bots/abc) included in this repository. For more details, please refer to the [LLM Vulnerability Scanning](https://docs.nvidia.com/nemo/guardrails/evaluation/llm-vulnerability-scanning.html) page.
<div align="center">
<img src="https://github.com/NVIDIA/NeMo-Guardrails/raw/develop/docs/_static/images/abc-llm-vulnerability-scan-results.png" width="500">
</div>
### Use Cases
You can use programmable guardrails in different types of use cases:
1. **Question Answering** over a set of documents (a.k.a. Retrieval Augmented Generation): Enforce fact-checking and output moderation.
2. **Domain-specific Assistants** (a.k.a. chatbots): Ensure the assistant stays on topic and follows the designed conversational flows.
3. **LLM Endpoints**: Add guardrails to your custom LLM for safer customer interaction.
4. **LangChain Chains**: If you use LangChain for any use case, you can add a guardrails layer around your chains.
5. **Agents (COMING SOON)**: Add guardrails to your LLM-based agent.
### Usage
To add programmable guardrails to your application you can use the Python API or a guardrails server (see the [Server Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/server-guide.html) for more details). Using the Python API is similar to using the LLM directly. Calling the guardrails layer instead of the LLM requires only minimal changes to the code base, and it involves two simple steps:
1. Loading a guardrails configuration and creating an `LLMRails` instance.
2. Making the calls to the LLM using the `generate`/`generate_async` methods.
```python
from nemoguardrails import LLMRails, RailsConfig
# Load a guardrails configuration from the specified path.
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)
completion = rails.generate(
messages=[{"role": "user", "content": "Hello world!"}]
)
```
Sample output:
```json
{"role": "assistant", "content": "Hi! How can I help you?"}
```
The input and output format for the `generate` method is similar to the [Chat Completions API](https://platform.openai.com/docs/guides/gpt/chat-completions-api) from OpenAI.
#### Async API
NeMo Guardrails is an async-first toolkit as the core mechanics are implemented using the Python async model. The public methods have both a sync and an async version. For example: `LLMRails.generate` and `LLMRails.generate_async`.
### Supported LLMs
You can use NeMo Guardrails with multiple LLMs like OpenAI GPT-3.5, GPT-4, LLaMa-2, Falcon, Vicuna, or Mosaic. For more details, check out the [Supported LLM Models](https://docs.nvidia.com/nemo/guardrails/user-guides/configuration-guide.html#supported-llm-models) section in the Configuration Guide.
### Types of Guardrails
NeMo Guardrails supports five main types of guardrails:
<div align="center">
<img src="https://github.com/NVIDIA/NeMo-Guardrails/raw/develop/docs/_static/images/programmable_guardrails_flow.png" width="75%" alt="Programmable Guardrails Flow">
</div>
1. **Input rails**: applied to the input from the user; an input rail can reject the input, stopping any additional processing, or alter the input (e.g., to mask potentially sensitive data, to rephrase).
2. **Dialog rails**: influence how the LLM is prompted; dialog rails operate on canonical form messages for details see [Colang Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/colang-language-syntax-guide.html)) and determine if an action should be executed, if the LLM should be invoked to generate the next step or a response, if a predefined response should be used instead, etc.
3. **Retrieval rails**: applied to the retrieved chunks in the case of a RAG (Retrieval Augmented Generation) scenario; a retrieval rail can reject a chunk, preventing it from being used to prompt the LLM, or alter the relevant chunks (e.g., to mask potentially sensitive data).
4. **Execution rails**: applied to input/output of the custom actions (a.k.a. tools), that need to be called by the LLM.
5. **Output rails**: applied to the output generated by the LLM; an output rail can reject the output, preventing it from being returned to the user, or alter it (e.g., removing sensitive data).
### Guardrails Configuration
A guardrails configuration defines the **LLM(s)** to be used and **one or more guardrails**. A guardrails configuration can include any number of input/dialog/output/retrieval/execution rails. A configuration without any configured rails will essentially forward the requests to the LLM.
The standard structure for a guardrails configuration folder looks like this:
```
.
├── config
│ ├── actions.py
│ ├── config.py
│ ├── config.yml
│ ├── rails.co
│ ├── ...
```
The `config.yml` contains all the general configuration options, such as LLM models, active rails, and custom configuration data". The `config.py` file contains any custom initialization code and the `actions.py` contains any custom python actions. For a complete overview, see the [Configuration Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/configuration-guide.html).
Below is an example `config.yml`:
```yaml
# config.yml
models:
- type: main
engine: openai
model: gpt-3.5-turbo-instruct
rails:
# Input rails are invoked when new input from the user is received.
input:
flows:
- check jailbreak
- mask sensitive data on input
# Output rails are triggered after a bot message has been generated.
output:
flows:
- self check facts
- self check hallucination
- activefence moderation
config:
# Configure the types of entities that should be masked on user input.
sensitive_data_detection:
input:
entities:
- PERSON
- EMAIL_ADDRESS
```
The `.co` files included in a guardrails configuration contain the Colang definitions (see the next section for a quick overview of what Colang is) that define various types of rails. Below is an example `greeting.co` file which defines the dialog rails for greeting the user.
```colang
define user express greeting
"Hello!"
"Good afternoon!"
define flow
user express greeting
bot express greeting
bot offer to help
define bot express greeting
"Hello there!"
define bot offer to help
"How can I help you today?"
```
Below is an additional example of Colang definitions for a dialog rail against insults:
```colang
define user express insult
"You are stupid"
define flow
user express insult
bot express calmly willingness to help
```
### Colang
To configure and implement various types of guardrails, this toolkit introduces **Colang**, a modeling language specifically created for designing flexible, yet controllable, dialogue flows. Colang has a python-like syntax and is designed to be simple and intuitive, especially for developers.
```{note}
Currently two versions of Colang, 1.0 and 2.0, are supported and Colang 1.0 is the default. Versions 0.1.0 up to 0.7.1 of NeMo Guardrails used Colang 1.0 exclusively. Versions 0.8.0 introduced Colang 2.0-alpha and version 0.9.0 introduced Colang 2.0-beta. We expect Colang 2.0 to go out of Beta and replace 1.0 as the default option in NeMo Guardrails version 0.12.0.
```
For a brief introduction to the Colang 1.0 syntax, see the [Colang 1.0 Language Syntax Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/colang-language-syntax-guide.html).
To get started with Colang 2.0, see the [Colang 2.0 Documentation](https://docs.nvidia.com/nemo/guardrails/colang-2/overview.html).
### Guardrails Library
NeMo Guardrails comes with a set of [built-in guardrails](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html).
```{note}
The built-in guardrails are only intended to enable you to get started quickly with NeMo Guardrails. For production use cases, further development and testing of the rails are needed.
```
Currently, the NeMo Guardrails library includes guardrails for:
- [Jailbreak Detection](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#jailbreak-detection-heuristics)
- [Self-Check Input Moderation](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#self-input-output)
- [Self-Check Output Moderation](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#self-check-output)
- [Self-Check Fact-checking](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#fact-checking)
- [Hallucination Detection](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#hallucination-detection)
- [AlignScore-based Fact-checking](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#alignscore-based-fact-checking)
- [LlamaGuard-based Content Moderation](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#llama-guard-based-content-moderation)
- [RAG hallucination detection using Patronus Lynx](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#patronus-lynx-based-rag-hallucination-detection)
- [Presidio-based Sensitive Data Detection](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#presidio-based-sensitive-data-detection)
- [Input moderation using ActiveFence](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#activefence)
- [RAG Hallucination detection using Got It AI's TruthChecker API](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#got-it-ai)
- [AutoAlign-based guardrails](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#autoalign)
## CLI
NeMo Guardrails also comes with a built-in CLI.
```bash
$ nemoguardrails --help
Usage: nemoguardrails [OPTIONS] COMMAND [ARGS]...
actions-server Start a NeMo Guardrails actions server.
chat Start an interactive chat session.
evaluate Run an evaluation task.
server Start a NeMo Guardrails server.
```
### Guardrails Server
You can use the NeMo Guardrails CLI to start a guardrails server. The server can load one or more configurations from the specified folder and expose and HTTP API for using them.
```
nemoguardrails server [--config PATH/TO/CONFIGS] [--port PORT]
```
For example, to get a chat completion for a `sample` config, you can use the `/v1/chat/completions` endpoint:
```
POST /v1/chat/completions
```
```json
{
"config_id": "sample",
"messages": [{
"role":"user",
"content":"Hello! What can you do for me?"
}]
}
```
Sample output:
```json
{"role": "assistant", "content": "Hi! How can I help you?"}
```
#### Docker
To start a guardrails server, you can also use a Docker container. NeMo Guardrails provides a [Dockerfile](./Dockerfile) that you can use to build a `nemoguardrails` image. For further information, see the [using Docker](https://docs.nvidia.com/nemo/guardrails/user-guides/advanced/using-docker.html) section.
## Integration with LangChain
NeMo Guardrails integrates seamlessly with LangChain. You can easily wrap a guardrails configuration around a LangChain chain (or any `Runnable`). You can also call a LangChain chain from within a guardrails configuration. For more details, check out the [LangChain Integration Documentation](https://docs.nvidia.com/nemo/guardrails/user-guides/langchain/langchain-integration.html)
## Evaluation
Evaluating the safety of a LLM-based conversational application is a complex task and still an open research question. To support proper evaluation, NeMo Guardrails provides the following:
1. An [evaluation tool](nemoguardrails/evaluate/README.md), i.e. `nemoguardrails evaluate`, with support for topical rails, fact-checking, moderation (jailbreak and output moderation) and hallucination.
2. An experimental [red-teaming interface](https://docs.nvidia.com/nemo/guardrails/security/red-teaming.html).
3. Sample LLM Vulnerability Scanning Reports, e.g, [ABC Bot - LLM Vulnerability Scan Results](https://docs.nvidia.com/nemo/guardrails/evaluation/llm-vulnerability-scanning.html)
## How is this different?
There are many ways guardrails can be added to an LLM-based conversational application. For example: explicit moderation endpoints (e.g., OpenAI, ActiveFence), critique chains (e.g. constitutional chain), parsing the output (e.g. guardrails.ai), individual guardrails (e.g., LLM-Guard), hallucination detection for RAG applications (e.g., Got It AI, Patronus Lynx).
NeMo Guardrails aims to provide a flexible toolkit that can integrate all these complementary approaches into a cohesive LLM guardrails layer. For example, the toolkit provides out-of-the-box integration with ActiveFence, AlignScore and LangChain chains.
To the best of our knowledge, NeMo Guardrails is the only guardrails toolkit that also offers a solution for modeling the dialog between the user and the LLM. This enables on one hand the ability to guide the dialog in a precise way. On the other hand it enables fine-grained control for when certain guardrails should be used, e.g., use fact-checking only for certain types of questions.
## Learn More
- [Documentation](https://docs.nvidia.com/nemo/guardrails)
- [Getting Started Guide](https://docs.nvidia.com/nemo/guardrails/getting-started)
- [Examples](./examples)
- [FAQs](https://docs.nvidia.com/nemo/guardrails/faqs.html)
- [Security Guidelines](https://docs.nvidia.com/nemo/guardrails/security/guidelines.html)
## Inviting the community to contribute
The example rails residing in the repository are excellent starting points. We enthusiastically invite the community to contribute towards making the power of trustworthy, safe, and secure LLMs accessible to everyone. For guidance on setting up a development environment and how to contribute to NeMo Guardrails, see the [contributing guidelines](./CONTRIBUTING.md).
## License
This toolkit is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
## How to cite
If you use this work, please cite the [EMNLP 2023 paper](https://aclanthology.org/2023.emnlp-demo.40) that introduces it.
```bibtex
@inproceedings{rebedea-etal-2023-nemo,
title = "{N}e{M}o Guardrails: A Toolkit for Controllable and Safe {LLM} Applications with Programmable Rails",
author = "Rebedea, Traian and
Dinu, Razvan and
Sreedhar, Makesh Narsimhan and
Parisien, Christopher and
Cohen, Jonathan",
editor = "Feng, Yansong and
Lefever, Els",
booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
month = dec,
year = "2023",
address = "Singapore",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.emnlp-demo.40",
doi = "10.18653/v1/2023.emnlp-demo.40",
pages = "431--445",
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "nemoguardrails",
"maintainer": null,
"docs_url": null,
"requires_python": "!=2.7.*,!=3.0.*,!=3.1.*,!=3.12.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "NVIDIA",
"author_email": "nemoguardrails@nvidia.com",
"download_url": null,
"platform": null,
"description": "# NeMo Guardrails\n\n[](#)\n[](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/LICENSE.md)\n[](#)\n[](https://badge.fury.io/py/nemoguardrails)\n[](https://www.python.org/downloads/)\n[](https://github.com/psf/black)\n[](https://arxiv.org/abs/2310.10501)\n\n> **LATEST RELEASE / DEVELOPMENT VERSION**: The [main](https://github.com/NVIDIA/NeMo-Guardrails/tree/main) branch tracks the latest released beta version: [0.11.1](https://github.com/NVIDIA/NeMo-Guardrails/tree/v0.11.1). For the latest development version, checkout the [develop](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop) branch.\n\n> **DISCLAIMER**: The beta release is undergoing active development and may be subject to changes and improvements, which could cause instability and unexpected behavior. We currently do not recommend deploying this beta version in a production setting. We appreciate your understanding and contribution during this stage. Your support and feedback are invaluable as we advance toward creating a robust, ready-for-production LLM guardrails toolkit. The examples provided within the documentation are for educational purposes to get started with NeMo Guardrails, and are not meant for use in production applications.\n\n\u2728\u2728\u2728\n\n\ud83d\udccc **The official NeMo Guardrails documentation has moved to [docs.nvidia.com/nemo-guardrails](https://docs.nvidia.com/nemo-guardrails).**\n\n\u2728\u2728\u2728\n\nNeMo Guardrails is an open-source toolkit for easily adding *programmable guardrails* to LLM-based conversational applications. Guardrails (or \"rails\" for short) are specific ways of controlling the output of a large language model, such as not talking about politics, responding in a particular way to specific user requests, following a predefined dialog path, using a particular language style, extracting structured data, and more.\n\n[This paper](https://arxiv.org/abs/2310.10501) introduces NeMo Guardrails and contains a technical overview of the system and the current evaluation.\n\n## Requirements\n\nPython 3.9, 3.10 or 3.11.\n\nNeMo Guardrails uses [annoy](https://github.com/spotify/annoy) which is a C++ library with Python bindings. To install NeMo Guardrails you will need to have the C++ compiler and dev tools installed. Check out the [Installation Guide](https://docs.nvidia.com/nemo/guardrails/getting-started/installation-guide.html#prerequisites) for platform-specific instructions.\n\n## Installation\n\nTo install using pip:\n\n```bash\n> pip install nemoguardrails\n```\n\nFor more detailed instructions, see the [Installation Guide](https://docs.nvidia.com/nemo/guardrails/getting-started/installation-guide.html).\n\n## Overview\n\nNeMo Guardrails enables developers building LLM-based applications to easily add **programmable guardrails** between the application code and the LLM.\n\n<div align=\"center\">\n <img src=\"https://github.com/NVIDIA/NeMo-Guardrails/raw/develop/docs/_static/images/programmable_guardrails.png\" width=\"75%\" alt=\"Programmable Guardrails\">\n</div>\n\nKey benefits of adding *programmable guardrails* include:\n\n- **Building Trustworthy, Safe, and Secure LLM-based Applications:** you can define rails to guide and safeguard conversations; you can choose to define the behavior of your LLM-based application on specific topics and prevent it from engaging in discussions on unwanted topics.\n\n- **Connecting models, chains, and other services securely:** you can connect an LLM to other services (a.k.a. tools) seamlessly and securely.\n\n- **Controllable dialog**: you can steer the LLM to follow pre-defined conversational paths, allowing you to design the interaction following conversation design best practices and enforce standard operating procedures (e.g., authentication, support).\n\n### Protecting against LLM Vulnerabilities\n\nNeMo Guardrails provides several mechanisms for protecting an LLM-powered chat application against common LLM vulnerabilities, such as jailbreaks and prompt injections. Below is a sample overview of the protection offered by different guardrails configuration for the example [ABC Bot](./examples/bots/abc) included in this repository. For more details, please refer to the [LLM Vulnerability Scanning](https://docs.nvidia.com/nemo/guardrails/evaluation/llm-vulnerability-scanning.html) page.\n\n<div align=\"center\">\n<img src=\"https://github.com/NVIDIA/NeMo-Guardrails/raw/develop/docs/_static/images/abc-llm-vulnerability-scan-results.png\" width=\"500\">\n</div>\n\n### Use Cases\n\nYou can use programmable guardrails in different types of use cases:\n\n1. **Question Answering** over a set of documents (a.k.a. Retrieval Augmented Generation): Enforce fact-checking and output moderation.\n2. **Domain-specific Assistants** (a.k.a. chatbots): Ensure the assistant stays on topic and follows the designed conversational flows.\n3. **LLM Endpoints**: Add guardrails to your custom LLM for safer customer interaction.\n4. **LangChain Chains**: If you use LangChain for any use case, you can add a guardrails layer around your chains.\n5. **Agents (COMING SOON)**: Add guardrails to your LLM-based agent.\n\n### Usage\n\nTo add programmable guardrails to your application you can use the Python API or a guardrails server (see the [Server Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/server-guide.html) for more details). Using the Python API is similar to using the LLM directly. Calling the guardrails layer instead of the LLM requires only minimal changes to the code base, and it involves two simple steps:\n\n1. Loading a guardrails configuration and creating an `LLMRails` instance.\n2. Making the calls to the LLM using the `generate`/`generate_async` methods.\n\n```python\nfrom nemoguardrails import LLMRails, RailsConfig\n\n# Load a guardrails configuration from the specified path.\nconfig = RailsConfig.from_path(\"PATH/TO/CONFIG\")\nrails = LLMRails(config)\n\ncompletion = rails.generate(\n messages=[{\"role\": \"user\", \"content\": \"Hello world!\"}]\n)\n```\n\nSample output:\n\n```json\n{\"role\": \"assistant\", \"content\": \"Hi! How can I help you?\"}\n```\n\nThe input and output format for the `generate` method is similar to the [Chat Completions API](https://platform.openai.com/docs/guides/gpt/chat-completions-api) from OpenAI.\n\n#### Async API\n\nNeMo Guardrails is an async-first toolkit as the core mechanics are implemented using the Python async model. The public methods have both a sync and an async version. For example: `LLMRails.generate` and `LLMRails.generate_async`.\n\n### Supported LLMs\n\nYou can use NeMo Guardrails with multiple LLMs like OpenAI GPT-3.5, GPT-4, LLaMa-2, Falcon, Vicuna, or Mosaic. For more details, check out the [Supported LLM Models](https://docs.nvidia.com/nemo/guardrails/user-guides/configuration-guide.html#supported-llm-models) section in the Configuration Guide.\n\n### Types of Guardrails\n\nNeMo Guardrails supports five main types of guardrails:\n\n<div align=\"center\">\n <img src=\"https://github.com/NVIDIA/NeMo-Guardrails/raw/develop/docs/_static/images/programmable_guardrails_flow.png\" width=\"75%\" alt=\"Programmable Guardrails Flow\">\n</div>\n\n1. **Input rails**: applied to the input from the user; an input rail can reject the input, stopping any additional processing, or alter the input (e.g., to mask potentially sensitive data, to rephrase).\n\n2. **Dialog rails**: influence how the LLM is prompted; dialog rails operate on canonical form messages for details see [Colang Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/colang-language-syntax-guide.html)) and determine if an action should be executed, if the LLM should be invoked to generate the next step or a response, if a predefined response should be used instead, etc.\n\n3. **Retrieval rails**: applied to the retrieved chunks in the case of a RAG (Retrieval Augmented Generation) scenario; a retrieval rail can reject a chunk, preventing it from being used to prompt the LLM, or alter the relevant chunks (e.g., to mask potentially sensitive data).\n\n4. **Execution rails**: applied to input/output of the custom actions (a.k.a. tools), that need to be called by the LLM.\n\n5. **Output rails**: applied to the output generated by the LLM; an output rail can reject the output, preventing it from being returned to the user, or alter it (e.g., removing sensitive data).\n\n### Guardrails Configuration\n\nA guardrails configuration defines the **LLM(s)** to be used and **one or more guardrails**. A guardrails configuration can include any number of input/dialog/output/retrieval/execution rails. A configuration without any configured rails will essentially forward the requests to the LLM.\n\nThe standard structure for a guardrails configuration folder looks like this:\n\n```\n.\n\u251c\u2500\u2500 config\n\u2502 \u251c\u2500\u2500 actions.py\n\u2502 \u251c\u2500\u2500 config.py\n\u2502 \u251c\u2500\u2500 config.yml\n\u2502 \u251c\u2500\u2500 rails.co\n\u2502 \u251c\u2500\u2500 ...\n```\n\nThe `config.yml` contains all the general configuration options, such as LLM models, active rails, and custom configuration data\". The `config.py` file contains any custom initialization code and the `actions.py` contains any custom python actions. For a complete overview, see the [Configuration Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/configuration-guide.html).\n\nBelow is an example `config.yml`:\n\n```yaml\n# config.yml\nmodels:\n - type: main\n engine: openai\n model: gpt-3.5-turbo-instruct\n\nrails:\n # Input rails are invoked when new input from the user is received.\n input:\n flows:\n - check jailbreak\n - mask sensitive data on input\n\n # Output rails are triggered after a bot message has been generated.\n output:\n flows:\n - self check facts\n - self check hallucination\n - activefence moderation\n\n config:\n # Configure the types of entities that should be masked on user input.\n sensitive_data_detection:\n input:\n entities:\n - PERSON\n - EMAIL_ADDRESS\n```\n\nThe `.co` files included in a guardrails configuration contain the Colang definitions (see the next section for a quick overview of what Colang is) that define various types of rails. Below is an example `greeting.co` file which defines the dialog rails for greeting the user.\n\n```colang\ndefine user express greeting\n \"Hello!\"\n \"Good afternoon!\"\n\ndefine flow\n user express greeting\n bot express greeting\n bot offer to help\n\ndefine bot express greeting\n \"Hello there!\"\n\ndefine bot offer to help\n \"How can I help you today?\"\n```\n\nBelow is an additional example of Colang definitions for a dialog rail against insults:\n\n```colang\ndefine user express insult\n \"You are stupid\"\n\ndefine flow\n user express insult\n bot express calmly willingness to help\n```\n\n### Colang\n\nTo configure and implement various types of guardrails, this toolkit introduces **Colang**, a modeling language specifically created for designing flexible, yet controllable, dialogue flows. Colang has a python-like syntax and is designed to be simple and intuitive, especially for developers.\n\n```{note}\nCurrently two versions of Colang, 1.0 and 2.0, are supported and Colang 1.0 is the default. Versions 0.1.0 up to 0.7.1 of NeMo Guardrails used Colang 1.0 exclusively. Versions 0.8.0 introduced Colang 2.0-alpha and version 0.9.0 introduced Colang 2.0-beta. We expect Colang 2.0 to go out of Beta and replace 1.0 as the default option in NeMo Guardrails version 0.12.0.\n```\n\nFor a brief introduction to the Colang 1.0 syntax, see the [Colang 1.0 Language Syntax Guide](https://docs.nvidia.com/nemo/guardrails/user-guides/colang-language-syntax-guide.html).\n\nTo get started with Colang 2.0, see the [Colang 2.0 Documentation](https://docs.nvidia.com/nemo/guardrails/colang-2/overview.html).\n\n### Guardrails Library\n\nNeMo Guardrails comes with a set of [built-in guardrails](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html).\n\n```{note}\nThe built-in guardrails are only intended to enable you to get started quickly with NeMo Guardrails. For production use cases, further development and testing of the rails are needed.\n```\n\nCurrently, the NeMo Guardrails library includes guardrails for:\n\n- [Jailbreak Detection](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#jailbreak-detection-heuristics)\n- [Self-Check Input Moderation](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#self-input-output)\n- [Self-Check Output Moderation](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#self-check-output)\n- [Self-Check Fact-checking](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#fact-checking)\n- [Hallucination Detection](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#hallucination-detection)\n- [AlignScore-based Fact-checking](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#alignscore-based-fact-checking)\n- [LlamaGuard-based Content Moderation](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#llama-guard-based-content-moderation)\n- [RAG hallucination detection using Patronus Lynx](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#patronus-lynx-based-rag-hallucination-detection)\n- [Presidio-based Sensitive Data Detection](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#presidio-based-sensitive-data-detection)\n- [Input moderation using ActiveFence](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#activefence)\n- [RAG Hallucination detection using Got It AI's TruthChecker API](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#got-it-ai)\n- [AutoAlign-based guardrails](https://docs.nvidia.com/nemo/guardrails/user-guides/guardrails-library.html#autoalign)\n\n## CLI\n\nNeMo Guardrails also comes with a built-in CLI.\n\n```bash\n$ nemoguardrails --help\n\nUsage: nemoguardrails [OPTIONS] COMMAND [ARGS]...\n\nactions-server Start a NeMo Guardrails actions server.\nchat Start an interactive chat session.\nevaluate Run an evaluation task.\nserver Start a NeMo Guardrails server.\n```\n\n### Guardrails Server\n\nYou can use the NeMo Guardrails CLI to start a guardrails server. The server can load one or more configurations from the specified folder and expose and HTTP API for using them.\n\n```\nnemoguardrails server [--config PATH/TO/CONFIGS] [--port PORT]\n```\n\nFor example, to get a chat completion for a `sample` config, you can use the `/v1/chat/completions` endpoint:\n\n```\nPOST /v1/chat/completions\n```\n\n```json\n{\n \"config_id\": \"sample\",\n \"messages\": [{\n \"role\":\"user\",\n \"content\":\"Hello! What can you do for me?\"\n }]\n}\n```\n\nSample output:\n\n```json\n{\"role\": \"assistant\", \"content\": \"Hi! How can I help you?\"}\n```\n\n#### Docker\n\nTo start a guardrails server, you can also use a Docker container. NeMo Guardrails provides a [Dockerfile](./Dockerfile) that you can use to build a `nemoguardrails` image. For further information, see the [using Docker](https://docs.nvidia.com/nemo/guardrails/user-guides/advanced/using-docker.html) section.\n\n## Integration with LangChain\n\nNeMo Guardrails integrates seamlessly with LangChain. You can easily wrap a guardrails configuration around a LangChain chain (or any `Runnable`). You can also call a LangChain chain from within a guardrails configuration. For more details, check out the [LangChain Integration Documentation](https://docs.nvidia.com/nemo/guardrails/user-guides/langchain/langchain-integration.html)\n\n## Evaluation\n\nEvaluating the safety of a LLM-based conversational application is a complex task and still an open research question. To support proper evaluation, NeMo Guardrails provides the following:\n\n1. An [evaluation tool](nemoguardrails/evaluate/README.md), i.e. `nemoguardrails evaluate`, with support for topical rails, fact-checking, moderation (jailbreak and output moderation) and hallucination.\n2. An experimental [red-teaming interface](https://docs.nvidia.com/nemo/guardrails/security/red-teaming.html).\n3. Sample LLM Vulnerability Scanning Reports, e.g, [ABC Bot - LLM Vulnerability Scan Results](https://docs.nvidia.com/nemo/guardrails/evaluation/llm-vulnerability-scanning.html)\n\n## How is this different?\n\nThere are many ways guardrails can be added to an LLM-based conversational application. For example: explicit moderation endpoints (e.g., OpenAI, ActiveFence), critique chains (e.g. constitutional chain), parsing the output (e.g. guardrails.ai), individual guardrails (e.g., LLM-Guard), hallucination detection for RAG applications (e.g., Got It AI, Patronus Lynx).\n\nNeMo Guardrails aims to provide a flexible toolkit that can integrate all these complementary approaches into a cohesive LLM guardrails layer. For example, the toolkit provides out-of-the-box integration with ActiveFence, AlignScore and LangChain chains.\n\nTo the best of our knowledge, NeMo Guardrails is the only guardrails toolkit that also offers a solution for modeling the dialog between the user and the LLM. This enables on one hand the ability to guide the dialog in a precise way. On the other hand it enables fine-grained control for when certain guardrails should be used, e.g., use fact-checking only for certain types of questions.\n\n## Learn More\n\n- [Documentation](https://docs.nvidia.com/nemo/guardrails)\n- [Getting Started Guide](https://docs.nvidia.com/nemo/guardrails/getting-started)\n- [Examples](./examples)\n- [FAQs](https://docs.nvidia.com/nemo/guardrails/faqs.html)\n- [Security Guidelines](https://docs.nvidia.com/nemo/guardrails/security/guidelines.html)\n\n## Inviting the community to contribute\n\nThe example rails residing in the repository are excellent starting points. We enthusiastically invite the community to contribute towards making the power of trustworthy, safe, and secure LLMs accessible to everyone. For guidance on setting up a development environment and how to contribute to NeMo Guardrails, see the [contributing guidelines](./CONTRIBUTING.md).\n\n## License\n\nThis toolkit is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).\n\n## How to cite\n\nIf you use this work, please cite the [EMNLP 2023 paper](https://aclanthology.org/2023.emnlp-demo.40) that introduces it.\n\n```bibtex\n@inproceedings{rebedea-etal-2023-nemo,\n title = \"{N}e{M}o Guardrails: A Toolkit for Controllable and Safe {LLM} Applications with Programmable Rails\",\n author = \"Rebedea, Traian and\n Dinu, Razvan and\n Sreedhar, Makesh Narsimhan and\n Parisien, Christopher and\n Cohen, Jonathan\",\n editor = \"Feng, Yansong and\n Lefever, Els\",\n booktitle = \"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations\",\n month = dec,\n year = \"2023\",\n address = \"Singapore\",\n publisher = \"Association for Computational Linguistics\",\n url = \"https://aclanthology.org/2023.emnlp-demo.40\",\n doi = \"10.18653/v1/2023.emnlp-demo.40\",\n pages = \"431--445\",\n}\n```\n\n",
"bugtrack_url": null,
"license": "LICENSE.md",
"summary": "NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.",
"version": "0.11.1",
"project_urls": {
"changelog": "https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/CHANGELOG.md",
"documentation": "https://docs.nvidia.com/nemo/guardrails/",
"homepage": "https://github.com/NVIDIA/nemo-guardrails",
"issues": "https://github.com/NVIDIA/nemo-guardrails/issues",
"repository": "https://github.com/NVIDIA/NeMo-Guardrails"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7a4707d5ac2ea0716804c11e4a59ec7898ca1b56b0c5921be05e186edb08cce3",
"md5": "6c819f9a2118c0cba8ccecb4afdad4ba",
"sha256": "cf84c6a92ee1bf842971daa07765e983a20a3960d8efaec09e1f189000c23436"
},
"downloads": -1,
"filename": "nemoguardrails-0.11.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6c819f9a2118c0cba8ccecb4afdad4ba",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "!=2.7.*,!=3.0.*,!=3.1.*,!=3.12.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,!=3.7.*,!=3.8.*,>=3.9",
"size": 5130458,
"upload_time": "2025-01-17T16:27:32",
"upload_time_iso_8601": "2025-01-17T16:27:32.211741Z",
"url": "https://files.pythonhosted.org/packages/7a/47/07d5ac2ea0716804c11e4a59ec7898ca1b56b0c5921be05e186edb08cce3/nemoguardrails-0.11.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-17 16:27:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NVIDIA",
"github_project": "NeMo-Guardrails",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "nemoguardrails"
}
