<a name="readme-top"></a>
<div align="center">
<a href="https://github.com/Addepar/RedFlag">
<img src="https://raw.githubusercontent.com/Addepar/RedFlag/main/docs/images/RedFlag-Logo.png" alt="RedFlag" height="100">
</a>
[![Python][python-shield]][python-url]
[![Contributors][contributors-shield]][contributors-url]
[![MIT License][license-shield]][license-url]
[![LinkedIn][linkedin-shield]][linkedin-url]<br />
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
RedFlag leverages AI to determine high-risk code changes.
Run it in batch mode to scope manual security testing of release candidates,
or run it in your CI pipelines to flag PRs and add the appropriate reviewers.
Despite being a security tool, RedFlag can be leveraged for almost any team
as it's configuration makes it infinitely flexible.
<br />
<a href="https://addepar.com/blog/introducing-redflag-using-ai-to-scale-addepar-s-offensive-security-team"><strong>Read the blog post »</strong></a>
<br />
<br />
<a href="https://opensource.addepar.com/RedFlag/">View Sample Report</a>
·
<a href="https://github.com/Addepar/RedFlag/issues/new?labels=bug&template=bug-report---.md">Report Bug</a>
·
<a href="https://github.com/Addepar/RedFlag/issues/new?labels=enhancement&template=feature-request---.md">Request Feature</a>
</div>
<details>
<summary><strong>Table of Contents</strong></summary>
<ul>
<li><a href="#batch-mode">Batch Mode</a></li>
<li><a href="#ci-mode">CI Mode</a></li>
<li><a href="#evaluation-mode">Evaluation Mode</a></li>
<li><a href="#advanced-configuration">Advanced Configuration</a></li>
<li><a href="#contributing">Contributing</a></li>
<li><a href="#license">License</a></li>
<li><a href="#contact">Contact</a></li>
</ul>
</details>
---
# Batch Mode
RedFlag is able to analyze a large number of commits in a single run.
These commits can be specified using commit hashes, branch names, or tags.
This is useful for scoping manual security testing of logical groups of code,
such as release candidates.
## Workflow
![Batch Workflow][batch-workflow]
## Getting Started
### Installation
##### Use a Virtual Environment
1. Create a virtual environment:
```shell
python -m venv redflag-venv
source redflag-venv/bin/activate
```
1. Install RedFlag:
```shell
pip install addepar-redflag
```
Alternatively, if you'd like to use Poetry, clone the repo and use `poetry install` and then `poetry run redflag`.
### Setup Credentials
Credentials can be set using:
1. Environment variables
1. A `.env` file
1. CLI parameters
1. Configuration file (This is not recommended for security reasons!)
##### AWS Credentials
RedFlag uses [Boto3-Compatible Credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) using **Profiles** or **Env Vars**.
Ensure that your AWS IAM policy has `InvokeModel` and `InvokeModelWithResponseStream` permissions to [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_service-with-iam.html).
Lastly, *make sure you've [requested the necessary Claude models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html)!*
##### GitHub PAT
Use a [Personal Access Token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) with
`repo` permissions. Set the token as an environment variable:
```shell
export RF_GITHUB_TOKEN=your-token-here
```
##### Jira API Token *(Optional)*
First, set a Jira URL (`https://your-org.atlassian.net`) in the configuration file (`jira_url`), as a CLI parameter (`--jira-url`), or as an environment variable (`RF_JIRA_URL`).
Then create a [Jira API Token](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/) and
set it as an environment variable:
```shell
export RF_JIRA_USER=your-username-here
export RF_JIRA_TOKEN=your-token-here
```
##### Slack Token *(Optional)*
1. [Create a Slack App](https://api.slack.com/quickstart)
1. [Request required scopes](https://api.slack.com/quickstart#scopes)
1. [Install and authorize the App](https://api.slack.com/quickstart#installing)
1. Add environment variables or configuration entries when running RedFlag
```shell
export RF_SLACK_TOKEN=xoxb-slack-token-here
export RF_SLACK_CHANNEL=C0123456789
```
*Don't forget to invite the bot to the channel to avoid a `channel_not_found` error*.
### Usage
> **Note**: When running RedFlag, output messages will indicate it is retrieving and evaluating commits from Pull Requests (PRs). Typically, commits are made via PRs if you have [branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) or [repository rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) in place. Consequently, once a PR is squashed and merged, it appears as a single commit in the target branch. The terminology used may vary depending on how you choose to run RedFlag, as you can specify `--from` and `--to` commit SHAs or branches.
Here are some examples on how to run RedFlag in batch mode:
```shell
# Using branch names:
redflag --repo YourOrg/SomeRepo --from main --to dev
# Using commit hashes:
redflag --repo YouOrg/SomeRepo --from a1b2c3 --to d4e5f6
# With a custom configuration file:
redflag --config custom-config.yml
```
## Report Output
By default, RedFlag produces an HTML report that can be opened in a browser.
<a href="https://opensource.addepar.com/RedFlag/">
<img src="https://raw.githubusercontent.com/Addepar/RedFlag/main/docs/images/Report-Animated.gif">
</a>
<p align="right">(<a href="#readme-top">back to top</a>)</p>
---
# CI Mode
RedFlag can be run in CI pipelines to flag PRs and add the appropriate reviewers.
This mode uses GitHub Actions to run RedFlag on every PR and post a comment if
the PR requires a review. Additionally, CI Mode is best suited for Slack alerting.
[![CI Mode][docs-ci-mode]][docs-ci-mode-url]
<p align="right">(<a href="#readme-top">back to top</a>)</p>
---
# Evaluation Mode
RedFlag can be run in evaluation mode to evaluate the performance of the AI model using your own custom
dataset. This mode is useful for understanding how the model and prompts perform on your codebase and aids in security
risk evaluation.
[![Evaluation Mode][docs-eval-mode]][docs-eval-mode-url]
<p align="right">(<a href="#readme-top">back to top</a>)</p>
---
# Advanced Configuration
## Order of Precedence
1. CLI Parameters
1. Environment Variables
1. Configuration File
1. Default Values
On each execution, RedFlag will load the configuration in the order of precedence above and then
output a table that shows the final configuration and where each parameter was set.
## Configuration Options and Defaults
The following table shows configuration options for each parameter:
#### General Settings
| Parameter | CLI Param | Env Var | Config File | Default |
|---------------------------------------------------------------------------------------|-----------|---------|-------------|---------|
| [Configuration File](https://github.com/Addepar/RedFlag/blob/main/config.sample.yaml) | --config | - | - | - |
| Repository | --repo | RF_REPO | repo | - |
| Branch/Commit From | --from | RF_FROM | from | - |
| Branch/Commit To | --to | RF_TO | to | - |
#### Integration Settings
| Parameter | CLI Param | Env Var | Config File | Default |
|-------------------------------------------------------------------------------------------------------------------------------------|------------------|-------------------|---------------|---------|
| [GitHub Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) | --github-token | RF_GITHUB_TOKEN | github_token | - |
| Jira URL | --jira-url | RF_JIRA_URL | jira.url | - |
| Jira Username | --jira-user | RF_JIRA_USER | jira.user | - |
| [Jira Token](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/) | --jira-token | RF_JIRA_TOKEN | jira.token | - |
| [Slack Token](https://api.slack.com/concepts/token-types) | --slack-token | RF_SLACK_TOKEN | slack.token | - |
| [Slack Channel (ID)](https://slack.com/help/articles/221769328-Locate-your-Slack-URL-or-ID) | --slack-channel | RF_SLACK_CHANNEL | slack.channel | - |
| Slack Message Headline | --slack-headline | RF_SLACK_HEADLINE | slack.headline | - |
#### LLM Settings
| Parameter | CLI Param | Env Var | Config File | Default |
|--------------------------------------------------------------------------------------------------------------------|--------------------|---------------------|----------------------------|-------------------------------------------|
| Debug LLM | --debug-llm | - | - | `False` |
| [Bedrock Model ID](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) | --bedrock-model-id | RF_BEDROCK_MODEL_ID | bedrock.model_id | `anthropic.claude-3-sonnet-20240229-v1:0` |
| [Bedrock Profile](https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html) | --bedrock-profile | RF_BEDROCK_PROFILE | bedrock.profile | - |
| [Bedrock Region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) | --bedrock-region | RF_BEDROCK_REGION | bedrock.region | - |
| Review Prompt (Role) | - | - | prompts.review.role | Security review (see `sample.config.yaml`) |
| Review Prompt (Question) | - | - | prompts.review.question | Security review (see `sample.config.yaml`) |
| Test Plan Prompt (Role) | - | - | prompts.test_plan.role | Security review (see `sample.config.yaml`) |
| Test Plan Prompt (Question) | - | - | prompts.test_plan.question | Security review (see `sample.config.yaml`) |
#### Input/Output Settings
| Parameter | CLI Param | Env Var | Config File | Default |
|---------------------------|--------------------------|----------------|-------------------------|-----------|
| Output Directory | --output-dir | RF_OUTPUT_DIR | output_dir | `results` |
| Maximum Commits | --max-commits | RF_MAX_COMMITS | max_commits | `0` (∞) |
| Don't Output HTML | --no-output-html | - | - | - |
| Don't Output JSON | --no-output-json | - | - | - |
| Don't Show Progress Bar | --no-progress-bar | - | - | - |
| Don't Strip HTML Comments | --no-strip-html-comments | - | - | - |
| Filter Commit Titles | - | - | filter_commits.title | - |
| Filter Commit Users | - | - | filter_commits.user | - |
| Strip Description Lines | - | - | strip_description_lines | - |
#### Evaluation Parameters (`eval` Command)
| Parameter | CLI Param | Env Var | Config File | Default |
|---------------------------------------------------------------------------------------------------------|------------|------------------|--------------|---------|
| [Evaluation Dataset](https://github.com/Addepar/RedFlag/wiki/Evaluation-Mode#building-a-custom-dataset) | --dataset | RF_DATASET | dataset | - |
<p align="right">(<a href="#readme-top">back to top</a>)</p>
---
## Contributing
Contributions are what make the open source community such an amazing place to learn, inspire, and create.
Any contributions you make are **greatly appreciated**.
If you have a suggestion that would make this better, please fork the repo and create a pull request.
You can also simply open an issue with the tag "enhancement".
Don't forget to give the project a star! Thanks again!
1. Fork the Project
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the Branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
<p align="right">(<a href="#readme-top">back to top</a>)</p>
---
## License
Distributed under the MIT License. See [`LICENSE.md`](https://github.com/Addepar/RedFlag?tab=MIT-1-ov-file#readme) for more information.
<p align="right">(<a href="#readme-top">back to top</a>)</p>
---
## Contact
Say hi to Addepar Security Engineering at [security-engineering@addepar.com](mailto:security-engineering@addepar.com).
<p align="right">(<a href="#readme-top">back to top</a>)</p>
<!-- MARKDOWN LINKS & IMAGES -->
<!-- https://www.markdownguide.org/basic-syntax/#reference-style-links -->
[python-shield]: https://img.shields.io/badge/Python-3.11+-blue?style=for-the-badge&logo=python&logoColor=white
[python-url]: https://www.python.org/
[contributors-shield]: https://img.shields.io/github/contributors/Addepar/RedFlag.svg?style=for-the-badge
[contributors-url]: https://github.com/Addepar/RedFlag/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/Addepar/RedFlag.svg?style=for-the-badge
[forks-url]: https://github.com/Addepar/RedFlag/network/members
[stars-shield]: https://img.shields.io/github/stars/Addepar/RedFlag.svg?style=for-the-badge
[stars-url]: https://github.com/Addepar/RedFlag/stargazers
[issues-shield]: https://img.shields.io/github/issues/Addepar/RedFlag.svg?style=for-the-badge
[issues-url]: https://github.com/Addepar/RedFlag/issues
[license-shield]: https://img.shields.io/github/license/Addepar/RedFlag.svg?style=for-the-badge
[license-url]: https://github.com/Addepar/RedFlag/blob/main/LICENSE.md
[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555
[linkedin-url]: https://www.linkedin.com/company/addepar
[batch-workflow]: https://raw.githubusercontent.com/Addepar/RedFlag/main/docs/images/Batch-Workflow.png
[docs-ci-mode]: https://img.shields.io/badge/Read%20More-How%20To%20Use%20RedFlag%20In%20CI%20Mode-blue?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAAAXNSR0IArs4c6QAAAERlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAADbN2wMAAABcUlEQVRoBe1ZQU7DMBBsOPMC3gISDwDxAr5BJe5I/Ki3gkS58ROO9MKFdJZmLyOysWIzUNiVVs527Rl7JrnUi8V/iL7vr5CPyC1SFQ9NtMVu71U7Zp7qAwDQlLd4R94gT6pBCwCM0KJgajwFGPbaWCzjmW27e8o2B3gbwD6Vx/Ozg3/DuHEZHNvraDyKmugdU/+D6pZle2xXAuPBvkJ+BvuIl8hf9xF30Ttgu4/6ot6m67qzMa6pb8DXneNhhXz1H4Tj6Wwuc8BiDGDfHe+PrSv9vQS/1IFSTvm8PIBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQefnnHdiapPibW3IzM8e+KQdeBtDrOeA/vgbKXw6XDF/ekZVcQNQcogQ/vCMzcoDcYbit2UjtWtyRTe4z5MAhLpBrpF9841EWT+HmDr25A8AdiN2uwvv8AAAAAElFTkSuQmCC
[docs-ci-mode-url]: https://github.com/Addepar/RedFlag/wiki/CI-Mode
[docs-eval-mode]: https://img.shields.io/badge/Read%20More-How%20To%20Use%20RedFlag%20In%20Evaluation%20Mode-blue?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAAAXNSR0IArs4c6QAAAERlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAADbN2wMAAABcUlEQVRoBe1ZQU7DMBBsOPMC3gISDwDxAr5BJe5I/Ki3gkS58ROO9MKFdJZmLyOysWIzUNiVVs527Rl7JrnUi8V/iL7vr5CPyC1SFQ9NtMVu71U7Zp7qAwDQlLd4R94gT6pBCwCM0KJgajwFGPbaWCzjmW27e8o2B3gbwD6Vx/Ozg3/DuHEZHNvraDyKmugdU/+D6pZle2xXAuPBvkJ+BvuIl8hf9xF30Ttgu4/6ot6m67qzMa6pb8DXneNhhXz1H4Tj6Wwuc8BiDGDfHe+PrSv9vQS/1IFSTvm8PIBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQefnnHdiapPibW3IzM8e+KQdeBtDrOeA/vgbKXw6XDF/ekZVcQNQcogQ/vCMzcoDcYbit2UjtWtyRTe4z5MAhLpBrpF9841EWT+HmDr25A8AdiN2uwvv8AAAAAElFTkSuQmCC
[docs-eval-mode-url]: https://github.com/Addepar/RedFlag/wiki/Evaluation-Mode
Raw data
{
"_id": null,
"home_page": "https://github.com/Addepar/RedFlag",
"name": "addepar-redflag",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "security, code review, static analysis, AI",
"author": "Addepar Security Engineering",
"author_email": "security-engineering@addepar.com",
"download_url": "https://files.pythonhosted.org/packages/d1/fb/0f88f4d6d0a9d0223bc986961bee5fc11b9a7ab201768d8a6c6675be0edc/addepar_redflag-1.1.0.tar.gz",
"platform": null,
"description": "<a name=\"readme-top\"></a>\n\n<div align=\"center\">\n <a href=\"https://github.com/Addepar/RedFlag\">\n <img src=\"https://raw.githubusercontent.com/Addepar/RedFlag/main/docs/images/RedFlag-Logo.png\" alt=\"RedFlag\" height=\"100\">\n </a>\n\n\n[![Python][python-shield]][python-url]\n[![Contributors][contributors-shield]][contributors-url]\n[![MIT License][license-shield]][license-url]\n[![LinkedIn][linkedin-shield]][linkedin-url]<br />\n[![Forks][forks-shield]][forks-url]\n[![Stargazers][stars-shield]][stars-url]\n[![Issues][issues-shield]][issues-url]\n\nRedFlag leverages AI to determine high-risk code changes. \nRun it in batch mode to scope manual security testing of release candidates, \nor run it in your CI pipelines to flag PRs and add the appropriate reviewers. \nDespite being a security tool, RedFlag can be leveraged for almost any team \nas it's configuration makes it infinitely flexible.\n<br />\n<a href=\"https://addepar.com/blog/introducing-redflag-using-ai-to-scale-addepar-s-offensive-security-team\"><strong>Read the blog post \u00bb</strong></a>\n<br />\n<br />\n<a href=\"https://opensource.addepar.com/RedFlag/\">View Sample Report</a>\n\u00b7\n<a href=\"https://github.com/Addepar/RedFlag/issues/new?labels=bug&template=bug-report---.md\">Report Bug</a>\n\u00b7\n<a href=\"https://github.com/Addepar/RedFlag/issues/new?labels=enhancement&template=feature-request---.md\">Request Feature</a>\n\n</div>\n\n<details>\n<summary><strong>Table of Contents</strong></summary>\n<ul>\n <li><a href=\"#batch-mode\">Batch Mode</a></li>\n <li><a href=\"#ci-mode\">CI Mode</a></li>\n <li><a href=\"#evaluation-mode\">Evaluation Mode</a></li>\n <li><a href=\"#advanced-configuration\">Advanced Configuration</a></li>\n <li><a href=\"#contributing\">Contributing</a></li>\n <li><a href=\"#license\">License</a></li>\n <li><a href=\"#contact\">Contact</a></li>\n</ul>\n</details>\n\n---\n\n# Batch Mode\n\nRedFlag is able to analyze a large number of commits in a single run.\nThese commits can be specified using commit hashes, branch names, or tags.\nThis is useful for scoping manual security testing of logical groups of code,\nsuch as release candidates.\n\n## Workflow\n\n![Batch Workflow][batch-workflow]\n\n## Getting Started\n\n### Installation\n\n##### Use a Virtual Environment\n\n1. Create a virtual environment:\n ```shell\n python -m venv redflag-venv\n source redflag-venv/bin/activate\n ```\n \n1. Install RedFlag:\n ```shell\n pip install addepar-redflag\n ```\n\nAlternatively, if you'd like to use Poetry, clone the repo and use `poetry install` and then `poetry run redflag`.\n\n### Setup Credentials\n\nCredentials can be set using:\n1. Environment variables \n1. A `.env` file\n1. CLI parameters\n1. Configuration file (This is not recommended for security reasons!)\n\n##### AWS Credentials\n\nRedFlag uses [Boto3-Compatible Credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html) using **Profiles** or **Env Vars**.\nEnsure that your AWS IAM policy has `InvokeModel` and `InvokeModelWithResponseStream` permissions to [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_service-with-iam.html). \nLastly, *make sure you've [requested the necessary Claude models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html)!*\n\n##### GitHub PAT\n\nUse a [Personal Access Token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) with\n`repo` permissions. Set the token as an environment variable:\n\n```shell\nexport RF_GITHUB_TOKEN=your-token-here\n```\n\n##### Jira API Token *(Optional)*\nFirst, set a Jira URL (`https://your-org.atlassian.net`) in the configuration file (`jira_url`), as a CLI parameter (`--jira-url`), or as an environment variable (`RF_JIRA_URL`).\n\nThen create a [Jira API Token](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/) and\nset it as an environment variable:\n\n```shell\nexport RF_JIRA_USER=your-username-here\nexport RF_JIRA_TOKEN=your-token-here\n```\n\n##### Slack Token *(Optional)*\n\n1. [Create a Slack App](https://api.slack.com/quickstart)\n1. [Request required scopes](https://api.slack.com/quickstart#scopes)\n1. [Install and authorize the App](https://api.slack.com/quickstart#installing)\n1. Add environment variables or configuration entries when running RedFlag\n\n```shell\nexport RF_SLACK_TOKEN=xoxb-slack-token-here\nexport RF_SLACK_CHANNEL=C0123456789\n```\n\n*Don't forget to invite the bot to the channel to avoid a `channel_not_found` error*.\n\n### Usage\n\n> **Note**: When running RedFlag, output messages will indicate it is retrieving and evaluating commits from Pull Requests (PRs). Typically, commits are made via PRs if you have [branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) or [repository rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) in place. Consequently, once a PR is squashed and merged, it appears as a single commit in the target branch. The terminology used may vary depending on how you choose to run RedFlag, as you can specify `--from` and `--to` commit SHAs or branches.\n\nHere are some examples on how to run RedFlag in batch mode:\n\n```shell\n# Using branch names:\nredflag --repo YourOrg/SomeRepo --from main --to dev\n# Using commit hashes:\nredflag --repo YouOrg/SomeRepo --from a1b2c3 --to d4e5f6\n# With a custom configuration file:\nredflag --config custom-config.yml\n```\n\n## Report Output\n\nBy default, RedFlag produces an HTML report that can be opened in a browser.\n\n<a href=\"https://opensource.addepar.com/RedFlag/\">\n <img src=\"https://raw.githubusercontent.com/Addepar/RedFlag/main/docs/images/Report-Animated.gif\">\n</a>\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n---\n\n# CI Mode\n\nRedFlag can be run in CI pipelines to flag PRs and add the appropriate reviewers.\nThis mode uses GitHub Actions to run RedFlag on every PR and post a comment if\nthe PR requires a review. Additionally, CI Mode is best suited for Slack alerting.\n\n[![CI Mode][docs-ci-mode]][docs-ci-mode-url]\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n---\n\n# Evaluation Mode\n\nRedFlag can be run in evaluation mode to evaluate the performance of the AI model using your own custom\ndataset. This mode is useful for understanding how the model and prompts perform on your codebase and aids in security\nrisk evaluation.\n\n[![Evaluation Mode][docs-eval-mode]][docs-eval-mode-url]\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n---\n\n# Advanced Configuration\n\n## Order of Precedence\n\n1. CLI Parameters\n1. Environment Variables\n1. Configuration File\n1. Default Values\n\nOn each execution, RedFlag will load the configuration in the order of precedence above and then\noutput a table that shows the final configuration and where each parameter was set.\n\n## Configuration Options and Defaults\n\nThe following table shows configuration options for each parameter:\n\n#### General Settings\n\n| Parameter | CLI Param | Env Var | Config File | Default |\n|---------------------------------------------------------------------------------------|-----------|---------|-------------|---------|\n| [Configuration File](https://github.com/Addepar/RedFlag/blob/main/config.sample.yaml) | --config | - | - | - |\n| Repository | --repo | RF_REPO | repo | - |\n| Branch/Commit From | --from | RF_FROM | from | - |\n| Branch/Commit To | --to | RF_TO | to | - |\n\n#### Integration Settings\n\n| Parameter | CLI Param | Env Var | Config File | Default |\n|-------------------------------------------------------------------------------------------------------------------------------------|------------------|-------------------|---------------|---------|\n| [GitHub Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) | --github-token | RF_GITHUB_TOKEN | github_token | - | \n| Jira URL | --jira-url | RF_JIRA_URL | jira.url | - | \n| Jira Username | --jira-user | RF_JIRA_USER | jira.user | - | \n| [Jira Token](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/) | --jira-token | RF_JIRA_TOKEN | jira.token | - | \n| [Slack Token](https://api.slack.com/concepts/token-types) | --slack-token | RF_SLACK_TOKEN | slack.token | - | \n| [Slack Channel (ID)](https://slack.com/help/articles/221769328-Locate-your-Slack-URL-or-ID) | --slack-channel | RF_SLACK_CHANNEL | slack.channel | - |\n| Slack Message Headline | --slack-headline | RF_SLACK_HEADLINE | slack.headline | - |\n\n#### LLM Settings\n\n| Parameter | CLI Param | Env Var | Config File | Default |\n|--------------------------------------------------------------------------------------------------------------------|--------------------|---------------------|----------------------------|-------------------------------------------|\n| Debug LLM | --debug-llm | - | - | `False` |\n| [Bedrock Model ID](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) | --bedrock-model-id | RF_BEDROCK_MODEL_ID | bedrock.model_id | `anthropic.claude-3-sonnet-20240229-v1:0` |\n| [Bedrock Profile](https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html) | --bedrock-profile | RF_BEDROCK_PROFILE | bedrock.profile | - |\n| [Bedrock Region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) | --bedrock-region | RF_BEDROCK_REGION | bedrock.region | - |\n| Review Prompt (Role) | - | - | prompts.review.role | Security review (see `sample.config.yaml`) |\n| Review Prompt (Question) | - | - | prompts.review.question | Security review (see `sample.config.yaml`) |\n| Test Plan Prompt (Role) | - | - | prompts.test_plan.role | Security review (see `sample.config.yaml`) |\n| Test Plan Prompt (Question) | - | - | prompts.test_plan.question | Security review (see `sample.config.yaml`) |\n\n#### Input/Output Settings\n\n| Parameter | CLI Param | Env Var | Config File | Default |\n|---------------------------|--------------------------|----------------|-------------------------|-----------|\n| Output Directory | --output-dir | RF_OUTPUT_DIR | output_dir | `results` |\n| Maximum Commits | --max-commits | RF_MAX_COMMITS | max_commits | `0` (\u221e) |\n| Don't Output HTML | --no-output-html | - | - | - |\n| Don't Output JSON | --no-output-json | - | - | - |\n| Don't Show Progress Bar | --no-progress-bar | - | - | - |\n| Don't Strip HTML Comments | --no-strip-html-comments | - | - | - |\n| Filter Commit Titles | - | - | filter_commits.title | - |\n| Filter Commit Users | - | - | filter_commits.user | - |\n| Strip Description Lines | - | - | strip_description_lines | - |\n\n#### Evaluation Parameters (`eval` Command)\n\n| Parameter | CLI Param | Env Var | Config File | Default |\n|---------------------------------------------------------------------------------------------------------|------------|------------------|--------------|---------|\n| [Evaluation Dataset](https://github.com/Addepar/RedFlag/wiki/Evaluation-Mode#building-a-custom-dataset) | --dataset | RF_DATASET | dataset | - |\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n---\n\n## Contributing\n\nContributions are what make the open source community such an amazing place to learn, inspire, and create. \nAny contributions you make are **greatly appreciated**.\n\nIf you have a suggestion that would make this better, please fork the repo and create a pull request. \nYou can also simply open an issue with the tag \"enhancement\".\nDon't forget to give the project a star! Thanks again!\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n---\n\n## License\n\nDistributed under the MIT License. See [`LICENSE.md`](https://github.com/Addepar/RedFlag?tab=MIT-1-ov-file#readme) for more information.\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n---\n\n## Contact\n\nSay hi to Addepar Security Engineering at [security-engineering@addepar.com](mailto:security-engineering@addepar.com).\n\n\n<p align=\"right\">(<a href=\"#readme-top\">back to top</a>)</p>\n\n\n<!-- MARKDOWN LINKS & IMAGES -->\n<!-- https://www.markdownguide.org/basic-syntax/#reference-style-links -->\n\n[python-shield]: https://img.shields.io/badge/Python-3.11+-blue?style=for-the-badge&logo=python&logoColor=white\n[python-url]: https://www.python.org/\n[contributors-shield]: https://img.shields.io/github/contributors/Addepar/RedFlag.svg?style=for-the-badge\n[contributors-url]: https://github.com/Addepar/RedFlag/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/Addepar/RedFlag.svg?style=for-the-badge\n[forks-url]: https://github.com/Addepar/RedFlag/network/members\n[stars-shield]: https://img.shields.io/github/stars/Addepar/RedFlag.svg?style=for-the-badge\n[stars-url]: https://github.com/Addepar/RedFlag/stargazers\n[issues-shield]: https://img.shields.io/github/issues/Addepar/RedFlag.svg?style=for-the-badge\n[issues-url]: https://github.com/Addepar/RedFlag/issues\n[license-shield]: https://img.shields.io/github/license/Addepar/RedFlag.svg?style=for-the-badge\n[license-url]: https://github.com/Addepar/RedFlag/blob/main/LICENSE.md\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555\n[linkedin-url]: https://www.linkedin.com/company/addepar\n\n[batch-workflow]: https://raw.githubusercontent.com/Addepar/RedFlag/main/docs/images/Batch-Workflow.png\n[docs-ci-mode]: https://img.shields.io/badge/Read%20More-How%20To%20Use%20RedFlag%20In%20CI%20Mode-blue?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAAAXNSR0IArs4c6QAAAERlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAADbN2wMAAABcUlEQVRoBe1ZQU7DMBBsOPMC3gISDwDxAr5BJe5I/Ki3gkS58ROO9MKFdJZmLyOysWIzUNiVVs527Rl7JrnUi8V/iL7vr5CPyC1SFQ9NtMVu71U7Zp7qAwDQlLd4R94gT6pBCwCM0KJgajwFGPbaWCzjmW27e8o2B3gbwD6Vx/Ozg3/DuHEZHNvraDyKmugdU/+D6pZle2xXAuPBvkJ+BvuIl8hf9xF30Ttgu4/6ot6m67qzMa6pb8DXneNhhXz1H4Tj6Wwuc8BiDGDfHe+PrSv9vQS/1IFSTvm8PIBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQefnnHdiapPibW3IzM8e+KQdeBtDrOeA/vgbKXw6XDF/ekZVcQNQcogQ/vCMzcoDcYbit2UjtWtyRTe4z5MAhLpBrpF9841EWT+HmDr25A8AdiN2uwvv8AAAAAElFTkSuQmCC\n[docs-ci-mode-url]: https://github.com/Addepar/RedFlag/wiki/CI-Mode\n[docs-eval-mode]: https://img.shields.io/badge/Read%20More-How%20To%20Use%20RedFlag%20In%20Evaluation%20Mode-blue?style=for-the-badge&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAAAXNSR0IArs4c6QAAAERlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAMKADAAQAAAABAAAAMAAAAADbN2wMAAABcUlEQVRoBe1ZQU7DMBBsOPMC3gISDwDxAr5BJe5I/Ki3gkS58ROO9MKFdJZmLyOysWIzUNiVVs527Rl7JrnUi8V/iL7vr5CPyC1SFQ9NtMVu71U7Zp7qAwDQlLd4R94gT6pBCwCM0KJgajwFGPbaWCzjmW27e8o2B3gbwD6Vx/Ozg3/DuHEZHNvraDyKmugdU/+D6pZle2xXAuPBvkJ+BvuIl8hf9xF30Ttgu4/6ot6m67qzMa6pb8DXneNhhXz1H4Tj6Wwuc8BiDGDfHe+PrSv9vQS/1IFSTvm8PIBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQeZkOyCUnwnSABJGX6YBcciJMB0gQefnnHdiapPibW3IzM8e+KQdeBtDrOeA/vgbKXw6XDF/ekZVcQNQcogQ/vCMzcoDcYbit2UjtWtyRTe4z5MAhLpBrpF9841EWT+HmDr25A8AdiN2uwvv8AAAAAElFTkSuQmCC\n[docs-eval-mode-url]: https://github.com/Addepar/RedFlag/wiki/Evaluation-Mode\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "RedFlag uses AI to identify high-risk code changes. Run it in batch mode for release candidate testing or in CI pipelines to flag commits and add reviewers. RedFlag's flexible configuration makes it valuable for any team.",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/Addepar/RedFlag",
"Repository": "https://github.com/Addepar/RedFlag"
},
"split_keywords": [
"security",
" code review",
" static analysis",
" ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a886312597956bf33f80d9a1ab8230d7fbdcead6d8a9b6eb43216dae78da7708",
"md5": "a07ebf87bec4b3aa94efee5557c8f1a0",
"sha256": "a81fb6c34d0b5035077df9e3b66a128a6b82b7136473659f8d06951f380868c1"
},
"downloads": -1,
"filename": "addepar_redflag-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a07ebf87bec4b3aa94efee5557c8f1a0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 117204,
"upload_time": "2024-10-28T13:32:36",
"upload_time_iso_8601": "2024-10-28T13:32:36.167108Z",
"url": "https://files.pythonhosted.org/packages/a8/86/312597956bf33f80d9a1ab8230d7fbdcead6d8a9b6eb43216dae78da7708/addepar_redflag-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d1fb0f88f4d6d0a9d0223bc986961bee5fc11b9a7ab201768d8a6c6675be0edc",
"md5": "bd549044589d0a2629c34460107fee7c",
"sha256": "ab6fbeb3516dd0fb1c12890380e1d0a2b10ab4f3df28779d8ac84ba755d56f2d"
},
"downloads": -1,
"filename": "addepar_redflag-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "bd549044589d0a2629c34460107fee7c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 115793,
"upload_time": "2024-10-28T13:32:37",
"upload_time_iso_8601": "2024-10-28T13:32:37.914435Z",
"url": "https://files.pythonhosted.org/packages/d1/fb/0f88f4d6d0a9d0223bc986961bee5fc11b9a7ab201768d8a6c6675be0edc/addepar_redflag-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-28 13:32:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Addepar",
"github_project": "RedFlag",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "addepar-redflag"
}
JSON
