# Custodian policies for Infrastructure Code
This package allows cloud custodian to evaluate policies directly
against infrastructure as code source assets.
It also provides a separate cli for better command line ux for
source asset evaluation.
## Install
We currently only support python > 3.10 on mac and linux, to run on windows
we recommend using our docker images.
```shell
pip install c7n-left
```
We also provide signed docker images. These images are built on top of chainguard's [wolfi linux
distribution](https://www.chainguard.dev/unchained/introducing-wolfi-the-first-linux-un-distro) which
is designed to be minimal, auditable, and secure.
```shell
docker pull cloudcustodian/c7n-left:dev
```
Images signatures can be verified using [cosign](https://github.com/sigstore/cosign)
```
export IMAGE=$(docker image inspect cloudcustodian/c7n-left:dev -f '{{index .RepoDigests 0}}')
cosign verify $IMAGE \
--certificate-identity 'https://github.com/cloud-custodian/cloud-custodian/.github/workflows/docker.yml@refs/heads/main' \
--certificate-oidc-issuer 'https://token.actions.githubusercontent.com'
```
## Usage
```shell
❯ c7n-left run --help
Usage: c7n-left run [OPTIONS]
evaluate policies against IaC sources.
c7n-left -p policy_dir -d terraform_root --filters "severity=HIGH"
WARNING - CLI interface subject to change.
Options:
--format TEXT
--filters TEXT Filter policies or resources as k=v pairs
with globbing
--warn-on TEXT Select policies to log instead of fail on
via k=v pairs with globbing
-p, --policy-dir PATH Directory with policies
-d, --directory PATH IaC directory to evaluate
-o, --output [cli|github|json] Output format (default cli)
--output-file FILENAME Output file (default stdout)
--var-file FILE Load variables from the given file, can be
used more than once
--output-query TEXT Use a jmespath expression to filter json
output
--summary [policy|resource]
--help Show this message and exit.
```
We'll create an empty directory with a policy in it
```yaml
policies:
- name: test
resource: terraform.aws_s3_bucket
metadata:
severity: medium
filters:
- server_side_encryption_configuration: absent
```
And now we can use it to evaluate a terraform root module
```shell
❯ c7n-left run -p policies -d module
Running 1 policies on 1 resources
test - terraform.aws_s3_bucket
Failed
File: s3.tf:1-8
1 resource "aws_s3_bucket" "example" {
2 bucket = "my-custodian-test-bucket"
3 acl = "private"
4
5 tags = {
6 original-tag = "original-value"
7 }
8 }
Evaluation complete 0.00 seconds -> 1 Failures
Summary - By Policy
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Severity ┃ Policy ┃ Result ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ medium │ test │ 1 failed 0 passed │
└──────────┴────────┴───────────────────┘
0 compliant of 1 total, 1 resource has 1 policy violation
```
For running in docker, you'll need to use volume mounts to provide access to
the policy directory and terraform root module.
```shell
docker run -ti --rm -v $(pwd)/policies:/policies -v $(pwd)/root-module:/module \
cloudcustodian/c7n-left:dev run -p /policies -d /module
```
If the terraform root module has other remote module dependencies, you'll need to fetch those first using terraform
before running c7n-left.
```shell
terraform get -update
```
## CLI Filters
Which policies and which resources are evaluated can be controlled via
command line via `--filters` option.
Available filters
- `name` - policy name
- `category` - policy category
- `severity` - minimum policy severity (unknown, low, medium, high, critical)
- `type` - resource type, ie. aws_security_group
- `id` - resource id ie. aws_vpc.example
Multiple values for a given filter can be specified as comma separate values, and all filters
except severity support globbing.
Examples
```
# run all encryption policies on ebs volumes and sqs queues
c7n-left run -p policy_dir -d terraform --filters="category=encryption type=aws_ebs_volume,aws_sqs_queue"
# run all medium and higher level policies cost policies
c7n-left run -p policy_dir -d terraform --filters="severity=medium category=cost"
```
policy values for severity and category are specified in its metadata section. ie
```yaml
policies:
- name: check-encryption
resource: [terraform.aws_ebs_volume, terraform.aws_sqs_queue]
metadata:
category: [encryption, security]
severity: high
filters:
- kms_master_key_id: absent
```
## Outputs
if your using this in github actions, we have special output mode for
reporting annotations directly into pull requests with `--output
github`
We also display a summary output after displaying resource matches,
there are two summary displays available, the default policy summary,
and a resource summary which can be enabled via `--summary resource`.
By default any policy matches cause a run to exit code 1 to mark failure,
this behavior can be controlled via the `--warn-on` cli flag. ie. given a policy
with
```yaml
policies:
- name: check-encryption
resource: [terraform.aws_ebs_volume, terraform.aws_sqs_queue]
metadata:
category: [beta, security]
severity: high
```
running the policy with `--warn-on category=beta` will cause matches to be logged only instead
of causing an exit code 1.
## Policy Language
Standard Custodian filters ([value](https://cloudcustodian.io/docs/filters.html#value-filter), [list-item](https://cloudcustodian.io/docs/aws/resources/aws-common-filters.html#aws-common-filters-list-item), `and`, `or`, `not`, [`reduce`](https://cloudcustodian.io/docs/filters.html#reduce-filter) and `event`) are available
Policies for c7n-left support a few additional capabilities beyond what's common for custodian policies.
Policies can be specified against multiple resource types either as an array or glob.
```yaml
policies:
- name: check-encryption
resource: [aws_ebs_volume, aws_sqs_queue]
```
### taggable filter
A `taggable` filter is available that allows filtering to only resources that support tagging.
In combination with resource wild card support, this allows using a single policy to enforce
an organization's tag standards.
```yaml
policies:
- name: check-tag-policy
resource: "terraform.aws*"
filters:
- taggable
- or:
- tag:Env: absent
- tag:Owner: absent
- tag:App: absent
```
This filter supports resources from several terraform providers including aws, azure, gcp, oci, tencentcloud.
terraform providers that support default_tags have those values automatically available on the applicable resources.
### traverse filter
A `traverse` filter is available that allows for multi-hop graph traversal from a resource
to any related resource.
ie, here's a policy against an aws ec2 instance, that checks if any of the security
groups attached to the instance, have a permission defined that allows access from
0.0.0.0/0
```yaml
policies:
- name: check-security-group-open-cidr
resource: terraform.aws_instance
description: "EC2 should not be open to world on ssh"
filters:
- type: traverse
resources:
- aws_security_group
- aws_security_ingress_permission
attrs:
- Ipv4: 0.0.0.0/0
```
### terraform data resources
Policies can also be written against data resources, note data
resources are prefixed with `data.`.
```yaml
policies:
- name: check-ami-data
resource: terraform.data.aws_ami
filters:
- type: value
key: owners
op: contains
value: "099720109477" # Canonical/ubuntu
```
and you can `traverse` from a resource to its data usage as well.
```yaml
policies:
- name: check-owner-specified
resource: terraform.aws_instance
filters:
- type: traverse
resources: data.aws_ami
attrs:
- owners: present
```
### environment variables
c7n-left is typically run in CI systems, which provide a wealth
of information in environment variables. Policies can make use
of these environment variables in two different ways.
They can be used to interpolate the content of a policy, where they
will they will be substituted prior to the policy execution. Note this
uses python's [format capabilties](https://pyformat.info)
```yaml
policies:
- name: "check-{env[REPO]}-{env[PR_NUMBER]}"
resource: terraform.aws*
```
Additionally they can be evaluated by the policy using the `event` filter
```yaml
policies:
- name: check-aws
resource: terraform.aws*
filters:
- type: event
key: env.REPO
value: "cloud-custodian/cloud-custodian"
```
## Policy Testing
c7n-left supports writing and running tests for policies.
To create a test for a policy, create a tests directory next to your policy files.
Within that tests directory, create a sub directory with the policy name.
Next add terraform files to this sub directory. Typically you would add
both terraform files that would match the policy and those that should not.
Finally you add assertions in a `left.plan[.yaml|.json]` file. The
format of the file is an array of dictionaries. The dictionaries are
used to match against the policy findings. The data its matching
against is what is found by using `c7n-left run --output json`. Each
key/value pair in the dictionary is matched against the finding.
So putting it all together, we've setup our tests as follows
```shell
❯ tree policy-dir-a/
policy-dir-a/
├── alb.yaml
└── tests
└── alb-deletion-protection-disabled
├── left.plan.yaml
├── negative1.tf
└── positive1.tf
3 directories, 4 files
❯ cat policy-dir-a/alb.yaml
policies:
- name: alb-deletion-protection-disabled
resource: [terraform.aws_lb, terraform.aws_alb]
description: |
Application Load Balancer should have deletion protection enabled
metadata:
severity: low
category: "Insecure Configurations"
filters:
- enable_deletion_protection: empty
❯ cat policy-dir-a/tests/alb-deletion-protection-disabled/left.plan.yaml
- "resource.__tfmeta.filename": "positive1.tf"
```
and now we can run a test
```shell
❯ c7n-left test -p policy-dir-a/
Discovered 1 Tests
Failure alb-deletion-protection-disabled [{'resource.__tfmeta.filename':
'positive1.tf'}] checks not used
1 Test Complete (0.05s) 1 Failure
```
A test fails if either an assertion in the plan file does not match one policy finding, or if a policy finding is not matched by an assertion.
Raw data
{
"_id": null,
"home_page": "https://cloudcustodian.io",
"name": "c7n-left",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Cloud Custodian Project",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# Custodian policies for Infrastructure Code\n\n\nThis package allows cloud custodian to evaluate policies directly\nagainst infrastructure as code source assets.\n\nIt also provides a separate cli for better command line ux for\nsource asset evaluation.\n\n## Install\n\nWe currently only support python > 3.10 on mac and linux, to run on windows\nwe recommend using our docker images.\n\n```shell\npip install c7n-left\n```\n\nWe also provide signed docker images. These images are built on top of chainguard's [wolfi linux\ndistribution](https://www.chainguard.dev/unchained/introducing-wolfi-the-first-linux-un-distro) which\nis designed to be minimal, auditable, and secure.\n\n```shell\ndocker pull cloudcustodian/c7n-left:dev\n```\n\nImages signatures can be verified using [cosign](https://github.com/sigstore/cosign)\n\n```\nexport IMAGE=$(docker image inspect cloudcustodian/c7n-left:dev -f '{{index .RepoDigests 0}}')\ncosign verify $IMAGE \\\n --certificate-identity 'https://github.com/cloud-custodian/cloud-custodian/.github/workflows/docker.yml@refs/heads/main' \\\n --certificate-oidc-issuer 'https://token.actions.githubusercontent.com'\n```\n\n\n## Usage\n\n```shell\n\u276f c7n-left run --help\nUsage: c7n-left run [OPTIONS]\n\n evaluate policies against IaC sources.\n\n c7n-left -p policy_dir -d terraform_root --filters \"severity=HIGH\"\n\n WARNING - CLI interface subject to change.\n\nOptions:\n --format TEXT\n --filters TEXT Filter policies or resources as k=v pairs\n with globbing\n --warn-on TEXT Select policies to log instead of fail on\n via k=v pairs with globbing\t\t\t\t\t\t\t\t \n -p, --policy-dir PATH Directory with policies\n -d, --directory PATH IaC directory to evaluate\n -o, --output [cli|github|json] Output format (default cli)\n --output-file FILENAME Output file (default stdout)\n --var-file FILE Load variables from the given file, can be\n used more than once\n --output-query TEXT Use a jmespath expression to filter json\n output\n --summary [policy|resource]\n --help Show this message and exit.\n```\n\n\nWe'll create an empty directory with a policy in it\n\n```yaml\npolicies:\n - name: test\n resource: terraform.aws_s3_bucket\n metadata:\n severity: medium\n filters:\n - server_side_encryption_configuration: absent\n```\n\nAnd now we can use it to evaluate a terraform root module\n\n```shell\n\n\u276f c7n-left run -p policies -d module\nRunning 1 policies on 1 resources\ntest - terraform.aws_s3_bucket\n Failed\n File: s3.tf:1-8\n 1 resource \"aws_s3_bucket\" \"example\" { \n 2 bucket = \"my-custodian-test-bucket\" \n 3 acl = \"private\" \n 4 \n 5 tags = { \n 6 original-tag = \"original-value\" \n 7 } \n 8 } \n\nEvaluation complete 0.00 seconds -> 1 Failures\n Summary - By Policy \n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 Severity \u2503 Policy \u2503 Result \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 medium \u2502 test \u2502 1 failed 0 passed \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n0 compliant of 1 total, 1 resource has 1 policy violation\n```\n\nFor running in docker, you'll need to use volume mounts to provide access to \nthe policy directory and terraform root module.\n\n```shell\ndocker run -ti --rm -v $(pwd)/policies:/policies -v $(pwd)/root-module:/module \\\n cloudcustodian/c7n-left:dev run -p /policies -d /module\n```\n\nIf the terraform root module has other remote module dependencies, you'll need to fetch those first using terraform\nbefore running c7n-left.\n\n```shell\nterraform get -update\n```\n\n## CLI Filters\n\nWhich policies and which resources are evaluated can be controlled via\ncommand line via `--filters` option.\n\nAvailable filters\n\n- `name` - policy name\n- `category` - policy category\n- `severity` - minimum policy severity (unknown, low, medium, high, critical)\n- `type` - resource type, ie. aws_security_group\n- `id` - resource id ie. aws_vpc.example \n\nMultiple values for a given filter can be specified as comma separate values, and all filters\nexcept severity support globbing.\n\nExamples\n```\n# run all encryption policies on ebs volumes and sqs queues\nc7n-left run -p policy_dir -d terraform --filters=\"category=encryption type=aws_ebs_volume,aws_sqs_queue\"\n\n# run all medium and higher level policies cost policies\nc7n-left run -p policy_dir -d terraform --filters=\"severity=medium category=cost\"\n```\n\npolicy values for severity and category are specified in its metadata section. ie\n\n```yaml\n\npolicies:\n - name: check-encryption\n resource: [terraform.aws_ebs_volume, terraform.aws_sqs_queue]\n metadata:\n category: [encryption, security]\n severity: high\n filters:\n - kms_master_key_id: absent\n```\n\n\n## Outputs\n\nif your using this in github actions, we have special output mode for\nreporting annotations directly into pull requests with `--output\ngithub`\n\nWe also display a summary output after displaying resource matches,\nthere are two summary displays available, the default policy summary,\nand a resource summary which can be enabled via `--summary resource`.\n\nBy default any policy matches cause a run to exit code 1 to mark failure,\nthis behavior can be controlled via the `--warn-on` cli flag. ie. given a policy\nwith\n\n```yaml\npolicies:\n - name: check-encryption\n resource: [terraform.aws_ebs_volume, terraform.aws_sqs_queue]\n metadata:\n category: [beta, security]\n severity: high\n```\n\nrunning the policy with `--warn-on category=beta` will cause matches to be logged only instead\nof causing an exit code 1.\n\n\n## Policy Language\n\nStandard Custodian filters ([value](https://cloudcustodian.io/docs/filters.html#value-filter), [list-item](https://cloudcustodian.io/docs/aws/resources/aws-common-filters.html#aws-common-filters-list-item), `and`, `or`, `not`, [`reduce`](https://cloudcustodian.io/docs/filters.html#reduce-filter) and `event`) are available\n\nPolicies for c7n-left support a few additional capabilities beyond what's common for custodian policies.\n\n\nPolicies can be specified against multiple resource types either as an array or glob.\n\n```yaml\npolicies:\n - name: check-encryption\n resource: [aws_ebs_volume, aws_sqs_queue]\n```\n\n### taggable filter\n\nA `taggable` filter is available that allows filtering to only resources that support tagging.\n\nIn combination with resource wild card support, this allows using a single policy to enforce\nan organization's tag standards.\n\n```yaml\npolicies:\n - name: check-tag-policy\n resource: \"terraform.aws*\"\n filters:\n - taggable\n - or:\n - tag:Env: absent\n - tag:Owner: absent\n - tag:App: absent\n```\n\nThis filter supports resources from several terraform providers including aws, azure, gcp, oci, tencentcloud.\n\nterraform providers that support default_tags have those values automatically available on the applicable resources.\n\n### traverse filter\n\nA `traverse` filter is available that allows for multi-hop graph traversal from a resource\nto any related resource.\n\nie, here's a policy against an aws ec2 instance, that checks if any of the security\ngroups attached to the instance, have a permission defined that allows access from\n0.0.0.0/0\n\n```yaml\npolicies:\n - name: check-security-group-open-cidr\n resource: terraform.aws_instance\n description: \"EC2 should not be open to world on ssh\"\n filters:\n - type: traverse\n resources:\n - aws_security_group\n - aws_security_ingress_permission\n attrs:\n - Ipv4: 0.0.0.0/0\n```\n\n### terraform data resources\n\nPolicies can also be written against data resources, note data\nresources are prefixed with `data.`.\n\n```yaml\npolicies:\n - name: check-ami-data\n resource: terraform.data.aws_ami\n filters:\n - type: value\n key: owners\n op: contains\n value: \"099720109477\" # Canonical/ubuntu\n```\n\nand you can `traverse` from a resource to its data usage as well.\n\n```yaml\npolicies:\n - name: check-owner-specified\n resource: terraform.aws_instance\n filters:\n - type: traverse\n resources: data.aws_ami\n attrs:\n - owners: present\n```\n\n\n### environment variables\n\nc7n-left is typically run in CI systems, which provide a wealth\nof information in environment variables. Policies can make use\nof these environment variables in two different ways.\n\nThey can be used to interpolate the content of a policy, where they\nwill they will be substituted prior to the policy execution. Note this\nuses python's [format capabilties](https://pyformat.info)\n\n```yaml\npolicies:\n - name: \"check-{env[REPO]}-{env[PR_NUMBER]}\"\n resource: terraform.aws*\n```\n\nAdditionally they can be evaluated by the policy using the `event` filter\n\n```yaml\npolicies:\n - name: check-aws\n resource: terraform.aws*\n filters:\n - type: event\n key: env.REPO\n value: \"cloud-custodian/cloud-custodian\"\n```\n\n\n## Policy Testing\n\nc7n-left supports writing and running tests for policies.\n\nTo create a test for a policy, create a tests directory next to your policy files.\n\nWithin that tests directory, create a sub directory with the policy name.\n\nNext add terraform files to this sub directory. Typically you would add\nboth terraform files that would match the policy and those that should not.\n\nFinally you add assertions in a `left.plan[.yaml|.json]` file. The\nformat of the file is an array of dictionaries. The dictionaries are\nused to match against the policy findings. The data its matching\nagainst is what is found by using `c7n-left run --output json`. Each\nkey/value pair in the dictionary is matched against the finding.\n\nSo putting it all together, we've setup our tests as follows\n\n```shell\n\u276f tree policy-dir-a/\npolicy-dir-a/\n\u251c\u2500\u2500 alb.yaml\n\u2514\u2500\u2500 tests\n \u2514\u2500\u2500 alb-deletion-protection-disabled\n \u251c\u2500\u2500 left.plan.yaml\n \u251c\u2500\u2500 negative1.tf\n \u2514\u2500\u2500 positive1.tf\n\n3 directories, 4 files\n\n\u276f cat policy-dir-a/alb.yaml\npolicies:\n - name: alb-deletion-protection-disabled\n resource: [terraform.aws_lb, terraform.aws_alb]\n description: |\n Application Load Balancer should have deletion protection enabled\n metadata:\n severity: low\n category: \"Insecure Configurations\"\n filters:\n - enable_deletion_protection: empty\n\n\u276f cat policy-dir-a/tests/alb-deletion-protection-disabled/left.plan.yaml\n- \"resource.__tfmeta.filename\": \"positive1.tf\"\n\n```\n\nand now we can run a test\n\n```shell\n\u276f c7n-left test -p policy-dir-a/\nDiscovered 1 Tests\nFailure alb-deletion-protection-disabled [{'resource.__tfmeta.filename':\n'positive1.tf'}] checks not used\n\n1 Test Complete (0.05s) 1 Failure\n```\n\nA test fails if either an assertion in the plan file does not match one policy finding, or if a policy finding is not matched by an assertion.\n\n",
"bugtrack_url": null,
"license": "Apache-2",
"summary": "Custodian policies for IAAC definitions",
"version": "0.3.18",
"project_urls": {
"Documentation": "https://cloudcustodian.io/docs/",
"Homepage": "https://cloudcustodian.io",
"Repository": "https://github.com/cloud-custodian/cloud-custodian"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "78b0bd9b924be829c5be8ff1b02d6580459e4a55273171d412c1e6349c9548fa",
"md5": "68817c4c0e51a22087033b9a211386b8",
"sha256": "d6e355d618285b47c5ee46595be461393ff36a539bdad65695d1a77df27de5bb"
},
"downloads": -1,
"filename": "c7n_left-0.3.18-py3-none-any.whl",
"has_sig": false,
"md5_digest": "68817c4c0e51a22087033b9a211386b8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 39377,
"upload_time": "2024-04-26T15:37:49",
"upload_time_iso_8601": "2024-04-26T15:37:49.915251Z",
"url": "https://files.pythonhosted.org/packages/78/b0/bd9b924be829c5be8ff1b02d6580459e4a55273171d412c1e6349c9548fa/c7n_left-0.3.18-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-26 15:37:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cloud-custodian",
"github_project": "cloud-custodian",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "c7n-left"
}
JSON
