# c7n-org: Multi Account Custodian Execution
% [comment]: # ( !!! IMPORTANT !!! )
% [comment]: # (This file is moved during document generation.)
% [comment]: # (Only edit the original document at ./tools/c7n_org/README.md)
c7n-org is a tool to run Custodian against multiple AWS accounts,
Azure subscriptions, GCP projects, or OCI tenancies in parallel.
## Installation
```shell
pip install c7n-org
```
c7n-org has 3 run modes:
```shell
Usage: c7n-org [OPTIONS] COMMAND [ARGS]...
custodian organization multi-account runner.
Options:
--help Show this message and exit.
Commands:
report report on an AWS cross account policy execution
run run a custodian policy across accounts (AWS, Azure, GCP, OCI)
run-script run a script across AWS accounts
```
In order to run c7n-org against multiple accounts, a config file must
first be created containing pertinent information about the accounts:
Example AWS Config File:
```yaml
accounts:
- account_id: '123123123123'
name: account-1
regions:
- us-east-1
- us-west-2
role: arn:aws:iam::123123123123:role/CloudCustodian
vars:
charge_code: xyz
tags:
- type:prod
- division:some division
- partition:us
- scope:pci
...
```
Example Azure Config File:
```yaml
subscriptions:
- name: Subscription-1
subscription_id: a1b2c3d4-e5f6-g7h8i9...
- name: Subscription-2
subscription_id: 1z2y3x4w-5v6u-7t8s9r...
```
Example GCP Config File:
```yaml
projects:
- name: app-dev
project_id: app-203501
tags:
- label:env:dev
- name: app-prod
project_id: app-1291
tags:
- label:env:dev
```
Example OCI Config File:
```yaml
tenancies:
- name: dev-tenancy
profile: DEVTENANCY
regions:
- us-ashburn-1
- us-phoenix-1
vars:
environment: dev
tags:
- type:test
...
```
### Config File Generation
We also distribute scripts to generate the necessary config file in the [`scripts` folder](https://github.com/cloud-custodian/cloud-custodian/tree/main/tools/c7n_org/scripts).
**Note:** Currently these are distributed only via git. Per
<https://github.com/cloud-custodian/cloud-custodian/issues/2420>, we'll
be looking to incorporate them into a new c7n-org subcommand.
- For **AWS**, the script `orgaccounts.py` generates a config file
from the AWS Organizations API.
```shell
python orgaccounts.py -f accounts.yml
```
- For **Azure**, the script `azuresubs.py` generates a config file
from the Azure Resource Management API.
- Please see the [Additional Azure Instructions](#additional-azure-instructions) for initial setup and other important info.
```shell
python azuresubs.py -f subscriptions.yml
```
- For **GCP**, the script `gcpprojects.py` generates a config file from
the GCP Resource Management API.
```shell
python gcpprojects.py -f projects.yml
```
- For **OCI**, the script `ocitenancies.py` generates a config file
using OCI Configuration file and OCI Organizations API.
- Please refer to the [Additional OCI Instructions](#additional-oci-instructions) for additional information.
```shell
python ocitenancies.py -f tenancies.yml
```
## Running a Policy with c7n-org
To run a policy, the following arguments must be passed in:
```shell
-c | accounts|projects|subscriptions|tenancies config file
-s | output directory
-u | policy
```
For example:
```shell
c7n-org run -c accounts.yml -s output -u test.yml --dryrun
```
After running the above command, the following folder structure will be created:
```
output
|_ account-1
|_ us-east-1
|_ policy-name
|_ resources.json
|_ custodian-run.log
|_ us-west-2
|_ policy-name
|_ resources.json
|_ custodian-run.log
|- account-2
...
```
Use `c7n-org report` to generate a csv report from the output directory.
## Selecting accounts, regions, policies for execution
You can filter the accounts to be run against by either passing the
account name or id via the `-a` flag, which can be specified multiple
times, or alternatively with comma separated values.
Groups of accounts can also be selected for execution by specifying
the `-t` tag filter. Account tags are specified in the config
file. Given the above accounts config file, you can specify all prod
accounts with `-t type:prod`. You can specify the `-t` flag multiple
times or use a comma separated list.
You can specify which policies to use for execution by either
specifying `-p` or selecting groups of policies via their tags with
`-l`. Both options support being specified multiple times or using
comma separated values.
By default in AWS, c7n-org will execute in parallel across regions.
The `-r` flag can be specified multiple times and defaults to
`(us-east-1, us-west-2)`. A special value of `all` will execute across
all regions.
See `c7n-org run --help` for more information.
## Defining and using variables
Each account/subscription/project configuration in the config file can
also define a variables section `vars` that can be used in policies'
definitions and are interpolated at execution time. These are in
addition to the default runtime variables custodian provides like
`account_id`, `now`, and `region`.
Example of defining in c7n-org config file:
```yaml
accounts:
- account_id: '123123123123'
name: account-1
role: arn:aws:iam::123123123123:role/CloudCustodian
vars:
charge_code: xyz
```
Example of using in a policy file:
```yaml
policies:
- name: ec2-check-tag
resource: aws.ec2
filters:
- "tag:CostCenter": "{charge_code}"
```
Another enhancement for `c7n-org run-script` is to support a few vars in the script arg.
The available vars are `account`, `account_id`, `region` and `output_dir`.
```shell
c7n-org run-script -s . -c my-projects.yml gcp_check_{region}.sh
# or
c7n-org run-script -s . -c my-projects.yml use_another_policy_result.sh {output_dir}
```
**Note:** Variable interpolation is sensitive to proper quoting and spacing,
i.e., `{ charge_code }` would be invalid due to the extra white space. Additionally,
yaml parsing can transform a value like `{charge_code}` to null, unless it's quoted
in strings like the above example. Values that do interpolation into other content
don't require quoting, i.e., "my_{charge_code}".
## Other commands
c7n-org also supports running arbitrary scripts against accounts via
the run-script command. For AWS the standard AWS SDK credential
information is exported into the process environment before executing.
For Azure and GCP, only the environment variables
`AZURE_SUBSCRIPTION_ID` and `PROJECT_ID` are exported(in addition of
the system env variables).
c7n-org also supports generating reports for a given policy execution
across accounts via the `c7n-org report` subcommand.
## Additional Azure Instructions
If you're using an Azure Service Principal for executing c7n-org
you'll need to ensure that the principal has access to multiple
subscriptions.
For instructions on creating a service principal and granting access
across subscriptions, visit the [Azure authentication docs
page](https://cloudcustodian.io/docs/azure/authentication.html).
## Additional OCI Instructions
The script `ocitenancies.py` accepts an optional argument `--add-child-tenancies`
which adds all the child tenancies associated with the `DEFAULT` profile's tenancy
in the generated c7n-org configuration file. If the profile for child tenancy is not available in
the OCI configuration file, then either user can add the profile for the child tenancy to the
OCI configuration file and replace the `<ADD_PROFILE>` entry in the c7n-org configuration with the
corresponding profile name or the user can delete the child tenancy entry from the
c7n-org configuration file. For more info about config file, refer to this [page](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm).
If the user wants to query for the resources in the specific compartments in c7n-org, then the user
can pass the compartment OCID's to the `oci_compartments` under the `vars` section like below. If the
`oci_comparments` is not passed under `vars`, then the resources will be fetched from the tenancy level.
```yaml
tenancies:
- name: dev-tenancy
profile: DEVTENANCY
regions:
- us-ashburn-1
- us-phoenix-1
vars:
oci_compartments: ocid1.test.oc1..<unique_ID>EXAMPLE-compartmentId-2-Value,ocid1.test.oc1..<unique_ID>EXAMPLE-compartmentId-3-Value
environment: dev
- name: test-tenancy
profile: TESTTENANCY
regions:
- us-ashburn-1
vars:
environment: test
```
Raw data
{
"_id": null,
"home_page": "https://cloudcustodian.io",
"name": "c7n-org",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Cloud Custodian Project",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# c7n-org: Multi Account Custodian Execution\n\n% [comment]: # ( !!! IMPORTANT !!! )\n% [comment]: # (This file is moved during document generation.)\n% [comment]: # (Only edit the original document at ./tools/c7n_org/README.md)\n\nc7n-org is a tool to run Custodian against multiple AWS accounts,\nAzure subscriptions, GCP projects, or OCI tenancies in parallel.\n\n## Installation\n\n```shell\npip install c7n-org\n```\n\nc7n-org has 3 run modes:\n\n```shell\nUsage: c7n-org [OPTIONS] COMMAND [ARGS]...\n\n custodian organization multi-account runner.\n\nOptions:\n --help Show this message and exit.\n\nCommands:\n report report on an AWS cross account policy execution\n run run a custodian policy across accounts (AWS, Azure, GCP, OCI)\n run-script run a script across AWS accounts\n```\n\nIn order to run c7n-org against multiple accounts, a config file must\nfirst be created containing pertinent information about the accounts:\n\n\nExample AWS Config File:\n\n```yaml\naccounts:\n- account_id: '123123123123'\n name: account-1\n regions:\n - us-east-1\n - us-west-2\n role: arn:aws:iam::123123123123:role/CloudCustodian\n vars:\n charge_code: xyz\n tags:\n - type:prod\n - division:some division\n - partition:us\n - scope:pci\n...\n```\n\nExample Azure Config File:\n\n```yaml\nsubscriptions:\n- name: Subscription-1\n subscription_id: a1b2c3d4-e5f6-g7h8i9...\n- name: Subscription-2\n subscription_id: 1z2y3x4w-5v6u-7t8s9r...\n```\n\nExample GCP Config File:\n\n```yaml\nprojects:\n- name: app-dev\n project_id: app-203501\n tags:\n - label:env:dev \n- name: app-prod\n project_id: app-1291\n tags:\n - label:env:dev\n\n```\n\nExample OCI Config File:\n\n```yaml\ntenancies:\n- name: dev-tenancy\n profile: DEVTENANCY\n regions:\n - us-ashburn-1\n - us-phoenix-1\n vars:\n environment: dev\n tags: \n - type:test\n...\n\n```\n\n### Config File Generation\n\nWe also distribute scripts to generate the necessary config file in the [`scripts` folder](https://github.com/cloud-custodian/cloud-custodian/tree/main/tools/c7n_org/scripts).\n\n**Note:** Currently these are distributed only via git. Per\n<https://github.com/cloud-custodian/cloud-custodian/issues/2420>, we'll\nbe looking to incorporate them into a new c7n-org subcommand.\n\n- For **AWS**, the script `orgaccounts.py` generates a config file\n from the AWS Organizations API.\n\n```shell\npython orgaccounts.py -f accounts.yml\n```\n\n- For **Azure**, the script `azuresubs.py` generates a config file\n from the Azure Resource Management API.\n\n - Please see the [Additional Azure Instructions](#additional-azure-instructions) for initial setup and other important info.\n\n```shell\npython azuresubs.py -f subscriptions.yml\n```\n\n- For **GCP**, the script `gcpprojects.py` generates a config file from\n the GCP Resource Management API.\n\n```shell\npython gcpprojects.py -f projects.yml\n```\n\n- For **OCI**, the script `ocitenancies.py` generates a config file\n using OCI Configuration file and OCI Organizations API.\n \n - Please refer to the [Additional OCI Instructions](#additional-oci-instructions) for additional information.\n\n```shell\npython ocitenancies.py -f tenancies.yml\n```\n\n## Running a Policy with c7n-org\n\nTo run a policy, the following arguments must be passed in:\n\n```shell\n-c | accounts|projects|subscriptions|tenancies config file\n-s | output directory\n-u | policy\n```\n\nFor example:\n\n```shell\nc7n-org run -c accounts.yml -s output -u test.yml --dryrun\n```\n\nAfter running the above command, the following folder structure will be created:\n\n```\noutput\n |_ account-1\n |_ us-east-1\n |_ policy-name\n |_ resources.json\n |_ custodian-run.log\n |_ us-west-2\n |_ policy-name\n |_ resources.json\n |_ custodian-run.log\n |- account-2\n...\n```\n\nUse `c7n-org report` to generate a csv report from the output directory.\n\n## Selecting accounts, regions, policies for execution\n\nYou can filter the accounts to be run against by either passing the\naccount name or id via the `-a` flag, which can be specified multiple\ntimes, or alternatively with comma separated values.\n\nGroups of accounts can also be selected for execution by specifying\nthe `-t` tag filter. Account tags are specified in the config\nfile. Given the above accounts config file, you can specify all prod\naccounts with `-t type:prod`. You can specify the `-t` flag multiple\ntimes or use a comma separated list.\n\nYou can specify which policies to use for execution by either\nspecifying `-p` or selecting groups of policies via their tags with\n`-l`. Both options support being specified multiple times or using\ncomma separated values.\n\nBy default in AWS, c7n-org will execute in parallel across regions.\nThe `-r` flag can be specified multiple times and defaults to\n`(us-east-1, us-west-2)`. A special value of `all` will execute across\nall regions.\n\n\nSee `c7n-org run --help` for more information.\n\n## Defining and using variables\n\nEach account/subscription/project configuration in the config file can\nalso define a variables section `vars` that can be used in policies'\ndefinitions and are interpolated at execution time. These are in\naddition to the default runtime variables custodian provides like\n`account_id`, `now`, and `region`.\n\nExample of defining in c7n-org config file:\n\n```yaml\naccounts:\n- account_id: '123123123123'\n name: account-1\n role: arn:aws:iam::123123123123:role/CloudCustodian\n vars:\n charge_code: xyz\n```\n\nExample of using in a policy file:\n\n```yaml\npolicies:\n - name: ec2-check-tag\n resource: aws.ec2\n filters:\n - \"tag:CostCenter\": \"{charge_code}\"\n```\n\nAnother enhancement for `c7n-org run-script` is to support a few vars in the script arg.\nThe available vars are `account`, `account_id`, `region` and `output_dir`.\n\n```shell\nc7n-org run-script -s . -c my-projects.yml gcp_check_{region}.sh\n# or\nc7n-org run-script -s . -c my-projects.yml use_another_policy_result.sh {output_dir}\n```\n\n**Note:** Variable interpolation is sensitive to proper quoting and spacing,\ni.e., `{ charge_code }` would be invalid due to the extra white space. Additionally,\nyaml parsing can transform a value like `{charge_code}` to null, unless it's quoted\nin strings like the above example. Values that do interpolation into other content\ndon't require quoting, i.e., \"my_{charge_code}\".\n\n## Other commands\n\nc7n-org also supports running arbitrary scripts against accounts via\nthe run-script command. For AWS the standard AWS SDK credential\ninformation is exported into the process environment before executing.\nFor Azure and GCP, only the environment variables\n`AZURE_SUBSCRIPTION_ID` and `PROJECT_ID` are exported(in addition of\nthe system env variables).\n\nc7n-org also supports generating reports for a given policy execution\nacross accounts via the `c7n-org report` subcommand.\n\n## Additional Azure Instructions\n\nIf you're using an Azure Service Principal for executing c7n-org\nyou'll need to ensure that the principal has access to multiple\nsubscriptions.\n\nFor instructions on creating a service principal and granting access\nacross subscriptions, visit the [Azure authentication docs\npage](https://cloudcustodian.io/docs/azure/authentication.html).\n\n## Additional OCI Instructions\n\nThe script `ocitenancies.py` accepts an optional argument `--add-child-tenancies`\nwhich adds all the child tenancies associated with the `DEFAULT` profile's tenancy \nin the generated c7n-org configuration file. If the profile for child tenancy is not available in \nthe OCI configuration file, then either user can add the profile for the child tenancy to the\nOCI configuration file and replace the `<ADD_PROFILE>` entry in the c7n-org configuration with the\ncorresponding profile name or the user can delete the child tenancy entry from the\nc7n-org configuration file. For more info about config file, refer to this [page](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm).\n\nIf the user wants to query for the resources in the specific compartments in c7n-org, then the user\ncan pass the compartment OCID's to the `oci_compartments` under the `vars` section like below. If the \n`oci_comparments` is not passed under `vars`, then the resources will be fetched from the tenancy level.\n\n```yaml\ntenancies:\n- name: dev-tenancy\n profile: DEVTENANCY\n regions:\n - us-ashburn-1\n - us-phoenix-1\n vars:\n oci_compartments: ocid1.test.oc1..<unique_ID>EXAMPLE-compartmentId-2-Value,ocid1.test.oc1..<unique_ID>EXAMPLE-compartmentId-3-Value\n environment: dev\n- name: test-tenancy\n profile: TESTTENANCY\n regions:\n - us-ashburn-1\n vars:\n environment: test\n\n```\n\n\n\n\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Cloud Custodian - Parallel Execution",
"version": "0.6.34",
"project_urls": {
"Documentation": "https://cloudcustodian.io/docs/tools/c7n-org.html",
"Homepage": "https://cloudcustodian.io",
"Repository": "https://github.com/cloud-custodian/cloud-custodian"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "abbe6d46246655a67899a649b59465e2b68c42cfc46f16b619f675ae6c6585ff",
"md5": "f3587013a1436050a7ee5f4027c1016a",
"sha256": "09c70bf128bff890e21d04d5c0cdf06f61f58bfcbfd429d44b1fefbbb39b31e9"
},
"downloads": -1,
"filename": "c7n_org-0.6.34-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f3587013a1436050a7ee5f4027c1016a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 13193,
"upload_time": "2024-03-26T21:21:33",
"upload_time_iso_8601": "2024-03-26T21:21:33.178235Z",
"url": "https://files.pythonhosted.org/packages/ab/be/6d46246655a67899a649b59465e2b68c42cfc46f16b619f675ae6c6585ff/c7n_org-0.6.34-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-26 21:21:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cloud-custodian",
"github_project": "cloud-custodian",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "c7n-org"
}
JSON
