# Cloud Intelligence Dashboards (CUDOS Framework)
[](https://badge.fury.io/py/cid-cmd)
## Welcome to Cloud Intelligence Dashboards (CUDOS Framework) automation repository
This repository contains CloudFormation templates, Terraform modules, and a Command Line tool (cid-cmd) for managing various dashboards provided in AWS Well Architected LAB [Cloud Intelligence Dashboards](https://www.wellarchitectedlabs.com/cloud-intelligence-dashboards/).
There are several ways we can manage dashboards:
1. [CloudFormation Template](./cfn-templates/cid-cfn.yml) (using cid-cmd tool in lambda)
2. [Terraform module](./terraform-modules/cid-dashboards/README.md) (wrapper around CloudFormation Template)
3. Using cid-cmd tool from command line
We recommend cid-cmd tool via [AWS CloudShell](https://console.aws.amazon.com/cloudshell/home).
## Supported dashboards
---
| Dashboard documentation | Demo URL | Prerequisites URL |
| --- | --- | --- |
| [CUDOS Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi#cudos-dashboard) | [demo](https://cid.workshops.aws.dev/demo?dashboard=cudos) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |
| [Cost Intelligence Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi#cost-intelligence-dashboard-(cid)) | [demo](https://cid.workshops.aws.dev/demo?dashboard=cost_intelligence_dashboard) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |
| [Trusted Advisor Organisation (TAO) Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/trusted-advisor) | [demo](https://cid.workshops.aws.dev/demo?dashboard=tao-dashboard) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/trusted-advisor/prerequisites) |
| [Trends Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/additional/trends) | [demo](https://cid.workshops.aws.dev/demo?dashboard=trends-dashboard) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |
| [KPI Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi#kpi-dashboard) | [demo](https://cid.workshops.aws.dev/demo?dashboard=kpi) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |
| [Compute Optimizer Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/compute-optimizer) | [demo](https://cid.workshops.aws.dev/demo?dashboard=compute-optimizer-dashboard) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/compute-optimizer/prerequisites) |
| [Cost Anomaly Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/cost-anomaly) | [demo](https://cid.workshops.aws.dev/demo?dashboard=aws-cost-anomalies) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/cost-anomaly/prerequisites) |
| [Data Transfer Cost Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/additional/data-transfer) | [demo](https://cid.workshops.aws.dev/demo?dashboard=datatransfer-cost-analysis-dashboard) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi) |
See more dashboards on the [workshop page](https://catalog.workshops.aws/awscid/en-US/dashboards).
## Before you start
1. :heavy_exclamation_mark: Complete the prerequisites for respective dashboard (see above).
2. :heavy_exclamation_mark: [Specifying a Query Result Location Using a Workgroup](https://docs.aws.amazon.com/athena/latest/ug/querying.html#query-results-specify-location-workgroup)
3. :heavy_exclamation_mark: Make sure QuickSight [Enterprise edition](https://aws.amazon.com/premiumsupport/knowledge-center/quicksight-enterprise-account/) is activated.
## Command line tool cid-cmd
#### Demo: Deployment of Dashboards with cid-cmd tool
[](https://asciinema.org/a/467770)
#### Install
1. Launch [AWS CloudShell](https://console.aws.amazon.com/cloudshell/home) or your local shell
Automation requires Python 3
2. Make sure you have the latest pip package installed
```bash
python3 -m ensurepip --upgrade
```
4. Install CID Python automation PyPI package
```bash
pip3 install --upgrade cid-cmd
```
#### Dasbhoard Deployment
```bash
cid-cmd deploy
```
#### Update existing Dashboards
Update only Dashboard
```bash
cid-cmd update
```
Update dashboard and all dependencies (Datasets and Athena View). WARNING: this will override any customization of SQL files and Datasets.
```bash
cid-cmd update --force --recursive
```
#### Show Dashboard Status
Show dashboards status
```bash
cid-cmd status
```
[<img width="558" alt="status" src="https://github.com/aws-samples/aws-cudos-framework-deployment/assets/82834333/cae2015f-0f81-4593-80b3-c67ec1200fcd">](https://www.youtube.com/watch?v=ivr1MoGaApM)
#### Share QuickSight resources
```bash
cid-cmd share
```
#### Initialize Amazon QuickSight
One time action to initialize Amazon QuickSight Enterprise Edition.
```bash
cid-cmd init-qs
```
#### Initialize CUR
One time action to initialize Athena table and Crawler from s3 with CUR data.
```bash
cid-cmd init-cur
```
#### Delete Dashboard and all dependencies unused by other
Delete Dashboards and all dependencies unused by other CID-managed dashboards.(including QuickSight datasets, Athena views and tables)
```bash
cid-cmd delete
```
#### Delete Command Options:
```
--dashboard-id TEXT QuickSight dashboard id
--athena-database TEXT Athena database
```
#### Export
The command `export` lets you download or share a customized dashboard with another AWS Account. It takes the QuickSight Analysis as an input and generates all the assets needed to deploy your Analysis into another AWS Account. This command will generate a yaml file with a description of the Dashboard and all required Datasets. Also this command generates a QuickSight Template in the current AWS Account that can be used for Dashboard deployment in other accounts. The resource file can be used with all other cid commands. Both accounts must have relevant Athena Views and Tables.
Export from account A:
```
cid-cmd export
```
Deployment to account B:
```
cid-cmd deploy --resources ./mydashboard.yaml
```
#### See available commands and command line options
```
cid-cmd --help
```
#### CSV to Athena View
Generate and SQL file for Athena View for CSV file
```
cid-cmd csv2view --input my_mapping.csv --name my_mapping
```
This command generates a SQL file that you can execute. Please mind [Athena Service Limit for Query Size](https://docs.aws.amazon.com/athena/latest/ug/service-limits.html#service-limits-query-string-length).
## Cloud Formation
CID is also provided in a form of CloudFormation templates. See detailed instructions in the [Well Architected Labs](https://wellarchitectedlabs.com/cost/200_labs/200_cloud_intelligence/cost-usage-report-dashboards/dashboards/deploy_dashboards/) site.
## Terraform
CID offers a set of Terraform modules to deploy CUR replication and CID dashboards. These modules act as a wrapper around CloudFormation.
1. Create a bucket for consolidating CUR [terraform-modules/cur-setup-destination/](terraform-modules/cur-setup-destination/)
2. Create a CUR in Payer Account(s) [terraform-modules/cur-setup-source/](terraform-modules/cur-setup-source/)
3. Create Dashboards [terraform-modules/cid-dashboards/](terraform-modules/cid-dashboards/)
## Rights Management
The ownership of CID is usually with the FinOps team, who do not have administrative access. However, they require specific privileges to install and operate CID dashboards. To assist the Admin team in granting the necessary privileges to the CID owners, a CFN template is provided. This template, located at [CFN template](cfn-templates/cid-admin-policies.yaml), takes an IAM role name as a parameter and adds the required policies to the role.
## Troubleshooting and Support
If you experience unexpected behaviour of the `cid-cmd` script please run `cid-cmd` in debug mode:
```bash
cid-cmd -vv [command]
```
This will produce a log file in the same directory that were at the tile of launch of cid-cmd.
:heavy_exclamation_mark:Inspect the produced debug log for any sensitive information and anonymize it.
We encourage you to open [new issue](https://github.com/aws-samples/aws-cudos-framework-deployment/issues/new) with description of the problem and attached debug log file.
Raw data
{
"_id": null,
"home_page": "https://github.com/aws-samples/aws-cudos-framework-deployment",
"name": "cid-cmd",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "aws, cmd, cli, cost intelligence dashboards",
"author": "AWS CUDOS Team",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/91/78/8c8ddd2017040d8a4fd42a30b10a41df04c60dcd3310ee05d75d5d210b30/cid_cmd-0.3.3.tar.gz",
"platform": null,
"description": "# Cloud Intelligence Dashboards (CUDOS Framework)\n\n[](https://badge.fury.io/py/cid-cmd)\n\n## Welcome to Cloud Intelligence Dashboards (CUDOS Framework) automation repository\nThis repository contains CloudFormation templates, Terraform modules, and a Command Line tool (cid-cmd) for managing various dashboards provided in AWS Well Architected LAB [Cloud Intelligence Dashboards](https://www.wellarchitectedlabs.com/cloud-intelligence-dashboards/).\n\nThere are several ways we can manage dashboards:\n1. [CloudFormation Template](./cfn-templates/cid-cfn.yml) (using cid-cmd tool in lambda)\n2. [Terraform module](./terraform-modules/cid-dashboards/README.md) (wrapper around CloudFormation Template)\n3. Using cid-cmd tool from command line\n\nWe recommend cid-cmd tool via [AWS CloudShell](https://console.aws.amazon.com/cloudshell/home).\n\n## Supported dashboards\n---\n| Dashboard documentation | Demo URL | Prerequisites URL |\n| --- | --- | --- |\n| [CUDOS Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi#cudos-dashboard) | [demo](https://cid.workshops.aws.dev/demo?dashboard=cudos) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |\n| [Cost Intelligence Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi#cost-intelligence-dashboard-(cid)) | [demo](https://cid.workshops.aws.dev/demo?dashboard=cost_intelligence_dashboard) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |\n| [Trusted Advisor Organisation (TAO) Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/trusted-advisor) | [demo](https://cid.workshops.aws.dev/demo?dashboard=tao-dashboard) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/trusted-advisor/prerequisites) |\n| [Trends Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/additional/trends) | [demo](https://cid.workshops.aws.dev/demo?dashboard=trends-dashboard) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |\n| [KPI Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi#kpi-dashboard) | [demo](https://cid.workshops.aws.dev/demo?dashboard=kpi) | [link](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting) |\n| [Compute Optimizer Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/compute-optimizer) | [demo](https://cid.workshops.aws.dev/demo?dashboard=compute-optimizer-dashboard) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/compute-optimizer/prerequisites) |\n| [Cost Anomaly Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/cost-anomaly) | [demo](https://cid.workshops.aws.dev/demo?dashboard=aws-cost-anomalies) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/cost-anomaly/prerequisites) |\n| [Data Transfer Cost Dashboard](https://catalog.workshops.aws/awscid/en-US/dashboards/additional/data-transfer) | [demo](https://cid.workshops.aws.dev/demo?dashboard=datatransfer-cost-analysis-dashboard) | [link](https://catalog.workshops.aws/awscid/en-US/dashboards/foundational/cudos-cid-kpi) |\n\nSee more dashboards on the [workshop page](https://catalog.workshops.aws/awscid/en-US/dashboards).\n\n## Before you start\n1. :heavy_exclamation_mark: Complete the prerequisites for respective dashboard (see above).\n2. :heavy_exclamation_mark: [Specifying a Query Result Location Using a Workgroup](https://docs.aws.amazon.com/athena/latest/ug/querying.html#query-results-specify-location-workgroup)\n3. :heavy_exclamation_mark: Make sure QuickSight [Enterprise edition](https://aws.amazon.com/premiumsupport/knowledge-center/quicksight-enterprise-account/) is activated.\n\n## Command line tool cid-cmd\n\n#### Demo: Deployment of Dashboards with cid-cmd tool\n\n [](https://asciinema.org/a/467770)\n\n#### Install\n\n1. Launch [AWS CloudShell](https://console.aws.amazon.com/cloudshell/home) or your local shell\n\n Automation requires Python 3\n\n2. Make sure you have the latest pip package installed\n ```bash\n python3 -m ensurepip --upgrade\n ```\n\n4. Install CID Python automation PyPI package\n ```bash\n pip3 install --upgrade cid-cmd\n ```\n\n#### Dasbhoard Deployment\n\n```bash\ncid-cmd deploy\n```\n\n\n#### Update existing Dashboards\nUpdate only Dashboard\n```bash\ncid-cmd update\n```\nUpdate dashboard and all dependencies (Datasets and Athena View). WARNING: this will override any customization of SQL files and Datasets.\n```bash\ncid-cmd update --force --recursive\n```\n#### Show Dashboard Status\nShow dashboards status\n\n```bash\ncid-cmd status\n```\n[<img width=\"558\" alt=\"status\" src=\"https://github.com/aws-samples/aws-cudos-framework-deployment/assets/82834333/cae2015f-0f81-4593-80b3-c67ec1200fcd\">](https://www.youtube.com/watch?v=ivr1MoGaApM)\n\n\n\n\n#### Share QuickSight resources\n```bash\ncid-cmd share\n```\n\n#### Initialize Amazon QuickSight\nOne time action to initialize Amazon QuickSight Enterprise Edition.\n\n```bash\ncid-cmd init-qs\n```\n\n#### Initialize CUR\nOne time action to initialize Athena table and Crawler from s3 with CUR data.\n\n```bash\ncid-cmd init-cur\n```\n\n#### Delete Dashboard and all dependencies unused by other\nDelete Dashboards and all dependencies unused by other CID-managed dashboards.(including QuickSight datasets, Athena views and tables)\n```bash\ncid-cmd delete\n```\n\n#### Delete Command Options:\n```\n --dashboard-id TEXT QuickSight dashboard id\n --athena-database TEXT Athena database\n ```\n\n#### Export\nThe command `export` lets you download or share a customized dashboard with another AWS Account. It takes the QuickSight Analysis as an input and generates all the assets needed to deploy your Analysis into another AWS Account. This command will generate a yaml file with a description of the Dashboard and all required Datasets. Also this command generates a QuickSight Template in the current AWS Account that can be used for Dashboard deployment in other accounts. The resource file can be used with all other cid commands. Both accounts must have relevant Athena Views and Tables.\n\nExport from account A:\n```\ncid-cmd export\n```\n\nDeployment to account B:\n```\ncid-cmd deploy --resources ./mydashboard.yaml\n```\n\n#### See available commands and command line options\n```\ncid-cmd --help\n```\n\n#### CSV to Athena View\nGenerate and SQL file for Athena View for CSV file\n\n```\ncid-cmd csv2view --input my_mapping.csv --name my_mapping\n```\nThis command generates a SQL file that you can execute. Please mind [Athena Service Limit for Query Size](https://docs.aws.amazon.com/athena/latest/ug/service-limits.html#service-limits-query-string-length).\n\n\n## Cloud Formation\nCID is also provided in a form of CloudFormation templates. See detailed instructions in the [Well Architected Labs](https://wellarchitectedlabs.com/cost/200_labs/200_cloud_intelligence/cost-usage-report-dashboards/dashboards/deploy_dashboards/) site.\n\n## Terraform\nCID offers a set of Terraform modules to deploy CUR replication and CID dashboards. These modules act as a wrapper around CloudFormation.\n\n 1. Create a bucket for consolidating CUR [terraform-modules/cur-setup-destination/](terraform-modules/cur-setup-destination/)\n 2. Create a CUR in Payer Account(s) [terraform-modules/cur-setup-source/](terraform-modules/cur-setup-source/)\n 3. Create Dashboards [terraform-modules/cid-dashboards/](terraform-modules/cid-dashboards/)\n\n\n## Rights Management\nThe ownership of CID is usually with the FinOps team, who do not have administrative access. However, they require specific privileges to install and operate CID dashboards. To assist the Admin team in granting the necessary privileges to the CID owners, a CFN template is provided. This template, located at [CFN template](cfn-templates/cid-admin-policies.yaml), takes an IAM role name as a parameter and adds the required policies to the role.\n\n\n## Troubleshooting and Support\nIf you experience unexpected behaviour of the `cid-cmd` script please run `cid-cmd` in debug mode:\n\n```bash\ncid-cmd -vv [command]\n```\n \nThis will produce a log file in the same directory that were at the tile of launch of cid-cmd. \n\n:heavy_exclamation_mark:Inspect the produced debug log for any sensitive information and anonymize it.\n\nWe encourage you to open [new issue](https://github.com/aws-samples/aws-cudos-framework-deployment/issues/new) with description of the problem and attached debug log file.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Cloud Intelligence Dashboards deployment helper tool",
"version": "0.3.3",
"project_urls": {
"Homepage": "https://github.com/aws-samples/aws-cudos-framework-deployment"
},
"split_keywords": [
"aws",
" cmd",
" cli",
" cost intelligence dashboards"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6b113c5ecf8ba0fc68123b61739bc31c5a8a3bec7019031e851c40edc6fd4bf1",
"md5": "e29731a3df152a5457db48be25026c03",
"sha256": "525094bcb0502ade292ce957c49b4a4cdc8dbe4ceec2f75b799d3c37bdb1011f"
},
"downloads": -1,
"filename": "cid_cmd-0.3.3-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "e29731a3df152a5457db48be25026c03",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 229579,
"upload_time": "2024-04-25T09:08:16",
"upload_time_iso_8601": "2024-04-25T09:08:16.184189Z",
"url": "https://files.pythonhosted.org/packages/6b/11/3c5ecf8ba0fc68123b61739bc31c5a8a3bec7019031e851c40edc6fd4bf1/cid_cmd-0.3.3-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "91788c8ddd2017040d8a4fd42a30b10a41df04c60dcd3310ee05d75d5d210b30",
"md5": "b390ee20129b0b18e47e8a24f8b1a485",
"sha256": "332187ef44926d8e80e32e037359f836df2a44a73afb72ceee70b48d302e8577"
},
"downloads": -1,
"filename": "cid_cmd-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "b390ee20129b0b18e47e8a24f8b1a485",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 171163,
"upload_time": "2024-04-25T09:08:18",
"upload_time_iso_8601": "2024-04-25T09:08:18.545758Z",
"url": "https://files.pythonhosted.org/packages/91/78/8c8ddd2017040d8a4fd42a30b10a41df04c60dcd3310ee05d75d5d210b30/cid_cmd-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-25 09:08:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aws-samples",
"github_project": "aws-cudos-framework-deployment",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "boto3",
"specs": [
[
">=",
"1.29.5"
]
]
},
{
"name": "Click",
"specs": [
[
">=",
"8.0"
]
]
},
{
"name": "PyYAML",
"specs": []
},
{
"name": "requests",
"specs": []
},
{
"name": "six",
"specs": [
[
">=",
"1.15"
]
]
},
{
"name": "tqdm",
"specs": []
},
{
"name": "tzlocal",
"specs": [
[
">=",
"4.0"
]
]
},
{
"name": "questionary",
"specs": [
[
">=",
"1.10"
]
]
}
],
"lcname": "cid-cmd"
}
JSON
