# claranet-tfwrapper
[](CHANGELOG.md) [](LICENSE) [](https://pypi.org/project/claranet-tfwrapper/)
`tfwrapper` is a python wrapper for [OpenTofu](https://opentofu.org/) and legacy [Terraform](https://www.terraform.io/) which aims to simplify their usage and enforce best practices.
Note: the term _Terraform_ is used in this documentation when talking about generic concepts like providers, modules, stacks and the HCL based domain specific language.
## Table Of Contents
<!--TOC-->
- [claranet-tfwrapper](#claranet-tfwrapper)
- [Table Of Contents](#table-of-contents)
- [Features](#features)
- [Setup Dependencies](#setup-dependencies)
- [Runtime Dependencies](#runtime-dependencies)
- [Recommended setup](#recommended-setup)
- [Installation](#installation)
- [Setup command-line completion](#setup-command-line-completion)
- [Upgrade from tfwrapper v7 or older](#upgrade-from-tfwrapper-v7-or-older)
- [Required files](#required-files)
- [Configuration](#configuration)
- [tfwrapper configuration](#tfwrapper-configuration)
- [Stacks configurations](#stacks-configurations)
- [States centralization configuration](#states-centralization-configuration)
- [How to migrate from one backend to another for state centralization](#how-to-migrate-from-one-backend-to-another-for-state-centralization)
- [Stacks file structure](#stacks-file-structure)
- [Usage](#usage)
- [Stack bootstrap](#stack-bootstrap)
- [Working on stacks](#working-on-stacks)
- [Passing options](#passing-options)
- [Environment](#environment)
- [S3 state backend credentials](#s3-state-backend-credentials)
- [Azure Service Principal credentials](#azure-service-principal-credentials)
- [Azure authentication isolation](#azure-authentication-isolation)
- [GCP configuration](#gcp-configuration)
- [GKE configurations](#gke-configurations)
- [Stack configurations and credentials](#stack-configurations-and-credentials)
- [Stack path](#stack-path)
- [Development](#development)
- [Tests](#tests)
- [Debug command-line completion](#debug-command-line-completion)
- [Python code formatting](#python-code-formatting)
- [Checks](#checks)
- [README TOC](#readme-toc)
- [Using OpenTofu development builds](#using-opentofu-development-builds)
- [git pre-commit hooks](#git-pre-commit-hooks)
- [Review and merge open Dependabot PRs](#review-and-merge-open-dependabot-prs)
- [Tagging and publishing new releases to PyPI](#tagging-and-publishing-new-releases-to-pypi)
<!--TOC-->
## Features
- OpenTofu and Terraform behaviour overriding
- State centralization enforcement
- Standardized file structure
- Stack initialization from templates
- AWS credentials caching
- Azure credentials loading (both Service Principal or User)
- GCP and GKE user ADC support
- Plugins caching
- Tab completion
## Setup Dependencies
- `python3` `>= 3.8.1 <4.0`
- `python3-pip`
- `python3-venv`
- `pipx` (recommended)
## Runtime Dependencies
- `terraform` `>= 0.10` (`>= 0.15` for fully working Azure backend with isolation due to https://github.com/hashicorp/terraform/issues/25416)
- `azure-cli` when using context based Azure authentication
## Recommended setup
- OpenTofu 1.6+ (recommended) or Terraform 1.0+ (warning: versions above 1.6 are not open-source, and may cause legal issues depending on the context you are using it).
- An AWS S3 bucket and DynamoDB table for state centralization in AWS.
- An Azure Blob Storage container for state centralization in Azure.
## Installation
tfwrapper should installed using pipx (recommended) or pip:
```bash
pipx install claranet-tfwrapper
```
## Setup command-line completion
Add the following to your shell's interactive configuration file, e.g. `.bashrc` for bash:
```bash
eval "$(register-python-argcomplete tfwrapper -e tfwrapper)"
```
You can then press the completion key (usually `Tab ↹`) twice to get your partially typed `tfwrapper` commands completed.
Note: the `-e tfwrapper` parameter adds an suffix to the defined `_python_argcomplete` function to avoid clashes with other packages (see https://github.com/kislyuk/argcomplete/issues/310#issuecomment-697168326 for context).
## Upgrade from tfwrapper v7 or older
If you used versions of the wrapper older than v8, there is not much to do when upgrading to v8
except a little cleanup.
Indeed, the wrapper is no longer installed as a git submodule of your project like it used to be instructed and there is no longer any `Makefile` to activate it.
Just clean up each project by destroying the `.wrapper` submodule:
```bash
git rm -f Makefile
git submodule deinit .wrapper
rm -rf .git/modules/.wrapper
git rm -f .wrapper
```
Then check the staged changes and commit them.
### Required files
tfwrapper expects multiple files and directories at the root of a project.
#### conf
Stacks configurations are stored in the `conf` directory.
#### templates
The `templates` directory is used to store the state backend configuration template and the Terraform stack templates used to initialize new stacks. Using a git submodule is recommended.
The following files are required:
- `templates/{provider}/common/state.tf.jinja2`: AWS S3 or Azure Blob Storage state backend configuration template.
- `templates/{provider}/basic/main.tf`: the default Terraform configuration for new stacks. The whole `template/{provider}/basic` directory is copied on stack initialization.
For example with AWS:
```bash
mkdir -p templates/aws/common templates/aws/basic
# create state configuration template with AWS backend
cat << 'EOF' > templates/aws/common/state.tf.jinja2
{% if region is not none %}
{% set region = '/' + region + '/' %}
{% else %}
{% set region = '/' %}
{% endif %}
terraform {
backend "s3" {
bucket = "my-centralized-terraform-states-bucket"
key = "{{ client_name }}/{{ account }}/{{ environment }}{{ region }}{{ stack }}/terraform.state"
region = "eu-west-1"
dynamodb_table = "my-terraform-states-lock-table"
}
}
resource "null_resource" "state-test" {}
EOF
# create a default stack templates with support for AWS assume role
cat << 'EOF' > templates/aws/basic/main.tf
provider "aws" {
region = var.aws_region
access_key = var.aws_access_key
secret_key = var.aws_secret_key
token = var.aws_token
}
EOF
```
For example with Azure:
```bash
mkdir -p templates/azure/common templates/azure/basic
# create state configuration template with Azure backend
cat << 'EOF' > templates/azure/common/state.tf.jinja2
{% if region is not none %}
{% set region = '/' + region + '/' %}
{% else %}
{% set region = '/' %}
{% endif %}
terraform {
backend "azurerm" {
subscription_id = "00000000-0000-0000-0000-000000000000"
resource_group_name = "my-resource-group"
storage_account_name = "my-centralized-terraform-states-account"
container_name = "terraform-states"
key = "{{ client_name }}/{{ account }}/{{ environment }}{{ region }}{{ stack }}/terraform.state"
}
}
EOF
# create a default stack templates with support for Azure credentials
cat << 'EOF' > templates/azure/basic/main.tf
provider "azurerm" {
subscription_id = var.azure_subscription_id
tenant_id = var.azure_tenant_id
}
EOF
```
#### .run
The `.run` directory is used for credentials caching and plan storage.
```bash
mkdir .run
cat << 'EOF' > .run/.gitignore
*
!.gitignore
EOF
```
#### .gitignore
Adding the following `.gitignore` at the root of your project is recommended:
```bash
cat << 'EOF' > .gitignore
.terraform
terraform.tfstate
terraform.tfstate.backup
terraform.tfvars
EOF
```
## Configuration
tfwrapper uses yaml files stored in the `conf` directory of the project.
### tfwrapper configuration
tfwrapper uses some default behaviors that can be overridden or modified via a `config.yml` file in the `conf` directory.
```yaml
---
always_trigger_init: False # Always trigger `terraform init` first when launching `plan` or `apply` commands
pipe_plan_command: "cat" # Default command used when you're invoking tfwrapper with `--pipe-plan`
use_local_azure_session_directory: False # Use the current user's Azure configuration in `~/.azure`. By default, the wrapper uses a local `azure-cli` session and configuration in the local `.run` directory.
```
### Stacks configurations
Stacks configuration files use the following naming convention:
```bash
conf/${account}_${environment}_${region}_${stack}.yml
```
Here is an example for an AWS stack configuration:
```yaml
---
state_configuration_name: "aws" # use "aws" backend state configuration
aws:
general:
account: &aws_account "xxxxxxxxxxx" # aws account for this stack
region: &aws_region eu-west-1 # aws region for this stack
credentials:
profile: my-aws-profile # should be configured in .aws/config
terraform:
legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.
version: "1.0" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0
vars: # variables passed to terraform
aws_account: *aws_account
aws_region: *aws_region
client_name: my-client-name # arbitrary client name
```
Here is an example for a stack on Azure configuration using user mode and AWS S3 backend for state storage:
```yaml
---
state_configuration_name: "aws-demo" # use "aws" backend state configuration
azure:
general:
mode: user # Uses personal credentials with MFA
directory_id: &directory_id "00000000-0000-0000-0000-000000000000" # Azure Tenant/Directory UID
subscription_id: &subscription_id "11111111-1111-1111-1111-111111111111" # Azure Subscription UID
terraform:
legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.
version: "1.0" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0
vars:
subscription_id: *subscription_id
directory_id: *directory_id
client_name: client-name #Replace it with the name of your client
```
It is using your account linked to a Microsoft Account. You must have access to the Azure Subscription if you want to use Terraform.
Here is an example for a stack on Azure configuration using Service Principal mode:
```yaml
---
azure:
general:
mode: service_principal # Uses an Azure tenant Service Principal account
directory_id: &directory_id "00000000-0000-0000-0000-000000000000" # Azure Tenant/Directory UID
subscription_id: &subscription_id "11111111-1111-1111-1111-111111111111" # Azure Subscription UID
credentials:
profile: customer-profile # To stay coherent, create an AzureRM profile with the same name as the account-alias. Please checkout `azurerm_config.yml.sample` file for configuration structure.
terraform:
legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.
version: "1.0" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0
vars:
subscription_id: *subscription_id
directory_id: *directory_id
client_name: client-name # Replace it with the name of your client
```
The wrapper uses the Service Principal's credentials to connect the Azure subscription. The given Service Principal must have access to the subscription.
The wrapper loads `client_id`, `client_secret` and `tenant_id` properties from your `config.yml` file located in `~/.azurerm/config.yml`.
`~/.azurerm/config.yml` file structure example:
```yaml
---
claranet-sandbox:
client_id: aaaaaaaa-bbbb-cccc-dddd-zzzzzzzzzzzz
client_secret: AAbbbCCCzzz==
tenant_id: 00000000-0000-0000-0000-000000000000
customer-profile:
client_id: aaaaaaaa-bbbb-cccc-dddd-zzzzzzzzzzzz
client_secret: AAbbbCCCzzz==
tenant_id: 00000000-0000-0000-0000-000000000000
```
Here is an example for a GCP/GKE stack with user ADC and multiple GKE instances:
```yaml
---
gcp:
general:
mode: adc-user
project: &gcp_project project-name
gke:
- name: kubernetes-1
zone: europe-west1-c
- name: kubernetes-2
region: europe-west1
terraform:
legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.
version: "1.0" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0
vars:
gcp_region: europe-west1
gcp_zone: europe-west1-c
gcp_project: *gcp_project
client_name: client-name
```
You can declare multiple providers configurations, context is set up accordingly.
⚠ This feature is only supported for Azure stacks for now and only works with [Azure authentication isolation](#azure-authentication-isolation)
```yaml
---
azure:
general:
mode: service_principal # Uses an Azure tenant Service Principal account
directory_id: &directory_id "00000000-0000-0000-0000-000000000000" # Azure Tenant/Directory UID
subscription_id: &subscription_id "11111111-1111-1111-1111-111111111111" # Azure Subscription UID
credentials:
profile: customer-profile # To stay coherent, create an AzureRM profile with the same name as the account-alias. Please checkout `azurerm_config.yml.sample` file for configuration structure.
alternative:
mode: service_principal # Uses an Azure tenant Service Principal account
directory_id: "00000000-0000-0000-0000-000000000000" # Azure Tenant/Directory UID
subscription_id: "22222222-2222-2222-2222-222222222222" # Azure Subscription UID
credentials:
profile: claranet-sandbox # To stay coherent, create an AzureRM profile with the same name as the account-alias. Please checkout `azurerm_config.yml.sample` file for configuration structure.
terraform:
version: "1.0" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0
legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.
vars:
subscription_id: *subscription_id
directory_id: *directory_id
client_name: client-name # Replace it with the name of your client
```
This configuration is useful when having various service principals with a dedicated rights scope for each.
The wrapper will generate the following Terraform variables that can be used in your stack:
- `<config_name>_azure_subscription_id` with Azure subscription ID. From the example, variable is: `alternative_subscription_id = "22222222-2222-2222-2222-222222222222"`
- `<config_name>_azure_tenant_id` with Azure tenant ID. From the example, variable is: `alternative_tenant_id = "00000000-0000-0000-0000-000000000000"`
- `<config_name>_azure_client_id` with Service Principal client id. From the example, variable is: `alternative_client_id = "aaaaaaaa-bbbb-cccc-dddd-zzzzzzzzzzzz"`
- `<config_name>_azure_client_secret` with Service Principal client secret. From the example, variable is: `alternative_client_secret = "AAbbbCCCzzz=="`
Also, an isolation context is set to the local `.run/aure_<config_name>` directory for each configuration.
### States centralization configuration
The `conf/state.yml` configuration file defines the configuration used to connect to state backends.
These backends can be of [AWS S3](https://developer.hashicorp.com/terraform/language/settings/backends/s3) and/or [AzureRM](https://developer.hashicorp.com/terraform/language/settings/backends/azurerm) types.
The resources for these backends are not created by tfwrapper, and thus must exist beforehand:
* AWS: an S3 bucket (and optionally but highly recommended a DynamoDB table for locking). It is also recommended to enable versioning on the S3 bucket.
* Azure: a Blob storage account
You can use other backends (e.g. Google GCS or Hashicorp Consul) not specifically supported by the wrapper, if you manage authentication yourself and omit the `conf/state.yml` file or make it empty:
```yaml
---
```
Example configuration with both AWS and Azure backends defined:
```yaml
---
aws:
- name: "aws-demo"
general:
account: "xxxxxxxxxxx"
region: eu-west-1
credentials:
profile: my-state-aws-profile # should be configured in .aws/config
azure:
# This backend use storage keys for authentication
- name: "azure-backend"
general:
subscription_id: "xxxxxxx" # the Azure account to use for state storage
resource_group_name: "tfstates-xxxxx-rg" # The Azure resource group with state storage
storage_account_name: "tfstatesxxxxx"
- name: "azure-alternative"
general:
subscription_id: "xxxxxxx" # the Azure account to use for state storage
resource_group_name: "tfstates-xxxxx-rg" # The Azure resource group with state storage
storage_account_name: "tfstatesxxxxx"
# This backend use Azure AD authentication
- name: "azure-ad-auth"
general:
subscription_id: "xxxxxxx" # the Azure account to use for state storage
resource_group_name: "tfstates-xxxxx-rg" # The Azure resource group with state storage
storage_account_name: "tfstatesxxxxx"
azuread_auth: true
backend_parameters: # Parameters or options which can be used by `state.j2.tf` template file
state_snaphot: "false" # Example of Azure storage backend option
```
Note: the first backend will be the default one for stacks not defining `state_backend_type`.
### How to migrate from one backend to another for state centralization
If for example you have both an AWS and Azure state backend configured in your `conf/state.yml` file,
you can migrate your stack state from one backend to another.
Here is a quick howto:
1. Make sure your stack is clean:
```bash
$ cd account/path/env/your_stack
$ tfwrapper init
$ tfwrapper plan
# should return no changes
```
2. Change your backend in the stack configuration yaml file:
```yaml
---
#state_configuration_name: 'aws-demo' # previous backend
state_configuration_name: "azure-alternative" # new backend to use
```
3. Back in your stack directory, you can perform the change:
```bash
$ cd account/path/env/your_stack
$ rm -v state.tf # removing old state backend configuration
$ tfwrapper bootstrap # regen a new state backend configuration based on the stack yaml config file
$ tfwrapper init # Terraform will detect the new backend and propose to migrate it
$ tfwrapper plan
# should return the same changes diff as before
```
## Stacks file structure
Terraform stacks are organized based on their:
- `account`: an account alias which may refer to provider accounts or subscriptions, e.g. `project-a-prod`, `customer-b-dev`.
- `environment`: `production`, `preproduction`, `dev`, etc. With `global` as a special case eliminating the `region` part.
- `region`: `eu-west-1`, `westeurope`, etc.
- `stack`: defaults to `default`. `web`, `admin`, `tools`, etc.
The following file structure is then enforced:
```
# project root
└── account
│ └── environment
│ └── region
│ └── stack
└── account
└── _global
└── stack
```
A real-life example:
```
# project root
├── aws-account-1
│ ├── _global
│ │ └── default
│ │ └── main.tf
│ └── production
│ ├── eu-central-1
│ │ └── web
│ │ └── main.tf
│ └── eu-west-1
│ ├── default
│ │ └── main.t
│ └── tools
│ └── main.tf
└── aws-account-2
└── backup
└── eu-west-1
└── backup
└── main.tf
```
## Usage
### Stack bootstrap
After creating a `conf/${account}_${environment}_${region}_${stack}.yml` stack configuration file you can bootstrap it.
```bash
# you can bootstrap using the templates/{provider}/basic stack
tfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} bootstrap
# or another stack template, for example: templates/aws/foobar
tfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} bootstrap aws/foobar
# or from an existent stack, for example: customer/env/region/stack
tfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} bootstrap mycustomer/dev/eu-west/run
```
In the special case of a global stack, the configuration file should instead be named as `conf/${account}_global_${stack}.yml`.
### Working on stacks
You can work on stacks from their directory or from the root of the project.
```bash
# working from the root of the project
tfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} plan
# working from the root of a stack
cd ${account}/${environment}/${region}/${stack}
tfwrapper plan
```
You can also work on several stacks sequentially with the `foreach` subcommand from any directory under the root of the project.
By default, `foreach` selects all stacks under the current directory,
so if called from the root of the project without any filter,
it will select all stacks and execute the specified command in them, one after another:
```bash
# working from the root of the project
tfwrapper foreach -- tfwrapper init
```
Any combination of the `-a`, `-e`, `-r` and `-s` arguments can be used to select specific stacks,
e.g. all stacks for an account across all environments but in a specific region:
```bash
# working from the root of the project
tfwrapper -a ${account} -r ${region} foreach -- tfwrapper plan
```
The same can be achieved with:
```bash
# working from an account directory
cd ${account}
tfwrapper -r ${region} foreach -- tfwrapper plan
```
Complex commands can be executed in a sub-shell with the `-S`/`--shell` argument, e.g.:
```bash
# working from an environment directory
cd ${account}/${environment}
tfwrapper foreach -S 'pwd && tfwrapper init >/dev/null 2>&1 && tfwrapper plan 2>/dev/null -- -no-color | grep "^Plan: "'
```
### Passing options
You can pass anything you want to `terraform` using `--`.
```bash
tfwrapper plan -- -target resource1 -target resource2
```
## Environment
tfwrapper sets the following environment variables.
### S3 state backend credentials
The default AWS credentials of the environment are set to point to the S3 state backend. Those credentials are acquired from the profile defined in `conf/state.yml`
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_SESSION_TOKEN`
### Azure Service Principal credentials
Those AzureRM credentials are loaded only if you are using the Service Principal mode. They are acquired from the profile defined in `~/.azurerm/config.yml`
- `ARM_CLIENT_ID`
- `ARM_CLIENT_SECRET`
- `ARM_TENANT_ID`
### Azure authentication isolation
`AZURE_CONFIG_DIR` environment variable is set to the local `.run/azure` directory if global configuration value `use_local_azure_session_directory` is set to `true`, which is the default, which is the default.
If you have multiple configurations in your stacks, you also have `<CONFIG_NAME>_AZURE_CONFIG_DIR` which is set to the local `.run/azure_<config_name>` directory.
### GCP configuration
Those GCP related variables are available from the environment when using the example configuration:
- `TF_VAR_gcp_region`
- `TF_VAR_gcp_zone`
- `TF_VAR_gcp_project`
### GKE configurations
Each GKE instance has its own kubeconfig, the path to each configuration is available from the environment:
- `TF_VAR_gke_kubeconfig_${gke_cluster_name}`
kubeconfig is automatically fetched by the wrapper (using gcloud) and stored inside the `.run` directory of your project.
It is refreshed automatically at every run to ensure you point to correct Kubernetes endpoint.
You can disable this behaviour by setting `refresh_kubeconfig: never` in your cluster settings.
```yaml
---
gcp:
general:
mode: adc-user
project: &gcp_project project-name
gke:
- name: kubernetes-1
zone: europe-west1-c
refresh_kubeconfig: never
```
### Stack configurations and credentials
The `terraform['vars']` dictionary from the stack configuration is accessible as Terraform variables.
The profile defined in the stack configuration is used to acquire credentials accessible from Terraform.
There is two supported providers, the variables which will be loaded depends on the used provider.
- `TF_VAR_client_name` (if set in .yml stack configuration file)
- `TF_VAR_aws_account`
- `TF_VAR_aws_region`
- `TF_VAR_aws_access_key`
- `TF_VAR_aws_secret_key`
- `TF_VAR_aws_token`
- `TF_VAR_azurerm_region`
- `TF_VAR_azure_region`
- `TF_VAR_azure_subscription_id`
- `TF_VAR_azure_tenant_id`
- ~~`TF_VAR_azure_state_access_key`~~ (removed in `v11.0.0`)
### Stack path
The stack path is passed to Terraform. This is especially useful for resource naming and tagging.
- `TF_VAR_account`
- `TF_VAR_environment`
- `TF_VAR_region`
- `TF_VAR_stack`
# Development
## Tests
All new code contributions should come with unit and/or integrations tests.
To run those tests locally, use [tox](https://github.com/tox-dev/tox):
```bash
poetry run tox -e py
```
Linters are also used to ensure code respects our standards.
To run those linters locally:
```bash
poetry run tox -e lint
```
## Debug command-line completion
You can get verbose debugging information for `argcomplete` by defining the following environment variable:
```bash
export _ARC_DEBUG=1
```
## Python code formatting
Our code is formatted with [black](https://github.com/psf/black/).
Make sure to format all your code contributions with `black ${filename}`.
Hint: enable auto-format on save with `black` in your [favorite IDE](https://black.readthedocs.io/en/stable/editor_integration.html).
## Checks
To run code and documentation style checks, run `tox -e lint`.
In addition to `black --check`, code is also checked with:
- [flake8](https://github.com/pycqa/flake8), a wrapper for [pycodestyle](https://github.com/PyCQA/pycodestyle) and [pyflakes](https://github.com/PyCQA/pyflakes).
- [flake8-docstrings](https://github.com/pycqa/flake8-docstrings), a wrapper for [pydocstyle](https://github.com/PyCQA/pydocstyle).
## README TOC
This [README's table of content](#table-of-contents) is formatted with [md_toc](https://github.com/frnmst/md-toc).
Keep in mind to update it with `md_toc --in-place github README.md`.
## Using OpenTofu development builds
To build and use development versions of OpenTofu, put them in a `~/.terraform.d/versions/X.Y/X.Y.Z-dev/` folder:
```bash
# git clone https://github.com/opentofu/opentofu.git ~/go/src/github.com/opentofu/opentofu
# cd ~/go/src/github.com/opentofu/opentofu
# go build ./cmd/tofu/
# ./tofu version
OpenTofu v1.6.0-dev
on linux_amd64
# mkdir -p ~/.terraform.d/versions/1.6/1.6.0-dev
# mv ./opentofu ~/.terraform.d/versions/1.6/1.6.0-dev/
```
## git pre-commit hooks
Some git pre-commit hooks are configured in `.pre-commit-config.yaml` for use with the [pre-commit tool](https://pre-commit.com).
Using them helps avoiding to push changes that will fail the CI.
They can be installed locally with:
```bash
# pre-commit install
```
If updating hooks configuration, run checks against all files to make sure everything is fine:
```bash
# pre-commit run --all-files --show-diff-on-failure
```
Note: the `pre-commit` tool itself can be installed with `pip` or `pipx`.
## Review and merge open Dependabot PRs
Use the `scripts/merge-dependabot-mrs.sh` script from `master` branch to:
- list open Dependabot PRs that are mergeable,
- review, approve and merge them,
- pull changes from github and pushing them to origin.
Just invoke the script without any argument:
```bash
# ./scripts/merge-dependabot-mrs.sh
```
Check the help:
```bash
# ./scripts/merge-dependabot-mrs.sh --help
```
## Tagging and publishing new releases to PyPI
Use the `scripts/release.sh` script from `master` branch to:
- bump the version with poetry,
- update `CHANGELOG.md`,
- commit these changes,
- tag with last CHANGELOG.md item content as annotation,
- bump the version with poetry again to mark it for development,
- commit this change,
- push all commits and tags to all remote repositories.
This will trigger a Github Actions job to publish packages to PyPI.
To invoke the script, pass it the desired bump rule, e.g.:
```bash
# ./scripts/release.sh minor
```
For more options, check the help:
```bash
# ./scripts/release.sh --help
```
Raw data
{
"_id": null,
"home_page": "https://github.com/claranet/tfwrapper",
"name": "claranet-tfwrapper",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8.1,<4.0",
"maintainer_email": "",
"keywords": "terraform,wrapper",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/5e/eb/bae845cfe99072849c60d68e3e3eac8461e333c8e83ef2c21e3edf705275/claranet_tfwrapper-13.2.2.tar.gz",
"platform": null,
"description": "# claranet-tfwrapper\n\n[](CHANGELOG.md) [](LICENSE) [](https://pypi.org/project/claranet-tfwrapper/)\n\n`tfwrapper` is a python wrapper for [OpenTofu](https://opentofu.org/) and legacy [Terraform](https://www.terraform.io/) which aims to simplify their usage and enforce best practices.\n\nNote: the term _Terraform_ is used in this documentation when talking about generic concepts like providers, modules, stacks and the HCL based domain specific language.\n\n## Table Of Contents\n\n<!--TOC-->\n\n- [claranet-tfwrapper](#claranet-tfwrapper)\n - [Table Of Contents](#table-of-contents)\n - [Features](#features)\n - [Setup Dependencies](#setup-dependencies)\n - [Runtime Dependencies](#runtime-dependencies)\n - [Recommended setup](#recommended-setup)\n - [Installation](#installation)\n - [Setup command-line completion](#setup-command-line-completion)\n - [Upgrade from tfwrapper v7 or older](#upgrade-from-tfwrapper-v7-or-older)\n - [Required files](#required-files)\n - [Configuration](#configuration)\n - [tfwrapper configuration](#tfwrapper-configuration)\n - [Stacks configurations](#stacks-configurations)\n - [States centralization configuration](#states-centralization-configuration)\n - [How to migrate from one backend to another for state centralization](#how-to-migrate-from-one-backend-to-another-for-state-centralization)\n - [Stacks file structure](#stacks-file-structure)\n - [Usage](#usage)\n - [Stack bootstrap](#stack-bootstrap)\n - [Working on stacks](#working-on-stacks)\n - [Passing options](#passing-options)\n - [Environment](#environment)\n - [S3 state backend credentials](#s3-state-backend-credentials)\n - [Azure Service Principal credentials](#azure-service-principal-credentials)\n - [Azure authentication isolation](#azure-authentication-isolation)\n - [GCP configuration](#gcp-configuration)\n - [GKE configurations](#gke-configurations)\n - [Stack configurations and credentials](#stack-configurations-and-credentials)\n - [Stack path](#stack-path)\n- [Development](#development)\n - [Tests](#tests)\n - [Debug command-line completion](#debug-command-line-completion)\n - [Python code formatting](#python-code-formatting)\n - [Checks](#checks)\n - [README TOC](#readme-toc)\n - [Using OpenTofu development builds](#using-opentofu-development-builds)\n - [git pre-commit hooks](#git-pre-commit-hooks)\n - [Review and merge open Dependabot PRs](#review-and-merge-open-dependabot-prs)\n - [Tagging and publishing new releases to PyPI](#tagging-and-publishing-new-releases-to-pypi)\n\n<!--TOC-->\n\n## Features\n\n- OpenTofu and Terraform behaviour overriding\n- State centralization enforcement\n- Standardized file structure\n- Stack initialization from templates\n- AWS credentials caching\n- Azure credentials loading (both Service Principal or User)\n- GCP and GKE user ADC support\n- Plugins caching\n- Tab completion\n\n## Setup Dependencies\n\n- `python3` `>= 3.8.1 <4.0`\n- `python3-pip`\n- `python3-venv`\n- `pipx` (recommended)\n\n## Runtime Dependencies\n\n- `terraform` `>= 0.10` (`>= 0.15` for fully working Azure backend with isolation due to https://github.com/hashicorp/terraform/issues/25416)\n- `azure-cli` when using context based Azure authentication\n\n## Recommended setup\n\n- OpenTofu 1.6+ (recommended) or Terraform 1.0+ (warning: versions above 1.6 are not open-source, and may cause legal issues depending on the context you are using it).\n- An AWS S3 bucket and DynamoDB table for state centralization in AWS.\n- An Azure Blob Storage container for state centralization in Azure.\n\n## Installation\n\ntfwrapper should installed using pipx (recommended) or pip:\n\n```bash\npipx install claranet-tfwrapper\n```\n\n## Setup command-line completion\n\nAdd the following to your shell's interactive configuration file, e.g. `.bashrc` for bash:\n\n```bash\neval \"$(register-python-argcomplete tfwrapper -e tfwrapper)\"\n```\n\nYou can then press the completion key (usually `Tab \u21b9`) twice to get your partially typed `tfwrapper` commands completed.\n\nNote: the `-e tfwrapper` parameter adds an suffix to the defined `_python_argcomplete` function to avoid clashes with other packages (see https://github.com/kislyuk/argcomplete/issues/310#issuecomment-697168326 for context).\n\n## Upgrade from tfwrapper v7 or older\n\nIf you used versions of the wrapper older than v8, there is not much to do when upgrading to v8\nexcept a little cleanup.\nIndeed, the wrapper is no longer installed as a git submodule of your project like it used to be instructed and there is no longer any `Makefile` to activate it.\n\nJust clean up each project by destroying the `.wrapper` submodule:\n\n```bash\ngit rm -f Makefile\ngit submodule deinit .wrapper\nrm -rf .git/modules/.wrapper\ngit rm -f .wrapper\n```\n\nThen check the staged changes and commit them.\n\n### Required files\n\ntfwrapper expects multiple files and directories at the root of a project.\n\n#### conf\n\nStacks configurations are stored in the `conf` directory.\n\n#### templates\n\nThe `templates` directory is used to store the state backend configuration template and the Terraform stack templates used to initialize new stacks. Using a git submodule is recommended.\n\nThe following files are required:\n\n- `templates/{provider}/common/state.tf.jinja2`: AWS S3 or Azure Blob Storage state backend configuration template.\n- `templates/{provider}/basic/main.tf`: the default Terraform configuration for new stacks. The whole `template/{provider}/basic` directory is copied on stack initialization.\n\nFor example with AWS:\n\n```bash\nmkdir -p templates/aws/common templates/aws/basic\n\n# create state configuration template with AWS backend\ncat << 'EOF' > templates/aws/common/state.tf.jinja2\n{% if region is not none %}\n{% set region = '/' + region + '/' %}\n{% else %}\n{% set region = '/' %}\n{% endif %}\n\nterraform {\n backend \"s3\" {\n bucket = \"my-centralized-terraform-states-bucket\"\n key = \"{{ client_name }}/{{ account }}/{{ environment }}{{ region }}{{ stack }}/terraform.state\"\n region = \"eu-west-1\"\n\n dynamodb_table = \"my-terraform-states-lock-table\"\n }\n}\n\nresource \"null_resource\" \"state-test\" {}\nEOF\n\n# create a default stack templates with support for AWS assume role\ncat << 'EOF' > templates/aws/basic/main.tf\nprovider \"aws\" {\n region = var.aws_region\n access_key = var.aws_access_key\n secret_key = var.aws_secret_key\n token = var.aws_token\n}\nEOF\n```\n\nFor example with Azure:\n\n```bash\nmkdir -p templates/azure/common templates/azure/basic\n\n# create state configuration template with Azure backend\ncat << 'EOF' > templates/azure/common/state.tf.jinja2\n{% if region is not none %}\n{% set region = '/' + region + '/' %}\n{% else %}\n{% set region = '/' %}\n{% endif %}\n\nterraform {\n backend \"azurerm\" {\n subscription_id = \"00000000-0000-0000-0000-000000000000\"\n resource_group_name = \"my-resource-group\"\n storage_account_name = \"my-centralized-terraform-states-account\"\n container_name = \"terraform-states\"\n\n key = \"{{ client_name }}/{{ account }}/{{ environment }}{{ region }}{{ stack }}/terraform.state\"\n }\n}\nEOF\n\n# create a default stack templates with support for Azure credentials\ncat << 'EOF' > templates/azure/basic/main.tf\nprovider \"azurerm\" {\n subscription_id = var.azure_subscription_id\n tenant_id = var.azure_tenant_id\n}\nEOF\n```\n\n#### .run\n\nThe `.run` directory is used for credentials caching and plan storage.\n\n```bash\nmkdir .run\ncat << 'EOF' > .run/.gitignore\n*\n!.gitignore\nEOF\n```\n\n#### .gitignore\n\nAdding the following `.gitignore` at the root of your project is recommended:\n\n```bash\ncat << 'EOF' > .gitignore\n.terraform\nterraform.tfstate\nterraform.tfstate.backup\nterraform.tfvars\nEOF\n```\n\n## Configuration\n\ntfwrapper uses yaml files stored in the `conf` directory of the project.\n\n### tfwrapper configuration\n\ntfwrapper uses some default behaviors that can be overridden or modified via a `config.yml` file in the `conf` directory.\n\n```yaml\n---\nalways_trigger_init: False # Always trigger `terraform init` first when launching `plan` or `apply` commands\npipe_plan_command: \"cat\" # Default command used when you're invoking tfwrapper with `--pipe-plan`\nuse_local_azure_session_directory: False # Use the current user's Azure configuration in `~/.azure`. By default, the wrapper uses a local `azure-cli` session and configuration in the local `.run` directory.\n```\n\n### Stacks configurations\n\nStacks configuration files use the following naming convention:\n\n```bash\nconf/${account}_${environment}_${region}_${stack}.yml\n```\n\nHere is an example for an AWS stack configuration:\n\n```yaml\n---\nstate_configuration_name: \"aws\" # use \"aws\" backend state configuration\naws:\n general:\n account: &aws_account \"xxxxxxxxxxx\" # aws account for this stack\n region: &aws_region eu-west-1 # aws region for this stack\n credentials:\n profile: my-aws-profile # should be configured in .aws/config\n\nterraform:\n legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.\n version: \"1.0\" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0\n vars: # variables passed to terraform\n aws_account: *aws_account\n aws_region: *aws_region\n client_name: my-client-name # arbitrary client name\n```\n\nHere is an example for a stack on Azure configuration using user mode and AWS S3 backend for state storage:\n\n```yaml\n---\nstate_configuration_name: \"aws-demo\" # use \"aws\" backend state configuration\nazure:\n general:\n mode: user # Uses personal credentials with MFA\n directory_id: &directory_id \"00000000-0000-0000-0000-000000000000\" # Azure Tenant/Directory UID\n subscription_id: &subscription_id \"11111111-1111-1111-1111-111111111111\" # Azure Subscription UID\n\nterraform:\n legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.\n version: \"1.0\" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0\n vars:\n subscription_id: *subscription_id\n directory_id: *directory_id\n client_name: client-name #Replace it with the name of your client\n```\n\nIt is using your account linked to a Microsoft Account. You must have access to the Azure Subscription if you want to use Terraform.\n\nHere is an example for a stack on Azure configuration using Service Principal mode:\n\n```yaml\n---\nazure:\n general:\n mode: service_principal # Uses an Azure tenant Service Principal account\n directory_id: &directory_id \"00000000-0000-0000-0000-000000000000\" # Azure Tenant/Directory UID\n subscription_id: &subscription_id \"11111111-1111-1111-1111-111111111111\" # Azure Subscription UID\n\n credentials:\n profile: customer-profile # To stay coherent, create an AzureRM profile with the same name as the account-alias. Please checkout `azurerm_config.yml.sample` file for configuration structure.\n\nterraform:\n legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.\n version: \"1.0\" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0\n vars:\n subscription_id: *subscription_id\n directory_id: *directory_id\n client_name: client-name # Replace it with the name of your client\n```\n\nThe wrapper uses the Service Principal's credentials to connect the Azure subscription. The given Service Principal must have access to the subscription.\nThe wrapper loads `client_id`, `client_secret` and `tenant_id` properties from your `config.yml` file located in `~/.azurerm/config.yml`.\n\n`~/.azurerm/config.yml` file structure example:\n\n```yaml\n---\nclaranet-sandbox:\n client_id: aaaaaaaa-bbbb-cccc-dddd-zzzzzzzzzzzz\n client_secret: AAbbbCCCzzz==\n tenant_id: 00000000-0000-0000-0000-000000000000\n\ncustomer-profile:\n client_id: aaaaaaaa-bbbb-cccc-dddd-zzzzzzzzzzzz\n client_secret: AAbbbCCCzzz==\n tenant_id: 00000000-0000-0000-0000-000000000000\n```\n\nHere is an example for a GCP/GKE stack with user ADC and multiple GKE instances:\n\n```yaml\n---\ngcp:\n general:\n mode: adc-user\n project: &gcp_project project-name\n gke:\n - name: kubernetes-1\n zone: europe-west1-c\n - name: kubernetes-2\n region: europe-west1\n\nterraform:\n legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.\n version: \"1.0\" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0\n vars:\n gcp_region: europe-west1\n gcp_zone: europe-west1-c\n gcp_project: *gcp_project\n client_name: client-name\n```\n\nYou can declare multiple providers configurations, context is set up accordingly.\n\n\u26a0 This feature is only supported for Azure stacks for now and only works with [Azure authentication isolation](#azure-authentication-isolation)\n\n```yaml\n---\nazure:\n general:\n mode: service_principal # Uses an Azure tenant Service Principal account\n directory_id: &directory_id \"00000000-0000-0000-0000-000000000000\" # Azure Tenant/Directory UID\n subscription_id: &subscription_id \"11111111-1111-1111-1111-111111111111\" # Azure Subscription UID\n\n credentials:\n profile: customer-profile # To stay coherent, create an AzureRM profile with the same name as the account-alias. Please checkout `azurerm_config.yml.sample` file for configuration structure.\n\n alternative:\n mode: service_principal # Uses an Azure tenant Service Principal account\n directory_id: \"00000000-0000-0000-0000-000000000000\" # Azure Tenant/Directory UID\n subscription_id: \"22222222-2222-2222-2222-222222222222\" # Azure Subscription UID\n\n credentials:\n profile: claranet-sandbox # To stay coherent, create an AzureRM profile with the same name as the account-alias. Please checkout `azurerm_config.yml.sample` file for configuration structure.\n\nterraform:\n version: \"1.0\" # OpenTofu version that tfwrapper will use for this stack. Automatically downloaded if it's not available locally. Defaults to 1.0\n legacy: false # Use legacy version of the tool instead of OpenTofu, defaults to false. Only used if version >= 1.6.0.\n vars:\n subscription_id: *subscription_id\n directory_id: *directory_id\n client_name: client-name # Replace it with the name of your client\n```\n\nThis configuration is useful when having various service principals with a dedicated rights scope for each.\n\nThe wrapper will generate the following Terraform variables that can be used in your stack:\n\n- `<config_name>_azure_subscription_id` with Azure subscription ID. From the example, variable is: `alternative_subscription_id = \"22222222-2222-2222-2222-222222222222\"`\n- `<config_name>_azure_tenant_id` with Azure tenant ID. From the example, variable is: `alternative_tenant_id = \"00000000-0000-0000-0000-000000000000\"`\n- `<config_name>_azure_client_id` with Service Principal client id. From the example, variable is: `alternative_client_id = \"aaaaaaaa-bbbb-cccc-dddd-zzzzzzzzzzzz\"`\n- `<config_name>_azure_client_secret` with Service Principal client secret. From the example, variable is: `alternative_client_secret = \"AAbbbCCCzzz==\"`\n\nAlso, an isolation context is set to the local `.run/aure_<config_name>` directory for each configuration.\n\n### States centralization configuration\n\nThe `conf/state.yml` configuration file defines the configuration used to connect to state backends.\n\nThese backends can be of [AWS S3](https://developer.hashicorp.com/terraform/language/settings/backends/s3) and/or [AzureRM](https://developer.hashicorp.com/terraform/language/settings/backends/azurerm) types.\n\nThe resources for these backends are not created by tfwrapper, and thus must exist beforehand:\n* AWS: an S3 bucket (and optionally but highly recommended a DynamoDB table for locking). It is also recommended to enable versioning on the S3 bucket.\n* Azure: a Blob storage account\n\nYou can use other backends (e.g. Google GCS or Hashicorp Consul) not specifically supported by the wrapper, if you manage authentication yourself and omit the `conf/state.yml` file or make it empty:\n\n```yaml\n---\n```\n\nExample configuration with both AWS and Azure backends defined:\n\n```yaml\n---\naws:\n - name: \"aws-demo\"\n general:\n account: \"xxxxxxxxxxx\"\n region: eu-west-1\n credentials:\n profile: my-state-aws-profile # should be configured in .aws/config\nazure:\n # This backend use storage keys for authentication\n - name: \"azure-backend\"\n general:\n subscription_id: \"xxxxxxx\" # the Azure account to use for state storage\n resource_group_name: \"tfstates-xxxxx-rg\" # The Azure resource group with state storage\n storage_account_name: \"tfstatesxxxxx\"\n - name: \"azure-alternative\"\n general:\n subscription_id: \"xxxxxxx\" # the Azure account to use for state storage\n resource_group_name: \"tfstates-xxxxx-rg\" # The Azure resource group with state storage\n storage_account_name: \"tfstatesxxxxx\"\n # This backend use Azure AD authentication\n - name: \"azure-ad-auth\"\n general:\n subscription_id: \"xxxxxxx\" # the Azure account to use for state storage\n resource_group_name: \"tfstates-xxxxx-rg\" # The Azure resource group with state storage\n storage_account_name: \"tfstatesxxxxx\"\n azuread_auth: true\n\nbackend_parameters: # Parameters or options which can be used by `state.j2.tf` template file\n state_snaphot: \"false\" # Example of Azure storage backend option\n```\n\nNote: the first backend will be the default one for stacks not defining `state_backend_type`.\n\n### How to migrate from one backend to another for state centralization\n\nIf for example you have both an AWS and Azure state backend configured in your `conf/state.yml` file,\nyou can migrate your stack state from one backend to another.\n\nHere is a quick howto:\n\n1. Make sure your stack is clean:\n\n```bash\n$ cd account/path/env/your_stack\n$ tfwrapper init\n$ tfwrapper plan\n# should return no changes\n```\n\n2. Change your backend in the stack configuration yaml file:\n\n```yaml\n---\n#state_configuration_name: 'aws-demo' # previous backend\nstate_configuration_name: \"azure-alternative\" # new backend to use\n```\n\n3. Back in your stack directory, you can perform the change:\n\n```bash\n$ cd account/path/env/your_stack\n$ rm -v state.tf # removing old state backend configuration\n$ tfwrapper bootstrap # regen a new state backend configuration based on the stack yaml config file\n$ tfwrapper init # Terraform will detect the new backend and propose to migrate it\n$ tfwrapper plan\n# should return the same changes diff as before\n```\n\n## Stacks file structure\n\nTerraform stacks are organized based on their:\n\n- `account`: an account alias which may refer to provider accounts or subscriptions, e.g. `project-a-prod`, `customer-b-dev`.\n- `environment`: `production`, `preproduction`, `dev`, etc. With `global` as a special case eliminating the `region` part.\n- `region`: `eu-west-1`, `westeurope`, etc.\n- `stack`: defaults to `default`. `web`, `admin`, `tools`, etc.\n\n\nThe following file structure is then enforced:\n\n```\n# project root\n\u2514\u2500\u2500 account\n\u2502 \u2514\u2500\u2500 environment\n\u2502 \u2514\u2500\u2500 region\n\u2502 \u2514\u2500\u2500 stack\n\u2514\u2500\u2500 account\n \u2514\u2500\u2500 _global\n \u2514\u2500\u2500 stack\n```\n\nA real-life example:\n\n```\n# project root\n\u251c\u2500\u2500 aws-account-1\n\u2502 \u251c\u2500\u2500 _global\n\u2502 \u2502 \u2514\u2500\u2500 default\n\u2502 \u2502 \u2514\u2500\u2500 main.tf\n\u2502 \u2514\u2500\u2500 production\n\u2502 \u251c\u2500\u2500 eu-central-1\n\u2502 \u2502 \u2514\u2500\u2500 web\n\u2502 \u2502 \u2514\u2500\u2500 main.tf\n\u2502 \u2514\u2500\u2500 eu-west-1\n\u2502 \u251c\u2500\u2500 default\n\u2502 \u2502 \u2514\u2500\u2500 main.t\n\u2502 \u2514\u2500\u2500 tools\n\u2502 \u2514\u2500\u2500 main.tf\n\u2514\u2500\u2500 aws-account-2\n \u2514\u2500\u2500 backup\n \u2514\u2500\u2500 eu-west-1\n \u2514\u2500\u2500 backup\n \u2514\u2500\u2500 main.tf\n```\n\n## Usage\n\n### Stack bootstrap\n\nAfter creating a `conf/${account}_${environment}_${region}_${stack}.yml` stack configuration file you can bootstrap it.\n\n```bash\n# you can bootstrap using the templates/{provider}/basic stack\ntfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} bootstrap\n\n# or another stack template, for example: templates/aws/foobar\ntfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} bootstrap aws/foobar\n\n# or from an existent stack, for example: customer/env/region/stack\ntfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} bootstrap mycustomer/dev/eu-west/run\n```\n\nIn the special case of a global stack, the configuration file should instead be named as `conf/${account}_global_${stack}.yml`.\n\n### Working on stacks\n\nYou can work on stacks from their directory or from the root of the project.\n\n```bash\n# working from the root of the project\ntfwrapper -a ${account} -e ${environment} -r ${region} -s ${stack} plan\n\n# working from the root of a stack\ncd ${account}/${environment}/${region}/${stack}\ntfwrapper plan\n```\n\nYou can also work on several stacks sequentially with the `foreach` subcommand from any directory under the root of the project.\nBy default, `foreach` selects all stacks under the current directory,\nso if called from the root of the project without any filter,\nit will select all stacks and execute the specified command in them, one after another:\n\n```bash\n# working from the root of the project\ntfwrapper foreach -- tfwrapper init\n```\n\nAny combination of the `-a`, `-e`, `-r` and `-s` arguments can be used to select specific stacks,\ne.g. all stacks for an account across all environments but in a specific region:\n\n```bash\n# working from the root of the project\ntfwrapper -a ${account} -r ${region} foreach -- tfwrapper plan\n```\n\nThe same can be achieved with:\n\n```bash\n# working from an account directory\ncd ${account}\ntfwrapper -r ${region} foreach -- tfwrapper plan\n```\n\nComplex commands can be executed in a sub-shell with the `-S`/`--shell` argument, e.g.:\n\n```bash\n# working from an environment directory\ncd ${account}/${environment}\ntfwrapper foreach -S 'pwd && tfwrapper init >/dev/null 2>&1 && tfwrapper plan 2>/dev/null -- -no-color | grep \"^Plan: \"'\n```\n\n### Passing options\n\nYou can pass anything you want to `terraform` using `--`.\n\n```bash\ntfwrapper plan -- -target resource1 -target resource2\n```\n\n## Environment\n\ntfwrapper sets the following environment variables.\n\n### S3 state backend credentials\n\nThe default AWS credentials of the environment are set to point to the S3 state backend. Those credentials are acquired from the profile defined in `conf/state.yml`\n\n- `AWS_ACCESS_KEY_ID`\n- `AWS_SECRET_ACCESS_KEY`\n- `AWS_SESSION_TOKEN`\n\n### Azure Service Principal credentials\n\nThose AzureRM credentials are loaded only if you are using the Service Principal mode. They are acquired from the profile defined in `~/.azurerm/config.yml`\n\n- `ARM_CLIENT_ID`\n- `ARM_CLIENT_SECRET`\n- `ARM_TENANT_ID`\n\n### Azure authentication isolation\n\n`AZURE_CONFIG_DIR` environment variable is set to the local `.run/azure` directory if global configuration value `use_local_azure_session_directory` is set to `true`, which is the default, which is the default.\n\nIf you have multiple configurations in your stacks, you also have `<CONFIG_NAME>_AZURE_CONFIG_DIR` which is set to the local `.run/azure_<config_name>` directory.\n\n### GCP configuration\n\nThose GCP related variables are available from the environment when using the example configuration:\n\n- `TF_VAR_gcp_region`\n- `TF_VAR_gcp_zone`\n- `TF_VAR_gcp_project`\n\n### GKE configurations\n\nEach GKE instance has its own kubeconfig, the path to each configuration is available from the environment:\n\n- `TF_VAR_gke_kubeconfig_${gke_cluster_name}`\n\nkubeconfig is automatically fetched by the wrapper (using gcloud) and stored inside the `.run` directory of your project.\nIt is refreshed automatically at every run to ensure you point to correct Kubernetes endpoint.\nYou can disable this behaviour by setting `refresh_kubeconfig: never` in your cluster settings.\n\n```yaml\n---\ngcp:\n general:\n mode: adc-user\n project: &gcp_project project-name\n gke:\n - name: kubernetes-1\n zone: europe-west1-c\n refresh_kubeconfig: never\n```\n\n### Stack configurations and credentials\n\nThe `terraform['vars']` dictionary from the stack configuration is accessible as Terraform variables.\n\nThe profile defined in the stack configuration is used to acquire credentials accessible from Terraform.\nThere is two supported providers, the variables which will be loaded depends on the used provider.\n\n- `TF_VAR_client_name` (if set in .yml stack configuration file)\n- `TF_VAR_aws_account`\n- `TF_VAR_aws_region`\n- `TF_VAR_aws_access_key`\n- `TF_VAR_aws_secret_key`\n- `TF_VAR_aws_token`\n- `TF_VAR_azurerm_region`\n- `TF_VAR_azure_region`\n- `TF_VAR_azure_subscription_id`\n- `TF_VAR_azure_tenant_id`\n- ~~`TF_VAR_azure_state_access_key`~~ (removed in `v11.0.0`)\n\n### Stack path\n\nThe stack path is passed to Terraform. This is especially useful for resource naming and tagging.\n\n- `TF_VAR_account`\n- `TF_VAR_environment`\n- `TF_VAR_region`\n- `TF_VAR_stack`\n\n# Development\n\n## Tests\n\nAll new code contributions should come with unit and/or integrations tests.\n\nTo run those tests locally, use [tox](https://github.com/tox-dev/tox):\n\n```bash\npoetry run tox -e py\n```\n\nLinters are also used to ensure code respects our standards.\n\nTo run those linters locally:\n\n```bash\npoetry run tox -e lint\n```\n\n## Debug command-line completion\n\nYou can get verbose debugging information for `argcomplete` by defining the following environment variable:\n\n```bash\nexport _ARC_DEBUG=1\n```\n\n## Python code formatting\n\nOur code is formatted with [black](https://github.com/psf/black/).\n\nMake sure to format all your code contributions with `black ${filename}`.\n\nHint: enable auto-format on save with `black` in your [favorite IDE](https://black.readthedocs.io/en/stable/editor_integration.html).\n\n## Checks\n\nTo run code and documentation style checks, run `tox -e lint`.\n\nIn addition to `black --check`, code is also checked with:\n\n- [flake8](https://github.com/pycqa/flake8), a wrapper for [pycodestyle](https://github.com/PyCQA/pycodestyle) and [pyflakes](https://github.com/PyCQA/pyflakes).\n- [flake8-docstrings](https://github.com/pycqa/flake8-docstrings), a wrapper for [pydocstyle](https://github.com/PyCQA/pydocstyle).\n\n## README TOC\n\nThis [README's table of content](#table-of-contents) is formatted with [md_toc](https://github.com/frnmst/md-toc).\n\nKeep in mind to update it with `md_toc --in-place github README.md`.\n\n## Using OpenTofu development builds\n\nTo build and use development versions of OpenTofu, put them in a `~/.terraform.d/versions/X.Y/X.Y.Z-dev/` folder:\n\n```bash\n# git clone https://github.com/opentofu/opentofu.git ~/go/src/github.com/opentofu/opentofu\n# cd ~/go/src/github.com/opentofu/opentofu\n# go build ./cmd/tofu/\n# ./tofu version\nOpenTofu v1.6.0-dev\non linux_amd64\n# mkdir -p ~/.terraform.d/versions/1.6/1.6.0-dev\n# mv ./opentofu ~/.terraform.d/versions/1.6/1.6.0-dev/\n```\n\n## git pre-commit hooks\n\nSome git pre-commit hooks are configured in `.pre-commit-config.yaml` for use with the [pre-commit tool](https://pre-commit.com).\n\nUsing them helps avoiding to push changes that will fail the CI.\n\nThey can be installed locally with:\n\n```bash\n# pre-commit install\n```\n\nIf updating hooks configuration, run checks against all files to make sure everything is fine:\n\n```bash\n# pre-commit run --all-files --show-diff-on-failure\n```\n\nNote: the `pre-commit` tool itself can be installed with `pip` or `pipx`.\n\n## Review and merge open Dependabot PRs\n\nUse the `scripts/merge-dependabot-mrs.sh` script from `master` branch to:\n\n- list open Dependabot PRs that are mergeable,\n- review, approve and merge them,\n- pull changes from github and pushing them to origin.\n\nJust invoke the script without any argument:\n\n```bash\n# ./scripts/merge-dependabot-mrs.sh\n```\n\nCheck the help:\n\n```bash\n# ./scripts/merge-dependabot-mrs.sh --help\n```\n\n## Tagging and publishing new releases to PyPI\n\nUse the `scripts/release.sh` script from `master` branch to:\n\n- bump the version with poetry,\n- update `CHANGELOG.md`,\n- commit these changes,\n- tag with last CHANGELOG.md item content as annotation,\n- bump the version with poetry again to mark it for development,\n- commit this change,\n- push all commits and tags to all remote repositories.\n\nThis will trigger a Github Actions job to publish packages to PyPI.\n\nTo invoke the script, pass it the desired bump rule, e.g.:\n\n```bash\n# ./scripts/release.sh minor\n```\n\nFor more options, check the help:\n\n```bash\n# ./scripts/release.sh --help\n```\n\n",
"bugtrack_url": null,
"license": "Mozilla Public License Version 2.0",
"summary": "Claranet's `tfwrapper` is a wrapper for [Terraform](https://www.terraform.io/) implemented in python which aims to simplify Terraform usage and enforce best practices",
"version": "13.2.2",
"project_urls": {
"Homepage": "https://github.com/claranet/tfwrapper"
},
"split_keywords": [
"terraform",
"wrapper"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7fd2a5098f4f646c76b01ea5cabacf8dcaa5e18490b70fba3968f39f1239a735",
"md5": "75cf12633312b8711480192afbafc874",
"sha256": "e0b9e3db96cf626bcb0daaa74638d3c6df03ca5a688a798c9717c112472b08c4"
},
"downloads": -1,
"filename": "claranet_tfwrapper-13.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "75cf12633312b8711480192afbafc874",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8.1,<4.0",
"size": 34493,
"upload_time": "2024-03-13T07:41:34",
"upload_time_iso_8601": "2024-03-13T07:41:34.131478Z",
"url": "https://files.pythonhosted.org/packages/7f/d2/a5098f4f646c76b01ea5cabacf8dcaa5e18490b70fba3968f39f1239a735/claranet_tfwrapper-13.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5eebbae845cfe99072849c60d68e3e3eac8461e333c8e83ef2c21e3edf705275",
"md5": "6fa184babc7eddf24f397737c32153c6",
"sha256": "da5fc2628df2cd8402ff7bd98fc74d7bde0d070d9b23803c8ee7695b19ff0276"
},
"downloads": -1,
"filename": "claranet_tfwrapper-13.2.2.tar.gz",
"has_sig": false,
"md5_digest": "6fa184babc7eddf24f397737c32153c6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8.1,<4.0",
"size": 40896,
"upload_time": "2024-03-13T07:41:35",
"upload_time_iso_8601": "2024-03-13T07:41:35.498279Z",
"url": "https://files.pythonhosted.org/packages/5e/eb/bae845cfe99072849c60d68e3e3eac8461e333c8e83ef2c21e3edf705275/claranet_tfwrapper-13.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-13 07:41:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "claranet",
"github_project": "tfwrapper",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "claranet-tfwrapper"
}
JSON
