<div id="top"></div>
<!-- From https://github.com/othneildrew/Best-README-Template/ -->
[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![MIT License][license-shield]][license-url]
<br />
<div align="center">
<a href="https://github.com/jwoffindin/stk">
STK
</a>
<h3 align="center">CloudFormation doesn't have to suck 😉</h3>
<p align="center">
STK - improved infrastructure management with CloudFormation.</p>
</div>
## About The Project
STK provides an opinionated framework for better management of your AWS infrastructure with CloudFormation.
Some benefits of using `stk` for CloudFormation management.
* Encourages **composition of infrastructure through re-use of small, modular, sharable CloudFormation templates** over monolithic, complex templates. Think "lego" for infrastructure. Put together these Lego blocks in any number of combinations to deliver the infrastructure your business requires.
* **Decouple CloudFormation templates (code) from configuration**. It's as good for your infrastructure as it is for your applications. Templates are stored in external Git repositories separate from configuration. Compose from a variety of private and open source template repositories.
* **Reduce change risk through improved visibility** and simpler templates. Use of Jinja2 for templates and use of explicit configuration (directly interpolated variables) over implicit configuration (AWS parameters) means you can see exactly what is going to change.
### Local installation (Python/Pip)
STK is written in Python. Install it from GitHub using Python pip, or use the docker image for your CI/CD pipelines:
> **Note**:
> Requires **python 3.10+**
Install STK via pip
pip install cfn-stk
Alternatively
python3 -m pip install cfn-stk
Docker images are available from <https://hub.docker.com/repository/docker/johnwo/stk>.
<p align="right">(<a href="#top">back to top</a>)</p>
## Getting started
> **Note**: Quick start: The `cfn init` command will help you get started by creating a skeleton "configuration project":
```bash
cfn config init my-project --repo git@github.com:jwoffindin/stk-templates.git
cd my-project
```
> **Note**: `init` will configure an S3 bucket to store CloudFormation templates. You must select an AWS profile with currently valid credentials, otherwise the command will fail trying to list available buckets.
The GitHub project <https://github.com/jwoffindin/stk-templates> has several starter
templates that you can use. For example, to use the `vpc` template to deploy a VPC into
your AWS account:
```bash
cfn config add vpc
```
this will create a file `vpc.yml` with starter configuration. You can deploy to AWS with:
```bash
cfn deploy vpc dev
```
## Terminology
* **Environment**. What “level” are deploying a stack. E.g. development, test, stage, production, etc.
* **Template**. A YAML CloudFormation template (with Jinja2 templates, so yes — we're templating templates).
* **Archetype**. A common architectural building block that we deploy into each tier. I.e. same business function, but in dev, test, stage etc.
* **Configuration File**. A YAML file that defines what CloudFormation stacks are to be deployed. There are two flavours
of configuration file:
* Top-level configuration files. These define what *environments* a given stack can be deployed - i.e. 1:1 mapping of top-level config files to *archetype*.
* Include files. These are YAML files, mostly same structure as top-level configuration, but exist only to reduce copy
& paste between top-level configuration files. These are stored in an `includes/` directory.
* **Stack**. A CloudFormation stack. For our purpose, it is an instance of an Archetype - i.e. template + configuration file + tier.
## Commands
A full list of commands are available by running `stk help`
The most common commands that you would use are listed below. They are invoked as `cfn <command> <config-file> <environment>`
* `show-config` - show configuration that will be passed to a template.
* `show-template` - displays resulting CloudFormation template by applying configuration to template.
* `diff` - show the proposed changes between locally rendered CloudFormation Template and the currently deployed template. Displays a text diff between current and proposed templates
* `validate` - validates that a template is ready to be deployed. Performs validation of configuration and template, as well as submitting the template to AWS CloudFormation APIs for validation.
* `create` — deploys a new stack. Will fail if the stack exists already.
* `update` - update an existing stack
* `deploy` - an alias for create/update commands ("upsert"). Will update if the stack exists already, otherwise it will create a new stack.
### Configuration file
A configuration file is simply a YAML file that declares:
* How we deploy a CloudFormation template into an AWS account.
* Any configuration required for the template.
A minimal template may look like:
```yaml
# $CONFIG_PATH/foo.yml
aws:
region: ap-southeast-2
cfn_bucket: my-bucket-for-cloudformation-deployments
environments:
dev:
test:
prod:
```
This configuration file will allow us to deploy `dev`, `test` and `prod` instances of a CloudFormation stack from the template
`foo.yaml` into the `ap-southeast-2 `region.
Do deploy this stack `foo-dev` into `ap-southeast-2` using default credentials.
```sh
stk deploy foo dev
```
Create and apply a change set to the `foo-test` stack in ap-southeast-2 region.
```sh
stk update foo test
```
### CloudFormation Templates
The CloudFormation templates are YAML (JSON may work, I've not tried it). They are processed through `Jinja2` (much as is done with Ansible). Any variables declared in `vars:` section of a config file are available within the template.
For example:
```yaml
# $CONFIG_PATH/foo.yml
...
vars:
greeting: Hello World!
...
```
the corresponding template may look like:
```yaml
# $TEMPLATE_PATH/foo.yaml
Description: |
I'd like to say {{ greeting }}, to you all
```
To avoid confusion between YAML comments and Jinja2 block level operations, `stk` uses a `##` for Jinja2. For example, if we want to allow `greeting` to be optional, the template would look like:
```yaml
# $TEMPLATE_PATH/foo.yaml
## if greeting is defined
Description: |
I'd like to say {{ greeting }}, to you all
## endif
```
## Configuration
Configuration is generally stored in a different git repository from the actual templates. In the following examples, `$CONFIG_DIR` is used to represent where configuration files are stored, and `$TEMPLATE_DIR` where the CloudFormation templates live.
### Structure
The top-level keys that can be declared in a configuration file are:
* `includes` - include additional configuration files
* `template` - where to find the CloudFormation template
* `params` - AWS CloudFormation parameter definitions
* `vars` - Configuration variables (Jinja2)
* `refs` - Retrieve information from other CloudFormation stacks
* `tags` - Default tags to apply to the CloudFormation stack
* `aws` - Defines how to connect to AWS, what region to deploy etc.
* `environments` - Environment-specific overrides.
* `helpers` - Supports defining Python helpers in templates for more complex logic.
* `core` - Adjust some core `stk` settings.
These are explained in more detail below.
#### `includes`
Allows composition and sharing of common configuration to reduce unnecessary duplication.
It is a list of YAML files loaded from `includes/` directory. The `.yml`/`.yaml` extension is optional.
Includes are listed in highest-to-lowest precedence. Settings defined in any include file have a lower precedence than the current file.
```
# sample.yml
includes:
- a
- b
# includes/a.yml
vars:
foo: 'a'
# includes/b.yml
vars:
foo: 'b'
```
Two common use cases for include files are:
1. Environment specific configuration - e.g. AWS account details
1. Where multiple config files use the sample template with common settings.
### `template`
Defines where to stk can find the CloudFormation template. Supports local files, local git repositories and remote git repositories.
Templates are typically stored separately to the configuration project so they can be shared between teams.
Well-written templates are re-usable components that can be composed to build your final architecture (again, think Lego blocks).
The sources for templates are:
#### Plain filesystem (not git)
If `version` is not set, then the templates are assumed to come from local filesystem - useful
when developing templates or where templates don't need to be reused. In this case, set `root` to path.
E.g.
environments:
dev:
template:
version:
root: ./templates
During development, you can use an environment variable to define the template location:
environments:
dev:
template:
version:
root: {{ environ['TEMPLATE_PATH'] }}
#### Local Git Repository
Similar to above, but uses current commit `HEAD` in local filesystem:
environments:
dev:
template:
version: HEAD
repo: {{ environ['TEMPLATE_PATH'] }}
#### Remote Git Repository
Remote repositories are also supported. For example, the following will use `main` branch of a github repository:
template:
version: main
repo: git@github.com:jwoffindin/stk-templates.git
Note: `stk` creates a `.template-cache` directory when run - it's highly recommended that you add this to your `.gitignore` file.
### `params`
AWS Parameters that are passed into a stack. Minimize if possible, use `vars` instead.
Good use cases for `Params` include:
* passing secrets
...
* (that's about it?)
Example:
```
params:
MySubnetId: sub-0123456789
```
Params values are usually environment-specific. They can be overridden on a per environment basis, as illustrated in the following example:
```
params:
MySubnetId: sub-0123456789
environments:
dev:
test:
stage:
params:
MySubnetId: sub-abcdef
```
In this case deployments of `dev` and `test` will have `MySubnetId` of `sub-0123456789` (the default), and `stage` will be `sub-abcdef`
Parameters can be automatically JSON-encoded (for non-string values), by setting
`core.encode_params` to `true`. This will simplify passing JSON-style data as parameters.
For example:
```
core:
encode_params: true
params:
someComplexValue: { "key": "value" }
```
### `vars`
Jinja2 variables/values primarily used to pass configuration to to the CloudFormation Template.
`vars are the preferred way to manage configuration (as opposed to `params`) - e.g. use template conditions rather than 'native' AWS Template Conditions - they are unwieldy and can make changes hard to reason about.
As shown in examples above, vars are used in templates for interpolation, conditions etc. For example:
```yaml
# config file
vars:
queue_name: "MyQueue"
```
```yaml
# template
Resources:
MyQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: "{{ queue_name }}"
```
Like `params`, `vars` can have environment-specific overrides. Following the previous example, the following config file would allow deploying queues of different names to `dev` and `test`:
```yaml
# config file
environments:
dev:
vars:
queue_name: "MyDevQueue"
test:
vars:
queue_name: "MyTestQueue"
```
Configuration _values_ are processed though Jinja2 (although the file itself is *not* a Jinja template). Many
patterns are stolen from Ansible, so some concepts may be familiar if you've used Ansible.
For example:
```yaml
# $CONFIG_PATH/foo.yml
aws:
cfn_bucket: my-bucket-for-{{ environment }}-cloudformation-deployments
```
Declares a different bucket per tier.
Similarly:
```yaml
vars:
hello: "hello, {{ who | default('world') }}!"
environments:
dev:
test:
vars:
who: me
```
In this example, deploying to dev, `hello` will have the value `hello, world!`, but the value in test will be `hello, me!`.
### `refs` - Referencing other stack outputs
Use `refs:` to retrieve Outputs from other stacks and feeding them as inputs into your stack.
Refs is a map/dict of names and options. Refs are available to templates and configuration.
For example, if you have a VPC stack deployed from which you need to retrieve subnet IDs, which are exposed as `Subnets` output, you can do something like:
```yaml
refs:
vpc:
vars:
subnets: "{{ refs.vpc.Subnets }}"
```
will query the `Subnets` output from the vpc stack and pass in a list of subnets to the template as a var.
#### Optional references
Some stacks may be optional. A referenced output will return nil/blank value
if the stack doesn't exist.
For example:
```yaml
refs:
an_optional_stack:
optional: true
vars:
some_optional_var: "{{ refs.an_optional_stack.SomeOutput }}"
```
in this case, `some_optional_var` will be set iff the referenced stack exists, otherwise it is set to `nil`.
#### Overriding stack names
Normally, external stack will found using standard `stk` naming convention (`$environment-$stack_name`).
You can override this when needing to reference a stack that does not
follow this convernsion using the `stack_name` attribute.
Like all `refs:` settings, you can set this on a per-environment basis if required:
```yaml
environments:
prod:
vars:
foo: "{{ refs.some_stack.OtherOutput }}"
refs:
some_stack:
stack_name: "some-weirdly-named-stack"
```
### `tags`
Tags that are applied to the stack (and thus resources within the stack)
### `aws`
Information about the AWS account being deployed into. At a minimum needs region and name of the s3 bucket for uploading deployment artifacts.
### `environments`
If deploying into multiple environments, any environment-specific configuration goes here.
You need to declare at least one environment in a top-level configuration file.
Each sub-key of `environments` is the environment name.
Within a configuration file, environment-specific values take precedence over top-level configuration.
For example:
```
vars:
a: 'foo'
environments:
dev:
vars:
a: 'bar'
test:
prod:
```
In this case, a dev deployment `a` has value `bar`, and test/prod deployments `a` will value the value `foo`.
The following sections can have environment-specific overrides:
* `vars`
* `params`
* `aws`
* `refs`
* `tags`
For example:
```
...
environments:
dev:
tags:
Owner: "mary.jane@acme.com"
test:
tags:
Owner: "bob.smith@acme.com"
tags:
Environment: "{{ environment }}"
```
#### Valid deployment environments
Within an top-level config file, `environments` defines the allowable deployment
targets. Environments defined in include files are not used for this purpose.
In the following example, `a` can only be deployed as `dev` and `prod`, whereas `b` supports
`dev`, `test` and `prod` as deployment environments:
# a.yml
includes:
- common
environments:
dev:
prod:
# b.yml
includes:
- common
environments:
dev:
test:
prod:
# includes/common.yml
environments:
dev:
test:
prod:
### `helpers`
Custom helper functions may be used by templates. Since we're injecting code into our runtime, the configuration file must explicitly declare any helpers here.
### `core`
Configuration that changes behavior of the 'stk' application rather configuration/template deployment.
## Configuration Hierarchy
Obviously, providing deploying the same template into dev, test, and production is not overly useful. We need to be
able to deploy environment-specific configuration to each stack.
Under each environment (e.g. `environments` -> `dev`), we can apply almost all the top-level configuration items to
override the defaults.
For example:
```yaml
vars:
foo: fiz
environments:
dev:
vars:
foo: fuz
test:
prod:
```
In this example, `foo` is `fuz` for development deployments, and `fiz` for test and prod.
Most configuration options can have environment specific overrides.
For example:
```yaml
environments:
dev:
aws:
region: ap-southeast-2
template:
version: # use version from working directory
test:
aws:
region: us-east-1
template:
version: main
```
<p align="right">(<a href="#top">back to top</a>)</p>
## Configuration Detail
<p align="right">(<a href="#top">back to top</a>)</p>
## Environment Variables
| Name | Purpose |
| ---- | ------- |
| `CONFIG_PATH` | Default path to configuration files, override with `--config-path` |
| `TEMPLATE_PATH` | Default path to templates, usually overridden by config (`template.*`) or `--template-path` cli argument |
| `TEMPLATE_CACHE` | Override the path to template cache (local copy of template project for git projects only) |
## Logging
By default internal logs are sent to syslog with WARN.
The following environment variables can be used to change this behaviour
| Environment variable | Purpose | Example |
| --- | --- | --- |
| `CFN_LOG_LEVEL` | Change log level | `DEBUG`, `INFO`, `ERROR` |
| `LOG_LEVEL` | Fallback log level. Also used by boto, so will change boto logging too | `INFO`, `WARN` |
| `LOG_FILE` | Direct logging to this file rather than syslog | `./cfn.log` |
## Exit codes
| Exit code | Reason |
| --- | --- |
| -1 | General failure |
| -2 | Could not generate change set |
| -9 | No change to apply |
## Roadmap
See the [open issues](https://github.com/jwoffindin/stk/issues) for a full list of proposed features (and known issues).
<p align="right">(<a href="#top">back to top</a>)</p>
## Alternatives
`stk` is rather opinionated about managing CloudFormation. If this approach doesn't work for you, check out these alternatives:
* [Sceptre](https://github.com/Sceptre/sceptre) - a more mature project that takes a similar, less opinionated, approach to the problem. Supports stack sets. Only supports local (to the filesystem) templates.
> Sceptre is a tool to drive AWS CloudFormation. It automates the mundane, repetitive and error-prone tasks, enabling you to concentrate on building better infrastructure.
<!-- CONTRIBUTING -->
## Contributing
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".
Don't forget to give the project a star! Thanks again!
1. Fork the Project
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the Branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
<p align="right">(<a href="#top">back to top</a>)</p>
## License
Distributed under the MIT License. See `LICENSE.txt` for more information.
<p align="right">(<a href="#top">back to top</a>)</p>
## Contact
John Woffindin - [@john_woffindin](https://twitter.com/john_woffindin) - j.woffindin@gmail.com
Project Link: [https://github.com/jwoffindin/stk](https://github.com/jwoffindin/stk)
<p align="right">(<a href="#top">back to top</a>)</p>
## Acknowledgments
TODO
<p align="right">(<a href="#top">back to top</a>)</p>
[contributors-shield]: https://img.shields.io/github/contributors/jwoffindin/stk.svg?style=for-the-badge
[contributors-url]: https://github.com/jwoffindin/stk/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/jwoffindin/stk.svg?style=for-the-badge
[forks-url]: https://github.com/jwoffindin/stk/network/members
[stars-shield]: https://img.shields.io/github/stars/jwoffindin/stk.svg?style=for-the-badge
[stars-url]: https://github.com/jwoffindin/stk/stargazers
[issues-shield]: https://img.shields.io/github/issues/jwoffindin/stk.svg?style=for-the-badge
[issues-url]: https://github.com/jwoffindin/stk/issues
[license-shield]: https://img.shields.io/github/license/jwoffindin/stk.svg?style=for-the-badge
[license-url]: https://github.com/jwoffindin/stk/blob/master/LICENSE.txt
[product-screenshot]: images/screenshot.png
Raw data
{
"_id": null,
"home_page": "https://github.com/jwoffindin/stk",
"name": "cfn-stk",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "cloudformation,aws,cfn,cli",
"author": "John Woffindin",
"author_email": "j.woffindin@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c6/cb/1bc6a8c72e95666b68262a6d4b76c144e7eefa0ae43e9a079a3d1bef4ddf/cfn-stk-0.15.2.tar.gz",
"platform": null,
"description": "<div id=\"top\"></div>\n<!-- From https://github.com/othneildrew/Best-README-Template/ -->\n\n\n\n[![Contributors][contributors-shield]][contributors-url]\n[![Forks][forks-shield]][forks-url]\n[![Stargazers][stars-shield]][stars-url]\n[![Issues][issues-shield]][issues-url]\n[![MIT License][license-shield]][license-url]\n\n<br />\n<div align=\"center\">\n <a href=\"https://github.com/jwoffindin/stk\">\n STK\n </a>\n\n<h3 align=\"center\">CloudFormation doesn't have to suck \ud83d\ude09</h3>\n\n <p align=\"center\">\n STK - improved infrastructure management with CloudFormation.</p>\n</div>\n\n## About The Project\n\nSTK provides an opinionated framework for better management of your AWS infrastructure with CloudFormation.\n\nSome benefits of using `stk` for CloudFormation management.\n\n* Encourages **composition of infrastructure through re-use of small, modular, sharable CloudFormation templates** over monolithic, complex templates. Think \"lego\" for infrastructure. Put together these Lego blocks in any number of combinations to deliver the infrastructure your business requires.\n* **Decouple CloudFormation templates (code) from configuration**. It's as good for your infrastructure as it is for your applications. Templates are stored in external Git repositories separate from configuration. Compose from a variety of private and open source template repositories.\n* **Reduce change risk through improved visibility** and simpler templates. Use of Jinja2 for templates and use of explicit configuration (directly interpolated variables) over implicit configuration (AWS parameters) means you can see exactly what is going to change.\n\n### Local installation (Python/Pip)\n\nSTK is written in Python. Install it from GitHub using Python pip, or use the docker image for your CI/CD pipelines:\n\n> **Note**:\n> Requires **python 3.10+**\n\nInstall STK via pip\n\n pip install cfn-stk\n\nAlternatively\n\n python3 -m pip install cfn-stk\n\nDocker images are available from <https://hub.docker.com/repository/docker/johnwo/stk>.\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n\n## Getting started\n\n> **Note**: Quick start: The `cfn init` command will help you get started by creating a skeleton \"configuration project\":\n\n```bash\ncfn config init my-project --repo git@github.com:jwoffindin/stk-templates.git\ncd my-project\n```\n\n> **Note**: `init` will configure an S3 bucket to store CloudFormation templates. You must select an AWS profile with currently valid credentials, otherwise the command will fail trying to list available buckets.\n\nThe GitHub project <https://github.com/jwoffindin/stk-templates> has several starter\ntemplates that you can use. For example, to use the `vpc` template to deploy a VPC into\nyour AWS account:\n\n```bash\ncfn config add vpc\n```\n\nthis will create a file `vpc.yml` with starter configuration. You can deploy to AWS with:\n\n```bash\ncfn deploy vpc dev\n```\n\n\n## Terminology\n\n* **Environment**. What \u201clevel\u201d are deploying a stack. E.g. development, test, stage, production, etc.\n* **Template**. A YAML CloudFormation template (with Jinja2 templates, so yes \u2014 we're templating templates).\n* **Archetype**. A common architectural building block that we deploy into each tier. I.e. same business function, but in dev, test, stage etc.\n* **Configuration File**. A YAML file that defines what CloudFormation stacks are to be deployed. There are two flavours\n of configuration file:\n * Top-level configuration files. These define what *environments* a given stack can be deployed - i.e. 1:1 mapping of top-level config files to *archetype*.\n * Include files. These are YAML files, mostly same structure as top-level configuration, but exist only to reduce copy\n & paste between top-level configuration files. These are stored in an `includes/` directory.\n* **Stack**. A CloudFormation stack. For our purpose, it is an instance of an Archetype - i.e. template + configuration file + tier.\n\n## Commands\n\nA full list of commands are available by running `stk help`\n\nThe most common commands that you would use are listed below. They are invoked as `cfn <command> <config-file> <environment>`\n\n* `show-config` - show configuration that will be passed to a template.\n* `show-template` - displays resulting CloudFormation template by applying configuration to template.\n* `diff` - show the proposed changes between locally rendered CloudFormation Template and the currently deployed template. Displays a text diff between current and proposed templates\n* `validate` - validates that a template is ready to be deployed. Performs validation of configuration and template, as well as submitting the template to AWS CloudFormation APIs for validation.\n* `create` \u2014 deploys a new stack. Will fail if the stack exists already.\n* `update` - update an existing stack\n* `deploy` - an alias for create/update commands (\"upsert\"). Will update if the stack exists already, otherwise it will create a new stack.\n\n### Configuration file\n\nA configuration file is simply a YAML file that declares:\n\n* How we deploy a CloudFormation template into an AWS account.\n* Any configuration required for the template.\n\nA minimal template may look like:\n\n```yaml\n# $CONFIG_PATH/foo.yml\naws:\n region: ap-southeast-2\n cfn_bucket: my-bucket-for-cloudformation-deployments\n\nenvironments:\n dev:\n test:\n prod:\n```\n\nThis configuration file will allow us to deploy `dev`, `test` and `prod` instances of a CloudFormation stack from the template\n`foo.yaml` into the `ap-southeast-2 `region.\n\nDo deploy this stack `foo-dev` into `ap-southeast-2` using default credentials.\n\n```sh\nstk deploy foo dev\n```\n\nCreate and apply a change set to the `foo-test` stack in ap-southeast-2 region.\n\n```sh\nstk update foo test\n```\n\n### CloudFormation Templates\n\nThe CloudFormation templates are YAML (JSON may work, I've not tried it). They are processed through `Jinja2` (much as is done with Ansible). Any variables declared in `vars:` section of a config file are available within the template.\n\nFor example:\n\n```yaml\n# $CONFIG_PATH/foo.yml\n...\nvars:\n greeting: Hello World!\n...\n```\n\nthe corresponding template may look like:\n\n```yaml\n# $TEMPLATE_PATH/foo.yaml\nDescription: |\n I'd like to say {{ greeting }}, to you all\n```\n\nTo avoid confusion between YAML comments and Jinja2 block level operations, `stk` uses a `##` for Jinja2. For example, if we want to allow `greeting` to be optional, the template would look like:\n\n```yaml\n# $TEMPLATE_PATH/foo.yaml\n\n## if greeting is defined\nDescription: |\n I'd like to say {{ greeting }}, to you all\n## endif\n\n```\n\n## Configuration\n\nConfiguration is generally stored in a different git repository from the actual templates. In the following examples, `$CONFIG_DIR` is used to represent where configuration files are stored, and `$TEMPLATE_DIR` where the CloudFormation templates live.\n### Structure\n\nThe top-level keys that can be declared in a configuration file are:\n\n* `includes` - include additional configuration files\n* `template` - where to find the CloudFormation template\n* `params` - AWS CloudFormation parameter definitions\n* `vars` - Configuration variables (Jinja2)\n* `refs` - Retrieve information from other CloudFormation stacks\n* `tags` - Default tags to apply to the CloudFormation stack\n* `aws` - Defines how to connect to AWS, what region to deploy etc.\n* `environments` - Environment-specific overrides.\n* `helpers` - Supports defining Python helpers in templates for more complex logic.\n* `core` - Adjust some core `stk` settings.\n\nThese are explained in more detail below.\n\n#### `includes`\n\nAllows composition and sharing of common configuration to reduce unnecessary duplication.\n\nIt is a list of YAML files loaded from `includes/` directory. The `.yml`/`.yaml` extension is optional.\n\nIncludes are listed in highest-to-lowest precedence. Settings defined in any include file have a lower precedence than the current file.\n\n\n```\n# sample.yml\nincludes:\n - a\n - b\n\n# includes/a.yml\nvars:\n foo: 'a'\n\n# includes/b.yml\nvars:\n foo: 'b'\n```\n\nTwo common use cases for include files are:\n\n1. Environment specific configuration - e.g. AWS account details\n1. Where multiple config files use the sample template with common settings.\n\n\n### `template`\n\nDefines where to stk can find the CloudFormation template. Supports local files, local git repositories and remote git repositories.\n\nTemplates are typically stored separately to the configuration project so they can be shared between teams.\nWell-written templates are re-usable components that can be composed to build your final architecture (again, think Lego blocks).\n\nThe sources for templates are:\n#### Plain filesystem (not git)\n\nIf `version` is not set, then the templates are assumed to come from local filesystem - useful\nwhen developing templates or where templates don't need to be reused. In this case, set `root` to path.\n\nE.g.\n\n environments:\n dev:\n template:\n version:\n root: ./templates\n\nDuring development, you can use an environment variable to define the template location:\n\n environments:\n dev:\n template:\n version:\n root: {{ environ['TEMPLATE_PATH'] }}\n\n#### Local Git Repository\n\nSimilar to above, but uses current commit `HEAD` in local filesystem:\n\n environments:\n dev:\n template:\n version: HEAD\n repo: {{ environ['TEMPLATE_PATH'] }}\n\n#### Remote Git Repository\n\nRemote repositories are also supported. For example, the following will use `main` branch of a github repository:\n\n template:\n version: main\n repo: git@github.com:jwoffindin/stk-templates.git\n\nNote: `stk` creates a `.template-cache` directory when run - it's highly recommended that you add this to your `.gitignore` file.\n\n\n\n### `params`\n\nAWS Parameters that are passed into a stack. Minimize if possible, use `vars` instead.\n\nGood use cases for `Params` include:\n * passing secrets\n ...\n * (that's about it?)\n\nExample:\n\n```\nparams:\n MySubnetId: sub-0123456789\n```\n\nParams values are usually environment-specific. They can be overridden on a per environment basis, as illustrated in the following example:\n\n```\nparams:\n MySubnetId: sub-0123456789\n\nenvironments:\n dev:\n test:\n stage:\n params:\n MySubnetId: sub-abcdef\n```\n\nIn this case deployments of `dev` and `test` will have `MySubnetId` of `sub-0123456789` (the default), and `stage` will be `sub-abcdef`\n\nParameters can be automatically JSON-encoded (for non-string values), by setting\n`core.encode_params` to `true`. This will simplify passing JSON-style data as parameters.\n\nFor example:\n\n```\ncore:\n encode_params: true\n\nparams:\n someComplexValue: { \"key\": \"value\" }\n```\n\n\n### `vars`\n\nJinja2 variables/values primarily used to pass configuration to to the CloudFormation Template.\n\n`vars are the preferred way to manage configuration (as opposed to `params`) - e.g. use template conditions rather than 'native' AWS Template Conditions - they are unwieldy and can make changes hard to reason about.\n\nAs shown in examples above, vars are used in templates for interpolation, conditions etc. For example:\n\n```yaml\n# config file\nvars:\n queue_name: \"MyQueue\"\n```\n\n```yaml\n# template\nResources:\n MyQueue:\n Type: AWS::SQS::Queue\n Properties:\n QueueName: \"{{ queue_name }}\"\n```\n\nLike `params`, `vars` can have environment-specific overrides. Following the previous example, the following config file would allow deploying queues of different names to `dev` and `test`:\n\n```yaml\n# config file\nenvironments:\n dev:\n vars:\n queue_name: \"MyDevQueue\"\n test:\n vars:\n queue_name: \"MyTestQueue\"\n```\n\nConfiguration _values_ are processed though Jinja2 (although the file itself is *not* a Jinja template). Many\npatterns are stolen from Ansible, so some concepts may be familiar if you've used Ansible.\n\nFor example:\n\n```yaml\n# $CONFIG_PATH/foo.yml\naws:\n cfn_bucket: my-bucket-for-{{ environment }}-cloudformation-deployments\n```\n\nDeclares a different bucket per tier.\n\nSimilarly:\n\n```yaml\nvars:\n hello: \"hello, {{ who | default('world') }}!\"\n\nenvironments:\n dev:\n test:\n vars:\n who: me\n```\n\nIn this example, deploying to dev, `hello` will have the value `hello, world!`, but the value in test will be `hello, me!`.\n\n### `refs` - Referencing other stack outputs\n\nUse `refs:` to retrieve Outputs from other stacks and feeding them as inputs into your stack.\n\nRefs is a map/dict of names and options. Refs are available to templates and configuration.\n\nFor example, if you have a VPC stack deployed from which you need to retrieve subnet IDs, which are exposed as `Subnets` output, you can do something like:\n\n```yaml\nrefs:\n vpc:\n\nvars:\n subnets: \"{{ refs.vpc.Subnets }}\"\n```\n\nwill query the `Subnets` output from the vpc stack and pass in a list of subnets to the template as a var.\n\n#### Optional references\n\nSome stacks may be optional. A referenced output will return nil/blank value\nif the stack doesn't exist.\n\nFor example:\n\n```yaml\nrefs:\n an_optional_stack:\n optional: true\n\nvars:\n some_optional_var: \"{{ refs.an_optional_stack.SomeOutput }}\"\n```\n\nin this case, `some_optional_var` will be set iff the referenced stack exists, otherwise it is set to `nil`.\n\n#### Overriding stack names\n\nNormally, external stack will found using standard `stk` naming convention (`$environment-$stack_name`).\n\nYou can override this when needing to reference a stack that does not\nfollow this convernsion using the `stack_name` attribute.\n\nLike all `refs:` settings, you can set this on a per-environment basis if required:\n\n```yaml\nenvironments:\n prod:\n vars:\n foo: \"{{ refs.some_stack.OtherOutput }}\"\n refs:\n some_stack:\n stack_name: \"some-weirdly-named-stack\"\n```\n\n### `tags`\n\nTags that are applied to the stack (and thus resources within the stack)\n\n### `aws`\n\nInformation about the AWS account being deployed into. At a minimum needs region and name of the s3 bucket for uploading deployment artifacts.\n\n\n### `environments`\n\nIf deploying into multiple environments, any environment-specific configuration goes here.\nYou need to declare at least one environment in a top-level configuration file.\n\nEach sub-key of `environments` is the environment name.\n\nWithin a configuration file, environment-specific values take precedence over top-level configuration.\n\nFor example:\n\n```\nvars:\n a: 'foo'\n\nenvironments:\n dev:\n vars:\n a: 'bar'\n test:\n prod:\n```\n\nIn this case, a dev deployment `a` has value `bar`, and test/prod deployments `a` will value the value `foo`.\n\nThe following sections can have environment-specific overrides:\n\n* `vars`\n* `params`\n* `aws`\n* `refs`\n* `tags`\n\nFor example:\n\n```\n...\nenvironments:\n dev:\n tags:\n Owner: \"mary.jane@acme.com\"\n test:\n tags:\n Owner: \"bob.smith@acme.com\"\ntags:\n Environment: \"{{ environment }}\"\n```\n\n#### Valid deployment environments\n\nWithin an top-level config file, `environments` defines the allowable deployment\ntargets. Environments defined in include files are not used for this purpose.\n\nIn the following example, `a` can only be deployed as `dev` and `prod`, whereas `b` supports\n`dev`, `test` and `prod` as deployment environments:\n\n # a.yml\n includes:\n - common\n environments:\n dev:\n prod:\n\n # b.yml\n includes:\n - common\n environments:\n dev:\n test:\n prod:\n\n # includes/common.yml\n environments:\n dev:\n test:\n prod:\n\n\n### `helpers`\n\nCustom helper functions may be used by templates. Since we're injecting code into our runtime, the configuration file must explicitly declare any helpers here.\n\n### `core`\n\nConfiguration that changes behavior of the 'stk' application rather configuration/template deployment.\n\n## Configuration Hierarchy\n\nObviously, providing deploying the same template into dev, test, and production is not overly useful. We need to be\nable to deploy environment-specific configuration to each stack.\n\nUnder each environment (e.g. `environments` -> `dev`), we can apply almost all the top-level configuration items to\noverride the defaults.\n\nFor example:\n```yaml\nvars:\n foo: fiz\n\nenvironments:\n dev:\n vars:\n foo: fuz\n test:\n prod:\n```\n\nIn this example, `foo` is `fuz` for development deployments, and `fiz` for test and prod.\n\nMost configuration options can have environment specific overrides.\n\nFor example:\n\n```yaml\nenvironments:\n dev:\n aws:\n region: ap-southeast-2\n template:\n version: # use version from working directory\n test:\n aws:\n region: us-east-1\n template:\n version: main\n```\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n## Configuration Detail\n\n\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n## Environment Variables\n\n| Name | Purpose |\n| ---- | ------- |\n| `CONFIG_PATH` | Default path to configuration files, override with `--config-path` |\n| `TEMPLATE_PATH` | Default path to templates, usually overridden by config (`template.*`) or `--template-path` cli argument |\n| `TEMPLATE_CACHE` | Override the path to template cache (local copy of template project for git projects only) |\n\n## Logging\n\nBy default internal logs are sent to syslog with WARN.\n\nThe following environment variables can be used to change this behaviour\n\n| Environment variable | Purpose | Example |\n| --- | --- | --- |\n| `CFN_LOG_LEVEL` | Change log level | `DEBUG`, `INFO`, `ERROR` |\n| `LOG_LEVEL` | Fallback log level. Also used by boto, so will change boto logging too | `INFO`, `WARN` |\n| `LOG_FILE` | Direct logging to this file rather than syslog | `./cfn.log` |\n\n## Exit codes\n\n| Exit code | Reason |\n| --- | --- |\n| -1 | General failure |\n| -2 | Could not generate change set |\n| -9 | No change to apply |\n\n\n## Roadmap\n\nSee the [open issues](https://github.com/jwoffindin/stk/issues) for a full list of proposed features (and known issues).\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n## Alternatives\n\n`stk` is rather opinionated about managing CloudFormation. If this approach doesn't work for you, check out these alternatives:\n\n* [Sceptre](https://github.com/Sceptre/sceptre) - a more mature project that takes a similar, less opinionated, approach to the problem. Supports stack sets. Only supports local (to the filesystem) templates.\n > Sceptre is a tool to drive AWS CloudFormation. It automates the mundane, repetitive and error-prone tasks, enabling you to concentrate on building better infrastructure.\n\n\n<!-- CONTRIBUTING -->\n## Contributing\n\nContributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**.\n\nIf you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag \"enhancement\".\nDon't forget to give the project a star! Thanks again!\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n## License\n\nDistributed under the MIT License. See `LICENSE.txt` for more information.\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n\n## Contact\n\nJohn Woffindin - [@john_woffindin](https://twitter.com/john_woffindin) - j.woffindin@gmail.com\n\nProject Link: [https://github.com/jwoffindin/stk](https://github.com/jwoffindin/stk)\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n## Acknowledgments\n\nTODO\n\n<p align=\"right\">(<a href=\"#top\">back to top</a>)</p>\n\n\n[contributors-shield]: https://img.shields.io/github/contributors/jwoffindin/stk.svg?style=for-the-badge\n[contributors-url]: https://github.com/jwoffindin/stk/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/jwoffindin/stk.svg?style=for-the-badge\n[forks-url]: https://github.com/jwoffindin/stk/network/members\n[stars-shield]: https://img.shields.io/github/stars/jwoffindin/stk.svg?style=for-the-badge\n[stars-url]: https://github.com/jwoffindin/stk/stargazers\n[issues-shield]: https://img.shields.io/github/issues/jwoffindin/stk.svg?style=for-the-badge\n[issues-url]: https://github.com/jwoffindin/stk/issues\n[license-shield]: https://img.shields.io/github/license/jwoffindin/stk.svg?style=for-the-badge\n[license-url]: https://github.com/jwoffindin/stk/blob/master/LICENSE.txt\n[product-screenshot]: images/screenshot.png\n",
"bugtrack_url": null,
"license": "",
"summary": "Opinionated CloudFormation deployments",
"version": "0.15.2",
"project_urls": {
"Homepage": "https://github.com/jwoffindin/stk"
},
"split_keywords": [
"cloudformation",
"aws",
"cfn",
"cli"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "020ae6d538fa2f098a677fa7d0e9e218b7f0970520d722e125269d6566666b46",
"md5": "b5daf7c9643c5bb1b399c10856802e9f",
"sha256": "46f8a4e4b54efff05a855c7f3f4a0a3207bcef8d86602903c2fc8d831c1bc234"
},
"downloads": -1,
"filename": "cfn_stk-0.15.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b5daf7c9643c5bb1b399c10856802e9f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 65326,
"upload_time": "2023-05-27T23:21:51",
"upload_time_iso_8601": "2023-05-27T23:21:51.350793Z",
"url": "https://files.pythonhosted.org/packages/02/0a/e6d538fa2f098a677fa7d0e9e218b7f0970520d722e125269d6566666b46/cfn_stk-0.15.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c6cb1bc6a8c72e95666b68262a6d4b76c144e7eefa0ae43e9a079a3d1bef4ddf",
"md5": "bd5e11bf56936337d7c21993c1f8dbef",
"sha256": "a1412941eaad3144fe060ba58643478665ad80060798f13312dc2930c7b99660"
},
"downloads": -1,
"filename": "cfn-stk-0.15.2.tar.gz",
"has_sig": false,
"md5_digest": "bd5e11bf56936337d7c21993c1f8dbef",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 54078,
"upload_time": "2023-05-27T23:21:53",
"upload_time_iso_8601": "2023-05-27T23:21:53.467957Z",
"url": "https://files.pythonhosted.org/packages/c6/cb/1bc6a8c72e95666b68262a6d4b76c144e7eefa0ae43e9a079a3d1bef4ddf/cfn-stk-0.15.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-27 23:21:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jwoffindin",
"github_project": "stk",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cfn-stk"
}
JSON
