# echostream-terrafy
EchoStream provides a graphical way to design, build and update processing and integration
networks in real time.
However, there are teams that wish to manage all resources (EchoStream and others, such as
Azure or AWS) in Infrastructure as Code (IaC). If you have designed a large network of
processing resource in EchoStream, the ideal solution would be to introspect and extract
those resources into Terraform scripts.
`echostream-terrafy` is the tool that does this, creating:
- A set of Terraform scripts in HCL JSON format that exactly matches the current state
of your Tenant. These may be used as-is or may be converted into a module for use in
a larger Terraform script.
- A `terraform.tfstate` file that has all of the resources in the Terraform scripts imported.
## Installation
`echostream-terrafy` is a Python package that provides an executable. To install it:
1. Install [Python](https://www.python.org/downloads/) >= 3.9 for your device.
2. Install your preferred HCL client.
1. Install [Terraform](https://developer.hashicorp.com/terraform/install) >= 1.5.7 for your device.
> Warning - if you do not install Terraform to a location in your PATH, make note of the installation location!
-OR-
2. Install [OpenTofu](https://opentofu.org/docs/intro/install/) >= 1.6.2 for your device.
> Warning - if you do not install OponTofu to a location in your PATH, make note of the installation location!
3. Install `echostream-terrafy`.
1. Using [pipx](https://pipx.pypa.io/stable/) - **RECOMMENDED**
```shell
pipx install echostream-terrafy
```
2. Using [pip](https://pip.pypa.io/en/stable/)
1. Create a Python virtual environment for the `echostream-terrafy` installation.
```shell
python -m venv echostream-terrafy
```
2. Activate the virtual environment and install `echostream-terrafy` into it.
```shell
source echostream-terrafy/bin/activate
pip install echostream-terrafy
```
> Note - to deactivate the virtual environment, simply type `deactivate` at the command prompt.
## Usage
In your EchoStream Tenant, create an [ApiUser](https://docs.echo.stream/docs/api-users) with the admin role.
Make note of the following in your ApiUser's credentials:
- GraphQL Appsync Endpoint
- Client Id
- Username
- Password
- User Pool Id
You may execute `echostream-terrafy` using either environment variables, command-line parameters, or a combination of both. If a parameter is present in both the environment and on the command line, the command line takes precedence.
All output from `echostream-terrafy` executions will be written to the current working directory, with existing files (including the `terraform.tfstate` file) being overwritten.
> Note - it is recommended that you create a directory for the output of `echostream-terrafy` and execute it within that directory.
> Warning - if you did not install `terraform` or `tofu` into your PATH, you must let `echostream-terrafy` know where to find it. This may be accomplished by specifying the `--cli` command-line parameter with the full path to the `terraform` or `tofu` executable.
> **Warning** - while the EchoStream Terraform Provider will run in a non-POSIX environment (e.g. - Windows), `echostream-terrafy` will NOT generate correct paths in a non-POSIX environment. Path construction is OS dependent. Because of this, `echostream-terrafy` will not run in a non-POSIX envrionment.
### Executing with environment variables
```shell
source echostream-terrafy/bin/activate # OPTIONAL - for use with venv
export ECHOSTREAM_APPSYNC_ENDPOINT=<api_user_appsync_endpoint>
export ECHOSTREAM_CLIENT_ID=<api_user_client_id>
export ECHOSTREAM_PASSWORD=<api_user_password>
export ECHOSTREAM_TENANT=<my_tenant_name>
export ECHOSTREAM_USER_POOL_ID=<api_user_user_pool_id>
export ECHOSTREAM_USERNAME=<api_user_username>
echostream-terrafy
deactivate # OPTIONAL - for use with venv
```
### Executing using command-line variables
```shell
source echostream-terrafy/bin/activate # OPTIONAL - for use with venv
echostream-terrafy \
--appsync-endpoint <api_user_appsync_endpoint> \
--client-id <api_user_client_id> \
--password <api_user_password> \
--tenant <my_tenant_name> \
--user-pool-id <api_user_user_pool_id> \
--username <api_user_username>
deactivate # OPTIONAL - for use with venv
```
## Output
`echostream-terrafy` will generate the following files.
| Filename | Content |
| --- | --- |
| artifacts | Directory of Python code, config JSON and readme artifacts for resources |
| api-users.tf.json | The ApiUser resources |
| apps.tf.json | The App resources |
| functions.tf.json | The Function data sources and resources |
| kms-keys.tf.json | The KmsKey resources (except the Tenant default KmsKey) |
| main.tf.json | The `terraform` block |
| managed-node-types.tf.json | The ManagedNodeType data sources and resources |
| message-types.tf.json | The MessageType data sources and resources |
| nodes.tf.json | The Node data sources and resources |
| provider.tf.json | The `provider` block |
| tenant-users.tf.json | The TenantUser resources |
| tenant.json | The Tenant resource |
| terraform.tfstate | The current state, imported |
### Using the output as-is
You may use the output from `echostream-terrafy` as-is to manage your Tenant.
Simply make any changes that you wish to it and run `terraform plan` or `terraform apply`.
> Warning - rerunning `echostream-terrafy` after you have made manual changes will result in those changes being overwritten!
### Using the output as a Terraform [module](https://developer.hashicorp.com/terraform/language/modules)
1. Copy all of the `.tf.json` files and the `artifacts` directory to another folder.
2. Remove `provider.tf.json`. The provider should be passed in by the module caller.
3. Add a `variables.tf` file and variablize any input (e.g. - configs) that you wish to be modifiable by module users.
4. Add an `outputs.tf` file and output any information that needs to be accessed by module users.
5. Register the module with a public/private Terraform registry or include it in a `modules` directory (either directly or as a `git` submodule) in another Terraform project.
Raw data
{
"_id": null,
"home_page": "https://github.com/Echo-Stream/echostream-terrafy",
"name": "echostream-terrafy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "echostream, terraform, opentofu, tofu, terrafy",
"author": "EchoStream",
"author_email": "support@echo.stream",
"download_url": "https://files.pythonhosted.org/packages/2e/dd/b1b34d73ccb13fd6e0793e8fc1a89a3bf96edad29d8d943691906a12f8c0/echostream_terrafy-0.0.12.tar.gz",
"platform": "posix",
"description": "# echostream-terrafy\n\nEchoStream provides a graphical way to design, build and update processing and integration\nnetworks in real time. \n\nHowever, there are teams that wish to manage all resources (EchoStream and others, such as \nAzure or AWS) in Infrastructure as Code (IaC). If you have designed a large network of \nprocessing resource in EchoStream, the ideal solution would be to introspect and extract\nthose resources into Terraform scripts.\n\n`echostream-terrafy` is the tool that does this, creating:\n\n- A set of Terraform scripts in HCL JSON format that exactly matches the current state\nof your Tenant. These may be used as-is or may be converted into a module for use in\na larger Terraform script.\n- A `terraform.tfstate` file that has all of the resources in the Terraform scripts imported.\n\n## Installation\n`echostream-terrafy` is a Python package that provides an executable. To install it:\n\n1. Install [Python](https://www.python.org/downloads/) >= 3.9 for your device.\n2. Install your preferred HCL client.\n 1. Install [Terraform](https://developer.hashicorp.com/terraform/install) >= 1.5.7 for your device.\n > Warning - if you do not install Terraform to a location in your PATH, make note of the installation location!\n\n -OR-\n\n 2. Install [OpenTofu](https://opentofu.org/docs/intro/install/) >= 1.6.2 for your device.\n > Warning - if you do not install OponTofu to a location in your PATH, make note of the installation location!\n3. Install `echostream-terrafy`.\n 1. Using [pipx](https://pipx.pypa.io/stable/) - **RECOMMENDED**\n ```shell\n pipx install echostream-terrafy\n ```\n 2. Using [pip](https://pip.pypa.io/en/stable/)\n 1. Create a Python virtual environment for the `echostream-terrafy` installation.\n ```shell\n python -m venv echostream-terrafy\n ```\n 2. Activate the virtual environment and install `echostream-terrafy` into it.\n ```shell\n source echostream-terrafy/bin/activate\n pip install echostream-terrafy\n ```\n > Note - to deactivate the virtual environment, simply type `deactivate` at the command prompt.\n\n## Usage\n\nIn your EchoStream Tenant, create an [ApiUser](https://docs.echo.stream/docs/api-users) with the admin role.\n\nMake note of the following in your ApiUser's credentials:\n- GraphQL Appsync Endpoint\n- Client Id\n- Username\n- Password\n- User Pool Id\n\nYou may execute `echostream-terrafy` using either environment variables, command-line parameters, or a combination of both. If a parameter is present in both the environment and on the command line, the command line takes precedence.\n\nAll output from `echostream-terrafy` executions will be written to the current working directory, with existing files (including the `terraform.tfstate` file) being overwritten.\n\n> Note - it is recommended that you create a directory for the output of `echostream-terrafy` and execute it within that directory.\n\n> Warning - if you did not install `terraform` or `tofu` into your PATH, you must let `echostream-terrafy` know where to find it. This may be accomplished by specifying the `--cli` command-line parameter with the full path to the `terraform` or `tofu` executable.\n\n> **Warning** - while the EchoStream Terraform Provider will run in a non-POSIX environment (e.g. - Windows), `echostream-terrafy` will NOT generate correct paths in a non-POSIX environment. Path construction is OS dependent. Because of this, `echostream-terrafy` will not run in a non-POSIX envrionment.\n\n### Executing with environment variables\n```shell\nsource echostream-terrafy/bin/activate # OPTIONAL - for use with venv\nexport ECHOSTREAM_APPSYNC_ENDPOINT=<api_user_appsync_endpoint>\nexport ECHOSTREAM_CLIENT_ID=<api_user_client_id>\nexport ECHOSTREAM_PASSWORD=<api_user_password>\nexport ECHOSTREAM_TENANT=<my_tenant_name>\nexport ECHOSTREAM_USER_POOL_ID=<api_user_user_pool_id>\nexport ECHOSTREAM_USERNAME=<api_user_username>\nechostream-terrafy\ndeactivate # OPTIONAL - for use with venv\n```\n\n### Executing using command-line variables\n```shell\nsource echostream-terrafy/bin/activate # OPTIONAL - for use with venv\nechostream-terrafy \\\n --appsync-endpoint <api_user_appsync_endpoint> \\\n --client-id <api_user_client_id> \\\n --password <api_user_password> \\\n --tenant <my_tenant_name> \\\n --user-pool-id <api_user_user_pool_id> \\\n --username <api_user_username>\ndeactivate # OPTIONAL - for use with venv\n```\n\n## Output\n`echostream-terrafy` will generate the following files.\n\n| Filename | Content |\n| --- | --- |\n| artifacts | Directory of Python code, config JSON and readme artifacts for resources |\n| api-users.tf.json | The ApiUser resources |\n| apps.tf.json | The App resources |\n| functions.tf.json | The Function data sources and resources |\n| kms-keys.tf.json | The KmsKey resources (except the Tenant default KmsKey) |\n| main.tf.json | The `terraform` block |\n| managed-node-types.tf.json | The ManagedNodeType data sources and resources |\n| message-types.tf.json | The MessageType data sources and resources |\n| nodes.tf.json | The Node data sources and resources |\n| provider.tf.json | The `provider` block |\n| tenant-users.tf.json | The TenantUser resources |\n| tenant.json | The Tenant resource |\n| terraform.tfstate | The current state, imported |\n\n### Using the output as-is\nYou may use the output from `echostream-terrafy` as-is to manage your Tenant.\n\nSimply make any changes that you wish to it and run `terraform plan` or `terraform apply`.\n\n> Warning - rerunning `echostream-terrafy` after you have made manual changes will result in those changes being overwritten!\n\n### Using the output as a Terraform [module](https://developer.hashicorp.com/terraform/language/modules)\n1. Copy all of the `.tf.json` files and the `artifacts` directory to another folder.\n2. Remove `provider.tf.json`. The provider should be passed in by the module caller.\n3. Add a `variables.tf` file and variablize any input (e.g. - configs) that you wish to be modifiable by module users.\n4. Add an `outputs.tf` file and output any information that needs to be accessed by module users.\n5. Register the module with a public/private Terraform registry or include it in a `modules` directory (either directly or as a `git` submodule) in another Terraform project.\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Takes an existing EchoStream Tenant and creates a Terraform script and statefile for it",
"version": "0.0.12",
"project_urls": {
"Bug Tracker": "https://github.com/Echo-Stream/echostream-terrafy/issues",
"Homepage": "https://github.com/Echo-Stream/echostream-terrafy",
"Source Code": "https://github.com/Echo-Stream/echostream-terrafy"
},
"split_keywords": [
"echostream",
" terraform",
" opentofu",
" tofu",
" terrafy"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "30ee55e058b70e095ef87a399cc69ea6ed4659e0929c048cb5fcbad0b2f6e79b",
"md5": "f63457d5385bbef5f7bc73e213388714",
"sha256": "e2aef0549baf71cfe1f323c7e401d508909f6b4eb3a20026b150ffe40dd5e0e6"
},
"downloads": -1,
"filename": "echostream_terrafy-0.0.12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f63457d5385bbef5f7bc73e213388714",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 18261,
"upload_time": "2025-01-22T21:12:22",
"upload_time_iso_8601": "2025-01-22T21:12:22.567445Z",
"url": "https://files.pythonhosted.org/packages/30/ee/55e058b70e095ef87a399cc69ea6ed4659e0929c048cb5fcbad0b2f6e79b/echostream_terrafy-0.0.12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "2eddb1b34d73ccb13fd6e0793e8fc1a89a3bf96edad29d8d943691906a12f8c0",
"md5": "e0ff644658e06142b3bfd0a24572abd1",
"sha256": "fa630466ee67c7e786e5e425de789a3b235766628cb69804ff0b56e12152d2fc"
},
"downloads": -1,
"filename": "echostream_terrafy-0.0.12.tar.gz",
"has_sig": false,
"md5_digest": "e0ff644658e06142b3bfd0a24572abd1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 18945,
"upload_time": "2025-01-22T21:12:24",
"upload_time_iso_8601": "2025-01-22T21:12:24.908428Z",
"url": "https://files.pythonhosted.org/packages/2e/dd/b1b34d73ccb13fd6e0793e8fc1a89a3bf96edad29d8d943691906a12f8c0/echostream_terrafy-0.0.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-22 21:12:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Echo-Stream",
"github_project": "echostream-terrafy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "echostream-terrafy"
}
JSON
