# j2subst - Jinja2 template substitution tool
j2subst is a command-line tool for processing Jinja2 templates with configuration data from multiple sources. It's designed for use in CI/CD pipelines and configuration management workflows.
## Features
- **Multiple configuration formats**: Supports YAML, JSON, and TOML configuration files
- **Flexible template resolution**: Automatic template path resolution with placeholders
- **Environment variables**: Access to environment variables within templates
- **Python module integration**: Import Python modules for advanced template logic
- **Built-in functions & filters**: Comprehensive set of built-in Jinja2 filters and functions
- **CI/CD aware**: Special behavior for CI/CD environments
- **Dump mode**: Export configuration data for debugging and inspection
## Installation
### Using Docker
```sh
docker pull docker.io/rockdrilla/j2subst
```
### From PyPI
```sh
pip install j2subst
```
## Quick start
### Docker usage
Docker image `docker.io/rockdrilla/j2subst` has several extra things done:
1) entrypoint is set to `j2subst` script;
2) current working directory is set to "`/w`" and it's volume.
3) environment variable `J2SUBST_PYTHON_MODULES` is set to `"netaddr psutil"` - these packages are installed via `pip` and they are not dependencies in any kind, but provided for better usability; see file `docker/requirements-extra.txt`.
To simplify usage, the one may define shell alias:
```sh
alias j2subst='docker run --rm -v "${PWD}:/w" docker.io/rockdrilla/j2subst '
```
### Basic usage
Process a single template file:
```sh
j2subst template.j2
```
This will process `template.j2` and output to `template` (removing the `.j2` extension).
### Input/output specification
Process template with explicit input and output:
```sh
j2subst input.j2 output.txt
```
Use stdin/stdout:
```sh
cat template.j2 | j2subst - > output.txt
j2subst input.j2 -
```
### Directory processing
Process all templates in a directory (no recursion):
```sh
j2subst /path/to/templates/
```
Control recursion depth:
```sh
j2subst --depth 3 /path/to/templates/
```
## Configuration
### Configuration files/directories
j2subst can load configuration from multiple sources:
```sh
j2subst -c config.yml template.j2
j2subst -c /path/to/configs/ template.j2
```
Supported formats:
- YAML (`.yaml`, `.yml`)
- JSON (`.json`)
- TOML (`.toml`)
### Template paths
Specify template search paths:
```sh
j2subst -t /custom/templates:/other/templates template.j2
```
Default template path is `"@{ORIGIN}:@{CWD}"`.
Template path placeholders:
- `@{CWD}` - current working directory
- `@{ORIGIN}` - directory containing the currently processed template file. This placeholder dynamically updates when processing multiple templates, always pointing to the directory of the template currently being rendered.
*Nota bene*: `@{ORIGIN}` is unavailable when processing template from stdin.
## Template context
Templates have access to two main dictionaries:
- `{{ cfg }}` - Configuration data from files
- `{{ env }}` - Environment variables
### Example template
```jinja2
# config.j2
server_name: {{ cfg.server.name }}
database_url: {{ cfg.database.url }}
environment: {{ env.ENVIRONMENT }}
```
With configuration file `config.yml`:
```yaml
server:
name: myserver
database:
url: postgresql://localhost/mydb
```
Along with dictionaries above, the following variables are available:
- `{{ is_ci }}` - `True` if running in CI/CD environment, `False` otherwise
- `{{ j2subst_file }}` - path to the currently processed template
- `{{ j2subst_origin }}` - directory of the currently processed template
*Nota bene*: `j2subst_file` and `j2subst_origin` are set to `None` when processing template from stdin.
Example:
```jinja2
{% if is_ci %}
Running in CI environment
{% endif %}
{%- if j2subst_origin %}
{#- accessing files "near" template file -#}
{%- endif %}
```
## Command line options
### Core options
- `--verbose, -v` - Increase verbosity (can be used multiple times)
- `--quiet, -q` - Enable quiet mode (overrides "`--verbose`")
- `--debug, -D` - Enable debug mode (prints debug messages to stderr; overrides "`--quiet`")
- `--strict, -s` - Enable strict mode (warnings become errors)
- `--force, -f` - Enable force mode (overwrite existing files)
- `--unlink, -u` - Delete template files after processing
### Configuration options
- `--config-path, -c PATH` - Colon-separated list of config files/directories
- `--template-path, -t PATH` - Colon-separated list of template directories
- `--depth, -d INTEGER` - Set recursion depth for directory processing (1-20)
### Advanced options
- `--dump [FORMAT]` - Dump configuration to stdout (YAML/JSON) and exit
- `--python-modules LIST` - Space-separated list of Python modules to import
- `--fn-filters` - Propagate J2subst filters as functions too
- `--dict-name-cfg NAME` - Custom name for configuration dictionary
- `--dict-name-env NAME` - Custom name for environment dictionary
### Help options
- `--help-cicd` - Show help for CI/CD behavior
- `--help-click` - Show help for Click behavior
- `--help-dump` - Show help for dump mode
- `--help-env` - Show help for environment variables
- `--help-template-path` - Show help for template paths
## Environment variables
Corresponding environment variables are also supported:
```
|------------------------+------------------+---------|
| Environment variable | Flag option | Type |
|------------------------+------------------+---------|
| J2SUBST_VERBOSE | --verbose | integer |
| J2SUBST_QUIET | --quiet | flag |
| J2SUBST_DEBUG | --debug | flag |
| J2SUBST_STRICT | --strict | flag |
| J2SUBST_FORCE | --force | flag |
| J2SUBST_UNLINK | --unlink | flag |
| J2SUBST_DEPTH | --depth | integer |
| J2SUBST_CONFIG_PATH | --config-path | string |
| J2SUBST_TEMPLATE_PATH | --template-path | string |
| J2SUBST_PYTHON_MODULES | --python-modules | string |
| J2SUBST_FN_FILTERS | --fn-filters | flag |
| J2SUBST_DICT_NAME_CFG | --dict-name-cfg | string |
| J2SUBST_DICT_NAME_ENV | --dict-name-env | string |
|------------------------+------------------+---------|
```
See [Click documentation](https://click.palletsprojects.com/en/stable/options/#values-from-environment-variables) for more details about how Click handles environment variables, especially for flag options.
### CI/CD behavior
- option `--depth` / variable `J2SUBST_DEPTH` defaults to `20` if running in CI/CD and `1` otherwise.
- if argument list is empty then it set to current working directory.
## Built-in Python modules
The following Python modules are available by default in templates:
- `datetime`
- `hashlib`
- `os_path` (alias for `os.path`)
- `pathlib`
- `re`
- `secrets`
- `string`
## Built-in functions
Available built-in functions:
- `bool`, `filter`, `isinstance`, `len`, `list`, `repr`, `set`, `sorted`, `str`, `type`
Reference: [Built-in Functions](https://docs.python.org/3/library/functions.html)
## Built-in filters
Along with default [Jinja2 filters](https://jinja.palletsprojects.com/en/stable/templates/#builtin-filters), J2subst provides extra filters, see dedicated documentation [here](doc/filters.md).
Direct link to GitHub: [doc/filters.md](https://github.com/rockdrilla/j2subst/blob/main/doc/filters.md)
## Examples
### Dump configuration
```sh
j2subst --dump
j2subst --dump json
j2subst -c config.yml --dump
```
### Import custom Python modules
```sh
j2subst --python-modules "myjson:json math" template.j2
```
This imports:
- `json` module as `myjson`
- `math` module as `math`
### Custom dictionary names
```sh
j2subst --dict-name-cfg config --dict-name-env environment template.j2
```
Now use in templates:
```jinja2
{{ config.server.name }}
{{ environment.HOME }}
```
### Propagate J2subst filters as functions
```sh
j2subst --fn-filters template.j2
```
Consider the following template:
```jinja2
{% set x = j2subst_origin | join_prefix('file.html') %}
{# only available with "--fn-filters" #}
{% set y = join_prefix(j2subst_origin, 'file.html') %}
```
## Development
### Building Docker image
```sh
export IMAGE_VERSION=wip
./docker/build-scripts/image-base.sh
./docker/build-scripts/image.sh
```
## License
Apache-2.0
- [spdx.org](https://spdx.org/licenses/Apache-2.0.html)
- [opensource.org](https://opensource.org/licenses/Apache-2.0)
- [apache.org](https://www.apache.org/licenses/LICENSE-2.0)
Raw data
{
"_id": null,
"home_page": null,
"name": "j2subst",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "automation, ci-cd, configuration, deployment, environment, infrastructure, jinja2, json, substitution, templates, toml, yaml",
"author": null,
"author_email": "Konstantin Demin <rockdrilla@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/bd/4c/ca375dc96667220169efdefa01da1d09ec6229ca47d438c7fc16c21c838b/j2subst-0.0.4.tar.gz",
"platform": null,
"description": "# j2subst - Jinja2 template substitution tool\n\nj2subst is a command-line tool for processing Jinja2 templates with configuration data from multiple sources. It's designed for use in CI/CD pipelines and configuration management workflows.\n\n## Features\n\n- **Multiple configuration formats**: Supports YAML, JSON, and TOML configuration files\n- **Flexible template resolution**: Automatic template path resolution with placeholders\n- **Environment variables**: Access to environment variables within templates\n- **Python module integration**: Import Python modules for advanced template logic\n- **Built-in functions & filters**: Comprehensive set of built-in Jinja2 filters and functions\n- **CI/CD aware**: Special behavior for CI/CD environments\n- **Dump mode**: Export configuration data for debugging and inspection\n\n## Installation\n\n### Using Docker\n\n```sh\ndocker pull docker.io/rockdrilla/j2subst\n```\n\n### From PyPI\n\n```sh\npip install j2subst\n```\n\n## Quick start\n\n### Docker usage\n\nDocker image `docker.io/rockdrilla/j2subst` has several extra things done:\n1) entrypoint is set to `j2subst` script;\n2) current working directory is set to \"`/w`\" and it's volume.\n3) environment variable `J2SUBST_PYTHON_MODULES` is set to `\"netaddr psutil\"` - these packages are installed via `pip` and they are not dependencies in any kind, but provided for better usability; see file `docker/requirements-extra.txt`.\n\nTo simplify usage, the one may define shell alias:\n```sh\nalias j2subst='docker run --rm -v \"${PWD}:/w\" docker.io/rockdrilla/j2subst '\n```\n\n### Basic usage\n\nProcess a single template file:\n\n```sh\nj2subst template.j2\n```\n\nThis will process `template.j2` and output to `template` (removing the `.j2` extension).\n\n### Input/output specification\n\nProcess template with explicit input and output:\n\n```sh\nj2subst input.j2 output.txt\n```\n\nUse stdin/stdout:\n\n```sh\ncat template.j2 | j2subst - > output.txt\nj2subst input.j2 -\n```\n\n### Directory processing\n\nProcess all templates in a directory (no recursion):\n\n```sh\nj2subst /path/to/templates/\n```\n\nControl recursion depth:\n\n```sh\nj2subst --depth 3 /path/to/templates/\n```\n\n## Configuration\n\n### Configuration files/directories\n\nj2subst can load configuration from multiple sources:\n\n```sh\nj2subst -c config.yml template.j2\nj2subst -c /path/to/configs/ template.j2\n```\n\nSupported formats:\n- YAML (`.yaml`, `.yml`)\n- JSON (`.json`)\n- TOML (`.toml`)\n\n### Template paths\n\nSpecify template search paths:\n\n```sh\nj2subst -t /custom/templates:/other/templates template.j2\n```\n\nDefault template path is `\"@{ORIGIN}:@{CWD}\"`.\n\nTemplate path placeholders:\n- `@{CWD}` - current working directory\n- `@{ORIGIN}` - directory containing the currently processed template file. This placeholder dynamically updates when processing multiple templates, always pointing to the directory of the template currently being rendered.\n\n*Nota bene*: `@{ORIGIN}` is unavailable when processing template from stdin.\n\n## Template context\n\nTemplates have access to two main dictionaries:\n\n- `{{ cfg }}` - Configuration data from files\n- `{{ env }}` - Environment variables\n\n### Example template\n\n```jinja2\n# config.j2\nserver_name: {{ cfg.server.name }}\ndatabase_url: {{ cfg.database.url }}\nenvironment: {{ env.ENVIRONMENT }}\n```\n\nWith configuration file `config.yml`:\n```yaml\nserver:\n name: myserver\ndatabase:\n url: postgresql://localhost/mydb\n```\n\nAlong with dictionaries above, the following variables are available:\n\n- `{{ is_ci }}` - `True` if running in CI/CD environment, `False` otherwise\n- `{{ j2subst_file }}` - path to the currently processed template\n- `{{ j2subst_origin }}` - directory of the currently processed template\n\n*Nota bene*: `j2subst_file` and `j2subst_origin` are set to `None` when processing template from stdin.\n\nExample:\n```jinja2\n{% if is_ci %}\n Running in CI environment\n{% endif %}\n\n{%- if j2subst_origin %}\n {#- accessing files \"near\" template file -#}\n{%- endif %}\n```\n\n## Command line options\n\n### Core options\n\n- `--verbose, -v` - Increase verbosity (can be used multiple times)\n- `--quiet, -q` - Enable quiet mode (overrides \"`--verbose`\")\n- `--debug, -D` - Enable debug mode (prints debug messages to stderr; overrides \"`--quiet`\")\n- `--strict, -s` - Enable strict mode (warnings become errors)\n- `--force, -f` - Enable force mode (overwrite existing files)\n- `--unlink, -u` - Delete template files after processing\n\n### Configuration options\n\n- `--config-path, -c PATH` - Colon-separated list of config files/directories\n- `--template-path, -t PATH` - Colon-separated list of template directories\n- `--depth, -d INTEGER` - Set recursion depth for directory processing (1-20)\n\n### Advanced options\n\n- `--dump [FORMAT]` - Dump configuration to stdout (YAML/JSON) and exit\n\n- `--python-modules LIST` - Space-separated list of Python modules to import\n- `--fn-filters` - Propagate J2subst filters as functions too\n- `--dict-name-cfg NAME` - Custom name for configuration dictionary\n- `--dict-name-env NAME` - Custom name for environment dictionary\n\n### Help options\n\n- `--help-cicd` - Show help for CI/CD behavior\n- `--help-click` - Show help for Click behavior\n- `--help-dump` - Show help for dump mode\n- `--help-env` - Show help for environment variables\n- `--help-template-path` - Show help for template paths\n\n## Environment variables\n\nCorresponding environment variables are also supported:\n```\n|------------------------+------------------+---------|\n| Environment variable | Flag option | Type |\n|------------------------+------------------+---------|\n| J2SUBST_VERBOSE | --verbose | integer |\n| J2SUBST_QUIET | --quiet | flag |\n| J2SUBST_DEBUG | --debug | flag |\n| J2SUBST_STRICT | --strict | flag |\n| J2SUBST_FORCE | --force | flag |\n| J2SUBST_UNLINK | --unlink | flag |\n| J2SUBST_DEPTH | --depth | integer |\n| J2SUBST_CONFIG_PATH | --config-path | string |\n| J2SUBST_TEMPLATE_PATH | --template-path | string |\n| J2SUBST_PYTHON_MODULES | --python-modules | string |\n| J2SUBST_FN_FILTERS | --fn-filters | flag |\n| J2SUBST_DICT_NAME_CFG | --dict-name-cfg | string |\n| J2SUBST_DICT_NAME_ENV | --dict-name-env | string |\n|------------------------+------------------+---------|\n```\n\nSee [Click documentation](https://click.palletsprojects.com/en/stable/options/#values-from-environment-variables) for more details about how Click handles environment variables, especially for flag options.\n\n### CI/CD behavior\n\n- option `--depth` / variable `J2SUBST_DEPTH` defaults to `20` if running in CI/CD and `1` otherwise.\n- if argument list is empty then it set to current working directory.\n\n## Built-in Python modules\n\nThe following Python modules are available by default in templates:\n\n- `datetime`\n- `hashlib`\n- `os_path` (alias for `os.path`)\n- `pathlib`\n- `re`\n- `secrets`\n- `string`\n\n## Built-in functions\n\nAvailable built-in functions:\n\n- `bool`, `filter`, `isinstance`, `len`, `list`, `repr`, `set`, `sorted`, `str`, `type`\n\nReference: [Built-in Functions](https://docs.python.org/3/library/functions.html)\n\n## Built-in filters\n\nAlong with default [Jinja2 filters](https://jinja.palletsprojects.com/en/stable/templates/#builtin-filters), J2subst provides extra filters, see dedicated documentation [here](doc/filters.md).\n\nDirect link to GitHub: [doc/filters.md](https://github.com/rockdrilla/j2subst/blob/main/doc/filters.md)\n\n## Examples\n\n### Dump configuration\n\n```sh\nj2subst --dump\nj2subst --dump json\nj2subst -c config.yml --dump\n```\n\n### Import custom Python modules\n\n```sh\nj2subst --python-modules \"myjson:json math\" template.j2\n```\n\nThis imports:\n- `json` module as `myjson`\n- `math` module as `math`\n\n### Custom dictionary names\n\n```sh\nj2subst --dict-name-cfg config --dict-name-env environment template.j2\n```\n\nNow use in templates:\n```jinja2\n{{ config.server.name }}\n{{ environment.HOME }}\n```\n\n### Propagate J2subst filters as functions\n\n```sh\nj2subst --fn-filters template.j2\n```\n\nConsider the following template:\n```jinja2\n{% set x = j2subst_origin | join_prefix('file.html') %}\n\n{# only available with \"--fn-filters\" #}\n{% set y = join_prefix(j2subst_origin, 'file.html') %}\n```\n\n## Development\n\n### Building Docker image\n\n```sh\nexport IMAGE_VERSION=wip\n./docker/build-scripts/image-base.sh\n./docker/build-scripts/image.sh\n```\n\n## License\n\nApache-2.0\n\n- [spdx.org](https://spdx.org/licenses/Apache-2.0.html)\n- [opensource.org](https://opensource.org/licenses/Apache-2.0)\n- [apache.org](https://www.apache.org/licenses/LICENSE-2.0)\n",
"bugtrack_url": null,
"license": null,
"summary": "Jinja2 template substitution tool",
"version": "0.0.4",
"project_urls": {
"Bug Reports": "https://github.com/rockdrilla/j2subst/issues",
"Docker Image": "https://hub.docker.com/r/rockdrilla/j2subst/tags",
"Homepage": "https://github.com/rockdrilla/j2subst",
"Source Code": "https://github.com/rockdrilla/j2subst"
},
"split_keywords": [
"automation",
" ci-cd",
" configuration",
" deployment",
" environment",
" infrastructure",
" jinja2",
" json",
" substitution",
" templates",
" toml",
" yaml"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ad79a70eda45b8df742572bc094ffd3be1042c46b9374cc53f895198f3fc1083",
"md5": "8ee4f8770842cf73ddba422bbca6f478",
"sha256": "2451b1d13087cca1698447295170c2a3800a085a22b5a19878ec5af67244278f"
},
"downloads": -1,
"filename": "j2subst-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8ee4f8770842cf73ddba422bbca6f478",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 25204,
"upload_time": "2025-10-14T04:28:26",
"upload_time_iso_8601": "2025-10-14T04:28:26.493090Z",
"url": "https://files.pythonhosted.org/packages/ad/79/a70eda45b8df742572bc094ffd3be1042c46b9374cc53f895198f3fc1083/j2subst-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "bd4cca375dc96667220169efdefa01da1d09ec6229ca47d438c7fc16c21c838b",
"md5": "65a3c2b6ee4e8fa2ee31ecbf3491252a",
"sha256": "691a847a49f11e36becf7ca7d3025ae80e9141255349c3581832db24938a5e4a"
},
"downloads": -1,
"filename": "j2subst-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "65a3c2b6ee4e8fa2ee31ecbf3491252a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 25664,
"upload_time": "2025-10-14T04:28:27",
"upload_time_iso_8601": "2025-10-14T04:28:27.622167Z",
"url": "https://files.pythonhosted.org/packages/bd/4c/ca375dc96667220169efdefa01da1d09ec6229ca47d438c7fc16c21c838b/j2subst-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-14 04:28:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rockdrilla",
"github_project": "j2subst",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "j2subst"
}
JSON
