Mergeconfigs is a package that help managing configuration files.
**For now only YAML is supported.**
Mergeconfigs makes easy to merge several yaml files into one, using includes, placeholders and extends.
That way you can split your configurations into several files and merge them into one single yaml.
# Install
Run `pip install mergeconfigs` to install.
Run `python -m mergeconfigs --help` for help.
# Syntax
In a YAML file, one can use the following syntax:
- Include this file in another file: `#extends path/to/file.yaml`, if the current file contain
common keys with the other file, the current file will override it.
- Load context from a file: `#load path/to/file.yaml`, all the variables of this file will be loaded into a
context and can be used as a placeholder.
- Include the content of another file: `#include path/to/file.yaml`, if the file contain keys
that are shared by the current file, it will override it.
- Use a variable from the context: `${namespace@variable}`, namespace here refer to
the name of the file (without its extension) that contain the variable. It supports accessing
dictionary keys, for example a variable `foo["bar"]` from `logging.yaml` can be used with `${logging@foo.bar}`.
The current file can be referenced with the special namespace `this`: `${this@myvar}`.
> **Note:** The file is read line by line, which mean if an #include is used in a file,
> some values can still be overriden in the following lines.
# Hierarchy descriptor
In most cases your configs will follow some hierarachy, we didn't wanna impose anything here so we give to the user the possibility to
pass a `hierarchy` file which contain the hierarchical order of the environments. Every environment should be in a single line, with the
root level at the top, i.e. if a file is not found in a directory, it will be searched in the directory above him. For example:
```
prod
stage
dev
local
```
# Example
Mergeconfigs use basic templating functions like variables insertion and
including or extending other templates.
It also provides support for different environment.
It is best to explain how it works with an example:
Say we have an application directory with two settings config file,
`settings-prod.yaml` is the production settings file and `settings-local.yaml`
is a copy of `settings-prod.yaml` but use a local mongoDB:
```
| app/
| src/
| my_program.py
| settings-prod.yaml
| settings-local.yaml
```
The config files contain the following content:
**settings-prod.yaml**
```
app_name: "MY_APP"
app_port: 5000
app_secret_key = "my-very-secret-key"
my_service:
health_check_url: "http://my_service.com/check"
mongodb:
uri: "mongodb://user:passwd@
auto_connect: false
print_errors: true
db_alias: "my_database_alias"
log_messages: true
log_level: info
log_prefix: "my_app_logger:"
```
**settings-local.yaml**
```
app_name: "MY_APP"
app_port: 5000
app_secret_key = "my-very-secret-key"
my_service:
health_check_url: "http://my_service.com/check"
mongodb:
uri: "mongodb://127.0.0.1:27017/my_database?readpreferenceaen=rest"
auto_connect: false
print_errors: true
db_alias: "my_database_alias"
log_messages: true
log_level: info
log_prefix: "my_app_logger:"
```
Big config files can be overwhelming, especially if you need to update/edit them to fit
your needs. Mergeconfigs help to split them to multiple files, here
is an example of how it can be used, add a `config` directory to `app/`,
move all the core content into `core.yaml` and split the other contents
into other files, adding a `hosts.yaml` file with all the hosts can be very useful:
```
| app/
| src/
| | my_program.py
| config/
| core.yaml
| logging.yaml
| prod
| | hosts.yaml
| local
| | hosts.yaml
| .hierarchy
```
Then generate a single `settings.yaml` file using:
```
python -m mergeconfigs --workdir config --file core.yaml --env local --hierarchy .hierarchy
```
Here is the content of the files:
**.hierarchy**
```
prod
local
```
**core.yaml**
```
#load $$ENV$$/hosts.yaml
app_name: "MY_APP"
app_port: 5000
app_secret_key = "my-very-secret-key"
my_service:
health_check_url: "http://${hosts@my_service.host}/check"
mongodb:
uri: "mongodb://${hosts@mongodb.host}/my_database?readpreferenceaen=rest"
auto_connect: false
print_errors: true
db_alias: "my_database_alias"
#include logging.yaml
```
The first line load all the variables in the `$$ENV$$/hosts.yaml` file, here
`$$ENV$$` is replaced by `local` because of the `--env local` parameter. Therefore,
it will load the `local/hosts.yaml` file, which itself extends the
`prod/hosts.yaml` file.
The hosts are then replaced using placeholders, like `${hosts@my_service.host}`
Then `logging.yaml` is added to the content of the yaml file.
**logging.yaml**
```
log_messages: true
log_level: info
log_prefix: "my_app_logger:"
```
**prod/hosts.yaml**
```
my_service:
host: my_service.com
mongodb:
host: mymongodb.net:27017
```
**local/hosts.yaml**
```
#extends prod/hosts.yaml
mongodb:
host: 127.0.0.1:27017
```
The `local/hosts.yaml` file extends `prod/hosts.yaml` file, it only
overrides one of its attribute, the `mongodb.host` one.
# TODO
- Add support for list indices in variables: `${file@variable.0}`
Raw data
{
"_id": null,
"home_page": "https://github.com/x41lakazam/mergeconfigs",
"name": "mergeconfigs",
"maintainer": "Eyal Chocron",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "x41lakazam@gmail.com",
"keywords": "config,yaml,inheritance,templates",
"author": "Eyal Chocron",
"author_email": "x41lakazam@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/97/c8/246fbb5b2dc6df846643bdb8f8fcb3ceb2003bbe7ae49b609ab6a2c2122e/mergeconfigs-0.10.tar.gz",
"platform": null,
"description": "Mergeconfigs is a package that help managing configuration files.\n\n**For now only YAML is supported.**\n\nMergeconfigs makes easy to merge several yaml files into one, using includes, placeholders and extends.\nThat way you can split your configurations into several files and merge them into one single yaml. \n\n# Install\n\nRun `pip install mergeconfigs` to install. \n\nRun `python -m mergeconfigs --help` for help.\n\n# Syntax\nIn a YAML file, one can use the following syntax:\n\n- Include this file in another file: `#extends path/to/file.yaml`, if the current file contain\ncommon keys with the other file, the current file will override it.\n- Load context from a file: `#load path/to/file.yaml`, all the variables of this file will be loaded into a\ncontext and can be used as a placeholder.\n- Include the content of another file: `#include path/to/file.yaml`, if the file contain keys \nthat are shared by the current file, it will override it.\n- Use a variable from the context: `${namespace@variable}`, namespace here refer to \nthe name of the file (without its extension) that contain the variable. It supports accessing \ndictionary keys, for example a variable `foo[\"bar\"]` from `logging.yaml` can be used with `${logging@foo.bar}`.\nThe current file can be referenced with the special namespace `this`: `${this@myvar}`.\n\n> **Note:** The file is read line by line, which mean if an #include is used in a file, \n> some values can still be overriden in the following lines.\n\n# Hierarchy descriptor\nIn most cases your configs will follow some hierarachy, we didn't wanna impose anything here so we give to the user the possibility to \npass a `hierarchy` file which contain the hierarchical order of the environments. Every environment should be in a single line, with the \nroot level at the top, i.e. if a file is not found in a directory, it will be searched in the directory above him. For example:\n```\nprod\nstage\ndev\nlocal\n```\n\n# Example\n\nMergeconfigs use basic templating functions like variables insertion and \nincluding or extending other templates. \nIt also provides support for different environment. \nIt is best to explain how it works with an example:\n\nSay we have an application directory with two settings config file, \n`settings-prod.yaml` is the production settings file and `settings-local.yaml`\nis a copy of `settings-prod.yaml` but use a local mongoDB:\n\n```\n| app/\n | src/\n | my_program.py\n | settings-prod.yaml \n | settings-local.yaml \n```\n\nThe config files contain the following content:\n\n**settings-prod.yaml**\n```\napp_name: \"MY_APP\"\napp_port: 5000\napp_secret_key = \"my-very-secret-key\"\n\nmy_service:\n health_check_url: \"http://my_service.com/check\"\n \nmongodb:\n uri: \"mongodb://user:passwd@\n auto_connect: false\n print_errors: true\n db_alias: \"my_database_alias\"\n \nlog_messages: true\nlog_level: info \nlog_prefix: \"my_app_logger:\"\n```\n\n**settings-local.yaml**\n```\napp_name: \"MY_APP\"\napp_port: 5000\napp_secret_key = \"my-very-secret-key\"\n\nmy_service:\n health_check_url: \"http://my_service.com/check\"\n \nmongodb:\n uri: \"mongodb://127.0.0.1:27017/my_database?readpreferenceaen=rest\"\n auto_connect: false\n print_errors: true\n db_alias: \"my_database_alias\"\n \nlog_messages: true\nlog_level: info \nlog_prefix: \"my_app_logger:\"\n```\n\nBig config files can be overwhelming, especially if you need to update/edit them to fit\nyour needs. Mergeconfigs help to split them to multiple files, here\nis an example of how it can be used, add a `config` directory to `app/`,\nmove all the core content into `core.yaml` and split the other contents\ninto other files, adding a `hosts.yaml` file with all the hosts can be very useful:\n\n```\n| app/\n | src/\n | | my_program.py\n | config/\n | core.yaml\n | logging.yaml\n | prod\n | | hosts.yaml\n | local\n | | hosts.yaml\n | .hierarchy\n```\nThen generate a single `settings.yaml` file using: \n\n```\npython -m mergeconfigs --workdir config --file core.yaml --env local --hierarchy .hierarchy\n```\n\n\nHere is the content of the files:\n\n**.hierarchy**\n```\nprod\nlocal\n```\n\n**core.yaml**\n```\n#load $$ENV$$/hosts.yaml\n\napp_name: \"MY_APP\"\napp_port: 5000\napp_secret_key = \"my-very-secret-key\"\n\nmy_service:\n health_check_url: \"http://${hosts@my_service.host}/check\"\n \nmongodb:\n uri: \"mongodb://${hosts@mongodb.host}/my_database?readpreferenceaen=rest\"\n auto_connect: false\n print_errors: true\n db_alias: \"my_database_alias\"\n\n#include logging.yaml\n```\n\nThe first line load all the variables in the `$$ENV$$/hosts.yaml` file, here\n`$$ENV$$` is replaced by `local` because of the `--env local` parameter. Therefore,\nit will load the `local/hosts.yaml` file, which itself extends the\n`prod/hosts.yaml` file.\n\nThe hosts are then replaced using placeholders, like `${hosts@my_service.host}` \n\nThen `logging.yaml` is added to the content of the yaml file.\n\n**logging.yaml**\n```\nlog_messages: true\nlog_level: info \nlog_prefix: \"my_app_logger:\"\n```\n\n**prod/hosts.yaml**\n```\nmy_service:\n host: my_service.com\n \nmongodb:\n host: mymongodb.net:27017\n```\n\n**local/hosts.yaml**\n```\n#extends prod/hosts.yaml\n\nmongodb:\n host: 127.0.0.1:27017\n```\n\nThe `local/hosts.yaml` file extends `prod/hosts.yaml` file, it only\noverrides one of its attribute, the `mongodb.host` one.\n\n\n\n# TODO\n- Add support for list indices in variables: `${file@variable.0}`\n",
"bugtrack_url": null,
"license": "",
"summary": "Help developers with config files",
"version": "0.10",
"split_keywords": [
"config",
"yaml",
"inheritance",
"templates"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "11dfa0106277d31e5b7846adf7e6e8862d761c766f1e288a0f8d906f89e215a9",
"md5": "f368182f3116d016b75ae66350fb4e5e",
"sha256": "f405096914bbadc649d071c4dc59bf8e8d2d347eccaa9af3a551a8c8283d23b2"
},
"downloads": -1,
"filename": "mergeconfigs-0.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f368182f3116d016b75ae66350fb4e5e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 6788,
"upload_time": "2023-04-16T10:03:12",
"upload_time_iso_8601": "2023-04-16T10:03:12.735037Z",
"url": "https://files.pythonhosted.org/packages/11/df/a0106277d31e5b7846adf7e6e8862d761c766f1e288a0f8d906f89e215a9/mergeconfigs-0.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "97c8246fbb5b2dc6df846643bdb8f8fcb3ceb2003bbe7ae49b609ab6a2c2122e",
"md5": "869244e6bf2a3ede9ceba77ccd415b45",
"sha256": "bb46818f3e3bfb70945cadbbf65005b1976542fe207ed1de1d50b49edb2cfbc1"
},
"downloads": -1,
"filename": "mergeconfigs-0.10.tar.gz",
"has_sig": false,
"md5_digest": "869244e6bf2a3ede9ceba77ccd415b45",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 6233,
"upload_time": "2023-04-16T10:03:14",
"upload_time_iso_8601": "2023-04-16T10:03:14.812300Z",
"url": "https://files.pythonhosted.org/packages/97/c8/246fbb5b2dc6df846643bdb8f8fcb3ceb2003bbe7ae49b609ab6a2c2122e/mergeconfigs-0.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-16 10:03:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "x41lakazam",
"github_project": "mergeconfigs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mergeconfigs"
}
JSON
