| Name | variconf JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | |
| Summary | Wrapper around OmegaConf for loading configuration from various types of files. |
| upload_time | 2023-02-10 13:03:00 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | |
| keywords |
config
configuration
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
VariConf - Load Configurations from Various Types of Files
==========================================================
VariConf provides a wrapper around [OmegaConf](https://omegaconf.readthedocs.io) for
loading configuration from various types of files.
Supported file types are JSON, YAML and TOML. Support for more file types can easily be
added by registering a custom loader function.
Thanks to the power of OmegaConf, you can provide a configuration schema which the
defines expected parameters, default values and optionally expected types.
Design Goals
------------
This package is developed with the following goals in mind:
- Load configuration from files with expected parameters and default values provided in
an easy way.
- Provide config as a simple Namespace-like object (with option to convert to
dictionary).
- Do not commit to a specific file format. All formats that can easily be loaded into a
dictionary should be supported (json, toml, yaml, ...).
- Optionally check types.
- Optionally check for unknown parameters in the file (to raise an error).
- Keep it simple. Prefer fewer features over too complicated API.
Installation
------------
Basic (does not include dependencies for YAML and TOML):
```
pip install variconf
```
With optional dependencies:
```bash
# with TOML support
pip install "variconf[toml]"
# with YAML support
pip install "variconf[yaml]"
# to include everything:
pip install "variconf[all]"
```
Usage
-----
The package provides a class `WConf` for loading and merging configurations from
different sources. When creating an instance of it, a configuration "schema" needs to be
given, i.e. a structure (dictionary or dataclass) that defines what sections and
parameters the configuration has and that provides default values.
A number of "load"-methods is provided to load configurations from different sources
(e.g. JSON files, YAML files, dictionaries from other sources, ...). When called,
the corresponding parameters are merged into the existing configuration, overwriting
existing values. This means that an input does not need to provide all parameters,
in this case the default values are kept. Further, if calling multiple
"load"-methods after another, the later calls will overwrite values set by previous
ones.
All "load"-methods return ``self``, so they can be chained:
```python
import variconf
schema = {"sec1": {"foo": 42, "bar": 13}, "sec2": {"bla": ""}}
wconf = variconf.WConf(schema)
config = (
wconf.load_file("~/global_config.toml")
.load_file("./local_config.yml", fail_if_not_found=False)
.load_dotlist(sys.argv[1:]) # easily allow overwriting parameters via
# command-line arguments
.get() # return the final config object
)
```
### Allow Unknown Parameters
By default an error is raised if the loaded configuration contains parameters that are
not declared in the schema. If you want to allow these unknown parameters, initialise
`WConf` with `strict=False`:
```python
wconf = variconf.WConf(schema, strict=False)
```
This will result in the unknown parameters being merged into the config object.
With this you can even omit the schema altogether by simply passing an empty dictionary:
```python
wconf = variconf.WConf({}, strict=False)
```
### Search File
Assuming an application where the config file can be located in one of several places
(e.g. `~`, `~/.config` or `/etc/myapp`). This situation is supported by the optional
`search_paths` argument of `load_file()`:
```python
wconf.load_file(
"config.yml",
search_paths=[os.expanduser("~"), os.expanduser("~/.config"), "/etc/myapp"],
fail_if_not_found=False,
)
```
This will search for a file "config.yml" in the listed directories (in the given order)
and use the first match.
By setting `fail_if_not_found=False`, we specify that it's okay if the file is not found
in any of these directories. In this case, we simply keep the default values of all
parameters.
### Using XDG Base Directory Specification
If your application follows the [XDG Base Directory
Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html)
you can use ``load_xdg_config()`` (currently not supported on Windows!):
```python
wconf.load_xdg_config("myapp/config.toml")
```
Will search for the file in the directories specified in the environment variables
`XDG_CONFIG_HOME` and `XDG_CONFIG_DIRS` (defaulting to `~/.config`).
Like for `load_file()` there is an argument `fail_if_not_found` but here it defaults to
False as providing a config in `XDG_CONFIG_HOME` is typically optional.
### Supported File Types
Supported file types are JSON, YAML and TOML. Support for custom file types can be
added by providing a loader function. Example for adding XML support:
```python
import xml.etree.ElementTree as ET
def xml_loader(fp: typing.IO) -> dict:
xml_str = fp.read()
xml_tree = ET.fromstring(xml_str)
# do some magic to convert XML tree to dictionary
xml_dict = tree_to_dict(xml_tree)
return xml_dict
wconf.add_file_loader("xml", [".xml"], xml_loader)
# now, XML files can be read by WConf.load and WConf.load_file
wconf.load_file("config.xml")
```
### Type Checking
OmegaConf supports type-checking by providing a schema as dataclass with type hints:
```python
@dataclasses.dataclass
class ConfigSchema:
foo: int = 42
bar: str = 13
wconf = variconf.WConf(ConfigSchema)
# raises ValidationError: Value 'hi' of type 'str' could not be converted to Integer
wconf.load_dict({"foo": "hi"})
```
### Required Values
Required parameters without default value are supported through OmegaConf's concept of
missing values.
When using a dictionary schema:
```python
schema = {
"optional_param": "default value",
"required_param": "???",
}
```
When using a dataclass schema:
```python
@dataclasses.dataclass
class Schema:
required_param1: float # not providing a default makes it required
optional_param: str = "default value"
required_param2: int = omegaconf.MISSING # alternative for required parameters
```
If there is a required parameter for which no value has been provided by any of the
`load*`-methods, calling `get()` will raise an error.
You can avoid that error by using `get(allow_missing=True)`. However, the error is
still raised when trying to access the actual value of the missing parameter.
### Variable Interpolation
OmegaConf has a feature called [variable interpolation](https://omegaconf.readthedocs.io/en/latest/usage.html#variable-interpolation)
that allows to refer to other fields within the config file:
```yaml
server:
host: localhost
port: 80
client:
url: http://${server.host}:${server.port}/
server_port: ${server.port}
# relative interpolation
description: Client of ${.url}
```
See the documentation of OmegaConf for more information.
Copyright and License
---------------------
VariConf is written and maintained by Felix Kloss at the Max Planck Institute for
Intelligent Systems, Tübingen.
Copyright (c) 2022, Max Planck Gesellschaft. All rights reserved.
License: [BSD 3-clause](LICENSE)
Raw data
{
"_id": null,
"home_page": "",
"name": "variconf",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "Felix Kloss <felix.widmaier@tue.mpg.de>",
"keywords": "config,configuration",
"author": "",
"author_email": "Felix Kloss <felix.widmaier@tue.mpg.de>",
"download_url": "https://files.pythonhosted.org/packages/15/4f/fcdeaa5e6d24c0256d5c4a941f521c36999c359d75ebe14e828e01e59c3b/variconf-1.0.1.tar.gz",
"platform": null,
"description": "VariConf - Load Configurations from Various Types of Files\n==========================================================\n\nVariConf provides a wrapper around [OmegaConf](https://omegaconf.readthedocs.io) for\nloading configuration from various types of files.\n\nSupported file types are JSON, YAML and TOML. Support for more file types can easily be\nadded by registering a custom loader function.\n\nThanks to the power of OmegaConf, you can provide a configuration schema which the\ndefines expected parameters, default values and optionally expected types.\n\n\nDesign Goals\n------------\n\nThis package is developed with the following goals in mind:\n\n- Load configuration from files with expected parameters and default values provided in\n an easy way.\n- Provide config as a simple Namespace-like object (with option to convert to\n dictionary).\n- Do not commit to a specific file format. All formats that can easily be loaded into a\n dictionary should be supported (json, toml, yaml, ...).\n- Optionally check types.\n- Optionally check for unknown parameters in the file (to raise an error).\n- Keep it simple. Prefer fewer features over too complicated API.\n\n\nInstallation\n------------\n\nBasic (does not include dependencies for YAML and TOML):\n```\npip install variconf\n```\n\nWith optional dependencies:\n```bash\n# with TOML support\npip install \"variconf[toml]\"\n\n# with YAML support\npip install \"variconf[yaml]\"\n\n# to include everything:\npip install \"variconf[all]\"\n```\n\n\nUsage\n-----\n\nThe package provides a class `WConf` for loading and merging configurations from\ndifferent sources. When creating an instance of it, a configuration \"schema\" needs to be\ngiven, i.e. a structure (dictionary or dataclass) that defines what sections and\nparameters the configuration has and that provides default values.\n\nA number of \"load\"-methods is provided to load configurations from different sources\n(e.g. JSON files, YAML files, dictionaries from other sources, ...). When called,\nthe corresponding parameters are merged into the existing configuration, overwriting\nexisting values. This means that an input does not need to provide all parameters,\nin this case the default values are kept. Further, if calling multiple\n\"load\"-methods after another, the later calls will overwrite values set by previous\nones.\n\nAll \"load\"-methods return ``self``, so they can be chained:\n\n```python\nimport variconf\n\nschema = {\"sec1\": {\"foo\": 42, \"bar\": 13}, \"sec2\": {\"bla\": \"\"}}\nwconf = variconf.WConf(schema)\nconfig = (\n wconf.load_file(\"~/global_config.toml\")\n .load_file(\"./local_config.yml\", fail_if_not_found=False)\n .load_dotlist(sys.argv[1:]) # easily allow overwriting parameters via\n # command-line arguments\n .get() # return the final config object\n)\n```\n\n\n### Allow Unknown Parameters\n\nBy default an error is raised if the loaded configuration contains parameters that are\nnot declared in the schema. If you want to allow these unknown parameters, initialise\n`WConf` with `strict=False`:\n\n```python\nwconf = variconf.WConf(schema, strict=False)\n```\n\nThis will result in the unknown parameters being merged into the config object.\n\nWith this you can even omit the schema altogether by simply passing an empty dictionary:\n```python\nwconf = variconf.WConf({}, strict=False)\n```\n\n\n### Search File\n\nAssuming an application where the config file can be located in one of several places\n(e.g. `~`, `~/.config` or `/etc/myapp`). This situation is supported by the optional\n`search_paths` argument of `load_file()`:\n\n```python\nwconf.load_file(\n \"config.yml\",\n search_paths=[os.expanduser(\"~\"), os.expanduser(\"~/.config\"), \"/etc/myapp\"],\n fail_if_not_found=False,\n)\n```\nThis will search for a file \"config.yml\" in the listed directories (in the given order)\nand use the first match.\nBy setting `fail_if_not_found=False`, we specify that it's okay if the file is not found\nin any of these directories. In this case, we simply keep the default values of all\nparameters.\n\n\n### Using XDG Base Directory Specification\n\nIf your application follows the [XDG Base Directory\nSpecification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html)\nyou can use ``load_xdg_config()`` (currently not supported on Windows!):\n\n```python\nwconf.load_xdg_config(\"myapp/config.toml\")\n```\nWill search for the file in the directories specified in the environment variables\n`XDG_CONFIG_HOME` and `XDG_CONFIG_DIRS` (defaulting to `~/.config`).\n\nLike for `load_file()` there is an argument `fail_if_not_found` but here it defaults to\nFalse as providing a config in `XDG_CONFIG_HOME` is typically optional.\n\n\n### Supported File Types\n\nSupported file types are JSON, YAML and TOML. Support for custom file types can be\nadded by providing a loader function. Example for adding XML support:\n\n```python\nimport xml.etree.ElementTree as ET\n\ndef xml_loader(fp: typing.IO) -> dict:\n xml_str = fp.read()\n xml_tree = ET.fromstring(xml_str)\n # do some magic to convert XML tree to dictionary\n xml_dict = tree_to_dict(xml_tree)\n return xml_dict\n\nwconf.add_file_loader(\"xml\", [\".xml\"], xml_loader)\n\n# now, XML files can be read by WConf.load and WConf.load_file\nwconf.load_file(\"config.xml\")\n```\n\n\n### Type Checking\n\nOmegaConf supports type-checking by providing a schema as dataclass with type hints:\n\n```python\n@dataclasses.dataclass\nclass ConfigSchema:\n foo: int = 42\n bar: str = 13\n\nwconf = variconf.WConf(ConfigSchema)\n\n# raises ValidationError: Value 'hi' of type 'str' could not be converted to Integer\nwconf.load_dict({\"foo\": \"hi\"})\n```\n\n### Required Values\n\nRequired parameters without default value are supported through OmegaConf's concept of\nmissing values.\n\nWhen using a dictionary schema:\n\n```python\nschema = {\n \"optional_param\": \"default value\",\n \"required_param\": \"???\",\n}\n```\n\nWhen using a dataclass schema:\n\n```python\n@dataclasses.dataclass\nclass Schema:\n required_param1: float # not providing a default makes it required\n optional_param: str = \"default value\"\n required_param2: int = omegaconf.MISSING # alternative for required parameters\n```\n\nIf there is a required parameter for which no value has been provided by any of the\n`load*`-methods, calling `get()` will raise an error.\n\nYou can avoid that error by using `get(allow_missing=True)`. However, the error is\nstill raised when trying to access the actual value of the missing parameter.\n\n\n### Variable Interpolation\n\nOmegaConf has a feature called [variable interpolation](https://omegaconf.readthedocs.io/en/latest/usage.html#variable-interpolation)\nthat allows to refer to other fields within the config file:\n\n```yaml\nserver:\n host: localhost\n port: 80\n\nclient:\n url: http://${server.host}:${server.port}/\n server_port: ${server.port}\n # relative interpolation\n description: Client of ${.url}\n```\nSee the documentation of OmegaConf for more information.\n\n\nCopyright and License\n---------------------\n\nVariConf is written and maintained by Felix Kloss at the Max Planck Institute for\nIntelligent Systems, T\u00fcbingen.\n\nCopyright (c) 2022, Max Planck Gesellschaft. All rights reserved.\n\nLicense: [BSD 3-clause](LICENSE)\n",
"bugtrack_url": null,
"license": "",
"summary": "Wrapper around OmegaConf for loading configuration from various types of files.",
"version": "1.0.1",
"split_keywords": [
"config",
"configuration"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e3015716521b719ae5b72aa9f94de3602c0cab2744f31477faafdf2d5195728b",
"md5": "9f7369b5b2d89573ee8a2520a2b60d94",
"sha256": "4ae7b484b256e2eccf72d8c8b526e61d2e714779ab3de1bec740ea8e1a503fcd"
},
"downloads": -1,
"filename": "variconf-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9f7369b5b2d89573ee8a2520a2b60d94",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 9956,
"upload_time": "2023-02-10T13:02:58",
"upload_time_iso_8601": "2023-02-10T13:02:58.922539Z",
"url": "https://files.pythonhosted.org/packages/e3/01/5716521b719ae5b72aa9f94de3602c0cab2744f31477faafdf2d5195728b/variconf-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "154ffcdeaa5e6d24c0256d5c4a941f521c36999c359d75ebe14e828e01e59c3b",
"md5": "de7e44068ddf8cc39a3ef2b66b342b5b",
"sha256": "4848da3f5335dd870b51221b68db42eb260216c1433d83a2e82ceb1401f69c45"
},
"downloads": -1,
"filename": "variconf-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "de7e44068ddf8cc39a3ef2b66b342b5b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 14555,
"upload_time": "2023-02-10T13:03:00",
"upload_time_iso_8601": "2023-02-10T13:03:00.361404Z",
"url": "https://files.pythonhosted.org/packages/15/4f/fcdeaa5e6d24c0256d5c4a941f521c36999c359d75ebe14e828e01e59c3b/variconf-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-02-10 13:03:00",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "variconf"
}
