| Name | onacol JSON |
| Version |
0.3.3
JSON |
| download |
| home_page | |
| Summary | Oh No! Another Configuration Library |
| upload_time | 2023-12-15 19:28:35 |
| maintainer | |
| docs_url | None |
| author | Josef Nevrly |
| requires_python | >=3.6,<4.0 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
|
| coveralls test coverage |
No coveralls.
|
=============================================
ONACOL (Oh No! Another COnfiguration Library)
=============================================
.. image:: https://badge.fury.io/py/onacol.svg
:target: https://badge.fury.io/py/onacol
.. image:: https://github.com/calcite/onacol/actions/workflows/test.yaml/badge.svg?branch=main
:target: https://github.com/calcite/onacol/actions/workflows/test.yaml
.. image:: https://readthedocs.org/projects/onacol/badge/?version=latest
:target: https://onacol.readthedocs.io/en/latest/?version=latest
:alt: Documentation Status
.. image:: https://coveralls.io/repos/github/calcite/onacol/badge.svg?branch=main
:target: https://coveralls.io/github/calcite/onacol?branch=main
:alt: Test coverage Status
.. image:: https://img.shields.io/lgtm/grade/python/g/calcite/onacol.svg?logo=lgtm&logoWidth=18
:target: https://lgtm.com/projects/g/calcite/onacol/context:python
:alt: Language grade: Python
.. image:: https://img.shields.io/pypi/pyversions/onacol
:alt: PyPI - Python Version
Onacol is a low-opinionated configuration management library with following
features:
* YAML (=structured and hierarchical) configuration file support
* Environment variables support (explicit and implicit)
* CLI arguments support
* Configuration merging/overwriting/layering
* Parameter validation (via Cerberus_)
* Configuration schema, documentation and default values are defined in
single YAML -> No code schema.
* Minimal dependencies
Comparison with other Python configuration libraries/frameworks
---------------------------------------------------------------
As the library name suggests, author is painfully aware this is not a unique
solution to the problem of application configuration. However, in the plethora
of existing solutions, none was completely fulfilling the features/requirements
mentioned above. So, with great reluctance,
`I had to make my own <https://xkcd.com/927/>`_.
Following table lists known/popular configuration frameworks and their
features relative to Onacol, but not comparing other features that some of those
libraries have and Onacol doesn't, so check them out - you may find it suits
your need better.
.. list-table:: Popular configuration framework comparison
:widths: 30 10 10 10 10 10 10
:header-rows: 1
* - Framework
- YAML
- ENV vars
- CLI args
- Merging
- Validation
- No code schema
* - Hydra_
- ✔️
- ✔️
- ❓
- ✔️
- ✔️
- ✖️
* - Pydantic_
- ❓
- ❓
- ✔️
- ✔️
- ✔️
- ✖️
* - Dynaconf_
- ✔️
- ❓
- ✔️
- ✔️
- ✔️
- ✖️
* - python-dotenv_
- ✖️
- ✔️
- ✖️
- ✖️
- ✖️
- ✖️
* - `Gin Config`_
- ❓
- ❓
- ❓
- ❓
- ✔️
- ✖️
* - `Python Decouple`_
- ✖️
- ✖️
- ✔️
- ✔️
- ✖️
- ✖️
* - OmegaConf_
- ✔️
- ✔️
- ✔️
- ✔️
- ✔️
- ✖️
* - Confuse_
- ✔️
- ✔️
- ❓
- ✔️
- ✔️
- ✖️
* - Everett_
- ✔️
- ✔️
- ✔️
- ❓
- ✔️
- ✖️
* - parse_it_
- ✔️
- ✔️
- ✔️
- ✔️
- ❓
- ✖️
* - Grift_
- ✖️
- ✖️
- ✖️
- ❓
- ✔️
- ✖️
* - profig_
- ✖️
- ✔️
- ✖️
- ❓
- ✔️
- ✖️
* - tweak_
- ✔️
- ✖️
- ✖️
- ✔️
- ✖️
- ✖️
* - Bison_
- ✔️
- ❓
- ✔️
- ✔️
- ✔️
- ✖️
* - Config-Man_
- ✖️
- ✔️
- ✔️
- ❓
- ✔️
- ✖️
* - figga_
- ✔️
- ✖️
- ✔️
- ❓
- ✖️
- ✖️
* - **Onacol**
- ✔️
- ✔️
- ✔️
- ✔️
- ✔️
- ✔️
Installation
------------
As usually with pip::
$ pip install onacol
Usage
-----
Default configuration file & schema
+++++++++++++++++++++++++++++++++++
The whole point of this library is the definition of both default configuration
and configuration schema in one YAML file (i.e. single source of configuration
truth).
Let's start with a simple ``default_config.yaml`` file that is part of an example
application's package. This example file contains default values for the
configuration.
.. code-block:: yaml
general:
# Logging level for this application.
log_level: INFO
ui:
# Address and port of the UI webserver
addr: 0.0.0.0
port: 8888
sensors:
sensor_reset_interval: 30.0 # Sensor reset interval in seconds
connected_units:
- id: 0 # Sensor ID <0, 16>
name: "Basic sensor"
min_trigger_limit: 30 # Minimal triggering limit [cm]
max_trigger_limit: 120 # Maximal triggering limit [cm]
- id: 1
name: "Additional sensor"
min_trigger_limit: 40
max_trigger_limit: 100
This file can be used as it is. However, we can add a schema definition to the
structure, that will allow parameter validation and automatic type conversion.
This is done by adding metadata to the YAML structure. Following metadata are
recognized by Onacol:
* ``oc_schema``: Cerberus_ validator/schema definitions.
* ``oc_default``: Default value (if metadata are attached to the YAML element, it
can no longer bear the value directly).
* ``oc_schema_id``: Definition of a schema reference (see
`Repeating schema elements`_)
Schema metadata are NOT MANDATORY. We can only provide them to parameters for
which we think validation (or type conversion) may be useful.
.. code-block:: yaml
general:
# Logging level for this application.
log_level: INFO
ui:
# Address and port of the UI webserver
addr:
oc_default: 0.0.0.0
oc_schema:
type: string
regex: "^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}$"
port:
oc_default: 8888
oc_schema:
type: integer
sensors:
sensor_reset_interval: # Sensor reset interval in seconds
oc_default: 30.0
oc_schema:
type: float
min: 0.0
max: 100.0
connected_units:
- id: # Sensor ID <0, 16>
oc_default: 0
oc_schema:
type: integer
min: 0
max: 16
name: "Basic sensor"
min_trigger_limit: # Minimal triggering limit [cm]
oc_default: 30
oc_schema:
type: integer
min: 0
max: 200
max_trigger_limit: # Maximal triggering limit [cm]
oc_default: 120
oc_schema:
type: integer
min: 0
max: 200
- id: 1
name: "Additional sensor"
min_trigger_limit: 40
max_trigger_limit: 100
Note that for list definitions, schema is added only to the first element of the
list. Other elements will be validated based on the first element's schema.
Loading and validating configuration in an application
++++++++++++++++++++++++++++++++++++++++++++++++++++++
Onacol is used by the application via the ``ConfigManager`` instance.
``ConfigManager`` can load configurations from multiple sources (files,
command line optional arguments, environment variables), but does not do it
automatically - the sources and order is up to the app implementation.
A complete minimalistic example of an application (using Click_ as a CLI
framework):
.. code-block:: python
"""Console script for onacol_test."""
import sys
import click
import pkg_resources
from onacol import ConfigManager
# Localizing the defaults/schema configuration YAML in the package
DEFAULT_CONFIG_FILE = pkg_resources.resource_filename("onacol_test",
"default_config.yaml")
# This must be here in order to retrieve args and options
# that are not Click related (see https://stackoverflow.com/a/32946412)
@click.command(context_settings=dict(
ignore_unknown_options=True,
allow_extra_args=True
))
@click.pass_context
# The rest is the usual Click stuff
@click.option("--config", type=click.Path(exists=True), default=None,
help="Path to the configuration file.")
@click.option("--get-config-template", type=click.File("w"), default=None,
help="Write default configuration template to the file.")
def main(ctx, config, get_config_template):
# Wrap optional config file into a list
user_config_file = [config] if config else []
# Instantiate config_manager
config_manager = ConfigManager(DEFAULT_CONFIG_FILE,
env_var_prefix="OCTEST",
optional_files=user_config_file
)
# Generate configuration for the --get-config-template option
# Then finish the application
if get_config_template:
config_manager.generate_config_example(get_config_template)
return
# Load (implicit) environment variables
config_manager.config_from_env_vars()
# Parse all extra command line options
config_manager.config_from_cli_args(ctx.args)
# Validate the config
config_manager.validate()
# Finally, let's review interesting bits of the config
print("---------<Application configuration>-------------")
print(f"Log level: {config_manager.config['general']['log_level']}")
print(f"UI: {config_manager.config['ui']['addr']} "
f"(port: {config_manager.config['ui']['port']})")
print(f"Sensor reset interval: "
f"{config_manager.config['sensors']['sensor_reset_interval']}")
print(f"Sensors:")
for sensor in config_manager.config["sensors"]["connected_units"]:
print(f"\t {sensor['name']} [{sensor['id']}] \t Trigger limits: "
f"({sensor['min_trigger_limit']}, {sensor['max_trigger_limit']})")
if __name__ == "__main__":
sys.exit(main()) # pragma: no cover
In this example, the application is bundling the ``default_config.yaml`` from
the examples above as the default configuration/schema file.
Then it accepts additional configuration file via command
option, and on the top it uses the environment variables and additional
configuration via command line options. Configuration from all sources
are layered/overwritten on the top of the current configuration.
As you can see in the code, the sources of configuration as well as their
prioritization depend on the order of which ``ConfigManager`` methods are
called, there is no default and even the validation must be called explicitly.
Configuration using additional file
+++++++++++++++++++++++++++++++++++
In the example app, additional config file are loaded with the ``--config``
optional command line argument, that is used in the ``ConfigManager``'s
``optional_files`` init option. There is also the ``ConfigManager.config_from_file``
method to do this anytime after init.
Let's use the following config file (``my_config.yaml``):
.. code-block:: yaml
general:
log_level: DEBUG
ui:
port: 127.0.0.1
And load it with the app::
$ python main.py --config my_config.yaml
---------<Application configuration>-------------
Log level: DEBUG
UI: 127.0.0.1 (port: 8888)
Sensor reset interval: 30.0
Sensors:
Basic sensor [0] Trigger limits: (30, 120)
Additional sensor [1] Trigger limits: (40, 100)
As you can see, the relevant default config parameters have been overwritten,
the others stay default. This layering works over configuration dicts of
unlimited depth, but does not work with lists (by design).
Configuration using environment variables
+++++++++++++++++++++++++++++++++++++++++
There are two ways how to use environment variables with Onacol:
* **Implicit way** - Onacol detects environment variables with defined prefix
and use them to overwrite current configuration.
* **Explicit way** - environment variables are referenced in the configuration
files and Onacol resolves the references upon loading the file.
Using environment variables implicitly
**************************************
In the example app source, we defined the ``env_var_prefix`` with value
``OCTEST``. Using the ``ConfigManager.config_from_env_vars`` method will then
make Onacol parse existing environment variables for names
starting with the chosen prefix, and then use the rest of the name as path for
the configuration structure (using uppercase and ``__`` as the level separator).
Let's continue with the previous example::
$ export OCTEST_SENSORS__SENSOR_RESET_INTERVAL=20.1
$ python main.py --config my_config.yaml
Log level: DEBUG
UI: 127.0.0.1 (port: 8888)
Sensor reset interval: 20.1
Sensors:
Basic sensor[0] Trigger limits: (30, 120)
Additional sensor[1] Trigger limits: (40, 100)
Again, environment variable overwrites the original value. Environment variable
values are always strings. However, as we defined schema and type for the
configuration parameter ``sensor_reset_interval``, it was automatically
converted to integer. Although schema is not mandatory, it's always useful for
parameters that can be configured via environment variables.
When schema is not defined, Onacol tries to apply JSON conversion rules to
the value of the environment variable. That helps in most cases, but can
cause problems if you pass value such as "1.2". In that case, it will be
automatically converted to float. If you want to receive it as string, you
must define schema for that particular config.
It is also possible to overwrite entire lists with environment variables.
To do that, use again JSON as format::
$ export OCTEST_SENSORS__CONNECTED_UNITS='[{"id": 2, "name": "JSON sensor", "min_trigger_limit": 10, "max_trigger_limit": 90}]'
$ python main.py --config my_config.yaml
---------<Application configuration>-------------
Log level: DEBUG
UI: 127.0.0.1 (port: 8888)
Sensor reset interval: 30.0
Sensors:
JSON sensor [2] Trigger limits: (10, 90)
As explained above, lists are always overwritten completely, no layering.
It is not possible to use JSON to overwrite dicts in the configuration
structure.
Using environment variables explicitly
**************************************
Environment variables can be also explicitly referred in the configuration YAML
file with syntax ``${oc_env:ENV_VAR}``:
.. code-block:: yaml
general:
log_level: DEBUG
ui:
addr: ${oc_env:MY_ADDR}
This reference is being resolved before the YAML is parsed (it's a primitive
regex substitution). Therefore the YAML type conversion is used for non-string
values. Explicit environment variable references can be only used in file-type
configuration sources. Example::
$ export MY_ADDR=192.168.1.10
$ python main.py --config my_config.yaml
---------<Application configuration>-------------
Log level: DEBUG
UI: 192.168.1.10 (port: 8888)
Sensor reset interval: 30.0
Sensors:
Basic sensor [0] Trigger limits: (30, 120)
Additional sensor [1] Trigger limits: (40, 100)
In explicitly used environment variables, where schema is not defined, then
of course YAML default conversion rules are used.
Configuration using command-line options
++++++++++++++++++++++++++++++++++++++++
Command-line optional arguments can be also parsed by Onacol to retrieve
configuration parameters. The logic is very similar to the implicit usage of
environment variables, but no prefix is used and the level separator is ``--``::
$ python main.py --config my_config.yaml --ui--port 8080 --sensors--sensor-reset-interval 15.8
---------<Application configuration>-------------
Log level: DEBUG
UI: 127.0.0.1 (port: 8080)
Sensor reset interval: 15.8
Sensors:
Basic sensor [0] Trigger limits: (30, 120)
Additional sensor [1] Trigger limits: (40, 100)
As with implicit environment variable, config parameters with defined schema get
automatically converted to their types. It's also allowed to use JSON lists.
Generation of an example/template config file
+++++++++++++++++++++++++++++++++++++++++++++
Default configuration/schema can be used to generate an example (template)
config file with ``ConfigManager.generate_config_example`` method. This file
has the schema information stripped, but retains the comments used in the
defaults YAML file.
The example app has the `--get-config-template` option to demonstrate it::
$ python main.py --get-config-template config_template.yaml
will generate following `config_template.yaml` file:
.. code-block:: yaml
general:
# Logging level for this application.
log_level: INFO
ui:
# Address and port of the UI webserver
addr: 0.0.0.0
port: 8888
sensors:
sensor_reset_interval: 30.0 # Sensor reset interval in seconds
connected_units:
- id: 0 # Sensor ID <0, 16>
name: Basic sensor
min_trigger_limit: 30 # Minimal triggering limit [cm]
max_trigger_limit: 120 # Maximal triggering limit [cm]
- id: 1
name: Additional sensor
min_trigger_limit: 40
max_trigger_limit: 100
The comments are retained by the magic of `Ruamel YAML`_, and there are some
limits. For proper retaining of comments, try to put the comments at the end
of line and avoid above-line comments where the preceding element is a schema
element.
Exporting current configuration to a config file
++++++++++++++++++++++++++++++++++++++++++++++++
The current state of the configuration can be dumped to a file using
the ``ConfigManager.export_current_config`` method.
Repeating schema elements
+++++++++++++++++++++++++
In case the configuration schema has repeating elements, it's possible to define
schema for just one element, declare a reference for it with ``oc_schema_id``
and then refer other elements to that schema definition directly with
``oc_schema``:
.. code-block:: yaml
network_interfaces:
ethernet_interface:
name: # Element name
oc_default: "eth0"
oc_schema:
type: string
id:
oc_default: 0
oc_schema:
type: integer
ip_addr:
oc_default: 192.168.1.2
oc_schema:
type: string
regex: "^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}$"
# Here we declare re-usable schema
oc_schema_id: network_interface_item
wifi_interface:
name: wifi
id: 1
ip_addr: 192.168.2.3
oc_schema: network_interface_item # Here we reference the previously declared schema:
Configuration layering
++++++++++++++++++++++
When default or current configuration gets overwritten with new config values,
the previous values are kept internally and can be accessed. This is done using
the cascading features of CascaDict_ (the configuration structure is kept in
``ConfigManager.config`` as ``CascaDict`` instance).
If you are not interested in this, just use it as if it was a regular ``dict``.
Other notes
+++++++++++
* For any sort of configuration with variable amount of elements, use lists,
not dicts. Onacol is written on assumption that the configuration tree
consists of more-or-less fixed dicts and variable length lists.
* To create a default config/schema that shall enforce the end user to overwrite
some parameters, use ``null`` as the default value and use schema with
``nullable: false`` - see `Cerberos docs <https://docs.python-cerberus.org/en/stable/validation-rules.html#nullable>`_.
Validation will then report error when this value is not overwritten.
License
-------
Free software: MIT license
Documentation
-------------
Full docs at https://onacol.readthedocs.io.
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
.. _Cerberus: https://docs.python-cerberus.org/en/stable/
.. _Hydra: https://hydra.cc/
.. _Config-Man: https://github.com/mmohaveri/config-man
.. _Dynaconf: https://github.com/rochacbruno/dynaconf
.. _Pydantic: https://pydantic-docs.helpmanual.io/
.. _python-dotenv: https://github.com/theskumar/python-dotenv
.. _`Gin Config`: https://github.com/google/gin-config
.. _OmegaConf: https://github.com/omry/omegaconf
.. _Confuse: https://github.com/beetbox/confuse
.. _`Python Decouple`: https://github.com/henriquebastos/python-decouple
.. _parse_it: https://github.com/naorlivne/parse_it
.. _grift: https://github.com/kensho-technologies/grift
.. _profig: https://github.com/dhagrow/profig
.. _tweak: https://github.com/kislyuk/tweak
.. _Bison: https://github.com/edaniszewski/bison
.. _figga: https://github.com/berislavlopac/figga
.. _Click: https://click.palletsprojects.com
.. _CascaDict: https://github.com/JNevrly/cascadict
.. _`Ruamel YAML`: https://yaml.readthedocs.io/en/latest/
.. _Everett: https://github.com/willkg/everett
Raw data
{
"_id": null,
"home_page": "",
"name": "onacol",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Josef Nevrly",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/ef/91/68c10e8613445b251c5de5b6b510cfcb7ac925744845d3c083996b5aee72/onacol-0.3.3.tar.gz",
"platform": null,
"description": "=============================================\nONACOL (Oh No! Another COnfiguration Library)\n=============================================\n\n.. image:: https://badge.fury.io/py/onacol.svg\n :target: https://badge.fury.io/py/onacol\n\n.. image:: https://github.com/calcite/onacol/actions/workflows/test.yaml/badge.svg?branch=main\n :target: https://github.com/calcite/onacol/actions/workflows/test.yaml\n\n.. image:: https://readthedocs.org/projects/onacol/badge/?version=latest\n :target: https://onacol.readthedocs.io/en/latest/?version=latest\n :alt: Documentation Status\n\n.. image:: https://coveralls.io/repos/github/calcite/onacol/badge.svg?branch=main\n :target: https://coveralls.io/github/calcite/onacol?branch=main\n :alt: Test coverage Status\n\n.. image:: https://img.shields.io/lgtm/grade/python/g/calcite/onacol.svg?logo=lgtm&logoWidth=18\n :target: https://lgtm.com/projects/g/calcite/onacol/context:python\n :alt: Language grade: Python\n\n.. image:: https://img.shields.io/pypi/pyversions/onacol\n :alt: PyPI - Python Version\n\nOnacol is a low-opinionated configuration management library with following\nfeatures:\n\n* YAML (=structured and hierarchical) configuration file support\n* Environment variables support (explicit and implicit)\n* CLI arguments support\n* Configuration merging/overwriting/layering\n* Parameter validation (via Cerberus_)\n* Configuration schema, documentation and default values are defined in\n single YAML -> No code schema.\n* Minimal dependencies\n\nComparison with other Python configuration libraries/frameworks\n---------------------------------------------------------------\n\nAs the library name suggests, author is painfully aware this is not a unique\nsolution to the problem of application configuration. However, in the plethora\nof existing solutions, none was completely fulfilling the features/requirements\nmentioned above. So, with great reluctance,\n`I had to make my own <https://xkcd.com/927/>`_.\n\nFollowing table lists known/popular configuration frameworks and their\nfeatures relative to Onacol, but not comparing other features that some of those\nlibraries have and Onacol doesn't, so check them out - you may find it suits\nyour need better.\n\n\n.. list-table:: Popular configuration framework comparison\n :widths: 30 10 10 10 10 10 10\n :header-rows: 1\n\n * - Framework\n - YAML\n - ENV vars\n - CLI args\n - Merging\n - Validation\n - No code schema\n * - Hydra_\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - Pydantic_\n - \u2753\n - \u2753\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - Dynaconf_\n - \u2714\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - python-dotenv_\n - \u2716\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n * - `Gin Config`_\n - \u2753\n - \u2753\n - \u2753\n - \u2753\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - `Python Decouple`_\n - \u2716\ufe0f\n - \u2716\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n * - OmegaConf_\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - Confuse_\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - Everett_\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - parse_it_\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2753\n - \u2716\ufe0f\n * - Grift_\n - \u2716\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - profig_\n - \u2716\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - tweak_\n - \u2714\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n - \u2716\ufe0f\n * - Bison_\n - \u2714\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - Config-Man_\n - \u2716\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2753\n - \u2714\ufe0f\n - \u2716\ufe0f\n * - figga_\n - \u2714\ufe0f\n - \u2716\ufe0f\n - \u2714\ufe0f\n - \u2753\n - \u2716\ufe0f\n - \u2716\ufe0f\n * - **Onacol**\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n - \u2714\ufe0f\n\nInstallation\n------------\n\nAs usually with pip::\n\n $ pip install onacol\n\nUsage\n-----\n\nDefault configuration file & schema\n+++++++++++++++++++++++++++++++++++\n\nThe whole point of this library is the definition of both default configuration\nand configuration schema in one YAML file (i.e. single source of configuration\ntruth).\n\nLet's start with a simple ``default_config.yaml`` file that is part of an example\napplication's package. This example file contains default values for the\nconfiguration.\n\n.. code-block:: yaml\n\n general:\n # Logging level for this application.\n log_level: INFO\n\n ui:\n # Address and port of the UI webserver\n addr: 0.0.0.0\n port: 8888\n\n sensors:\n sensor_reset_interval: 30.0 # Sensor reset interval in seconds\n connected_units:\n - id: 0 # Sensor ID <0, 16>\n name: \"Basic sensor\"\n min_trigger_limit: 30 # Minimal triggering limit [cm]\n max_trigger_limit: 120 # Maximal triggering limit [cm]\n - id: 1\n name: \"Additional sensor\"\n min_trigger_limit: 40\n max_trigger_limit: 100\n\nThis file can be used as it is. However, we can add a schema definition to the\nstructure, that will allow parameter validation and automatic type conversion.\n\nThis is done by adding metadata to the YAML structure. Following metadata are\nrecognized by Onacol:\n\n* ``oc_schema``: Cerberus_ validator/schema definitions.\n* ``oc_default``: Default value (if metadata are attached to the YAML element, it\n can no longer bear the value directly).\n* ``oc_schema_id``: Definition of a schema reference (see\n `Repeating schema elements`_)\n\nSchema metadata are NOT MANDATORY. We can only provide them to parameters for\nwhich we think validation (or type conversion) may be useful.\n\n.. code-block:: yaml\n\n general:\n # Logging level for this application.\n log_level: INFO\n\n ui:\n # Address and port of the UI webserver\n addr:\n oc_default: 0.0.0.0\n oc_schema:\n type: string\n regex: \"^(?:[0-9]{1,3}\\\\.){3}[0-9]{1,3}$\"\n\n port:\n oc_default: 8888\n oc_schema:\n type: integer\n\n sensors:\n sensor_reset_interval: # Sensor reset interval in seconds\n oc_default: 30.0\n oc_schema:\n type: float\n min: 0.0\n max: 100.0\n connected_units:\n - id: # Sensor ID <0, 16>\n oc_default: 0\n oc_schema:\n type: integer\n min: 0\n max: 16\n name: \"Basic sensor\"\n min_trigger_limit: # Minimal triggering limit [cm]\n oc_default: 30\n oc_schema:\n type: integer\n min: 0\n max: 200\n max_trigger_limit: # Maximal triggering limit [cm]\n oc_default: 120\n oc_schema:\n type: integer\n min: 0\n max: 200\n - id: 1\n name: \"Additional sensor\"\n min_trigger_limit: 40\n max_trigger_limit: 100\n\nNote that for list definitions, schema is added only to the first element of the\nlist. Other elements will be validated based on the first element's schema.\n\n\nLoading and validating configuration in an application\n++++++++++++++++++++++++++++++++++++++++++++++++++++++\n\nOnacol is used by the application via the ``ConfigManager`` instance.\n``ConfigManager`` can load configurations from multiple sources (files,\ncommand line optional arguments, environment variables), but does not do it\nautomatically - the sources and order is up to the app implementation.\n\nA complete minimalistic example of an application (using Click_ as a CLI\nframework):\n\n.. code-block:: python\n\n \"\"\"Console script for onacol_test.\"\"\"\n import sys\n import click\n import pkg_resources\n\n from onacol import ConfigManager\n\n # Localizing the defaults/schema configuration YAML in the package\n DEFAULT_CONFIG_FILE = pkg_resources.resource_filename(\"onacol_test\",\n \"default_config.yaml\")\n\n # This must be here in order to retrieve args and options\n # that are not Click related (see https://stackoverflow.com/a/32946412)\n @click.command(context_settings=dict(\n ignore_unknown_options=True,\n allow_extra_args=True\n ))\n @click.pass_context\n # The rest is the usual Click stuff\n @click.option(\"--config\", type=click.Path(exists=True), default=None,\n help=\"Path to the configuration file.\")\n @click.option(\"--get-config-template\", type=click.File(\"w\"), default=None,\n help=\"Write default configuration template to the file.\")\n def main(ctx, config, get_config_template):\n # Wrap optional config file into a list\n user_config_file = [config] if config else []\n\n # Instantiate config_manager\n config_manager = ConfigManager(DEFAULT_CONFIG_FILE,\n env_var_prefix=\"OCTEST\",\n optional_files=user_config_file\n )\n\n # Generate configuration for the --get-config-template option\n # Then finish the application\n if get_config_template:\n config_manager.generate_config_example(get_config_template)\n return\n\n # Load (implicit) environment variables\n config_manager.config_from_env_vars()\n\n # Parse all extra command line options\n config_manager.config_from_cli_args(ctx.args)\n\n # Validate the config\n config_manager.validate()\n\n # Finally, let's review interesting bits of the config\n print(\"---------<Application configuration>-------------\")\n print(f\"Log level: {config_manager.config['general']['log_level']}\")\n print(f\"UI: {config_manager.config['ui']['addr']} \"\n f\"(port: {config_manager.config['ui']['port']})\")\n print(f\"Sensor reset interval: \"\n f\"{config_manager.config['sensors']['sensor_reset_interval']}\")\n print(f\"Sensors:\")\n for sensor in config_manager.config[\"sensors\"][\"connected_units\"]:\n print(f\"\\t {sensor['name']} [{sensor['id']}] \\t Trigger limits: \"\n f\"({sensor['min_trigger_limit']}, {sensor['max_trigger_limit']})\")\n\n\n if __name__ == \"__main__\":\n sys.exit(main()) # pragma: no cover\n\nIn this example, the application is bundling the ``default_config.yaml`` from\nthe examples above as the default configuration/schema file.\nThen it accepts additional configuration file via command\noption, and on the top it uses the environment variables and additional\nconfiguration via command line options. Configuration from all sources\nare layered/overwritten on the top of the current configuration.\n\nAs you can see in the code, the sources of configuration as well as their\nprioritization depend on the order of which ``ConfigManager`` methods are\ncalled, there is no default and even the validation must be called explicitly.\n\nConfiguration using additional file\n+++++++++++++++++++++++++++++++++++\n\nIn the example app, additional config file are loaded with the ``--config``\noptional command line argument, that is used in the ``ConfigManager``'s\n``optional_files`` init option. There is also the ``ConfigManager.config_from_file``\nmethod to do this anytime after init.\n\nLet's use the following config file (``my_config.yaml``):\n\n.. code-block:: yaml\n\n general:\n log_level: DEBUG\n\n ui:\n port: 127.0.0.1\n\nAnd load it with the app::\n\n $ python main.py --config my_config.yaml\n ---------<Application configuration>-------------\n Log level: DEBUG\n UI: 127.0.0.1 (port: 8888)\n Sensor reset interval: 30.0\n Sensors:\n Basic sensor [0] Trigger limits: (30, 120)\n Additional sensor [1] Trigger limits: (40, 100)\n\nAs you can see, the relevant default config parameters have been overwritten,\nthe others stay default. This layering works over configuration dicts of\nunlimited depth, but does not work with lists (by design).\n\nConfiguration using environment variables\n+++++++++++++++++++++++++++++++++++++++++\n\nThere are two ways how to use environment variables with Onacol:\n\n* **Implicit way** - Onacol detects environment variables with defined prefix\n and use them to overwrite current configuration.\n* **Explicit way** - environment variables are referenced in the configuration\n files and Onacol resolves the references upon loading the file.\n\nUsing environment variables implicitly\n**************************************\n\nIn the example app source, we defined the ``env_var_prefix`` with value\n``OCTEST``. Using the ``ConfigManager.config_from_env_vars`` method will then\nmake Onacol parse existing environment variables for names\nstarting with the chosen prefix, and then use the rest of the name as path for\nthe configuration structure (using uppercase and ``__`` as the level separator).\n\nLet's continue with the previous example::\n\n $ export OCTEST_SENSORS__SENSOR_RESET_INTERVAL=20.1\n $ python main.py --config my_config.yaml\n Log level: DEBUG\n UI: 127.0.0.1 (port: 8888)\n Sensor reset interval: 20.1\n Sensors:\n Basic sensor[0] Trigger limits: (30, 120)\n Additional sensor[1] Trigger limits: (40, 100)\n\nAgain, environment variable overwrites the original value. Environment variable\nvalues are always strings. However, as we defined schema and type for the\nconfiguration parameter ``sensor_reset_interval``, it was automatically\nconverted to integer. Although schema is not mandatory, it's always useful for\nparameters that can be configured via environment variables.\n\nWhen schema is not defined, Onacol tries to apply JSON conversion rules to\nthe value of the environment variable. That helps in most cases, but can\ncause problems if you pass value such as \"1.2\". In that case, it will be\nautomatically converted to float. If you want to receive it as string, you\nmust define schema for that particular config.\n\nIt is also possible to overwrite entire lists with environment variables.\nTo do that, use again JSON as format::\n\n $ export OCTEST_SENSORS__CONNECTED_UNITS='[{\"id\": 2, \"name\": \"JSON sensor\", \"min_trigger_limit\": 10, \"max_trigger_limit\": 90}]'\n $ python main.py --config my_config.yaml\n ---------<Application configuration>-------------\n Log level: DEBUG\n UI: 127.0.0.1 (port: 8888)\n Sensor reset interval: 30.0\n Sensors:\n JSON sensor [2] Trigger limits: (10, 90)\n\nAs explained above, lists are always overwritten completely, no layering.\nIt is not possible to use JSON to overwrite dicts in the configuration\nstructure.\n\nUsing environment variables explicitly\n**************************************\n\nEnvironment variables can be also explicitly referred in the configuration YAML\nfile with syntax ``${oc_env:ENV_VAR}``:\n\n.. code-block:: yaml\n\n general:\n log_level: DEBUG\n\n ui:\n addr: ${oc_env:MY_ADDR}\n\nThis reference is being resolved before the YAML is parsed (it's a primitive\nregex substitution). Therefore the YAML type conversion is used for non-string\nvalues. Explicit environment variable references can be only used in file-type\nconfiguration sources. Example::\n\n $ export MY_ADDR=192.168.1.10\n $ python main.py --config my_config.yaml\n ---------<Application configuration>-------------\n Log level: DEBUG\n UI: 192.168.1.10 (port: 8888)\n Sensor reset interval: 30.0\n Sensors:\n Basic sensor [0] Trigger limits: (30, 120)\n Additional sensor [1] Trigger limits: (40, 100)\n\nIn explicitly used environment variables, where schema is not defined, then\nof course YAML default conversion rules are used.\n\nConfiguration using command-line options\n++++++++++++++++++++++++++++++++++++++++\n\nCommand-line optional arguments can be also parsed by Onacol to retrieve\nconfiguration parameters. The logic is very similar to the implicit usage of\nenvironment variables, but no prefix is used and the level separator is ``--``::\n\n $ python main.py --config my_config.yaml --ui--port 8080 --sensors--sensor-reset-interval 15.8\n ---------<Application configuration>-------------\n Log level: DEBUG\n UI: 127.0.0.1 (port: 8080)\n Sensor reset interval: 15.8\n Sensors:\n Basic sensor [0] Trigger limits: (30, 120)\n Additional sensor [1] Trigger limits: (40, 100)\n\nAs with implicit environment variable, config parameters with defined schema get\nautomatically converted to their types. It's also allowed to use JSON lists.\n\nGeneration of an example/template config file\n+++++++++++++++++++++++++++++++++++++++++++++\n\nDefault configuration/schema can be used to generate an example (template)\nconfig file with ``ConfigManager.generate_config_example`` method. This file\nhas the schema information stripped, but retains the comments used in the\ndefaults YAML file.\n\nThe example app has the `--get-config-template` option to demonstrate it::\n\n $ python main.py --get-config-template config_template.yaml\n\nwill generate following `config_template.yaml` file:\n\n.. code-block:: yaml\n\n general:\n # Logging level for this application.\n log_level: INFO\n\n ui:\n # Address and port of the UI webserver\n addr: 0.0.0.0\n port: 8888\n sensors:\n sensor_reset_interval: 30.0 # Sensor reset interval in seconds\n connected_units:\n - id: 0 # Sensor ID <0, 16>\n name: Basic sensor\n min_trigger_limit: 30 # Minimal triggering limit [cm]\n max_trigger_limit: 120 # Maximal triggering limit [cm]\n - id: 1\n name: Additional sensor\n min_trigger_limit: 40\n max_trigger_limit: 100\n\nThe comments are retained by the magic of `Ruamel YAML`_, and there are some\nlimits. For proper retaining of comments, try to put the comments at the end\nof line and avoid above-line comments where the preceding element is a schema\nelement.\n\nExporting current configuration to a config file\n++++++++++++++++++++++++++++++++++++++++++++++++\n\nThe current state of the configuration can be dumped to a file using\nthe ``ConfigManager.export_current_config`` method.\n\nRepeating schema elements\n+++++++++++++++++++++++++\n\nIn case the configuration schema has repeating elements, it's possible to define\nschema for just one element, declare a reference for it with ``oc_schema_id``\nand then refer other elements to that schema definition directly with\n``oc_schema``:\n\n.. code-block:: yaml\n\n network_interfaces:\n ethernet_interface:\n name: # Element name\n oc_default: \"eth0\"\n oc_schema:\n type: string\n id:\n oc_default: 0\n oc_schema:\n type: integer\n ip_addr:\n oc_default: 192.168.1.2\n oc_schema:\n type: string\n regex: \"^(?:[0-9]{1,3}\\\\.){3}[0-9]{1,3}$\"\n\n # Here we declare re-usable schema\n oc_schema_id: network_interface_item\n wifi_interface:\n name: wifi\n id: 1\n ip_addr: 192.168.2.3\n oc_schema: network_interface_item # Here we reference the previously declared schema:\n\nConfiguration layering\n++++++++++++++++++++++\n\nWhen default or current configuration gets overwritten with new config values,\nthe previous values are kept internally and can be accessed. This is done using\nthe cascading features of CascaDict_ (the configuration structure is kept in\n``ConfigManager.config`` as ``CascaDict`` instance).\n\nIf you are not interested in this, just use it as if it was a regular ``dict``.\n\nOther notes\n+++++++++++\n\n* For any sort of configuration with variable amount of elements, use lists,\n not dicts. Onacol is written on assumption that the configuration tree\n consists of more-or-less fixed dicts and variable length lists.\n* To create a default config/schema that shall enforce the end user to overwrite\n some parameters, use ``null`` as the default value and use schema with\n ``nullable: false`` - see `Cerberos docs <https://docs.python-cerberus.org/en/stable/validation-rules.html#nullable>`_.\n Validation will then report error when this value is not overwritten.\n\nLicense\n-------\nFree software: MIT license\n\nDocumentation\n-------------\n\nFull docs at https://onacol.readthedocs.io.\n\n.. _Cookiecutter: https://github.com/audreyr/cookiecutter\n.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage\n.. _Cerberus: https://docs.python-cerberus.org/en/stable/\n.. _Hydra: https://hydra.cc/\n.. _Config-Man: https://github.com/mmohaveri/config-man\n.. _Dynaconf: https://github.com/rochacbruno/dynaconf\n.. _Pydantic: https://pydantic-docs.helpmanual.io/\n.. _python-dotenv: https://github.com/theskumar/python-dotenv\n.. _`Gin Config`: https://github.com/google/gin-config\n.. _OmegaConf: https://github.com/omry/omegaconf\n.. _Confuse: https://github.com/beetbox/confuse\n.. _`Python Decouple`: https://github.com/henriquebastos/python-decouple\n.. _parse_it: https://github.com/naorlivne/parse_it\n.. _grift: https://github.com/kensho-technologies/grift\n.. _profig: https://github.com/dhagrow/profig\n.. _tweak: https://github.com/kislyuk/tweak\n.. _Bison: https://github.com/edaniszewski/bison\n.. _figga: https://github.com/berislavlopac/figga\n.. _Click: https://click.palletsprojects.com\n.. _CascaDict: https://github.com/JNevrly/cascadict\n.. _`Ruamel YAML`: https://yaml.readthedocs.io/en/latest/\n.. _Everett: https://github.com/willkg/everett\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Oh No! Another Configuration Library",
"version": "0.3.3",
"project_urls": {
"documentation": "https://onacol.readthedocs.io/",
"homepage": "https://github.com/calcite/onacol"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a43ade50ebcfdeb94467971d3e4640007967cfd5f9bd5847e26a258b48b6514b",
"md5": "9751f239eee123580473015411ae6e00",
"sha256": "eff5073b8588510ef124cf9d06bd9099db744a90c48b7ba13d670e051d0c74ad"
},
"downloads": -1,
"filename": "onacol-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9751f239eee123580473015411ae6e00",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6,<4.0",
"size": 17768,
"upload_time": "2023-12-15T19:28:33",
"upload_time_iso_8601": "2023-12-15T19:28:33.879064Z",
"url": "https://files.pythonhosted.org/packages/a4/3a/de50ebcfdeb94467971d3e4640007967cfd5f9bd5847e26a258b48b6514b/onacol-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ef9168c10e8613445b251c5de5b6b510cfcb7ac925744845d3c083996b5aee72",
"md5": "95ca9d1b1cc8c0161a2836622561894f",
"sha256": "f6c4482669f56d6080ef30b49abb5fbc39d29f6500569e4712872500d5c33376"
},
"downloads": -1,
"filename": "onacol-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "95ca9d1b1cc8c0161a2836622561894f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6,<4.0",
"size": 21208,
"upload_time": "2023-12-15T19:28:35",
"upload_time_iso_8601": "2023-12-15T19:28:35.710357Z",
"url": "https://files.pythonhosted.org/packages/ef/91/68c10e8613445b251c5de5b6b510cfcb7ac925744845d3c083996b5aee72/onacol-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-15 19:28:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "calcite",
"github_project": "onacol",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "onacol"
}
