| Name | simputils-config JSON |
| Version |
1.1.1
JSON |
| download |
| home_page | None |
| Summary | Simple Configs Manager |
| upload_time | 2024-10-02 05:15:45 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | MIT License Copyright (c) 2024 Ivan Ponomarev Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
simputils
config
files
utils
framework
simple
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
pyyaml
python-magic
python-dotenv
pytest
pytest-cov
flake8
pydantic
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Python SimpUtils Config
Simplifies working with configs and params.
## Installation
This will install the latest version of the major version `1`.
It's safe enough due to followed by the project Semantic Versioning paradigm.
```shell
pip install "simputils-config~=1.0"
```
> [!WARNING]
> Developer of this project has nothing to do with `simputils` package of pypi.
> And installation of both might cause broken code.
>
> On the end of this project, the namespace `simputils` is made "shareable".
> But the structure of another developer's package is not designed in such way
>
> Disclaimer about that you can find here [Potential package collision 2024](docs/disclaimers.md)
## Description
Class `simputils.config.models.ConfigStore` is the keystone of the library.
Object of it represents config, that could be used to sequentially apply different sets of key-value
pairs from different sources (other dicts, files of different kinds, environment variables, etc.).
Object will store the latest values of the applied sets,
where the earliest applied values could be overriden by the following ones.
Basically, just a dictionary.
But besides being like a dictionary, it records the "history" of applied configs, so it's easier to
debug and control from where which value came from.
The major purpose of the functionality comes from using multiple files as sources for your config,
but it still allows to apply sets directly from the code.
Project follows [Semantic Versioning](https://semver.org/) standard.
No breaking changes suppose to be within `minor` and `patch` versions of the same `major` version.
Exception is only if there is a bug, that gets fixed.
> [!CAUTION]
> `ConfigStore` object is behaving like a `dict`, so if you need to check if
> the variable with this object is None, always check it like `if conf is None:`,
> and never check it like `if not conf:`!
>
> The check is implicit, so when you use `not` or simple boolean check, it will check if the object
> is empty (does not contain any value)
> [!NOTE]
> To check if `ConfigStore` object contains at least one key-value pair,
> you can use simple `if conf:`
----
When working with files, keep in mind that the only supported files are `.yml`/`.yaml`, `.env` and `.json`.
If you need support for other types, you will have to implement your custom handler(s) for those file-types.
## Documentation
* [Changelog](docs/CHANGELOG.md) - Please make sure you check it for new features and changes
* [The overall example](docs/overall-example.md)
* [Working with enums and annotations](docs/working-with-enums-and-annotations.md)
* [Config Object Style Access](docs/config-object-style-access.md)
* [Preprocessing and filtering](docs/preprocessing-and-filtering.md)
* [Working with `ConfigHub`](docs/working-with-config-hub.md)
(Quick Start, recommended way to work with the configs)
* [Working with `ConfigStore`](docs/working-with-config-store.md)
* [Config Merging Strategies](docs/config-merging-strategies.md)
* [Some typical examples](examples)
### Config Modifiers
There are 3 config modifiers that could be applied: `preprocessor`, `filter` and annotation type-casting
> [!INFO]
> The order of config modifiers applied to each config:
> 1. `preprocessor`
> 2. `filter`
> 3. Annotation type-casting
## Generic examples
### The simplest usage
Aggregation from multiple sources (you can specify any number of sources).
> [!IMPORTANT]
> Keep in mind that order matters, keys/values from the latest might override already specified
From files:
```python
from simputils.config.components import ConfigHub
conf = ConfigHub.aggregate(
"config-1.yml",
"config-2.yml",
# ...
)
print(conf, conf.applied_confs)
```
From dictionaries:
```python
from simputils.config.components import ConfigHub
conf = ConfigHub.aggregate(
{"key1": "val1", "key2": "val2"},
{"key2": "redefined val2", "key3": "val3"},
# ...
)
print(conf, conf.applied_confs)
```
Accessing values:
```python
# Accessing values by name, if does not exist - None is returned
conf["key1"]
conf["key2"]
# Accessing values by name, if does not exist or None - `default` parameter value is returned
conf.get("key1", "My Default Value")
conf.get("key2")
# Iterating as a dictionary
for key, val in conf.items():
print(key, val)
```
### Enums as default config with filter
```python
import os
from typing import Annotated
from simputils.config.base import simputils_pp
from simputils.config.components import ConfigHub
from simputils.config.generic import BasicConfigEnum
from simputils.config.models import ConfigStore, AnnotatedConfigData
class MyEnum(BasicConfigEnum):
# Annotated without and with default values set
MY_E_KEY_1: Annotated[str, AnnotatedConfigData()] = "my-e-key-1"
MY_E_KEY_2: Annotated[str, AnnotatedConfigData(
default=3.1415
)] = "my-e-key-2"
# Non-annotated, so they will be None by default
MY_E_KEY_3 = "my-e-key-3"
MY_E_KEY_4 = "my-e-key-4"
MY_E_KEY_5 = "my-e-key-5"
# Some of them used in `app-conf.yml`
MY_FIRST_VAL = "val-1"
MY_SECOND_VAL = "VAL_2"
# Will be taken from os.environ,
# all other os.environ values will be excluded
ENV_USER_NAME = "USER"
conf = ConfigHub.aggregate(
"tests/data/config-1.yml",
os.environ,
target=ConfigStore(
MyEnum.defaults(),
preprocessor=simputils_pp,
filter=True
),
)
print("conf: ", conf)
```
```text
conf: {
'MY_E_KEY_1': None,
'MY_E_KEY_2': 3.1415,
'MY_E_KEY_3': None,
'MY_E_KEY_4': None,
'MY_E_KEY_5': None,
'VAL_1': 'My conf value 1',
'VAL_2': 'My conf value 2',
'USER': 'ivan'
}
```
### Enums and argparser support
`Enum` keys are supported out of the box, and `argparse.Namespace` could be used for `ConfigStore`
> [!NOTE]
> `BasicConfigEnum` is used for convenience.
> And it's suggested way, it allows to use `True` as a filter key
>
> You still can use Enums without that class,
> just make sure that enum is inherited from `str, Enum` in that order!
```python
from argparse import ArgumentParser
from simputils.config.base import simputils_pp
from simputils.config.components import ConfigHub
from simputils.config.models import ConfigStore
from simputils.config.generic import BasicConfigEnum
args_parser = ArgumentParser()
args_parser.add_argument("--name", "-n", default="PandaHugMonster")
args_parser.add_argument("--age", default="33")
args = args_parser.parse_args(["--name", "Oldie", "--age", "34"])
class MyEnum(BasicConfigEnum):
MY_1 = "key-1"
MY_2 = "key-2"
MY_3 = "key-3"
NAME = "name"
AGE = "age"
c = ConfigHub.aggregate(
{MyEnum.MY_2: "new val 2", "test": "test"},
args,
target=ConfigStore(
MyEnum.defaults(),
preprocessor=simputils_pp,
filter=True
)
)
```
### Simple Config and EnvVars
```python
import os
from simputils.config.components import ConfigHub
# Sequence of files/values matter!
app_conf = ConfigHub.aggregate(
"data/app-conf.yml",
# This one does not exist, and used only for local redefinitions of developer or stage
"data/app-conf-local.yml",
)
# Sequence of files/values matter!
app_env_vars = ConfigHub.aggregate(
"data/production.env",
# This one does not exist, and used only for local redefinitions of developer or stage
"data/production-local.env",
# This one does not exist, and used only for local redefinitions of developer or stage
".env",
# Environment Variables from OS
os.environ,
)
print("App Conf: ", app_conf)
print("App EnvVars: ", app_env_vars)
```
```text
App Conf: {
'val-1': 'My conf value 1',
'val-2': 'My conf value 2',
'val-3': 'My conf value 3'
}
App EnvVars: {
'APP_MY_ENV_VAR_1': '1',
'APP_MY_ENV_VAR_2': '2',
'APP_MY_ENV_VAR_3': '3',
...(values from OS env-vars are here)
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "simputils-config",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "simputils, config, files, utils, framework, simple",
"author": null,
"author_email": "Ivan Ponomarev <i.ponomarev@pandev.tech>",
"download_url": "https://files.pythonhosted.org/packages/36/67/e8507264a51bf5358ef5142d94ab5cc68d2673db8a2c3446e5c0fd5dbb52/simputils_config-1.1.1.tar.gz",
"platform": null,
"description": "# Python SimpUtils Config\nSimplifies working with configs and params.\n\n## Installation\n\nThis will install the latest version of the major version `1`.\nIt's safe enough due to followed by the project Semantic Versioning paradigm.\n\n```shell\npip install \"simputils-config~=1.0\"\n```\n\n> [!WARNING]\n> Developer of this project has nothing to do with `simputils` package of pypi.\n> And installation of both might cause broken code. \n> \n> On the end of this project, the namespace `simputils` is made \"shareable\".\n> But the structure of another developer's package is not designed in such way\n> \n> Disclaimer about that you can find here [Potential package collision 2024](docs/disclaimers.md)\n\n\n## Description\nClass `simputils.config.models.ConfigStore` is the keystone of the library.\nObject of it represents config, that could be used to sequentially apply different sets of key-value\npairs from different sources (other dicts, files of different kinds, environment variables, etc.).\n\nObject will store the latest values of the applied sets, \nwhere the earliest applied values could be overriden by the following ones.\nBasically, just a dictionary.\n\nBut besides being like a dictionary, it records the \"history\" of applied configs, so it's easier to\ndebug and control from where which value came from.\n\nThe major purpose of the functionality comes from using multiple files as sources for your config,\nbut it still allows to apply sets directly from the code.\n\nProject follows [Semantic Versioning](https://semver.org/) standard. \nNo breaking changes suppose to be within `minor` and `patch` versions of the same `major` version.\nException is only if there is a bug, that gets fixed.\n\n> [!CAUTION]\n> `ConfigStore` object is behaving like a `dict`, so if you need to check if\n> the variable with this object is None, always check it like `if conf is None:`,\n> and never check it like `if not conf:`!\n> \n> The check is implicit, so when you use `not` or simple boolean check, it will check if the object\n> is empty (does not contain any value)\n\n> [!NOTE]\n> To check if `ConfigStore` object contains at least one key-value pair, \n> you can use simple `if conf:`\n\n----\n\nWhen working with files, keep in mind that the only supported files are `.yml`/`.yaml`, `.env` and `.json`.\nIf you need support for other types, you will have to implement your custom handler(s) for those file-types.\n\n## Documentation\n* [Changelog](docs/CHANGELOG.md) - Please make sure you check it for new features and changes\n* [The overall example](docs/overall-example.md)\n* [Working with enums and annotations](docs/working-with-enums-and-annotations.md)\n* [Config Object Style Access](docs/config-object-style-access.md)\n* [Preprocessing and filtering](docs/preprocessing-and-filtering.md)\n* [Working with `ConfigHub`](docs/working-with-config-hub.md)\n (Quick Start, recommended way to work with the configs)\n* [Working with `ConfigStore`](docs/working-with-config-store.md)\n* [Config Merging Strategies](docs/config-merging-strategies.md)\n* [Some typical examples](examples)\n\n### Config Modifiers\n\nThere are 3 config modifiers that could be applied: `preprocessor`, `filter` and annotation type-casting \n\n> [!INFO]\n> The order of config modifiers applied to each config:\n> 1. `preprocessor`\n> 2. `filter`\n> 3. Annotation type-casting\n\n## Generic examples\n\n### The simplest usage\n\nAggregation from multiple sources (you can specify any number of sources).\n\n> [!IMPORTANT]\n> Keep in mind that order matters, keys/values from the latest might override already specified\n\nFrom files:\n```python\nfrom simputils.config.components import ConfigHub\n\nconf = ConfigHub.aggregate(\n \"config-1.yml\",\n \"config-2.yml\",\n # ...\n)\nprint(conf, conf.applied_confs)\n```\n\nFrom dictionaries:\n```python\nfrom simputils.config.components import ConfigHub\n\nconf = ConfigHub.aggregate(\n\t{\"key1\": \"val1\", \"key2\": \"val2\"},\n {\"key2\": \"redefined val2\", \"key3\": \"val3\"},\n # ...\n)\nprint(conf, conf.applied_confs)\n```\n\nAccessing values:\n```python\n\n# Accessing values by name, if does not exist - None is returned\nconf[\"key1\"]\nconf[\"key2\"]\n\n# Accessing values by name, if does not exist or None - `default` parameter value is returned\nconf.get(\"key1\", \"My Default Value\")\nconf.get(\"key2\")\n\n# Iterating as a dictionary\nfor key, val in conf.items():\n print(key, val)\n```\n\n### Enums as default config with filter\n\n```python\nimport os\nfrom typing import Annotated\n\nfrom simputils.config.base import simputils_pp\nfrom simputils.config.components import ConfigHub\nfrom simputils.config.generic import BasicConfigEnum\nfrom simputils.config.models import ConfigStore, AnnotatedConfigData\n\n\nclass MyEnum(BasicConfigEnum):\n\n # Annotated without and with default values set\n MY_E_KEY_1: Annotated[str, AnnotatedConfigData()] = \"my-e-key-1\"\n\n MY_E_KEY_2: Annotated[str, AnnotatedConfigData(\n default=3.1415\n )] = \"my-e-key-2\"\n\n # Non-annotated, so they will be None by default\n MY_E_KEY_3 = \"my-e-key-3\"\n MY_E_KEY_4 = \"my-e-key-4\"\n MY_E_KEY_5 = \"my-e-key-5\"\n\n # Some of them used in `app-conf.yml`\n MY_FIRST_VAL = \"val-1\"\n MY_SECOND_VAL = \"VAL_2\"\n\n # Will be taken from os.environ,\n # all other os.environ values will be excluded\n ENV_USER_NAME = \"USER\"\n\n\nconf = ConfigHub.aggregate(\n \"tests/data/config-1.yml\",\n\n os.environ,\n\n target=ConfigStore(\n MyEnum.defaults(),\n preprocessor=simputils_pp,\n filter=True\n ),\n)\n\nprint(\"conf: \", conf)\n```\n\n```text\nconf: {\n 'MY_E_KEY_1': None, \n 'MY_E_KEY_2': 3.1415, \n 'MY_E_KEY_3': None, \n 'MY_E_KEY_4': None, \n 'MY_E_KEY_5': None, \n 'VAL_1': 'My conf value 1', \n 'VAL_2': 'My conf value 2', \n 'USER': 'ivan'\n}\n```\n\n### Enums and argparser support\n`Enum` keys are supported out of the box, and `argparse.Namespace` could be used for `ConfigStore`\n\n> [!NOTE]\n> `BasicConfigEnum` is used for convenience. \n> And it's suggested way, it allows to use `True` as a filter key\n> \n> You still can use Enums without that class, \n> just make sure that enum is inherited from `str, Enum` in that order! \n\n```python\nfrom argparse import ArgumentParser\n\nfrom simputils.config.base import simputils_pp\nfrom simputils.config.components import ConfigHub\nfrom simputils.config.models import ConfigStore\nfrom simputils.config.generic import BasicConfigEnum\n\n\nargs_parser = ArgumentParser()\nargs_parser.add_argument(\"--name\", \"-n\", default=\"PandaHugMonster\")\nargs_parser.add_argument(\"--age\", default=\"33\")\n\nargs = args_parser.parse_args([\"--name\", \"Oldie\", \"--age\", \"34\"])\n\n\nclass MyEnum(BasicConfigEnum):\n\tMY_1 = \"key-1\"\n\tMY_2 = \"key-2\"\n\tMY_3 = \"key-3\"\n\n\tNAME = \"name\"\n\tAGE = \"age\"\n\nc = ConfigHub.aggregate(\n\t{MyEnum.MY_2: \"new val 2\", \"test\": \"test\"},\n\targs,\n\ttarget=ConfigStore(\n MyEnum.defaults(),\n\t\tpreprocessor=simputils_pp,\n\t\tfilter=True\n\t)\n)\n```\n\n### Simple Config and EnvVars\n\n```python\nimport os\n\nfrom simputils.config.components import ConfigHub\n\n# Sequence of files/values matter!\napp_conf = ConfigHub.aggregate(\n\t\"data/app-conf.yml\",\n\n\t# This one does not exist, and used only for local redefinitions of developer or stage\n\t\"data/app-conf-local.yml\",\n)\n\n# Sequence of files/values matter!\napp_env_vars = ConfigHub.aggregate(\n\t\"data/production.env\",\n\n\t# This one does not exist, and used only for local redefinitions of developer or stage\n\t\"data/production-local.env\",\n\n\t# This one does not exist, and used only for local redefinitions of developer or stage\n\t\".env\",\n\n\t# Environment Variables from OS\n\tos.environ,\n)\n\nprint(\"App Conf: \", app_conf)\nprint(\"App EnvVars: \", app_env_vars)\n```\n\n```text\nApp Conf: {\n 'val-1': 'My conf value 1', \n 'val-2': 'My conf value 2', \n 'val-3': 'My conf value 3'\n}\nApp EnvVars: {\n 'APP_MY_ENV_VAR_1': '1', \n 'APP_MY_ENV_VAR_2': '2', \n 'APP_MY_ENV_VAR_3': '3',\n \n ...(values from OS env-vars are here) \n}\n```\n\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Ivan Ponomarev Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Simple Configs Manager",
"version": "1.1.1",
"project_urls": {
"Homepage": "https://github.com/PandaHugMonster/py-simputils-config",
"Issues": "https://github.com/PandaHugMonster/py-simputils-config/issues"
},
"split_keywords": [
"simputils",
" config",
" files",
" utils",
" framework",
" simple"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cec197fa06cbaf46697279871bf2af4099b57fefa43ac016c0560a929b66988b",
"md5": "ef2d319cd4a8ef646ba8fabd6dc8e5a0",
"sha256": "f0237395ded0582922d04c8b94303238d3be1e29ceb9beecc1bcf9f7623a3158"
},
"downloads": -1,
"filename": "simputils_config-1.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ef2d319cd4a8ef646ba8fabd6dc8e5a0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 28666,
"upload_time": "2024-10-02T05:15:41",
"upload_time_iso_8601": "2024-10-02T05:15:41.526953Z",
"url": "https://files.pythonhosted.org/packages/ce/c1/97fa06cbaf46697279871bf2af4099b57fefa43ac016c0560a929b66988b/simputils_config-1.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3667e8507264a51bf5358ef5142d94ab5cc68d2673db8a2c3446e5c0fd5dbb52",
"md5": "beacc18c2826b01d63d28f5cf1fa794f",
"sha256": "dd1ed1454fe68132e03a0949cf64e73f9dcda417e55b64ebf4015578b6941942"
},
"downloads": -1,
"filename": "simputils_config-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "beacc18c2826b01d63d28f5cf1fa794f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 733782,
"upload_time": "2024-10-02T05:15:45",
"upload_time_iso_8601": "2024-10-02T05:15:45.631826Z",
"url": "https://files.pythonhosted.org/packages/36/67/e8507264a51bf5358ef5142d94ab5cc68d2673db8a2c3446e5c0fd5dbb52/simputils_config-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-02 05:15:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "PandaHugMonster",
"github_project": "py-simputils-config",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "pyyaml",
"specs": []
},
{
"name": "python-magic",
"specs": []
},
{
"name": "python-dotenv",
"specs": []
},
{
"name": "pytest",
"specs": []
},
{
"name": "pytest-cov",
"specs": []
},
{
"name": "flake8",
"specs": []
},
{
"name": "pydantic",
"specs": []
}
],
"lcname": "simputils-config"
}
