# eparams
_Simple_ and _configurable_ parameters/configuration class. No manual schema needed!
<br/> Use python3.6+ type hinting to declare the param type, or let eparams to infer it from the assignment, and keep you from assigning the wrong type down the line!
At its core it is designed as a drop-in replacement for `dataclass`, but with added features to make life easier.
## Installation
$`pip install eparams`
## Features
- Automatically type-check parameters using [typeguard](https://github.com/agronholm/typeguard).
- Auto typing - doesn't *force* you to add type hints everywhere.
- Define a custom / use our default preprocess function to cast your input to the correct type.
- Use mutable types (`list` `dict` etc) at class definition with no worries and no awkward mandatory factory objects.
- Protect from typos that would normally lead to assigning undeclared parameters.
- Define custom constraints.
- freeze / unfreeze class.
- History tracking.
- Track assignment to the class to see which attributes were assigned and from what values.
- Added helper methods:
- `_to_json()` `_to_yaml()` `_to_dict()` `_from_json()` `_from_yaml()` `_from_dict()`
- `copy()` `_freeze()` `_unfreeze()` `_history()`
- `__contains__()` `__eq__()` `__getitem__()` `__setitem__()`
- All above features are fully configurable per parameter and globally.
- Built to be nested, with `conf.sub_conf.some_val` structure, all methods work recursively. Plays well with `dataclass`.
- Helper functions:
- Recursive comparison: compare configs from different versions of your code / accross experiments
- Register partial-config functions in a dictionary for easy calls using cli and external configs.
## Example usage:
See **[sample_project](sample_project)** for an example usage in a "complete" project which includes:
- cli integration with `argparse`.
- delta-config registration.
- constraints.
- frozen class.
- yaml/dict export.
#### Basic usage:
```python
from eparams import params
# Basic usage
@params
class OptimizerParams:
learning_rate = 1e-2
weight_decay = 0.001
batch_size = 1024
@params
class Params:
name = 'default name'
tags = ['no', 'problem', 'with', 'list', 'here']
optim = OptimizerParams()
verbose = False
config = Params(name='new name')
config.optim.batch_size = 2048 # ok
config.optim.learning_rate = '0.001' # ok, cast to float
config.optim.batch_size = '32' # ok, cast to int
config.verbose = 'True' # ok, cast to bool
config['optim.weight_decay'] = 0 # we can set nested attributes like this as-well
print(config)
# prints:
# name='new name'
# tags=['no', 'problem', 'with', 'list', 'here']
# optim.learning_rate=0.001
# optim.weight_decay=0
# optim.batch_size=32
# verbose=True
config._to_yaml('/path/to/config.yaml') # save as yaml
# The following lines will raise an exception
config.optim.batch_size = 'string' # raises ValueError: invalid literal for int() with base 10: 'string'
config.optim.batch_size = 1.3 # raises TypeError: type of batch_size must be int; got float instead
config.verrrbose = False # raises ValueError: Cannot assign <verrrbose> to class <<class '__main__.Params'>>, missing from class definition (allow_dynamic_attribute=False)
```
#### Debug and compare configs from old/different runs:
```python
from eparams import params, params_compare
# type_verify=False does not check for typing errors.
# default_preprocessor=None removes the preprocessing step (a custom function is also valid here)
# allow_dynamic_attribute=True allows the class to dynamically set new attributes.
@params(type_verify=False, default_preprocessor=None, allow_dynamic_attribute=True)
class Params:
name = 'default name'
my_int = 0
old_config = Params()._from_yaml('/path/to/old/config.yaml', strict=False) # load some old config
print(old_config._history()) # show parameters that were loaded and their pre-loaded value
params_not_in_old_yaml = [k for k, hist in old_config._history(full=True).items() if not hist]
modified, added, removed = params_compare(old_config, Params()) # compare two versions of configs
```
#### Preprocessing example:
```python
import enum
from pathlib import Path
from eparams import params, Var
class MyEnum(enum.Enum):
a = 'a'
b = 'b'
@params
class Example:
num: int = 3
type = MyEnum.b
path: Path = '/user/home' # cast to Path
some_list = [1, 2, 3] # we copy the list before assignment
some_string = Var('yo_yo', preprocess_fn=str.lower) # lower string
ex = Example()
ex.num = '4' # cast to int
ex.type = 'a' # cast to MyEnum
ex.some_string = 'HELLO' # .lower()
assert ex.some_list is not Example.some_list
print(ex)
# prints:
# num=4
# type=<MyEnum.a: 'a'>
# path=PosixPath('/user/home')
# some_list=[1, 2, 3]
# some_string='hello'
```
### Other popular approaches/libraries to define parameters:
- Simply use a `dict`
- Simply use a `dataclass`
- [pydantic](https://pydantic-docs.helpmanual.io/)
- [attrs](https://www.attrs.org/en/stable/index.html)
Raw data
{
"_id": null,
"home_page": "https://github.com/amirfru/eparams",
"name": "eparams",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6, <4",
"maintainer_email": "",
"keywords": "params,config,development",
"author": "Amir Fruchtman",
"author_email": "amir.fru@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/33/a4/60e0105523f33ab3b2f3b64e286c50f1a10bb4895e26639d7799951731a1/eparams-0.2.tar.gz",
"platform": null,
"description": "# eparams\n_Simple_ and _configurable_ parameters/configuration class. No manual schema needed!\n<br/> Use python3.6+ type hinting to declare the param type, or let eparams to infer it from the assignment, and keep you from assigning the wrong type down the line!\n\nAt its core it is designed as a drop-in replacement for `dataclass`, but with added features to make life easier.\n\n## Installation \n$`pip install eparams`\n\n## Features\n- Automatically type-check parameters using [typeguard](https://github.com/agronholm/typeguard).\n- Auto typing - doesn't *force* you to add type hints everywhere.\n- Define a custom / use our default preprocess function to cast your input to the correct type.\n- Use mutable types (`list` `dict` etc) at class definition with no worries and no awkward mandatory factory objects.\n- Protect from typos that would normally lead to assigning undeclared parameters.\n- Define custom constraints.\n- freeze / unfreeze class.\n- History tracking.\n - Track assignment to the class to see which attributes were assigned and from what values.\n- Added helper methods: \n - `_to_json()` `_to_yaml()` `_to_dict()` `_from_json()` `_from_yaml()` `_from_dict()`\n - `copy()` `_freeze()` `_unfreeze()` `_history()`\n - `__contains__()` `__eq__()` `__getitem__()` `__setitem__()`\n- All above features are fully configurable per parameter and globally.\n- Built to be nested, with `conf.sub_conf.some_val` structure, all methods work recursively. Plays well with `dataclass`.\n- Helper functions:\n - Recursive comparison: compare configs from different versions of your code / accross experiments\n - Register partial-config functions in a dictionary for easy calls using cli and external configs. \n\n\n## Example usage:\nSee **[sample_project](sample_project)** for an example usage in a \"complete\" project which includes:\n- cli integration with `argparse`.\n- delta-config registration.\n- constraints.\n- frozen class.\n- yaml/dict export.\n\n#### Basic usage:\n```python\nfrom eparams import params\n\n# Basic usage\n@params\nclass OptimizerParams:\n learning_rate = 1e-2\n weight_decay = 0.001\n batch_size = 1024\n\n@params\nclass Params:\n name = 'default name'\n tags = ['no', 'problem', 'with', 'list', 'here']\n optim = OptimizerParams()\n verbose = False\n\nconfig = Params(name='new name')\nconfig.optim.batch_size = 2048 # ok\nconfig.optim.learning_rate = '0.001' # ok, cast to float\nconfig.optim.batch_size = '32' # ok, cast to int\nconfig.verbose = 'True' # ok, cast to bool\nconfig['optim.weight_decay'] = 0 # we can set nested attributes like this as-well\nprint(config)\n# prints: \n# name='new name'\n# tags=['no', 'problem', 'with', 'list', 'here']\n# optim.learning_rate=0.001\n# optim.weight_decay=0\n# optim.batch_size=32\n# verbose=True\nconfig._to_yaml('/path/to/config.yaml') # save as yaml\n\n# The following lines will raise an exception\nconfig.optim.batch_size = 'string' # raises ValueError: invalid literal for int() with base 10: 'string'\nconfig.optim.batch_size = 1.3 # raises TypeError: type of batch_size must be int; got float instead\nconfig.verrrbose = False # raises ValueError: Cannot assign <verrrbose> to class <<class '__main__.Params'>>, missing from class definition (allow_dynamic_attribute=False)\n```\n\n#### Debug and compare configs from old/different runs:\n```python\nfrom eparams import params, params_compare\n\n# type_verify=False does not check for typing errors.\n# default_preprocessor=None removes the preprocessing step (a custom function is also valid here)\n# allow_dynamic_attribute=True allows the class to dynamically set new attributes.\n@params(type_verify=False, default_preprocessor=None, allow_dynamic_attribute=True)\nclass Params:\n name = 'default name'\n my_int = 0\n\nold_config = Params()._from_yaml('/path/to/old/config.yaml', strict=False) # load some old config\nprint(old_config._history()) # show parameters that were loaded and their pre-loaded value\nparams_not_in_old_yaml = [k for k, hist in old_config._history(full=True).items() if not hist]\nmodified, added, removed = params_compare(old_config, Params()) # compare two versions of configs\n```\n\n#### Preprocessing example:\n```python\nimport enum\nfrom pathlib import Path\nfrom eparams import params, Var\n\n\nclass MyEnum(enum.Enum):\n a = 'a'\n b = 'b'\n\n@params\nclass Example:\n num: int = 3\n type = MyEnum.b\n path: Path = '/user/home' # cast to Path\n some_list = [1, 2, 3] # we copy the list before assignment\n some_string = Var('yo_yo', preprocess_fn=str.lower) # lower string\n\nex = Example()\nex.num = '4' # cast to int\nex.type = 'a' # cast to MyEnum\nex.some_string = 'HELLO' # .lower()\n\nassert ex.some_list is not Example.some_list\nprint(ex)\n# prints:\n# num=4\n# type=<MyEnum.a: 'a'>\n# path=PosixPath('/user/home')\n# some_list=[1, 2, 3]\n# some_string='hello'\n```\n\n### Other popular approaches/libraries to define parameters:\n- Simply use a `dict`\n- Simply use a `dataclass`\n- [pydantic](https://pydantic-docs.helpmanual.io/)\n- [attrs](https://www.attrs.org/en/stable/index.html)\n\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Parameter class for all",
"version": "0.2",
"split_keywords": [
"params",
"config",
"development"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3cf8e17b1469f54466712c78bd47bf64ecff0f8d32a19c5a374852915ad66555",
"md5": "c65cb6f2a319736988c611f6af03012d",
"sha256": "5b2c1940119fab479ff01cd6f0855dede483c7121e7ff2d46cd35723d43457b7"
},
"downloads": -1,
"filename": "eparams-0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c65cb6f2a319736988c611f6af03012d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6, <4",
"size": 11773,
"upload_time": "2023-03-25T20:38:38",
"upload_time_iso_8601": "2023-03-25T20:38:38.517114Z",
"url": "https://files.pythonhosted.org/packages/3c/f8/e17b1469f54466712c78bd47bf64ecff0f8d32a19c5a374852915ad66555/eparams-0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "33a460e0105523f33ab3b2f3b64e286c50f1a10bb4895e26639d7799951731a1",
"md5": "f3cd49c5f0472f38ca852fdaf150485f",
"sha256": "01d2bb46ade2766c202fb48201e7ab56be9af150c52a31ad8d065c728b5497d1"
},
"downloads": -1,
"filename": "eparams-0.2.tar.gz",
"has_sig": false,
"md5_digest": "f3cd49c5f0472f38ca852fdaf150485f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6, <4",
"size": 13376,
"upload_time": "2023-03-25T20:38:40",
"upload_time_iso_8601": "2023-03-25T20:38:40.702795Z",
"url": "https://files.pythonhosted.org/packages/33/a4/60e0105523f33ab3b2f3b64e286c50f1a10bb4895e26639d7799951731a1/eparams-0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-25 20:38:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "amirfru",
"github_project": "eparams",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "eparams"
}
JSON
