[](https://github.com/JeanKossaifi/configmypy/actions/workflows/test.yml)
# ConfigMyPy
Configure My Python: Easy configuration of Python projects.
## Quickstart
Just clone the repository and install it, here in editable mode:
```bash
git clone https://github.com/JeanKossaifi/configmypy
cd configmypy
python -m pip install -e .
```
Then, assuming you have a configuration file `config.yaml` in your folder, get your configuration using
```python
from configmypy import ConfigPipeline, YamlConfig, ArgparseConfig
pipe = ConfigPipeline([YamlConfig('./config.yaml'),
ArgparseConfig()])
config = pipe.read_conf()
```
Then just read your parameters from config:
```
arg = config.arg
arg2 = config['arg2']
...
```
The script will read config.yaml, as well as optional command-line arguments, which will overwrite the config.
## Rationale
How many times did you start having a large Python project, let's say to run some Machine Learning models, where you wanted to change quickly the parameters.
Then you want to track them. In you write a small logger. Then you write a small yaml config file to more easily manage the quickly growing list of parameters.
Then you realize you sometimes need to change some of these parameters on the fly.
In comes argparse, and you manually override an increasingly long list of paramters with a code becoming a long list of
```python
if args.parameter is not None:
config['parameter'] = args.parameter
```
...and there goes your sanity... Along with any hope of properly managing and tracking your experiments.
To address this frustration, I wrote this minimal package that makes this easy.
## Reading from a `.yaml` config file
Imagine you have a `config.yaml` file that looks like this:
```yaml
default:
opt:
optimizer: 'adam'
lr: 0.1
data:
dataset: 'ns'
batch_size: 12
test:
opt:
optimizer: 'SGD'
```
You can read that using `configmypy.YamlConfig`:
```yaml
reader = YamlConfig('./config.yaml', config_name='default')
config, _ = reader.read_conf()
```
## Bunch configurations
All our configurations return a Bunch, not a dict. A Bunch is simply a dictionary that exposes its parameters as attributes so you can access them, equivalently, as
`config['param']` or `config.param`.
## Configuration Pipeline
The real power of `configmypy` comes from the ConfigPipeline: you can have several steps called sequentially.
A typical usecase would be:
1) Read a default (.yaml) config
2) Optionally overwrite any of those parameters using argparse (command-line arguments)
3) If the user specified another configuration, update the parameters read so far with that new config
This would look like this with `configmypy`:
```python
pipe = ConfigPipeline([YamlConfig('./config.yaml', config_name='default'),
ArgparseConfig(config_file=None, config_name=None),
YamlConfig()])
config = pipe.read_conf()
```
This will:
1) first read the `default` section in `config.yaml`
2) update any parameters with those passed by the user, including additional parameters `config_file` and `config_name` which are automatically passed to the next step
3) If the user specified a `config_file`, that will be read by the next step and used to update the configuration.
You can check the configuration by calling `pipe.log()`:
```python
>>> pipe.log()
###############################
##### CONFIGURATION #####
###############################
Steps:
------
(1) YamlConfig with config_file=./config.yaml, config_name=default
(2) ArgparseConfig with additional config={'config_file': None, 'config_name': None}
(3) YamlConfig with config_file=config.yaml, config_name=test
-------------------------------
Configuration:
--------------
opt.optimizer=SGD
opt.lr=0.1
data.dataset=ns
data.batch_size=24
###############################
```
## Questions or issues
This is very much a project in development that I wrote for myself and decided to open-source so myself and others could easily reuse it for multiple projects, while knowing it is actually tested!
If you have any questions or find any bugs, please open an issue, or better yet, a pull-request!
Raw data
{
"_id": null,
"home_page": "https://github.com/JeanKossaifi/configmypy",
"name": "configmypy",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Jean Kossaifi",
"author_email": "jean.kossaifi@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e2/2f/df2bd442d1767a879da63091394b0dc89a5800da84cd9fb36ecb818bfcfe/configmypy-0.2.0.tar.gz",
"platform": null,
"description": "\n[](https://github.com/JeanKossaifi/configmypy/actions/workflows/test.yml)\n\n# ConfigMyPy\n\nConfigure My Python: Easy configuration of Python projects.\n\n## Quickstart\n\nJust clone the repository and install it, here in editable mode:\n\n```bash\ngit clone https://github.com/JeanKossaifi/configmypy\ncd configmypy\npython -m pip install -e .\n```\n\nThen, assuming you have a configuration file `config.yaml` in your folder, get your configuration using\n```python\nfrom configmypy import ConfigPipeline, YamlConfig, ArgparseConfig\n\npipe = ConfigPipeline([YamlConfig('./config.yaml'),\n ArgparseConfig()])\nconfig = pipe.read_conf()\n```\n\nThen just read your parameters from config:\n```\narg = config.arg\narg2 = config['arg2']\n...\n```\n\nThe script will read config.yaml, as well as optional command-line arguments, which will overwrite the config.\n\n## Rationale\n\nHow many times did you start having a large Python project, let's say to run some Machine Learning models, where you wanted to change quickly the parameters.\nThen you want to track them. In you write a small logger. Then you write a small yaml config file to more easily manage the quickly growing list of parameters.\nThen you realize you sometimes need to change some of these parameters on the fly. \nIn comes argparse, and you manually override an increasingly long list of paramters with a code becoming a long list of\n\n```python\nif args.parameter is not None:\n config['parameter'] = args.parameter\n```\n\n...and there goes your sanity... Along with any hope of properly managing and tracking your experiments. \n\nTo address this frustration, I wrote this minimal package that makes this easy.\n\n## Reading from a `.yaml` config file\n\nImagine you have a `config.yaml` file that looks like this:\n\n```yaml\ndefault:\n opt:\n optimizer: 'adam'\n lr: 0.1\n data:\n dataset: 'ns'\n batch_size: 12\n \ntest:\n opt:\n optimizer: 'SGD'\n```\n\nYou can read that using `configmypy.YamlConfig`: \n\n```yaml\nreader = YamlConfig('./config.yaml', config_name='default')\nconfig, _ = reader.read_conf()\n```\n\n## Bunch configurations\n\nAll our configurations return a Bunch, not a dict. A Bunch is simply a dictionary that exposes its parameters as attributes so you can access them, equivalently, as\n`config['param']` or `config.param`.\n\n## Configuration Pipeline\n\nThe real power of `configmypy` comes from the ConfigPipeline: you can have several steps called sequentially. \nA typical usecase would be: \n1) Read a default (.yaml) config\n2) Optionally overwrite any of those parameters using argparse (command-line arguments)\n3) If the user specified another configuration, update the parameters read so far with that new config\n\nThis would look like this with `configmypy`:\n\n```python\npipe = ConfigPipeline([YamlConfig('./config.yaml', config_name='default'),\n ArgparseConfig(config_file=None, config_name=None),\n YamlConfig()])\nconfig = pipe.read_conf()\n```\n\nThis will: \n1) first read the `default` section in `config.yaml`\n2) update any parameters with those passed by the user, including additional parameters `config_file` and `config_name` which are automatically passed to the next step\n3) If the user specified a `config_file`, that will be read by the next step and used to update the configuration.\n\nYou can check the configuration by calling `pipe.log()`:\n\n```python\n>>> pipe.log()\n###############################\n##### CONFIGURATION #####\n###############################\n\nSteps:\n------\n (1) YamlConfig with config_file=./config.yaml, config_name=default\n (2) ArgparseConfig with additional config={'config_file': None, 'config_name': None}\n (3) YamlConfig with config_file=config.yaml, config_name=test\n\n-------------------------------\n\nConfiguration:\n--------------\n\nopt.optimizer=SGD\nopt.lr=0.1\ndata.dataset=ns\ndata.batch_size=24\n\n###############################\n\n```\n\n## Questions or issues\nThis is very much a project in development that I wrote for myself and decided to open-source so myself and others could easily reuse it for multiple projects, while knowing it is actually tested!\n\nIf you have any questions or find any bugs, please open an issue, or better yet, a pull-request!\n\n\n\n",
"bugtrack_url": null,
"license": "Modified BSD",
"summary": "Fast and easy configuration of Python projects.",
"version": "0.2.0",
"project_urls": {
"Download": "https://github.com/JeanKossaifi/configmypy/tarball/0.2.0",
"Homepage": "https://github.com/JeanKossaifi/configmypy"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0d22d1d35a8d667592ac59b57e214ce92abd1019437f00e8362fef17d99eca2f",
"md5": "d4613bdd8429060b88a5984c9de98162",
"sha256": "c50e998edc052f3d3f10659338e28631cf3962a60778a55d318edcc4cebf2cfb"
},
"downloads": -1,
"filename": "configmypy-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d4613bdd8429060b88a5984c9de98162",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 14542,
"upload_time": "2024-06-09T17:16:13",
"upload_time_iso_8601": "2024-06-09T17:16:13.019143Z",
"url": "https://files.pythonhosted.org/packages/0d/22/d1d35a8d667592ac59b57e214ce92abd1019437f00e8362fef17d99eca2f/configmypy-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e22fdf2bd442d1767a879da63091394b0dc89a5800da84cd9fb36ecb818bfcfe",
"md5": "547bd9064a6fab5cc31ce8fbeda49866",
"sha256": "a517921d18656844dae6831665cbf7718f153876b9cbc526f8bca6a6fb8cd28a"
},
"downloads": -1,
"filename": "configmypy-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "547bd9064a6fab5cc31ce8fbeda49866",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 13034,
"upload_time": "2024-06-09T17:16:14",
"upload_time_iso_8601": "2024-06-09T17:16:14.364027Z",
"url": "https://files.pythonhosted.org/packages/e2/2f/df2bd442d1767a879da63091394b0dc89a5800da84cd9fb36ecb818bfcfe/configmypy-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-09 17:16:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "JeanKossaifi",
"github_project": "configmypy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "configmypy"
}
JSON
