# recap
[](https://github.com/georg-wolflein/recap/actions?query=workflow%3Abuild)
[](https://pypi.org/project/recap)
[](https://pypi.org/project/recap)
[](https://github.com/georg-wolflein/recap/blob/master/LICENSE)
[](http://recap.readthedocs.io/?badge=latest)
_recap_ is a tool for providing _REproducible Configurations for Any Project_.
Research should be reproducible.
Especially in deep learning, it is important to keep track of hyperparameters and configurations used in experiments.
This package aims at making that easier.
## Installing
Just install like any Python package:
```bash
pip install recap
```
## Overview
Recap provides two top-level concepts that would be imported as follows:
```python
from recap import URI, CfgNode as CN
```
The `CfgNode` is a subclass of [yacs](https://github.com/rbgirshick/yacs)' `CfgNode`.
It provides some additional features for parsing configurations that are inherited between files which is not possible with yacs.
Recap's `URI` class provides a mechanism for handling logical paths within your project more conveniently with an interface that is fully compatible with `pathlib.Path`.
## YAML configurations
Configurations are defined [just like in yacs](https://github.com/rbgirshick/yacs#usage), except that you need to import the `CfgNode` class from the recap package instead of yacs.
Consider the following YAML configuration that sets default values for all configuration options we will use in our project. We shall name it `_base.yaml` because our experiments will build on these values.
```yaml
SYSTEM:
NUM_GPUS: 4
NUM_WORKERS: 2
TRAIN:
LEARNING_RATE: 0.001
BATCH_SIZE: 32
SOME_OTHER_HYPERPARAMETER: 10
```
The equivalent configuration can be obtained programatically like so:
```python
from recap import CfgNode as CN
cfg = CN()
cfg.SYSTEM = CN()
cfg.SYSTEM.NUM_GPUS = 4
cfg.SYSTEM.NUM_WORKERS = 2
cfg.TRAIN = CN()
cfg.TRAIN.LEARNING_RATE = 1e-3
cfg.TRAIN.BATCH_SIZE = 32
cfg.TRAIN.SOME_OTHER_HYPERPARAMETER = 10
print(cfg)
```
### Inheriting configurations
Recap provides functionality for inheriting configuration options from other configuration files by setting the top-level `_BASE_` key.
So, we could create a configuration file `experiment_1.yaml` for an experiment where we try a different learning rate and batch size:
```yaml
_BASE_: _base.yaml
TRAIN:
LEARNING_RATE: 1e-2
BATCH_SIZE: 64
```
In our code, when we want to load the experiment configuration, we would use the `recap.CfgNode.load_yaml_with_base()` function:
```python
from recap import CfgNode as CN
cfg = CN.load_yaml_with_base("experiment_1.yaml")
print(cfg)
# Will output:
"""
SYSTEM:
NUM_GPUS: 4
NUM_WORKERS: 2
TRAIN:
LEARNING_RATE: 0.01
BATCH_SIZE: 64
SOME_OTHER_HYPERPARAMETER: 10
"""
```
Note that the `_BASE_` keys can be arbitrarily nested; however, circular references are prohibited.
## Logical URIs and the path manager
Recap includes a path manager for conveniently specifying paths to logical entities.
The path strings are set up like a URI where the scheme (i.e. `http` in the path string `http://google.com`) refers to a logical entity.
Each such entity needs to be set up as a `PathTranslator` that can translate the logical URI path to a physical path on the file system.
For example, we could set up a path translator for the `data` scheme to refer to the the path of a dataset on our file system located at `/path/to/dataset`. Then the recap URI `data://train/abc.txt` would be translated to `/path/to/dataset/train/abc.txt`.
The simplest way of setting that up is using the `register_translator` function (although more complex setups are possible with the `recap.path_manager.PathTranslator` class, allowing you to download files from the internet, for example):
```python
from recap.path_manager import register_translator
from pathlib import Path
register_translator("data", Path("/path/to/dataset"))
```
Then, we can use the `recap.URI` class just like any `pathlib.Path` object:
```python
from recap import URI
my_uri = URI("data://train/abc.txt")
# Here, str(my_uri) == "/path/to/dataset/train/abc.txt"
with my_uri.open("r") as f:
print(f.read())
```
### Logical URIs in inherited configurations
The `recap.URI` interface is fully compatible with the nested configurations.
This means that you can use recap `URI`s within the `_BASE_` field for inheriting configurations.
For example, you could register a path translator for the `config` scheme and then include `_BASE_: config://_base.yaml` in your configuration files.
Raw data
{
"_id": null,
"home_page": "http://recap.readthedocs.io",
"name": "recap",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "flake8,markdown,lint",
"author": "Georg W\u00f6lflein",
"author_email": "georgw7777@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/8e/02/f47099fdb481164f46dcfe017ccbc657225c3c9859b9b8b1c0adaec40c76/recap-0.1.7.tar.gz",
"platform": null,
"description": "# recap\n\n[](https://github.com/georg-wolflein/recap/actions?query=workflow%3Abuild)\n[](https://pypi.org/project/recap)\n[](https://pypi.org/project/recap)\n[](https://github.com/georg-wolflein/recap/blob/master/LICENSE)\n[](http://recap.readthedocs.io/?badge=latest)\n\n_recap_ is a tool for providing _REproducible Configurations for Any Project_.\n\nResearch should be reproducible.\nEspecially in deep learning, it is important to keep track of hyperparameters and configurations used in experiments.\nThis package aims at making that easier.\n\n## Installing\n\nJust install like any Python package:\n\n```bash\npip install recap\n```\n\n## Overview\n\nRecap provides two top-level concepts that would be imported as follows:\n\n```python\nfrom recap import URI, CfgNode as CN\n```\n\nThe `CfgNode` is a subclass of [yacs](https://github.com/rbgirshick/yacs)' `CfgNode`.\nIt provides some additional features for parsing configurations that are inherited between files which is not possible with yacs.\n\nRecap's `URI` class provides a mechanism for handling logical paths within your project more conveniently with an interface that is fully compatible with `pathlib.Path`.\n\n## YAML configurations\n\nConfigurations are defined [just like in yacs](https://github.com/rbgirshick/yacs#usage), except that you need to import the `CfgNode` class from the recap package instead of yacs.\nConsider the following YAML configuration that sets default values for all configuration options we will use in our project. We shall name it `_base.yaml` because our experiments will build on these values.\n\n```yaml\nSYSTEM:\n NUM_GPUS: 4\n NUM_WORKERS: 2\nTRAIN:\n LEARNING_RATE: 0.001\n BATCH_SIZE: 32\n SOME_OTHER_HYPERPARAMETER: 10\n```\n\nThe equivalent configuration can be obtained programatically like so:\n\n```python\nfrom recap import CfgNode as CN\n\ncfg = CN()\ncfg.SYSTEM = CN()\ncfg.SYSTEM.NUM_GPUS = 4\ncfg.SYSTEM.NUM_WORKERS = 2\ncfg.TRAIN = CN()\ncfg.TRAIN.LEARNING_RATE = 1e-3\ncfg.TRAIN.BATCH_SIZE = 32\ncfg.TRAIN.SOME_OTHER_HYPERPARAMETER = 10\n\nprint(cfg)\n```\n\n### Inheriting configurations\n\nRecap provides functionality for inheriting configuration options from other configuration files by setting the top-level `_BASE_` key.\nSo, we could create a configuration file `experiment_1.yaml` for an experiment where we try a different learning rate and batch size:\n\n```yaml\n_BASE_: _base.yaml\n\nTRAIN:\n LEARNING_RATE: 1e-2\n BATCH_SIZE: 64\n```\n\nIn our code, when we want to load the experiment configuration, we would use the `recap.CfgNode.load_yaml_with_base()` function:\n\n```python\nfrom recap import CfgNode as CN\n\ncfg = CN.load_yaml_with_base(\"experiment_1.yaml\")\n\nprint(cfg)\n\n# Will output:\n\"\"\"\nSYSTEM:\n NUM_GPUS: 4\n NUM_WORKERS: 2\nTRAIN:\n LEARNING_RATE: 0.01\n BATCH_SIZE: 64\n SOME_OTHER_HYPERPARAMETER: 10\n\"\"\"\n```\n\nNote that the `_BASE_` keys can be arbitrarily nested; however, circular references are prohibited.\n\n## Logical URIs and the path manager\n\nRecap includes a path manager for conveniently specifying paths to logical entities.\nThe path strings are set up like a URI where the scheme (i.e. `http` in the path string `http://google.com`) refers to a logical entity.\nEach such entity needs to be set up as a `PathTranslator` that can translate the logical URI path to a physical path on the file system.\n\nFor example, we could set up a path translator for the `data` scheme to refer to the the path of a dataset on our file system located at `/path/to/dataset`. Then the recap URI `data://train/abc.txt` would be translated to `/path/to/dataset/train/abc.txt`.\n\nThe simplest way of setting that up is using the `register_translator` function (although more complex setups are possible with the `recap.path_manager.PathTranslator` class, allowing you to download files from the internet, for example):\n\n```python\nfrom recap.path_manager import register_translator\nfrom pathlib import Path\n\nregister_translator(\"data\", Path(\"/path/to/dataset\"))\n```\n\nThen, we can use the `recap.URI` class just like any `pathlib.Path` object:\n\n```python\nfrom recap import URI\n\nmy_uri = URI(\"data://train/abc.txt\")\n# Here, str(my_uri) == \"/path/to/dataset/train/abc.txt\"\n\nwith my_uri.open(\"r\") as f:\n print(f.read())\n```\n\n### Logical URIs in inherited configurations\n\nThe `recap.URI` interface is fully compatible with the nested configurations.\nThis means that you can use recap `URI`s within the `_BASE_` field for inheriting configurations.\n\nFor example, you could register a path translator for the `config` scheme and then include `_BASE_: config://_base.yaml` in your configuration files.\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Reproducible configurations for any project",
"version": "0.1.7",
"project_urls": {
"Homepage": "http://recap.readthedocs.io",
"Repository": "https://github.com/georg-wolflein/recap"
},
"split_keywords": [
"flake8",
"markdown",
"lint"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "91016c37fcccd44869355073711c2a09b64b543e5a96475b6a5bab3a06f550e1",
"md5": "368315c882dee976e3bb65b7f07f414f",
"sha256": "c56e869fdf29eaddf287a360af1e2f856f370a91dc283f919f55dcb0ccf5fdc4"
},
"downloads": -1,
"filename": "recap-0.1.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "368315c882dee976e3bb65b7f07f414f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 14770,
"upload_time": "2023-10-30T19:06:52",
"upload_time_iso_8601": "2023-10-30T19:06:52.819745Z",
"url": "https://files.pythonhosted.org/packages/91/01/6c37fcccd44869355073711c2a09b64b543e5a96475b6a5bab3a06f550e1/recap-0.1.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8e02f47099fdb481164f46dcfe017ccbc657225c3c9859b9b8b1c0adaec40c76",
"md5": "5e718327c2c2c4747af09e32cd8928c0",
"sha256": "c66d6e2d38c0abc5278edaa939fee40debbeb5b30bee838ad9bc90c9b4f5dbbe"
},
"downloads": -1,
"filename": "recap-0.1.7.tar.gz",
"has_sig": false,
"md5_digest": "5e718327c2c2c4747af09e32cd8928c0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 9771,
"upload_time": "2023-10-30T19:06:54",
"upload_time_iso_8601": "2023-10-30T19:06:54.417222Z",
"url": "https://files.pythonhosted.org/packages/8e/02/f47099fdb481164f46dcfe017ccbc657225c3c9859b9b8b1c0adaec40c76/recap-0.1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-30 19:06:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "georg-wolflein",
"github_project": "recap",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "recap"
}
JSON
