spock-config


Namespock-config JSON
Version 2.5.2 PyPI version JSON
download
home_pagehttps://github.com/fidelity/spock
SummarySpock is a framework designed to help manage complex parameter configurations for Python applications
upload_time2022-05-17 18:12:34
maintainer
docs_urlNone
authorFMR LLC
requires_python>=3.6
license
keywords configuration argparse parameters machine learning deep learning reproducibility hyper-parameter tuning
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage
            <html>
<h1 align="center">
    <a href="https://fidelity.github.io/spock/"><img src="https://raw.githubusercontent.com/fidelity/spock/master/resources/images/logo.png"/></a>
    <h6 align="center">Managing complex configurations any other way would be highly illogical...</h6>
</h1>

<p align="center">
  <a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-9cf"/></a>
  <a href="https://bestpractices.coreinfrastructure.org/projects/5551"><img src="https://bestpractices.coreinfrastructure.org/projects/5551/badge"/></a>
  <a><img src="https://github.com/fidelity/spock/workflows/pytest/badge.svg?branch=master"/></a>
<a href="https://coveralls.io/github/fidelity/spock?branch=master"><img src="https://coveralls.io/repos/github/fidelity/spock/badge.svg?branch=master"/></a>
  <a><img src="https://github.com/fidelity/spock/workflows/docs/badge.svg"/></a>
</p>

<p align="center">
  <a><img src="https://img.shields.io/badge/python-3.6+-informational.svg"/></a>
  <a href="https://github.com/psf/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg"/></a>
  <a href="https://badge.fury.io/py/spock-config"><img src="https://badge.fury.io/py/spock-config.svg"/></a>
  <a href="https://pepy.tech/project/spock-config"><img src="https://static.pepy.tech/personalized-badge/spock-config?period=total&units=international_system&left_color=grey&right_color=orange&left_text=Downloads"/></a>
</p>

<h3 align="center">
  <a href="https://fidelity.github.io/spock/Quick-Start">Quick Start</a>
  <span> · </span>
  <a href="https://fidelity.github.io/spock/">Documentation</a>
  <span> · </span>
  <a href="https://github.com/fidelity/spock/blob/master/examples">Examples</a>
  <span> · </span>
  <a href="https://github.com/fidelity/spock/releases">Releases</a>
</h3>

</html>

## About

`spock` is a framework that helps users easily define, manage, and use complex parameter configurations within Python 
applications. It lets you focus on the code you need to write instead of re-implementing boilerplate code such as
creating ArgParsers, reading configuration files, handling dependencies, implementing type validation, 
maintaining traceability, etc.

`spock` configurations are normal python classes that are decorated with `@spock`. It supports 
inheritance, dynamic class dependencies, loading/saving configurations from/to multiple markdown formats, automatically 
generating CLI arguments, and hierarchical configuration by composition.

## 💥 Why You Should Use Spock 💥 

* Simple organized parameter definitions (i.e. a single line)
* Type checked (static-eqsue) & frozen parameters (i.e. fail early during long ML training runs)
* Complex parameter dependencies made simple (i.e. `@spock` class with a parameter that is an Enum of other 
`@spock` classes)
* Fully serializable parameter state(s) (i.e. exactly reproduce prior runtime parameter configurations)
* Automatic type checked CLI generation w/o argparser boilerplate (i.e click and/or typer for free!)
* Easily maintain parity between CLIs and Python APIs (i.e. single line changes between CLI and Python API definitions)
* Unified hyper-parameter definitions and interface (i.e. don't write different definitions for Ax or Optuna)
* Resolver that supports value definitions from environmental variables, dynamic template re-injection, and 
encryption of sensitive values

## Key Features

* [Simple Declaration](https://fidelity.github.io/spock/basics/Define): Type checked parameters are 
  defined within a `@spock` decorated class. Supports required/optional and automatic defaults.
* Easily Managed Parameter Groups: Each class automatically generates its own object within a single namespace.
* [Parameter Inheritance](https://fidelity.github.io/spock/advanced_features/Inheritance): Classes support 
  inheritance (w/ lazy evaluation of inheritance/dependencies) allowing for complex configurations derived from 
  a common base set of parameters.
* [Complex Types](https://fidelity.github.io/spock/advanced_features/Advanced-Types/): Nested Lists/Tuples, 
  List/Tuples of Enum of `@spock` classes, List of repeated `@spock` classes
* Multiple Configuration File Types: Configurations are specified from YAML, TOML, or JSON files.
* [Hierarchical Configuration](https://fidelity.github.io/spock/advanced_features/Composition/): Compose from 
  multiple configuration files via simple include statements.
* [Command-Line Overrides](https://fidelity.github.io/spock/advanced_features/Command-Line-Overrides/): Quickly 
  experiment by overriding a value with automatically generated command line arguments.
* Immutable: All classes are *frozen* preventing any misuse or accidental overwrites (to the extent they can be in 
  Python).
* [Tractability and Reproducibility](https://fidelity.github.io/spock/basics/Saving): Save runtime 
  parameter configuration to YAML, TOML, or JSON with a single chained command (with extra runtime info such as Git info, 
  Python version, machine FQDN, etc). The saved markdown file can be used as the configuration input to reproduce 
  prior runtime configurations.
* [Hyper-Parameter Tuner Addon](https://fidelity.github.io/spock/addons/tuner/About): Provides a unified
  interface for defining hyper-parameters (via `@spockTuner` decorator) that supports various tuning/algorithm 
  backends (currently: Optuna, Ax)
* [S3 Addon](https://fidelity.github.io/spock/addons/S3): Automatically detects `s3://` URI(s) and handles loading 
  and saving `spock` configuration files when an active `boto3.Session` is passed in (plus any additional 
  `S3Transfer` configurations)

## Quick Install

The basic install and `[s3]` extension require Python 3.6+ while the `[tune]` extension requires Python 3.7+

| Base | w/ S3 Extension | w/ Hyper-Parameter Tuner |
|------|-----------------|--------------------------|
| `pip install spock-config` | `pip install spock-config[s3]` | `pip install spock-config[tune]` |

## News/Releases

See [Releases](https://github.com/fidelity/spock/releases) for more information.

<html>
<h2 id="features"> 
  Recent Changes
</h2>
</html>

<details>

#### May 17th, 2022
* Added support for resolving value definitions from environmental variables with the following syntax, 
`${spock.env:name, default}`
* Added `.inject` annotation that will write back the original env notation to the saved output
* Added the `.crypto` annotation which provides a simple way to hide sensitive environmental
variables while still maintaining the written/loadable state of the spock config

#### March 17th, 2022
* Added support for `typing.Callable` types (includes advanced types such as `List[List[Callable]]`)
* Added support for `typing.Dict` types with type checking for types of both keys and values (includes advanced types
such as `Dict[str, Tuple[Callable, Callable]]`)
* Added support for post init hooks that allow for validation on parameters defined within `@spock` decorated classes. 
Additionally, added some common validation check to utils (within, greater than, less than, etc.)
* Updated unit tests to support Python 3.10

#### January 26th, 2022
* Added `evolve` support to the underlying `SpockBuilder` class. This provides functionality similar to the underlying
attrs library ([attrs.evolve](https://www.attrs.org/en/stable/api.html#attrs.evolve)). `evolve()` creates a new 
`Spockspace` instance based on differences between the underlying declared state and any passed in instantiated 
`@spock` decorated classes.

#### January 18th, 2022
* Support for lazy evaluation: (1) inherited classes do not need to be `@spock` decorated, (2) dependencies/references 
between `spock` classes can be lazily handled thus preventing the need for every `@spock` decorated classes to be 
passed into `*args` within the main `SpockBuilder` API
* Updated main API interface for better top-level imports (backwards compatible): `ConfigArgBuilder`->`SpockBuilder`
* Added stubs to the underlying decorator that should help with type hinting in VSCode (pylance/pyright)

</details>

## Original Implementation

`spock` was originally developed by the **Artificial Intelligence Center of Excellence at Fidelity Investments** by [Nicholas Cilfone](https://github.com/ncilfone) and [Siddharth Narayanan](https://github.com/sidnarayanan)





            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/fidelity/spock",
    "name": "spock-config",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.6",
    "maintainer_email": "",
    "keywords": "configuration,argparse,parameters,machine learning,deep learning,reproducibility,hyper-parameter tuning",
    "author": "FMR LLC",
    "author_email": "",
    "download_url": "https://files.pythonhosted.org/packages/1d/91/192ace87882d0676c1e74cf4dee8c8905970508d228db9445a79c62528ae/spock-config-2.5.2.tar.gz",
    "platform": null,
    "description": "<html>\n<h1 align=\"center\">\n    <a href=\"https://fidelity.github.io/spock/\"><img src=\"https://raw.githubusercontent.com/fidelity/spock/master/resources/images/logo.png\"/></a>\n    <h6 align=\"center\">Managing complex configurations any other way would be highly illogical...</h6>\n</h1>\n\n<p align=\"center\">\n  <a href=\"https://opensource.org/licenses/Apache-2.0\"><img src=\"https://img.shields.io/badge/License-Apache%202.0-9cf\"/></a>\n  <a href=\"https://bestpractices.coreinfrastructure.org/projects/5551\"><img src=\"https://bestpractices.coreinfrastructure.org/projects/5551/badge\"/></a>\n  <a><img src=\"https://github.com/fidelity/spock/workflows/pytest/badge.svg?branch=master\"/></a>\n<a href=\"https://coveralls.io/github/fidelity/spock?branch=master\"><img src=\"https://coveralls.io/repos/github/fidelity/spock/badge.svg?branch=master\"/></a>\n  <a><img src=\"https://github.com/fidelity/spock/workflows/docs/badge.svg\"/></a>\n</p>\n\n<p align=\"center\">\n  <a><img src=\"https://img.shields.io/badge/python-3.6+-informational.svg\"/></a>\n  <a href=\"https://github.com/psf/black\"><img src=\"https://img.shields.io/badge/code%20style-black-000000.svg\"/></a>\n  <a href=\"https://badge.fury.io/py/spock-config\"><img src=\"https://badge.fury.io/py/spock-config.svg\"/></a>\n  <a href=\"https://pepy.tech/project/spock-config\"><img src=\"https://static.pepy.tech/personalized-badge/spock-config?period=total&units=international_system&left_color=grey&right_color=orange&left_text=Downloads\"/></a>\n</p>\n\n<h3 align=\"center\">\n  <a href=\"https://fidelity.github.io/spock/Quick-Start\">Quick Start</a>\n  <span> \u00b7 </span>\n  <a href=\"https://fidelity.github.io/spock/\">Documentation</a>\n  <span> \u00b7 </span>\n  <a href=\"https://github.com/fidelity/spock/blob/master/examples\">Examples</a>\n  <span> \u00b7 </span>\n  <a href=\"https://github.com/fidelity/spock/releases\">Releases</a>\n</h3>\n\n</html>\n\n## About\n\n`spock` is a framework that helps users easily define, manage, and use complex parameter configurations within Python \napplications. It lets you focus on the code you need to write instead of re-implementing boilerplate code such as\ncreating ArgParsers, reading configuration files, handling dependencies, implementing type validation, \nmaintaining traceability, etc.\n\n`spock` configurations are normal python classes that are decorated with `@spock`. It supports \ninheritance, dynamic class dependencies, loading/saving configurations from/to multiple markdown formats, automatically \ngenerating CLI arguments, and hierarchical configuration by composition.\n\n## \ud83d\udca5 Why You Should Use Spock \ud83d\udca5 \n\n* Simple organized parameter definitions (i.e. a single line)\n* Type checked (static-eqsue) & frozen parameters (i.e. fail early during long ML training runs)\n* Complex parameter dependencies made simple (i.e. `@spock` class with a parameter that is an Enum of other \n`@spock` classes)\n* Fully serializable parameter state(s) (i.e. exactly reproduce prior runtime parameter configurations)\n* Automatic type checked CLI generation w/o argparser boilerplate (i.e click and/or typer for free!)\n* Easily maintain parity between CLIs and Python APIs (i.e. single line changes between CLI and Python API definitions)\n* Unified hyper-parameter definitions and interface (i.e. don't write different definitions for Ax or Optuna)\n* Resolver that supports value definitions from environmental variables, dynamic template re-injection, and \nencryption of sensitive values\n\n## Key Features\n\n* [Simple Declaration](https://fidelity.github.io/spock/basics/Define): Type checked parameters are \n  defined within a `@spock` decorated class. Supports required/optional and automatic defaults.\n* Easily Managed Parameter Groups: Each class automatically generates its own object within a single namespace.\n* [Parameter Inheritance](https://fidelity.github.io/spock/advanced_features/Inheritance): Classes support \n  inheritance (w/ lazy evaluation of inheritance/dependencies) allowing for complex configurations derived from \n  a common base set of parameters.\n* [Complex Types](https://fidelity.github.io/spock/advanced_features/Advanced-Types/): Nested Lists/Tuples, \n  List/Tuples of Enum of `@spock` classes, List of repeated `@spock` classes\n* Multiple Configuration File Types: Configurations are specified from YAML, TOML, or JSON files.\n* [Hierarchical Configuration](https://fidelity.github.io/spock/advanced_features/Composition/): Compose from \n  multiple configuration files via simple include statements.\n* [Command-Line Overrides](https://fidelity.github.io/spock/advanced_features/Command-Line-Overrides/): Quickly \n  experiment by overriding a value with automatically generated command line arguments.\n* Immutable: All classes are *frozen* preventing any misuse or accidental overwrites (to the extent they can be in \n  Python).\n* [Tractability and Reproducibility](https://fidelity.github.io/spock/basics/Saving): Save runtime \n  parameter configuration to YAML, TOML, or JSON with a single chained command (with extra runtime info such as Git info, \n  Python version, machine FQDN, etc). The saved markdown file can be used as the configuration input to reproduce \n  prior runtime configurations.\n* [Hyper-Parameter Tuner Addon](https://fidelity.github.io/spock/addons/tuner/About): Provides a unified\n  interface for defining hyper-parameters (via `@spockTuner` decorator) that supports various tuning/algorithm \n  backends (currently: Optuna, Ax)\n* [S3 Addon](https://fidelity.github.io/spock/addons/S3): Automatically detects `s3://` URI(s) and handles loading \n  and saving `spock` configuration files when an active `boto3.Session` is passed in (plus any additional \n  `S3Transfer` configurations)\n\n## Quick Install\n\nThe basic install and `[s3]` extension require Python 3.6+ while the `[tune]` extension requires Python 3.7+\n\n| Base | w/ S3 Extension | w/ Hyper-Parameter Tuner |\n|------|-----------------|--------------------------|\n| `pip install spock-config` | `pip install spock-config[s3]` | `pip install spock-config[tune]` |\n\n## News/Releases\n\nSee [Releases](https://github.com/fidelity/spock/releases) for more information.\n\n<html>\n<h2 id=\"features\"> \n  Recent Changes\n</h2>\n</html>\n\n<details>\n\n#### May 17th, 2022\n* Added support for resolving value definitions from environmental variables with the following syntax, \n`${spock.env:name, default}`\n* Added `.inject` annotation that will write back the original env notation to the saved output\n* Added the `.crypto` annotation which provides a simple way to hide sensitive environmental\nvariables while still maintaining the written/loadable state of the spock config\n\n#### March 17th, 2022\n* Added support for `typing.Callable` types (includes advanced types such as `List[List[Callable]]`)\n* Added support for `typing.Dict` types with type checking for types of both keys and values (includes advanced types\nsuch as `Dict[str, Tuple[Callable, Callable]]`)\n* Added support for post init hooks that allow for validation on parameters defined within `@spock` decorated classes. \nAdditionally, added some common validation check to utils (within, greater than, less than, etc.)\n* Updated unit tests to support Python 3.10\n\n#### January 26th, 2022\n* Added `evolve` support to the underlying `SpockBuilder` class. This provides functionality similar to the underlying\nattrs library ([attrs.evolve](https://www.attrs.org/en/stable/api.html#attrs.evolve)). `evolve()` creates a new \n`Spockspace` instance based on differences between the underlying declared state and any passed in instantiated \n`@spock` decorated classes.\n\n#### January 18th, 2022\n* Support for lazy evaluation: (1) inherited classes do not need to be `@spock` decorated, (2) dependencies/references \nbetween `spock` classes can be lazily handled thus preventing the need for every `@spock` decorated classes to be \npassed into `*args` within the main `SpockBuilder` API\n* Updated main API interface for better top-level imports (backwards compatible): `ConfigArgBuilder`->`SpockBuilder`\n* Added stubs to the underlying decorator that should help with type hinting in VSCode (pylance/pyright)\n\n</details>\n\n## Original Implementation\n\n`spock` was originally developed by the **Artificial Intelligence Center of Excellence at Fidelity Investments** by [Nicholas Cilfone](https://github.com/ncilfone) and [Siddharth Narayanan](https://github.com/sidnarayanan)\n\n\n\n\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "Spock is a framework designed to help manage complex parameter configurations for Python applications",
    "version": "2.5.2",
    "split_keywords": [
        "configuration",
        "argparse",
        "parameters",
        "machine learning",
        "deep learning",
        "reproducibility",
        "hyper-parameter tuning"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "7f90b5c4be4e4b15330a392b1aff6fa2",
                "sha256": "3b0aaa0ce2c4b5531168362251f1ea613f61dcb5eb81e943339ea9065ed88de0"
            },
            "downloads": -1,
            "filename": "spock_config-2.5.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "7f90b5c4be4e4b15330a392b1aff6fa2",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 89683,
            "upload_time": "2022-05-17T18:12:31",
            "upload_time_iso_8601": "2022-05-17T18:12:31.990742Z",
            "url": "https://files.pythonhosted.org/packages/30/05/937b1d2041c9661fcaf2b361a197bf8e7cc6ba79c165805990c534f6e7ef/spock_config-2.5.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "705b422413de75024bc1734cd8c73033",
                "sha256": "34f97debddcb482c805f82b96dd519dd775fb64c61575647f9459bce2e814ddf"
            },
            "downloads": -1,
            "filename": "spock-config-2.5.2.tar.gz",
            "has_sig": false,
            "md5_digest": "705b422413de75024bc1734cd8c73033",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 90447,
            "upload_time": "2022-05-17T18:12:34",
            "upload_time_iso_8601": "2022-05-17T18:12:34.095367Z",
            "url": "https://files.pythonhosted.org/packages/1d/91/192ace87882d0676c1e74cf4dee8c8905970508d228db9445a79c62528ae/spock-config-2.5.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2022-05-17 18:12:34",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "fidelity",
    "github_project": "spock",
    "travis_ci": false,
    "coveralls": true,
    "github_actions": true,
    "lcname": "spock-config"
}
        
Elapsed time: 0.36630s