# autoCLI_config
**Automatic Configuration and Documentation System for Python**
A powerful system for managing configurations and auto-generating documentation in Python applications. Define your documentation once in configuration dataclasses and automatically propagate it to CLI help text, Python docstrings, and API documentation.
## Features
- **Single Source of Truth**: Define parameter documentation once in `PARAM_DOCS` dictionaries
- **Automatic CLI Generation**: Works seamlessly with `argParseFromDoc` to generate rich CLI help
- **Config-Backed Defaults**: Bind function parameters to configuration dataclass attributes
- **Flexible Overrides**: Support for YAML files and command-line config overrides (e.g., `--config train.n_epochs=100`)
- **Type Safety**: Automatic type checking and conversion for config values
- **Hierarchical Configs**: Support for nested configuration dataclasses
- **Validation & Transformation**: Optional validators and transformers for parameter values
## Installation
```bash
pip install autoCLI_config
```
Or install from source:
```bash
git clone https://github.com/rsanchezgarc/autoCLI_config.git
cd autoCLI_config
pip install -e .
```
## Quick Start
### Basic Example
```python
from dataclasses import dataclass
from autoCLI_config import (
CONFIG_PARAM,
inject_defaults_from_config,
inject_docs_from_config_params
)
# 1. Define your configuration with PARAM_DOCS
@dataclass
class TrainConfig:
"""Training configuration."""
PARAM_DOCS = {
'learning_rate': 'Learning rate for the optimizer',
'batch_size': 'Number of samples per training batch',
'n_epochs': 'Number of training epochs',
}
learning_rate: float = 0.001
batch_size: int = 32
n_epochs: int = 100
# Create config instance
config = TrainConfig()
# 2. Use decorators to inject config and docs
@inject_docs_from_config_params
@inject_defaults_from_config(config, update_config_with_args=True)
def train(
learning_rate: float = CONFIG_PARAM(),
batch_size: int = CONFIG_PARAM(),
n_epochs: int = CONFIG_PARAM()
):
"""
Train a machine learning model.
Args:
learning_rate: {learning_rate}
batch_size: {batch_size}
n_epochs: {n_epochs}
"""
print(f"Training with lr={learning_rate}, batch_size={batch_size}, epochs={n_epochs}")
# Use the function
train() # Uses config defaults
train(learning_rate=0.01) # Override specific params
```
### CLI Integration
```python
from autoCLI_config import ConfigArgumentParser
# Create parser with config integration
parser = ConfigArgumentParser(config_obj=config, verbose=True)
parser.add_args_from_function(train)
args, config_args = parser.parse_args()
# Call function with parsed args
train(**vars(args))
```
Command-line usage:
```bash
# Use defaults from config
python train.py
# Override individual parameters
python train.py --learning-rate 0.01 --batch-size 64
# Override via config system (dot notation)
python train.py --config learning_rate=0.01 batch_size=64
# Load config from YAML file
python train.py --config my_config.yaml
# Show all available config options
python train.py --show-config
# Export current config to YAML
python train.py --export-config output.yaml
```
## Advanced Features
### Hierarchical Configurations
```python
from dataclasses import dataclass, field
@dataclass
class OptimizerConfig:
PARAM_DOCS = {
'learning_rate': 'Learning rate for optimizer',
'weight_decay': 'L2 regularization weight decay',
}
learning_rate: float = 0.001
weight_decay: float = 1e-5
@dataclass
class DataConfig:
PARAM_DOCS = {
'batch_size': 'Number of samples per batch',
'num_workers': 'Number of data loading workers',
}
batch_size: int = 32
num_workers: int = 4
@dataclass
class MainConfig:
optimizer: OptimizerConfig = field(default_factory=OptimizerConfig)
data: DataConfig = field(default_factory=DataConfig)
config = MainConfig()
# Access nested config values
@inject_defaults_from_config(config.optimizer)
def train(learning_rate: float = CONFIG_PARAM()):
"""Train with optimizer config."""
pass
# Or reference across configs
@inject_defaults_from_config(config.optimizer)
def train_advanced(
learning_rate: float = CONFIG_PARAM(), # from optimizer config
batch_size: int = CONFIG_PARAM(config=config.data) # from data config
):
"""Train with multiple configs."""
pass
```
### Config Overrides
Override nested config values from command line:
```bash
# Dot notation for nested configs
python train.py --config optimizer.learning_rate=0.01 data.batch_size=64
# Mix YAML and individual overrides
python train.py --config base_config.yaml optimizer.learning_rate=0.01
```
### Validators and Transformers
```python
def positive_int(x: int) -> bool:
return isinstance(x, int) and x > 0
def to_float(x) -> float:
return float(x)
@inject_defaults_from_config(config)
def train(
batch_size: int = CONFIG_PARAM(
validator=positive_int,
doc="Batch size (must be positive)"
),
learning_rate: float = CONFIG_PARAM(
transform=to_float,
doc="Learning rate"
)
):
"""Train with validation."""
pass
```
### Enum Support
```python
from enum import Enum
class OptimizerType(str, Enum):
ADAM = "adam"
SGD = "sgd"
RMSPROP = "rmsprop"
@dataclass
class TrainConfig:
PARAM_DOCS = {
'optimizer': 'Optimizer algorithm to use',
}
optimizer: OptimizerType = OptimizerType.ADAM
# Automatic string-to-enum conversion
python train.py --optimizer sgd # Converts "sgd" to OptimizerType.SGD
```
## How It Works
The system consists of three main components:
### 1. CONFIG_PARAM Descriptor
A descriptor class that binds function parameters to configuration attributes:
```python
class CONFIG_PARAM:
def __init__(self, validator=None, transform=None, doc=None, config=None):
# Stores validation, transformation, and documentation
...
```
### 2. Decorators
**`inject_defaults_from_config(config, update_config_with_args=False)`**
- Binds CONFIG_PARAM instances to config attributes
- Provides default values from configuration
- Optionally updates config when function is called with new values
**`inject_docs_from_config_params(func)`**
- Injects parameter documentation into function docstrings
- Replaces `{param_name}` placeholders with actual documentation
- Works for both CONFIG_PARAM and regular parameters
### 3. Configuration System
**`ConfigOverrideSystem`**
- Parses config assignments (`key=value`)
- Loads YAML configuration files
- Applies overrides to config objects
- Handles type conversion (int, float, Path, Enum, etc.)
**`ConfigArgumentParser`**
- Extends `argParseFromDoc.AutoArgumentParser`
- Adds `--config`, `--show-config`, and `--export-config` arguments
- Manages precedence: direct args > `--config` > defaults
- Detects and reports conflicts
## Documentation Flow
1. **Define** documentation in `PARAM_DOCS` dictionary in config dataclass
2. **Bind** CONFIG_PARAMs to config attributes via `inject_defaults_from_config`
3. **Inject** documentation into docstrings via `inject_docs_from_config_params`
4. **Generate** CLI help automatically with `ConfigArgumentParser`
```
PARAM_DOCS (config class)
↓
CONFIG_PARAM.doc (auto-populated)
↓
Function docstring (via {placeholder})
↓
CLI help text (via argParseFromDoc)
```
## API Reference
### Core Classes
- `CONFIG_PARAM`: Descriptor for config-backed parameters
- `ConfigOverrideSystem`: Utility class for config manipulation
- `ConfigArgumentParser`: CLI parser with config integration
### Decorators
- `inject_defaults_from_config(config, update_config_with_args=False)`
- `inject_docs_from_config_params(func)`
### Utilities
- `merge_dicts(d1, d2)`: Recursively merge dictionaries
- `dataclass_to_dict(obj)`: Convert dataclass to dictionary
- `export_config_to_yaml(config, filepath)`: Export config to YAML
- `check_type_match(expected_type, actual_value)`: Type checking utility
## Performance overhead
Using the decorators adds a non-negligible overhead to each function call, hence we only recommend using them in class builders and outer-loop functions.
## Examples
See the `examples/` directory for complete working examples:
- `basic_example.py`: Simple configuration and CLI
- `multi_config_example.py`: Hierarchical configurations
- `full_workflow.py`: Complete training script pattern
## Testing
Run tests with pytest:
```bash
# Install dev dependencies
pip install -e ".[dev]"
# Run tests
pytest
# Run with coverage
pytest --cov=autoCLI_config --cov-report=term-missing
```
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## License
MIT License
## Author
Ruben Sanchez-Garcia (rsanchezgarcia@faculty.ie.edu)
## Acknowledgments
This system was originally developed as part of the [cryoPARES](https://github.com/rsanchezgarc/cryoPARES) project for cryo-EM structure determination, and has been extracted into a standalone package for broader use.
Raw data
{
"_id": null,
"home_page": "https://github.com/rsanchezgarc/autoCLI_config",
"name": "autoCLI-config",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "configuration, cli, documentation, argparse, dataclass",
"author": "Ruben Sanchez-Garcia",
"author_email": "Ruben Sanchez-Garcia <rsanchezgarcia@faculty.ie.edu>",
"download_url": "https://files.pythonhosted.org/packages/d8/26/d56d18d33e2995747329231349f885239110c91bcf410af23c1d8a96b2ba/autocli_config-0.1.5.tar.gz",
"platform": null,
"description": "# autoCLI_config\n\n**Automatic Configuration and Documentation System for Python**\n\nA powerful system for managing configurations and auto-generating documentation in Python applications. Define your documentation once in configuration dataclasses and automatically propagate it to CLI help text, Python docstrings, and API documentation.\n\n## Features\n\n- **Single Source of Truth**: Define parameter documentation once in `PARAM_DOCS` dictionaries\n- **Automatic CLI Generation**: Works seamlessly with `argParseFromDoc` to generate rich CLI help\n- **Config-Backed Defaults**: Bind function parameters to configuration dataclass attributes\n- **Flexible Overrides**: Support for YAML files and command-line config overrides (e.g., `--config train.n_epochs=100`)\n- **Type Safety**: Automatic type checking and conversion for config values\n- **Hierarchical Configs**: Support for nested configuration dataclasses\n- **Validation & Transformation**: Optional validators and transformers for parameter values\n\n## Installation\n\n```bash\npip install autoCLI_config\n```\n\nOr install from source:\n\n```bash\ngit clone https://github.com/rsanchezgarc/autoCLI_config.git\ncd autoCLI_config\npip install -e .\n```\n\n## Quick Start\n\n### Basic Example\n\n```python\nfrom dataclasses import dataclass\nfrom autoCLI_config import (\n CONFIG_PARAM,\n inject_defaults_from_config,\n inject_docs_from_config_params\n)\n\n# 1. Define your configuration with PARAM_DOCS\n@dataclass\nclass TrainConfig:\n \"\"\"Training configuration.\"\"\"\n\n PARAM_DOCS = {\n 'learning_rate': 'Learning rate for the optimizer',\n 'batch_size': 'Number of samples per training batch',\n 'n_epochs': 'Number of training epochs',\n }\n\n learning_rate: float = 0.001\n batch_size: int = 32\n n_epochs: int = 100\n\n# Create config instance\nconfig = TrainConfig()\n\n# 2. Use decorators to inject config and docs\n@inject_docs_from_config_params\n@inject_defaults_from_config(config, update_config_with_args=True)\ndef train(\n learning_rate: float = CONFIG_PARAM(),\n batch_size: int = CONFIG_PARAM(),\n n_epochs: int = CONFIG_PARAM()\n):\n \"\"\"\n Train a machine learning model.\n\n Args:\n learning_rate: {learning_rate}\n batch_size: {batch_size}\n n_epochs: {n_epochs}\n \"\"\"\n print(f\"Training with lr={learning_rate}, batch_size={batch_size}, epochs={n_epochs}\")\n\n# Use the function\ntrain() # Uses config defaults\ntrain(learning_rate=0.01) # Override specific params\n```\n\n### CLI Integration\n\n```python\nfrom autoCLI_config import ConfigArgumentParser\n\n# Create parser with config integration\nparser = ConfigArgumentParser(config_obj=config, verbose=True)\nparser.add_args_from_function(train)\nargs, config_args = parser.parse_args()\n\n# Call function with parsed args\ntrain(**vars(args))\n```\n\nCommand-line usage:\n\n```bash\n# Use defaults from config\npython train.py\n\n# Override individual parameters\npython train.py --learning-rate 0.01 --batch-size 64\n\n# Override via config system (dot notation)\npython train.py --config learning_rate=0.01 batch_size=64\n\n# Load config from YAML file\npython train.py --config my_config.yaml\n\n# Show all available config options\npython train.py --show-config\n\n# Export current config to YAML\npython train.py --export-config output.yaml\n```\n\n## Advanced Features\n\n### Hierarchical Configurations\n\n```python\nfrom dataclasses import dataclass, field\n\n@dataclass\nclass OptimizerConfig:\n PARAM_DOCS = {\n 'learning_rate': 'Learning rate for optimizer',\n 'weight_decay': 'L2 regularization weight decay',\n }\n learning_rate: float = 0.001\n weight_decay: float = 1e-5\n\n@dataclass\nclass DataConfig:\n PARAM_DOCS = {\n 'batch_size': 'Number of samples per batch',\n 'num_workers': 'Number of data loading workers',\n }\n batch_size: int = 32\n num_workers: int = 4\n\n@dataclass\nclass MainConfig:\n optimizer: OptimizerConfig = field(default_factory=OptimizerConfig)\n data: DataConfig = field(default_factory=DataConfig)\n\nconfig = MainConfig()\n\n# Access nested config values\n@inject_defaults_from_config(config.optimizer)\ndef train(learning_rate: float = CONFIG_PARAM()):\n \"\"\"Train with optimizer config.\"\"\"\n pass\n\n# Or reference across configs\n@inject_defaults_from_config(config.optimizer)\ndef train_advanced(\n learning_rate: float = CONFIG_PARAM(), # from optimizer config\n batch_size: int = CONFIG_PARAM(config=config.data) # from data config\n):\n \"\"\"Train with multiple configs.\"\"\"\n pass\n```\n\n### Config Overrides\n\nOverride nested config values from command line:\n\n```bash\n# Dot notation for nested configs\npython train.py --config optimizer.learning_rate=0.01 data.batch_size=64\n\n# Mix YAML and individual overrides\npython train.py --config base_config.yaml optimizer.learning_rate=0.01\n```\n\n### Validators and Transformers\n\n```python\ndef positive_int(x: int) -> bool:\n return isinstance(x, int) and x > 0\n\ndef to_float(x) -> float:\n return float(x)\n\n@inject_defaults_from_config(config)\ndef train(\n batch_size: int = CONFIG_PARAM(\n validator=positive_int,\n doc=\"Batch size (must be positive)\"\n ),\n learning_rate: float = CONFIG_PARAM(\n transform=to_float,\n doc=\"Learning rate\"\n )\n):\n \"\"\"Train with validation.\"\"\"\n pass\n```\n\n### Enum Support\n\n```python\nfrom enum import Enum\n\nclass OptimizerType(str, Enum):\n ADAM = \"adam\"\n SGD = \"sgd\"\n RMSPROP = \"rmsprop\"\n\n@dataclass\nclass TrainConfig:\n PARAM_DOCS = {\n 'optimizer': 'Optimizer algorithm to use',\n }\n optimizer: OptimizerType = OptimizerType.ADAM\n\n# Automatic string-to-enum conversion\npython train.py --optimizer sgd # Converts \"sgd\" to OptimizerType.SGD\n```\n\n## How It Works\n\nThe system consists of three main components:\n\n### 1. CONFIG_PARAM Descriptor\n\nA descriptor class that binds function parameters to configuration attributes:\n\n```python\nclass CONFIG_PARAM:\n def __init__(self, validator=None, transform=None, doc=None, config=None):\n # Stores validation, transformation, and documentation\n ...\n```\n\n### 2. Decorators\n\n**`inject_defaults_from_config(config, update_config_with_args=False)`**\n- Binds CONFIG_PARAM instances to config attributes\n- Provides default values from configuration\n- Optionally updates config when function is called with new values\n\n**`inject_docs_from_config_params(func)`**\n- Injects parameter documentation into function docstrings\n- Replaces `{param_name}` placeholders with actual documentation\n- Works for both CONFIG_PARAM and regular parameters\n\n### 3. Configuration System\n\n**`ConfigOverrideSystem`**\n- Parses config assignments (`key=value`)\n- Loads YAML configuration files\n- Applies overrides to config objects\n- Handles type conversion (int, float, Path, Enum, etc.)\n\n**`ConfigArgumentParser`**\n- Extends `argParseFromDoc.AutoArgumentParser`\n- Adds `--config`, `--show-config`, and `--export-config` arguments\n- Manages precedence: direct args > `--config` > defaults\n- Detects and reports conflicts\n\n## Documentation Flow\n\n1. **Define** documentation in `PARAM_DOCS` dictionary in config dataclass\n2. **Bind** CONFIG_PARAMs to config attributes via `inject_defaults_from_config`\n3. **Inject** documentation into docstrings via `inject_docs_from_config_params`\n4. **Generate** CLI help automatically with `ConfigArgumentParser`\n\n```\nPARAM_DOCS (config class)\n \u2193\nCONFIG_PARAM.doc (auto-populated)\n \u2193\nFunction docstring (via {placeholder})\n \u2193\nCLI help text (via argParseFromDoc)\n```\n\n## API Reference\n\n### Core Classes\n\n- `CONFIG_PARAM`: Descriptor for config-backed parameters\n- `ConfigOverrideSystem`: Utility class for config manipulation\n- `ConfigArgumentParser`: CLI parser with config integration\n\n### Decorators\n\n- `inject_defaults_from_config(config, update_config_with_args=False)`\n- `inject_docs_from_config_params(func)`\n\n### Utilities\n\n- `merge_dicts(d1, d2)`: Recursively merge dictionaries\n- `dataclass_to_dict(obj)`: Convert dataclass to dictionary\n- `export_config_to_yaml(config, filepath)`: Export config to YAML\n- `check_type_match(expected_type, actual_value)`: Type checking utility\n\n\n## Performance overhead\nUsing the decorators adds a non-negligible overhead to each function call, hence we only recommend using them in class builders and outer-loop functions.\n\n## Examples\n\nSee the `examples/` directory for complete working examples:\n\n- `basic_example.py`: Simple configuration and CLI\n- `multi_config_example.py`: Hierarchical configurations\n- `full_workflow.py`: Complete training script pattern\n\n## Testing\n\nRun tests with pytest:\n\n```bash\n# Install dev dependencies\npip install -e \".[dev]\"\n\n# Run tests\npytest\n\n# Run with coverage\npytest --cov=autoCLI_config --cov-report=term-missing\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nMIT License\n\n## Author\n\nRuben Sanchez-Garcia (rsanchezgarcia@faculty.ie.edu)\n\n## Acknowledgments\n\nThis system was originally developed as part of the [cryoPARES](https://github.com/rsanchezgarc/cryoPARES) project for cryo-EM structure determination, and has been extracted into a standalone package for broader use.\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Automatic configuration management and documentation generation system",
"version": "0.1.5",
"project_urls": {
"Bug Reports": "https://github.com/rsanchezgarc/autoCLI_config/issues",
"Homepage": "https://github.com/rsanchezgarc/autoCLI_config",
"Source": "https://github.com/rsanchezgarc/autoCLI_config"
},
"split_keywords": [
"configuration",
" cli",
" documentation",
" argparse",
" dataclass"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "27126cd952a3a3108a45005f808e6aed2fcebd468cd267ef3f1540d1e223ff07",
"md5": "35c63f5aac42d8b89e8772a35334c4a2",
"sha256": "9d3d87ca0811f88c0d90459e44491dc48a0984d050c3937bd4f52c3d8fadd67e"
},
"downloads": -1,
"filename": "autocli_config-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "35c63f5aac42d8b89e8772a35334c4a2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 32484,
"upload_time": "2025-11-03T20:07:20",
"upload_time_iso_8601": "2025-11-03T20:07:20.086757Z",
"url": "https://files.pythonhosted.org/packages/27/12/6cd952a3a3108a45005f808e6aed2fcebd468cd267ef3f1540d1e223ff07/autocli_config-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d826d56d18d33e2995747329231349f885239110c91bcf410af23c1d8a96b2ba",
"md5": "495071cb9e59e8210311d56f450f3705",
"sha256": "5c42fd3fd3a79605071be857aa4e6b6e3ececed3d7229435bb5a0db66c0f6c8a"
},
"downloads": -1,
"filename": "autocli_config-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "495071cb9e59e8210311d56f450f3705",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 44049,
"upload_time": "2025-11-03T20:07:21",
"upload_time_iso_8601": "2025-11-03T20:07:21.073002Z",
"url": "https://files.pythonhosted.org/packages/d8/26/d56d18d33e2995747329231349f885239110c91bcf410af23c1d8a96b2ba/autocli_config-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-11-03 20:07:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rsanchezgarc",
"github_project": "autoCLI_config",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "argParseFromDoc",
"specs": [
[
">=",
"0.0.8"
]
]
},
{
"name": "PyYAML",
"specs": [
[
">=",
"5.1"
]
]
}
],
"lcname": "autocli-config"
}
JSON
