| Name | libinephany JSON |
| Version |
0.15.3
JSON |
| download |
| home_page | None |
| Summary | Inephany library containing code commonly used by multiple subpackages. |
| upload_time | 2025-08-20 09:29:38 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | Apache 2.0 |
| keywords |
libinephany
library
utilities
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Inephany Common Library
The Inephany Common Library (`libinephany`) is a core utility package that provides shared functionality, data models, and utilities used across multiple Inephany packages. It contains essential components for hyperparameter optimization, model observation, data serialization, and common utilities.
## Features
- **Pydantic Data Models**: Comprehensive schemas for hyperparameters, observations, and API communications
- **Utility Functions**: Common utilities for PyTorch, optimization, transforms, and more
- **Observation System**: Tools for collecting and managing model statistics and observations
- **Constants and Enums**: Standardized constants and enumerations for agent types, model families, and module types
- **AWS Integration**: Utilities for AWS services integration
- **Web Application Utilities**: Common web app functionality and endpoints
## Installation
### Prerequisites
- Python 3.10+
- Make (for build automation)
#### Ubuntu / Debian
```bash
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.12
```
#### MacOS with brew
```bash
brew install python@3.12
```
### For Developers (Monorepo)
If you're working within the Inephany monorepo, the package is already available and will be installed automatically when you run the installation commands in dependent packages.
### For Clients (Standalone Installation)
`libinephany` is available on PyPI and can be installed directly:
```bash
pip install libinephany
```
For development installations with additional dependencies:
```bash
pip install libinephany[dev]
```
## Key Components
### Pydantic Models
The package provides comprehensive data models for:
- **Hyperparameter Configurations**: `HParamConfig`, `HParamConfigs`
- **Observation Models**: `ObservationInputs`, tensor statistics
- **API Schemas**: Request/response models for client-server communication
- **State Management**: Hyperparameter states and update callbacks
### Utility Functions
#### Agent Utilities (`agent_utils.py`)
- Agent ID generation and parsing
- Hyperparameter group management
- Agent type validation
#### Constants (`constants.py`)
- Hyperparameter type constants (learning_rate, weight_decay, etc.)
- Agent prefixes and suffixes
- API key headers and timestamp formats
#### Enums (`enums.py`)
- `AgentTypes`: Learning rate, weight decay, dropout, etc.
- `ModelFamilies`: GPT, BERT, OLMo
- `ModuleTypes`: Convolutional, attention, linear, embedding
#### Optimization Utilities (`optim_utils.py`)
- PyTorch optimizer utilities
- Parameter group management
- Learning rate scheduler utilities
#### PyTorch Utilities (`torch_utils.py`)
- Tensor operations
- Model utilities
- Distributed training helpers
### Observation System
The observation system provides tools for collecting and managing model statistics:
- **StatisticManager**: Centralized statistics collection and management
- **ObserverPipeline**: Configurable observation pipelines
- **PipelineCoordinator**: Coordinates multiple observers
- **StatisticTrackers**: Specialized trackers for different metric types
## Usage Examples
### Basic Import Examples
```python
# Import common constants
from libinephany.utils.constants import LEARNING_RATE, WEIGHT_DECAY, AGENT_PREFIX_LR
# Import enums
from libinephany.utils.enums import AgentTypes, ModelFamilies, ModuleTypes
# Import utility functions
from libinephany.utils import agent_utils, optim_utils, torch_utils
# Import data models
from libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig
from libinephany.pydantic_models.schemas.response_schemas import ClientPolicySchemaResponse
```
### Working with Agent Types
```python
from libinephany.utils.enums import AgentTypes
# Check if an agent type is valid
agent_type = "learning_rate"
if agent_type in [agent.value for agent in AgentTypes]:
print(f"{agent_type} is a valid agent type")
# Get agent type by index
lr_agent = AgentTypes.get_from_index(0) # LearningRateAgent
```
### Using Constants
```python
from libinephany.utils.constants import AGENT_PREFIX_LR, LEARNING_RATE
# Generate agent ID
agent_id = f"{AGENT_PREFIX_LR}_agent_001"
hyperparam_type = LEARNING_RATE
```
### Working with Pydantic Models
```python
from libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig
# Create a hyperparameter configuration
config = HParamConfig(
name="learning_rate",
value=0.001,
min_value=1e-6,
max_value=1.0
)
```
## Development
### Running Tests
```bash
make execute-unit-tests
```
### Code Quality
```bash
make lint # Run all linters
make fix-black # Fix formatting
make fix-isort # Fix imports
```
### Version Management
```bash
make increment-patch-version # Increment patch version
make increment-minor-version # Increment minor version
make increment-major-version # Increment major version
make increment-pre-release-version # Increment pre-release version
```
## Dependencies
### Core Dependencies
- `pydantic==2.8.2` - Data validation and serialization
- `torch==2.7.1` - PyTorch for tensor operations
- `numpy==1.26.4` - Numerical computing
- `requests==2.32.4` - HTTP client
- `loguru==0.7.2` - Logging
### Optional Dependencies
- `boto3<=1.38.44` - AWS SDK
- `fastapi==0.115.11` - Web framework
- `slack-sdk==3.35.0` - Slack integration
- `transformers==4.52.4` - Hugging Face transformers
- `accelerate==1.4.0` - Hugging Face accelerate
- `gymnasium==1.0.0` - RL environments
## Troubleshooting
### Common Issues
1. **Import Errors**: Ensure you're in the virtual environment and have installed the package correctly.
2. **Version Conflicts**: If you encounter dependency conflicts, try installing in a fresh virtual environment:
```bash
python -m venv fresh_env
source fresh_env/bin/activate
make install-dev
```
3. **Make Command Not Found**: Ensure you have `make` installed on your system.
4. **Python Version Issues**: This package requires Python 3.12+. Ensure you're using the correct version.
### Getting Help
- Check the example scripts in the repository
- Review the test files for usage examples
- Ensure all dependencies are installed correctly
- Verify your Python version is 3.12+
## Contributing
When contributing to `libinephany`:
1. Follow the existing code style (Black, isort, flake8)
2. Add appropriate type hints
3. Include unit tests for new functionality
4. Update documentation for new features
5. Ensure all tests pass before submitting
## License
This package is licensed under the Apache License, Version 2.0. See the LICENSE file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "libinephany",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "libinephany, library, utilities",
"author": null,
"author_email": "Inephany <info@inephany.com>",
"download_url": "https://files.pythonhosted.org/packages/e4/39/3b00686923e46ccd251f67bafd7ad6282080c4b059afcf38c0456f12c497/libinephany-0.15.3.tar.gz",
"platform": null,
"description": "# Inephany Common Library\n\nThe Inephany Common Library (`libinephany`) is a core utility package that provides shared functionality, data models, and utilities used across multiple Inephany packages. It contains essential components for hyperparameter optimization, model observation, data serialization, and common utilities.\n\n## Features\n\n- **Pydantic Data Models**: Comprehensive schemas for hyperparameters, observations, and API communications\n- **Utility Functions**: Common utilities for PyTorch, optimization, transforms, and more\n- **Observation System**: Tools for collecting and managing model statistics and observations\n- **Constants and Enums**: Standardized constants and enumerations for agent types, model families, and module types\n- **AWS Integration**: Utilities for AWS services integration\n- **Web Application Utilities**: Common web app functionality and endpoints\n\n## Installation\n\n### Prerequisites\n\n- Python 3.10+\n- Make (for build automation)\n\n#### Ubuntu / Debian\n```bash\nsudo add-apt-repository ppa:deadsnakes/ppa\nsudo apt update\nsudo apt install python3.12\n```\n\n#### MacOS with brew\n```bash\nbrew install python@3.12\n```\n\n### For Developers (Monorepo)\n\nIf you're working within the Inephany monorepo, the package is already available and will be installed automatically when you run the installation commands in dependent packages.\n\n### For Clients (Standalone Installation)\n\n`libinephany` is available on PyPI and can be installed directly:\n\n```bash\npip install libinephany\n```\n\nFor development installations with additional dependencies:\n\n```bash\npip install libinephany[dev]\n```\n\n## Key Components\n\n### Pydantic Models\n\nThe package provides comprehensive data models for:\n\n- **Hyperparameter Configurations**: `HParamConfig`, `HParamConfigs`\n- **Observation Models**: `ObservationInputs`, tensor statistics\n- **API Schemas**: Request/response models for client-server communication\n- **State Management**: Hyperparameter states and update callbacks\n\n### Utility Functions\n\n#### Agent Utilities (`agent_utils.py`)\n- Agent ID generation and parsing\n- Hyperparameter group management\n- Agent type validation\n\n#### Constants (`constants.py`)\n- Hyperparameter type constants (learning_rate, weight_decay, etc.)\n- Agent prefixes and suffixes\n- API key headers and timestamp formats\n\n#### Enums (`enums.py`)\n- `AgentTypes`: Learning rate, weight decay, dropout, etc.\n- `ModelFamilies`: GPT, BERT, OLMo\n- `ModuleTypes`: Convolutional, attention, linear, embedding\n\n#### Optimization Utilities (`optim_utils.py`)\n- PyTorch optimizer utilities\n- Parameter group management\n- Learning rate scheduler utilities\n\n#### PyTorch Utilities (`torch_utils.py`)\n- Tensor operations\n- Model utilities\n- Distributed training helpers\n\n### Observation System\n\nThe observation system provides tools for collecting and managing model statistics:\n\n- **StatisticManager**: Centralized statistics collection and management\n- **ObserverPipeline**: Configurable observation pipelines\n- **PipelineCoordinator**: Coordinates multiple observers\n- **StatisticTrackers**: Specialized trackers for different metric types\n\n## Usage Examples\n\n### Basic Import Examples\n\n```python\n# Import common constants\nfrom libinephany.utils.constants import LEARNING_RATE, WEIGHT_DECAY, AGENT_PREFIX_LR\n\n# Import enums\nfrom libinephany.utils.enums import AgentTypes, ModelFamilies, ModuleTypes\n\n# Import utility functions\nfrom libinephany.utils import agent_utils, optim_utils, torch_utils\n\n# Import data models\nfrom libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig\nfrom libinephany.pydantic_models.schemas.response_schemas import ClientPolicySchemaResponse\n```\n\n### Working with Agent Types\n\n```python\nfrom libinephany.utils.enums import AgentTypes\n\n# Check if an agent type is valid\nagent_type = \"learning_rate\"\nif agent_type in [agent.value for agent in AgentTypes]:\n print(f\"{agent_type} is a valid agent type\")\n\n# Get agent type by index\nlr_agent = AgentTypes.get_from_index(0) # LearningRateAgent\n```\n\n### Using Constants\n\n```python\nfrom libinephany.utils.constants import AGENT_PREFIX_LR, LEARNING_RATE\n\n# Generate agent ID\nagent_id = f\"{AGENT_PREFIX_LR}_agent_001\"\nhyperparam_type = LEARNING_RATE\n```\n\n### Working with Pydantic Models\n\n```python\nfrom libinephany.pydantic_models.configs.hyperparameter_configs import HParamConfig\n\n# Create a hyperparameter configuration\nconfig = HParamConfig(\n name=\"learning_rate\",\n value=0.001,\n min_value=1e-6,\n max_value=1.0\n)\n```\n\n## Development\n\n### Running Tests\n```bash\nmake execute-unit-tests\n```\n\n### Code Quality\n```bash\nmake lint # Run all linters\nmake fix-black # Fix formatting\nmake fix-isort # Fix imports\n```\n\n### Version Management\n```bash\nmake increment-patch-version # Increment patch version\nmake increment-minor-version # Increment minor version\nmake increment-major-version # Increment major version\nmake increment-pre-release-version # Increment pre-release version\n```\n\n## Dependencies\n\n### Core Dependencies\n- `pydantic==2.8.2` - Data validation and serialization\n- `torch==2.7.1` - PyTorch for tensor operations\n- `numpy==1.26.4` - Numerical computing\n- `requests==2.32.4` - HTTP client\n- `loguru==0.7.2` - Logging\n\n### Optional Dependencies\n- `boto3<=1.38.44` - AWS SDK\n- `fastapi==0.115.11` - Web framework\n- `slack-sdk==3.35.0` - Slack integration\n- `transformers==4.52.4` - Hugging Face transformers\n- `accelerate==1.4.0` - Hugging Face accelerate\n- `gymnasium==1.0.0` - RL environments\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Import Errors**: Ensure you're in the virtual environment and have installed the package correctly.\n\n2. **Version Conflicts**: If you encounter dependency conflicts, try installing in a fresh virtual environment:\n ```bash\n python -m venv fresh_env\n source fresh_env/bin/activate\n make install-dev\n ```\n\n3. **Make Command Not Found**: Ensure you have `make` installed on your system.\n\n4. **Python Version Issues**: This package requires Python 3.12+. Ensure you're using the correct version.\n\n### Getting Help\n\n- Check the example scripts in the repository\n- Review the test files for usage examples\n- Ensure all dependencies are installed correctly\n- Verify your Python version is 3.12+\n\n## Contributing\n\nWhen contributing to `libinephany`:\n\n1. Follow the existing code style (Black, isort, flake8)\n2. Add appropriate type hints\n3. Include unit tests for new functionality\n4. Update documentation for new features\n5. Ensure all tests pass before submitting\n\n## License\n\nThis package is licensed under the Apache License, Version 2.0. See the LICENSE file for details.\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Inephany library containing code commonly used by multiple subpackages.",
"version": "0.15.3",
"project_urls": null,
"split_keywords": [
"libinephany",
" library",
" utilities"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "23b50fd34ac70ca16d81eedf2779df5cd9bfbef812e05dbb7ba59ef40758d055",
"md5": "d46c8a18ae0ad77bcc0b65b261f2e617",
"sha256": "5b9a3e25040a2fa43ba68ce76f0219d768923d3b0a3a77494598098f5ee4fae6"
},
"downloads": -1,
"filename": "libinephany-0.15.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d46c8a18ae0ad77bcc0b65b261f2e617",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 76758,
"upload_time": "2025-08-20T09:29:36",
"upload_time_iso_8601": "2025-08-20T09:29:36.924968Z",
"url": "https://files.pythonhosted.org/packages/23/b5/0fd34ac70ca16d81eedf2779df5cd9bfbef812e05dbb7ba59ef40758d055/libinephany-0.15.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "e4393b00686923e46ccd251f67bafd7ad6282080c4b059afcf38c0456f12c497",
"md5": "db3431192888a5c0b382f2483477a5a9",
"sha256": "641cf501ccbde304b2a813ac4c741ba0ad9951f8f2ba6d9d713b73cf2bf0506b"
},
"downloads": -1,
"filename": "libinephany-0.15.3.tar.gz",
"has_sig": false,
"md5_digest": "db3431192888a5c0b382f2483477a5a9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 61469,
"upload_time": "2025-08-20T09:29:38",
"upload_time_iso_8601": "2025-08-20T09:29:38.241503Z",
"url": "https://files.pythonhosted.org/packages/e4/39/3b00686923e46ccd251f67bafd7ad6282080c4b059afcf38c0456f12c497/libinephany-0.15.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-20 09:29:38",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "libinephany"
}
