# SnapperML
[](https://snapperml.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.org/project/snapper-ml/)
SnapperML is a comprehensive framework for experiment tracking and machine learning operationalization (MLOps), built using well-supported technologies like [Mlflow](https://mlflow.org/), [Ray](https://github.com/ray-project/ray/), Docker, and more. It provides an opinionated workflow designed to facilitate both local and cloud-based experimentation.
## Key Features
- **Automatic Tracking**: Seamless integration with MLflow for parameter and metric tracking.
- **Distributed Training**: First-class support for distributed training and hyperparameter optimization using Optuna and Ray.
- **CLI-Based Execution**: Easily package and execute projects within containers using our intuitive Command Line Interface (CLI).
- **Web Interface**: A modern web interface developed with Vite, React, TypeScript, and Bootstrap for managing experiment configurations.
## Project Goals
SnapperML aims to:
1. **Enhance Maintainability**: By addressing technical debt and improving the codebase, making it cleaner and more efficient.
1. **Improve Scalability**: Ensure the system can handle large-scale experiments and concurrent requests smoothly.
1. **Provide a Robust Web UI**: A user-friendly interface that simplifies the setup and execution of ML experiments.
1. **Ensure Reproducibility**: Leverage MLOps principles to ensure experiments can be replicated easily.
## Architecture
### Overview
SnapperML integrates several components to streamline machine learning workflows:
- **CLI Framework**: Facilitates command-based interactions and logging for experiment execution.
- **Flask API**: Manages requests from the frontend and interfaces with backend processes.
- **Vite-Powered Web UI**: An accessible and intuitive web application that handles experiment configurations and tracks real-time logs.
- **Containerized Databases**: Securely stores experiment results using containerized MLflow and Optuna databases.
> [!IMPORTANT]
> Be sure to configure your databases and network settings carefully to ensure the security and integrity of your experiment data.
## Installation
### Prerequisites
- docker
- python 3.12+
- node.js (for UI development)
### Install
The python package can be install using **pip**:
```
pip install snapper-ml
```
Or from this repo:
```
pip install .
```
> [!NOTE]
> Python 3.12 or later is required. Ensure that Docker is installed and running on your system for full functionality.
## Deploy
To run SnapperML, you first need to deploy MLflow and Optuna databases. Execute:
> [!TIP]
> To use the SnapperML web interface, deploy it with:
```
snapper-ml make docker
```
Once the deploy finished you can execute `snapper-ml` in the CLI. For an ilustrative example, check the [example section](#Example).
To use snapperML web interface you need to deploy it too.
```
snapper-ml make UI
```
Open [localhost:4000](http://localhost:4000/) and upload your firsts experiments!
To stop snapper UI just execute:
```
make stop_UI
```
And to stop mlflow and optuna databases execute:
```
make stop_docker
```
> [!CAUTION]
> Running make stop_UI also stops the Docker containers for the databases, so ensure you have saved all necessary data.
## Documentation
The documentation is available [here](https://snapperml.readthedocs.io/en/latest/)
> [!TIP]
> Visit the documentation for more examples and detailed instructions.
## Example
```python
# train_svm.py
from snapper_ml import job
@job
def main(C, kernel, gamma='scale'):
np.random.seed(1234)
X_train, X_val, y_train, y_val = load_data()
model = SVC(C=C, gamma=gamma, kernel=kernel)
model.fit(X_train, y_train)
accuracy = model.score(X_val, y_val)
return {'val_accuracy': accuracy}
if __name__ == '__main__':
main()
```
```yaml
# train_svm.yaml
name: "SVM"
kind: "group"
num_trials: 12
sampler: TPE
param_space:
C: loguniform(0.01, 1000)
gamma: choice(['scale', 'auto'])
metric:
name: val_accuracy
direction: maximize
ray_config:
num_cpus: 4
data:
folder: data/
files: ["*QGSJet.txt"]
run:
- train_svm.py
```
```
snapper-ml run --config_file=train_svm.yaml
```
> [!WARNING]
> Make sure the configuration files are correctly set to avoid runtime errors. Misconfigured parameters could lead to unexpected behavior.
There are more examples in the [examples folder](https://github.com/yerasiito/SnapperML/tree/master/examples).
Raw data
{
"_id": null,
"home_page": null,
"name": "snapper-ml",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": "machine learning, reproducibility, automation",
"author": null,
"author_email": "Antonio Molner Domenech <antonio.molner@correo.ugr.es>, Yeray L\u00f3pez Ram\u00edrez <ylopezramirez@correo.ugr.es>",
"download_url": "https://files.pythonhosted.org/packages/78/eb/3013e61c2d3adabc09314a9653f38dc525233467840aba9b08d282bcd3d6/snapper_ml-0.4.1.tar.gz",
"platform": null,
"description": "# SnapperML\n\n[](https://snapperml.readthedocs.io/en/latest/?badge=latest)\n\n[](https://pypi.org/project/snapper-ml/)\n\n\n\nSnapperML is a comprehensive framework for experiment tracking and machine learning operationalization (MLOps), built using well-supported technologies like [Mlflow](https://mlflow.org/), [Ray](https://github.com/ray-project/ray/), Docker, and more. It provides an opinionated workflow designed to facilitate both local and cloud-based experimentation.\n\n## Key Features\n\n- **Automatic Tracking**: Seamless integration with MLflow for parameter and metric tracking.\n- **Distributed Training**: First-class support for distributed training and hyperparameter optimization using Optuna and Ray.\n- **CLI-Based Execution**: Easily package and execute projects within containers using our intuitive Command Line Interface (CLI).\n- **Web Interface**: A modern web interface developed with Vite, React, TypeScript, and Bootstrap for managing experiment configurations.\n\n## Project Goals\n\nSnapperML aims to:\n\n1. **Enhance Maintainability**: By addressing technical debt and improving the codebase, making it cleaner and more efficient.\n1. **Improve Scalability**: Ensure the system can handle large-scale experiments and concurrent requests smoothly.\n1. **Provide a Robust Web UI**: A user-friendly interface that simplifies the setup and execution of ML experiments.\n1. **Ensure Reproducibility**: Leverage MLOps principles to ensure experiments can be replicated easily.\n\n## Architecture\n\n### Overview\n\nSnapperML integrates several components to streamline machine learning workflows:\n\n- **CLI Framework**: Facilitates command-based interactions and logging for experiment execution.\n- **Flask API**: Manages requests from the frontend and interfaces with backend processes.\n- **Vite-Powered Web UI**: An accessible and intuitive web application that handles experiment configurations and tracks real-time logs.\n- **Containerized Databases**: Securely stores experiment results using containerized MLflow and Optuna databases.\n > [!IMPORTANT]\n > Be sure to configure your databases and network settings carefully to ensure the security and integrity of your experiment data.\n\n\n\n## Installation\n\n### Prerequisites\n\n- docker\n- python 3.12+\n- node.js (for UI development)\n\n### Install\n\nThe python package can be install using **pip**:\n\n```\npip install snapper-ml\n```\n\nOr from this repo:\n\n```\npip install .\n```\n\n> [!NOTE]\n> Python 3.12 or later is required. Ensure that Docker is installed and running on your system for full functionality.\n\n## Deploy\n\nTo run SnapperML, you first need to deploy MLflow and Optuna databases. Execute:\n\n> [!TIP]\n> To use the SnapperML web interface, deploy it with:\n\n```\nsnapper-ml make docker\n```\n\nOnce the deploy finished you can execute `snapper-ml` in the CLI. For an ilustrative example, check the [example section](#Example).\n\nTo use snapperML web interface you need to deploy it too.\n\n```\nsnapper-ml make UI\n```\n\nOpen [localhost:4000](http://localhost:4000/) and upload your firsts experiments!\n\n\n\nTo stop snapper UI just execute:\n\n```\nmake stop_UI\n```\n\nAnd to stop mlflow and optuna databases execute:\n\n```\nmake stop_docker\n```\n\n> [!CAUTION]\n> Running make stop_UI also stops the Docker containers for the databases, so ensure you have saved all necessary data.\n\n## Documentation\n\nThe documentation is available [here](https://snapperml.readthedocs.io/en/latest/)\n\n> [!TIP]\n> Visit the documentation for more examples and detailed instructions.\n\n## Example\n\n```python\n# train_svm.py\n\nfrom snapper_ml import job\n\n@job\ndef main(C, kernel, gamma='scale'):\n np.random.seed(1234)\n X_train, X_val, y_train, y_val = load_data()\n model = SVC(C=C, gamma=gamma, kernel=kernel)\n model.fit(X_train, y_train)\n accuracy = model.score(X_val, y_val)\n return {'val_accuracy': accuracy}\n\n\nif __name__ == '__main__':\n main()\n```\n\n```yaml\n# train_svm.yaml\n\nname: \"SVM\"\nkind: \"group\"\nnum_trials: 12\nsampler: TPE\n\nparam_space:\n C: loguniform(0.01, 1000)\n gamma: choice(['scale', 'auto'])\n\nmetric:\n name: val_accuracy\n direction: maximize\n\nray_config:\n num_cpus: 4\n\ndata:\n folder: data/\n files: [\"*QGSJet.txt\"]\n\nrun:\n - train_svm.py\n```\n\n```\n\nsnapper-ml run --config_file=train_svm.yaml\n```\n\n> [!WARNING]\n> Make sure the configuration files are correctly set to avoid runtime errors. Misconfigured parameters could lead to unexpected behavior.\n\nThere are more examples in the [examples folder](https://github.com/yerasiito/SnapperML/tree/master/examples).\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A framework for reproducible machine learning",
"version": "0.4.1",
"project_urls": {
"homepage": "https://github.com/SnapperML/SnapperML"
},
"split_keywords": [
"machine learning",
" reproducibility",
" automation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b3c03264fe1a0a1e32369105a68d2046db1cdad5fd64be7159477ff2a0535455",
"md5": "073f68d3ac550906a3803b70a4b0164c",
"sha256": "97b386e821c92142b888e00157389ca6df3d408a20ba3d550e5fd0b7840523a8"
},
"downloads": -1,
"filename": "snapper_ml-0.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "073f68d3ac550906a3803b70a4b0164c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 2306744,
"upload_time": "2024-11-12T18:25:49",
"upload_time_iso_8601": "2024-11-12T18:25:49.294648Z",
"url": "https://files.pythonhosted.org/packages/b3/c0/3264fe1a0a1e32369105a68d2046db1cdad5fd64be7159477ff2a0535455/snapper_ml-0.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "78eb3013e61c2d3adabc09314a9653f38dc525233467840aba9b08d282bcd3d6",
"md5": "dddede3225f685a4ac89800e6d20f518",
"sha256": "e162449ddf15eb757c7fc19822acfaafe96ff497606e2d674de8568fb0e9f092"
},
"downloads": -1,
"filename": "snapper_ml-0.4.1.tar.gz",
"has_sig": false,
"md5_digest": "dddede3225f685a4ac89800e6d20f518",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 2298236,
"upload_time": "2024-11-12T18:25:51",
"upload_time_iso_8601": "2024-11-12T18:25:51.624117Z",
"url": "https://files.pythonhosted.org/packages/78/eb/3013e61c2d3adabc09314a9653f38dc525233467840aba9b08d282bcd3d6/snapper_ml-0.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-12 18:25:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SnapperML",
"github_project": "SnapperML",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "snapper-ml"
}
JSON
