# seamlessvalidation
Automating the validation process for machine learning models is crucial for efficient deployment and monitoring. Depending on the phase (pre-deployment or monitoring) and the model type (such as Classifiers, Regressors, Clusters), a tailored approach is necessary for selecting appropriate validation types and methods.
During the pre-deployment phase, the seamlessvalidation package intelligently selects validation types like Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE) for regression models, while for classification models, it opts for Receiver Operating Characteristic (ROC) curves, Confusion Matrices, Precision-Recall Scores, or F1 scores based on the model's nature. Moreover, it selects validation methods such as k-fold cross-validation, stratified cross-validation, leave-one-out cross-validation, bootstrapping, or holdout validation to ensure robustness in assessing model performance.
In the monitoring phase, the system continues to adapt its validation strategy based on the model type and current deployment context. It automatically selects appropriate validation types and methods to evaluate the model's performance over time. Additionally, it provides continuous monitoring results on data drift, enabling users to assess the model's effectiveness and population shift in real-world scenarios.
By automating the selection of validation types and methods based on the model type and deployment phase, the package ensures thorough and accurate validation, leading to more reliable model deployment and monitoring processes.
## Installation
You can install your package using pip: pip install seamlessvalidation
## Usage
Here's how you can use your package:
```python
import seamlessvalidation
# Example usage code here
seamlessvalidation(your_model,X=X_class,y=y_class,n_splits=number_of_splits
,cv_strategy='stratified_k_fold',avg_strategy='weighted',phase='validation')
seamlessvalidation(your_model,X=X_class,y=y_class
,p_value=0.5
,data_compare=old_data_to_compare, data_new=new_data
,phase='post_deployment_monitoring')
```
## Features
Auto selection on validation metrics based on model type and development stage. Easily selection of validation strategy. Flexible intergration comparison method and data drift monitoring
## Dependencies
'scikit-learn',
'pandas',
'numpy',
'matplotlib',
'pickle',
'scipy'
## Contributing
Contributions are welcome! Here's how you can contribute:
- Fork the repository
- Make your changes
- Submit a pull request
## License
Copyright (c) 2023, The Hugging Face Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
## Authors
ZhuZheng(Iverson) ZHOU
## Support or Contact
zzho044@aucklanduni.ac.nz
Raw data
{
"_id": null,
"home_page": null,
"name": "seamlessvalidation",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "machine learning validation monitoring",
"author": "ZhuZheng(Iverson) ZHOU",
"author_email": "zzho044@aucklanduni.ac.nz",
"download_url": "https://files.pythonhosted.org/packages/ac/cd/0bb4523a7c6bb451a83f96767cefebfcc62f86341fee813f8952136bceb1/seamlessvalidation-0.12.tar.gz",
"platform": null,
"description": "# seamlessvalidation\n\nAutomating the validation process for machine learning models is crucial for efficient deployment and monitoring. Depending on the phase (pre-deployment or monitoring) and the model type (such as Classifiers, Regressors, Clusters), a tailored approach is necessary for selecting appropriate validation types and methods.\n\nDuring the pre-deployment phase, the seamlessvalidation package intelligently selects validation types like Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE) for regression models, while for classification models, it opts for Receiver Operating Characteristic (ROC) curves, Confusion Matrices, Precision-Recall Scores, or F1 scores based on the model's nature. Moreover, it selects validation methods such as k-fold cross-validation, stratified cross-validation, leave-one-out cross-validation, bootstrapping, or holdout validation to ensure robustness in assessing model performance.\n\nIn the monitoring phase, the system continues to adapt its validation strategy based on the model type and current deployment context. It automatically selects appropriate validation types and methods to evaluate the model's performance over time. Additionally, it provides continuous monitoring results on data drift, enabling users to assess the model's effectiveness and population shift in real-world scenarios.\n\nBy automating the selection of validation types and methods based on the model type and deployment phase, the package ensures thorough and accurate validation, leading to more reliable model deployment and monitoring processes.\n\n\n## Installation\n\nYou can install your package using pip: pip install seamlessvalidation\n\n\n\n## Usage\n\nHere's how you can use your package:\n\n```python\nimport seamlessvalidation\n\n# Example usage code here\nseamlessvalidation(your_model,X=X_class,y=y_class,n_splits=number_of_splits\n ,cv_strategy='stratified_k_fold',avg_strategy='weighted',phase='validation')\n\nseamlessvalidation(your_model,X=X_class,y=y_class\n ,p_value=0.5\n ,data_compare=old_data_to_compare, data_new=new_data\n ,phase='post_deployment_monitoring')\n```\n\n## Features\nAuto selection on validation metrics based on model type and development stage. Easily selection of validation strategy. Flexible intergration comparison method and data drift monitoring\n\n## Dependencies\n'scikit-learn',\n'pandas',\n'numpy',\n'matplotlib',\n'pickle',\n'scipy'\n\n## Contributing\nContributions are welcome! Here's how you can contribute:\n\n- Fork the repository\n- Make your changes\n- Submit a pull request\n\n## License\n Copyright (c) 2023, The Hugging Face Team. All rights reserved.\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n\n\n## Authors\nZhuZheng(Iverson) ZHOU\n\n## Support or Contact\nzzho044@aucklanduni.ac.nz\n",
"bugtrack_url": null,
"license": null,
"summary": "A package for easy validation and post deployment monitoring of common linear and non linear ML models and clustering model",
"version": "0.12",
"project_urls": null,
"split_keywords": [
"machine",
"learning",
"validation",
"monitoring"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4354de3208d0ec7511a1f96abd56509feb3730d7a102e87a9f8102043376e316",
"md5": "867d5c9ba8c5bca0ebc8595307c798a7",
"sha256": "5bf43d29be0e9fb5b0d8bc81978cf971833ca21e67885efcaacda0d8b5af6f63"
},
"downloads": -1,
"filename": "seamlessvalidation-0.12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "867d5c9ba8c5bca0ebc8595307c798a7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 17612,
"upload_time": "2024-03-29T13:28:43",
"upload_time_iso_8601": "2024-03-29T13:28:43.565581Z",
"url": "https://files.pythonhosted.org/packages/43/54/de3208d0ec7511a1f96abd56509feb3730d7a102e87a9f8102043376e316/seamlessvalidation-0.12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "accd0bb4523a7c6bb451a83f96767cefebfcc62f86341fee813f8952136bceb1",
"md5": "edb15f465d90ab8feaa8d7af5f70d326",
"sha256": "5948d16b9d4cf98d7254372103e9e6975f81ea5d5194eaf23ae0f8610575e811"
},
"downloads": -1,
"filename": "seamlessvalidation-0.12.tar.gz",
"has_sig": false,
"md5_digest": "edb15f465d90ab8feaa8d7af5f70d326",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 13498,
"upload_time": "2024-03-29T13:28:45",
"upload_time_iso_8601": "2024-03-29T13:28:45.947769Z",
"url": "https://files.pythonhosted.org/packages/ac/cd/0bb4523a7c6bb451a83f96767cefebfcc62f86341fee813f8952136bceb1/seamlessvalidation-0.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-29 13:28:45",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "seamlessvalidation"
}
JSON