| Name | ml-exp-comparator JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | None |
| Summary | Compare ML experiments |
| upload_time | 2024-08-02 19:13:16 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.7 |
| license | The MIT License (MIT) Copyright © 2024 <copyright holders> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
ml
ds
machine learning
data science
experiments
compare
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ML experiments comparator
ML experiments comparator is an easy tool for comparing different ML experiments.
## Installation
You can install the ML experiments comparator from [PyPI](https://pypi.org/project/ml-exp-comparator/):
`pip install ml_exp_comparator`
## How to use
```
# 1. create or select ModelRunner for your experiments
# 2. load trained experimental models into model runners
model_runner1, model_runner2 = get_model_runners(...)
# 3. create or select DatasetStore containing your dataset
dataset_store = get_dataset_store(...)
# 4. create or select MetricRunner for your dataset
metrics_runners = get_metrics_runners(...)
# 5. create or select ResultsAggregator for collecting metrics
results_aggregator = get_results_aggregator(...)
# 6. init ExperimentRunner
exp_runner = ExperimentsRunner(exp_name='exp_name',
storage_folder='storage_folder',
batch_size=100,
exp_models_runners=[model_runner1, model_runner2],
dataset_store=dataset_store,
metrics_runners=metrics_runners,
result_aggregator=results_aggregator)
# 7. run comparison and store the results
exp_runner.run_and_store()
```
Examples: `examples/.`
## Modules
### Experiments runner
`from ml_exp_comparator.experiments_runner import ExperimentsRunner`
Runs experimental models on the provided dataset using selected metrics.
### Dataset stores
`from ml_exp_comparator.dataset_stores.dataset_store import DatasetStore`
Dataset wrapper. Yields batches of a specific dataset.
### Metrics runners
`from ml_exp_comparator.metrics_runners.metric_runner import MetricRunner`
Calculates metrics for each output based on true and pred outputs. Aggregates calculated metrics.
### Models runners
`from ml_exp_comparator.models_runners.model_runner import ModelRunner`
Runs trained model on dataset true_input batches.
### Results aggregator
`from ml_exp_comparator.results_aggregators.results_aggregator import ResultsAggregator`
Collects single and aggregated metrics results. Dumps metric results into a local storage.
Raw data
{
"_id": null,
"home_page": null,
"name": "ml-exp-comparator",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "ML, DS, Machine Learning, Data Science, Experiments, Compare",
"author": null,
"author_email": "Max Kalganov <maksim.kalganov@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/2f/22/600de4373f14a5b0564a51d5f28195ba5a0991cc22e0b3acd2c8bc2c3923/ml_exp_comparator-1.0.1.tar.gz",
"platform": null,
"description": "# ML experiments comparator\n\nML experiments comparator is an easy tool for comparing different ML experiments.\n\n## Installation\n\nYou can install the ML experiments comparator from [PyPI](https://pypi.org/project/ml-exp-comparator/):\n\n`pip install ml_exp_comparator`\n\n## How to use\n\n```\n# 1. create or select ModelRunner for your experiments\n# 2. load trained experimental models into model runners \n\nmodel_runner1, model_runner2 = get_model_runners(...)\n\n# 3. create or select DatasetStore containing your dataset\ndataset_store = get_dataset_store(...)\n\n# 4. create or select MetricRunner for your dataset\nmetrics_runners = get_metrics_runners(...)\n\n# 5. create or select ResultsAggregator for collecting metrics\nresults_aggregator = get_results_aggregator(...)\n\n# 6. init ExperimentRunner\nexp_runner = ExperimentsRunner(exp_name='exp_name', \n storage_folder='storage_folder', \n batch_size=100,\n exp_models_runners=[model_runner1, model_runner2],\n dataset_store=dataset_store,\n metrics_runners=metrics_runners,\n result_aggregator=results_aggregator)\n\n# 7. run comparison and store the results \nexp_runner.run_and_store()\n```\nExamples: `examples/.`\n\n## Modules\n\n### Experiments runner \n\n`from ml_exp_comparator.experiments_runner import ExperimentsRunner`\n\nRuns experimental models on the provided dataset using selected metrics.\n\n### Dataset stores\n\n`from ml_exp_comparator.dataset_stores.dataset_store import DatasetStore`\n\nDataset wrapper. Yields batches of a specific dataset.\n\n### Metrics runners\n\n`from ml_exp_comparator.metrics_runners.metric_runner import MetricRunner`\n\nCalculates metrics for each output based on true and pred outputs. Aggregates calculated metrics.\n\n### Models runners\n\n`from ml_exp_comparator.models_runners.model_runner import ModelRunner`\n\nRuns trained model on dataset true_input batches.\n\n### Results aggregator\n\n`from ml_exp_comparator.results_aggregators.results_aggregator import ResultsAggregator`\n\nCollects single and aggregated metrics results. Dumps metric results into a local storage.\n\n\n",
"bugtrack_url": null,
"license": "The MIT License (MIT) Copyright \u00a9 2024 <copyright holders> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \u201cSoftware\u201d), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \u201cAS IS\u201d, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "Compare ML experiments",
"version": "1.0.1",
"project_urls": {
"repository": "https://github.com/max-kalganov/ml_exp_comparator"
},
"split_keywords": [
"ml",
" ds",
" machine learning",
" data science",
" experiments",
" compare"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2b6247f78ca43a2dce93d2280b88521fc3b404033243e79334bb8b319eeaa8fd",
"md5": "f9999afefe1d6a3ed76ec819da879b02",
"sha256": "433234199354c38699b29495a8414c80f36aebef54be6a0c27fa0c0a366b5006"
},
"downloads": -1,
"filename": "ml_exp_comparator-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f9999afefe1d6a3ed76ec819da879b02",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 11515,
"upload_time": "2024-08-02T19:13:13",
"upload_time_iso_8601": "2024-08-02T19:13:13.910514Z",
"url": "https://files.pythonhosted.org/packages/2b/62/47f78ca43a2dce93d2280b88521fc3b404033243e79334bb8b319eeaa8fd/ml_exp_comparator-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2f22600de4373f14a5b0564a51d5f28195ba5a0991cc22e0b3acd2c8bc2c3923",
"md5": "ab8ec6e4f5368a65f96db4c667d4b54a",
"sha256": "97a18590c18f071238b5bdeda9cfe29a7e601ea05d3a4b7bf8977be9a1e15691"
},
"downloads": -1,
"filename": "ml_exp_comparator-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "ab8ec6e4f5368a65f96db4c667d4b54a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 10265,
"upload_time": "2024-08-02T19:13:16",
"upload_time_iso_8601": "2024-08-02T19:13:16.176179Z",
"url": "https://files.pythonhosted.org/packages/2f/22/600de4373f14a5b0564a51d5f28195ba5a0991cc22e0b3acd2c8bc2c3923/ml_exp_comparator-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-02 19:13:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "max-kalganov",
"github_project": "ml_exp_comparator",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "ml-exp-comparator"
}
