| Name | ldimbenchmark JSON |
| Version |
0.2.67
JSON |
| download |
| home_page | |
| Summary | |
| upload_time | 2023-07-29 22:03:28 |
| maintainer | |
| docs_url | None |
| author | DanielHabenicht |
| requires_python | >=3.8,<4 |
| license | |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://pypi.org/project/ldimbenchmark)
[](https://tumt2022.github.io/LDIMBench/)
# LDIMBenchmark
Leakage Detection and Isolation Methods Benchmark
> Instead of collecting all the different dataset to benchmark different methods on. We wanted to create a Benchmarking Tool which makes it easy to reproduce the results of the different methods locally on your own dataset.
It provides a close to real-world conditions environment and forces researchers to provide a reproducible method implementation, which is supposed to run automated on any input dataset, thus hindering custom solutions which work well in one specific case.
## Usage
### Installation
```bash
pip install ldimbenchmark
```
### Python
```python
from ldimbenchmark.datasets import DatasetLibrary, DATASETS
from ldimbenchmark import (
LDIMBenchmark,
BenchmarkData,
BenchmarkLeakageResult,
)
from ldimbenchmark.classes import LDIMMethodBase
from typing import List
class YourCustomLDIMMethod(LDIMMethodBase):
def __init__(self):
super().__init__(
name="YourCustomLDIMMethod",
version="0.1.0"
)
def train(self, data: BenchmarkData):
pass
def detect(self, data: BenchmarkData) -> List[BenchmarkLeakageResult]:
return [
{
"leak_start": "2020-01-01",
"leak_end": "2020-01-02",
"leak_area": 0.2,
"pipe_id": "test",
}
]
def detect_datapoint(self, evaluation_data) -> BenchmarkLeakageResult:
return {}
datasets = DatasetLibrary("datasets").download(DATASETS.BATTLEDIM)
local_methods = [YourCustomLDIMMethod()]
hyperparameters = {}
benchmark = LDIMBenchmark(
hyperparameters, datasets, results_dir="./benchmark-results"
)
benchmark.add_local_methods(local_methods)
benchmark.run_benchmark()
benchmark.evaluate()
```
### CLI
```bash
ldimbenchmark --help
```
## Roadmap
- v1: Just Leakage Detection
- v2: Provides Benchmark of Isolation Methods
https://mathspp.com/blog/how-to-create-a-python-package-in-2022
## Development
https://python-poetry.org/docs/basic-usage/
```bash
# python 3.10
# Use Environment
poetry config virtualenvs.in-project true
poetry shell
poetry install --without ci # --with ci
# Test
poetry build
cp -r dist tests/dist
cd tests
docker build . -t testmethod
pytest -s -o log_cli=true
pytest tests/test_derivation.py -k 'test_mything'
pytest --testmon
pytest --snapshot-update
# Pytest watch
ptw
ptw -- --testmon
# Watch a file during development
npm install -g nodemon
nodemon -L experiments/auto_hyperparameter.py
# Test-Publish
poetry config repositories.testpypi https://test.pypi.org/legacy/
poetry config http-basic.testpypi __token__ pypi-your-api-token-here
poetry build
poetry publish -r testpypi
# Real Publish
poetry config pypi-token.pypi pypi-your-token-here
```
### Documentation
https://squidfunk.github.io/mkdocs-material/
https://click.palletsprojects.com/en/8.1.x/
```
poetry shell
mkdocs serve
```
# TODO
LDIMBenchmark:
Data Cleansing before working with them
- per sensor type, e.g. waterflow (cut of at 0)
- removing datapoints which are clearly a malfunction
Raw data
{
"_id": null,
"home_page": "",
"name": "ldimbenchmark",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4",
"maintainer_email": "",
"keywords": "",
"author": "DanielHabenicht",
"author_email": "daniel-habenicht@outlook.de",
"download_url": "https://files.pythonhosted.org/packages/34/90/6623ad082b92048f043026e9ce592a33637678e864bb3cb0b717f7ceb122/ldimbenchmark-0.2.67.tar.gz",
"platform": null,
"description": "[](https://pypi.org/project/ldimbenchmark)\n[](https://tumt2022.github.io/LDIMBench/)\n\n# LDIMBenchmark\n\nLeakage Detection and Isolation Methods Benchmark\n\n> Instead of collecting all the different dataset to benchmark different methods on. We wanted to create a Benchmarking Tool which makes it easy to reproduce the results of the different methods locally on your own dataset.\n\nIt provides a close to real-world conditions environment and forces researchers to provide a reproducible method implementation, which is supposed to run automated on any input dataset, thus hindering custom solutions which work well in one specific case.\n\n## Usage\n\n### Installation\n\n```bash\npip install ldimbenchmark\n```\n\n### Python\n\n```python\nfrom ldimbenchmark.datasets import DatasetLibrary, DATASETS\nfrom ldimbenchmark import (\n LDIMBenchmark,\n BenchmarkData,\n BenchmarkLeakageResult,\n)\nfrom ldimbenchmark.classes import LDIMMethodBase\nfrom typing import List\n\nclass YourCustomLDIMMethod(LDIMMethodBase):\n def __init__(self):\n super().__init__(\n name=\"YourCustomLDIMMethod\",\n version=\"0.1.0\"\n )\n\n def train(self, data: BenchmarkData):\n pass\n\n def detect(self, data: BenchmarkData) -> List[BenchmarkLeakageResult]:\n return [\n {\n \"leak_start\": \"2020-01-01\",\n \"leak_end\": \"2020-01-02\",\n \"leak_area\": 0.2,\n \"pipe_id\": \"test\",\n }\n ]\n\n def detect_datapoint(self, evaluation_data) -> BenchmarkLeakageResult:\n return {}\n\n\ndatasets = DatasetLibrary(\"datasets\").download(DATASETS.BATTLEDIM)\n\nlocal_methods = [YourCustomLDIMMethod()]\n\nhyperparameters = {}\n\nbenchmark = LDIMBenchmark(\n hyperparameters, datasets, results_dir=\"./benchmark-results\"\n)\nbenchmark.add_local_methods(local_methods)\n\nbenchmark.run_benchmark()\n\nbenchmark.evaluate()\n```\n\n### CLI\n\n```bash\nldimbenchmark --help\n```\n\n## Roadmap\n\n- v1: Just Leakage Detection\n- v2: Provides Benchmark of Isolation Methods\n\nhttps://mathspp.com/blog/how-to-create-a-python-package-in-2022\n\n## Development\n\nhttps://python-poetry.org/docs/basic-usage/\n\n```bash\n# python 3.10\n# Use Environment\npoetry config virtualenvs.in-project true\npoetry shell\npoetry install --without ci # --with ci\n\n\n# Test\npoetry build\ncp -r dist tests/dist\ncd tests\ndocker build . -t testmethod\npytest -s -o log_cli=true\npytest tests/test_derivation.py -k 'test_mything'\npytest --testmon\npytest --snapshot-update\n\n# Pytest watch\nptw\nptw -- --testmon\n\n# Watch a file during development\nnpm install -g nodemon\nnodemon -L experiments/auto_hyperparameter.py\n\n# Test-Publish\npoetry config repositories.testpypi https://test.pypi.org/legacy/\npoetry config http-basic.testpypi __token__ pypi-your-api-token-here\npoetry build\npoetry publish -r testpypi\n\n# Real Publish\npoetry config pypi-token.pypi pypi-your-token-here\n```\n\n### Documentation\n\nhttps://squidfunk.github.io/mkdocs-material/\nhttps://click.palletsprojects.com/en/8.1.x/\n\n```\npoetry shell\nmkdocs serve\n```\n\n# TODO\n\nLDIMBenchmark:\nData Cleansing before working with them\n\n- per sensor type, e.g. waterflow (cut of at 0)\n- removing datapoints which are clearly a malfunction\n\n",
"bugtrack_url": null,
"license": "",
"summary": "",
"version": "0.2.67",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d50ae40253cd450fbd91e00aeb54174422981490c24c42f5e6071933ee45e627",
"md5": "6210c19c3194425db0f8a2a9ffc92525",
"sha256": "085c388cb12044bfe32e9c688bf224162782b3219b9ab09bfd5f758648c0421e"
},
"downloads": -1,
"filename": "ldimbenchmark-0.2.67-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6210c19c3194425db0f8a2a9ffc92525",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4",
"size": 90509,
"upload_time": "2023-07-29T22:03:27",
"upload_time_iso_8601": "2023-07-29T22:03:27.004674Z",
"url": "https://files.pythonhosted.org/packages/d5/0a/e40253cd450fbd91e00aeb54174422981490c24c42f5e6071933ee45e627/ldimbenchmark-0.2.67-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "34906623ad082b92048f043026e9ce592a33637678e864bb3cb0b717f7ceb122",
"md5": "c12a47ef113c0e6fba078f64ab658c78",
"sha256": "e62d58e7104a6328d0c22e25d0bc49818417159b0fb6b64037bf5f3a42de9a1e"
},
"downloads": -1,
"filename": "ldimbenchmark-0.2.67.tar.gz",
"has_sig": false,
"md5_digest": "c12a47ef113c0e6fba078f64ab658c78",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4",
"size": 72760,
"upload_time": "2023-07-29T22:03:28",
"upload_time_iso_8601": "2023-07-29T22:03:28.193114Z",
"url": "https://files.pythonhosted.org/packages/34/90/6623ad082b92048f043026e9ce592a33637678e864bb3cb0b717f7ceb122/ldimbenchmark-0.2.67.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-29 22:03:28",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ldimbenchmark"
}
