# DOVER-Lap
Official implementation for [DOVER-Lap: A method for combining overlap-aware diarization outputs](https://arxiv.org/pdf/2011.01997.pdf).
## Installation
```shell
pip install dover-lap
```
## How to run
After installation, run
```shell
dover-lap [OPTIONS] OUTPUT_RTTM [INPUT_RTTMS]...
```
Example:
```shell
dover-lap egs/ami/rttm_dl_test egs/ami/rttm_test_*
```
## Usage instructions
```shell
Usage: dover-lap [OPTIONS] OUTPUT_RTTM [INPUT_RTTMS]...
Apply the DOVER-Lap algorithm on the input RTTM files.
Options:
--gaussian-filter-std FLOAT Standard deviation for Gaussian filter
applied before voting. This can help reduce
the effect of outliers in the input RTTMs.
For quick turn-taking, set this to a small
value (e.g. 0.1). 0.5 is a good value for
most cases. Set this to a very small value,
e.g. 0.01, to remove filtering. [default:
0.5]
--custom-weight TEXT Weights for input RTTMs
--dover-weight FLOAT DOVER weighting factor [default: 0.1]
--weight-type [rank|custom|norm]
Specify whether to use rank weighting or
provide custom weights [default: rank]
--voting-method [average] Choose voting method to use: average: use
weighted average to combine input RTTMs
[default: average]
--second-maximal If this flag is set, run a second iteration
of the maximal matching for greedy label
mapping [default: False]
--label-mapping [hungarian|greedy]
Choose label mapping algorithm to use
[default: greedy]
--random-seed INTEGER
-c, --channel INTEGER Use this value for output channel IDs
[default: 1]
-u, --uem-file PATH UEM file path
--help Show this message and exit.
```
**Note:**
1. If `--weight-type custom` is used, then `--custom-weight` must be provided. For example:
```shell
dover-lap egs/ami/rttm_dl_test egs/ami/rttm_test_* --weight-type custom --custom-weight '[0.4,0.3,0.3]'
```
2. `label-mapping` can be set to `greedy` (default) or `hungarian`, which is a modified version of the mapping
technique originally proposed in [DOVER](https://arxiv.org/abs/1909.08090).
## Results
We provide a sample result on the AMI mix-headset test set. The results can be
obtained using [`spyder`](https://github.com/desh2608/spyder), which is automatically
installed with `dover-lap`:
```shell
dover-lap egs/ami/rttm_dl_test egs/ami/rttm_test_*
spyder egs/ami/ref_rttm_test egs/ami/rttm_dl_test
```
and similarly for the input hypothesis. The DER results are shown below.
| | MS | FA | Conf. | DER |
|-----------------------------------|:-----:|:----:|:-----:|:-----:|
| Overlap-aware VB resegmentation | 9.84 | **2.06** | 9.60 | 21.50 |
| Overlap-aware spectral clustering | 11.48 | 2.27 | 9.81 | 23.56 |
| Region Proposal Network | **9.49** | 7.68 | 8.25 | 25.43 |
| DOVER-Lap (Hungarian mapping) | 9.98 | 2.13 | 8.25 | 20.35 |
| DOVER-Lap (Greedy mapping)* | 9.96 | 2.16 | **7.75** | **19.86** |
_* The Greedy label mapping is exponential in number of inputs (see [this paper](https://arxiv.org/abs/2104.01954))._
## Running time
The algorithm is implemented in pure Python with NumPy for tensor computations.
The time complexity is expected to increase exponentially with the number of
inputs, but it should be reasonable for combining up to 10 input hypotheses. For
combining more than 10 inputs, we recommend setting `--label-mapping hungarian`.
For smaller number of inputs (up to 5), the algorithm should take only a few seconds
to run on a laptop.
## Combining 2 systems with DOVER-Lap
DOVER-Lap is meant to be used to combine **more than 2 systems**, since
black-box voting between 2 systems does not make much sense. Still, if 2 systems
are provided as input, we fall back on the Hungarian algorithm for label mapping,
since it is provably optimal for this case. Both the systems are assigned equal
weights, and in case of voting conflicts, the region is assigned to both
labels. This is not the intended use case and will almost certainly lead
to performance degradation.
## Citation
```
@article{Raj2021Doverlap,
title={{DOVER-Lap}: A Method for Combining Overlap-aware Diarization Outputs},
author={D.Raj and P.Garcia and Z.Huang and S.Watanabe and D.Povey and A.Stolcke and S.Khudanpur},
journal={2021 IEEE Spoken Language Technology Workshop (SLT)},
year={2021}
}
@article{Raj2021ReformulatingDL,
title={Reformulating {DOVER-Lap} Label Mapping as a Graph Partitioning Problem},
author={Desh Raj and S. Khudanpur},
journal={INTERSPEECH},
year={2021},
}
```
## Contact
For issues/bug reports, please raise an Issue in this repository, or reach out to me at `draj@cs.jhu.edu`.
Raw data
{
"_id": null,
"home_page": "https://github.com/desh2608/dover-lap",
"name": "dover-lap",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "diarization dover",
"author": "Desh Raj",
"author_email": "r.desh26@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/a4/9e/cf07cf46dee1117bc913396ba2f5f714f934e2bfa37738325d5f44a91f47/dover-lap-1.3.1.tar.gz",
"platform": null,
"description": "# DOVER-Lap\nOfficial implementation for [DOVER-Lap: A method for combining overlap-aware diarization outputs](https://arxiv.org/pdf/2011.01997.pdf).\n\n## Installation\n\n```shell\npip install dover-lap\n```\n\n## How to run\n\nAfter installation, run\n\n```shell\ndover-lap [OPTIONS] OUTPUT_RTTM [INPUT_RTTMS]...\n```\n\nExample:\n\n```shell\ndover-lap egs/ami/rttm_dl_test egs/ami/rttm_test_*\n```\n\n## Usage instructions\n\n```shell\nUsage: dover-lap [OPTIONS] OUTPUT_RTTM [INPUT_RTTMS]...\n\n Apply the DOVER-Lap algorithm on the input RTTM files.\n\nOptions:\n --gaussian-filter-std FLOAT Standard deviation for Gaussian filter\n applied before voting. This can help reduce\n the effect of outliers in the input RTTMs.\n For quick turn-taking, set this to a small\n value (e.g. 0.1). 0.5 is a good value for\n most cases. Set this to a very small value,\n e.g. 0.01, to remove filtering. [default:\n 0.5]\n\n --custom-weight TEXT Weights for input RTTMs\n --dover-weight FLOAT DOVER weighting factor [default: 0.1]\n --weight-type [rank|custom|norm]\n Specify whether to use rank weighting or\n provide custom weights [default: rank]\n\n --voting-method [average] Choose voting method to use: average: use\n weighted average to combine input RTTMs\n [default: average]\n\n --second-maximal If this flag is set, run a second iteration\n of the maximal matching for greedy label\n mapping [default: False]\n\n --label-mapping [hungarian|greedy]\n Choose label mapping algorithm to use\n [default: greedy]\n\n --random-seed INTEGER\n -c, --channel INTEGER Use this value for output channel IDs\n [default: 1]\n\n -u, --uem-file PATH UEM file path\n --help Show this message and exit.\n```\n\n**Note:** \n\n1. If `--weight-type custom` is used, then `--custom-weight` must be provided. For example:\n\n```shell\ndover-lap egs/ami/rttm_dl_test egs/ami/rttm_test_* --weight-type custom --custom-weight '[0.4,0.3,0.3]'\n```\n\n2. `label-mapping` can be set to `greedy` (default) or `hungarian`, which is a modified version of the mapping\ntechnique originally proposed in [DOVER](https://arxiv.org/abs/1909.08090).\n\n## Results\n\nWe provide a sample result on the AMI mix-headset test set. The results can be \nobtained using [`spyder`](https://github.com/desh2608/spyder), which is automatically\ninstalled with `dover-lap`:\n\n```shell\ndover-lap egs/ami/rttm_dl_test egs/ami/rttm_test_*\nspyder egs/ami/ref_rttm_test egs/ami/rttm_dl_test\n```\n\nand similarly for the input hypothesis. The DER results are shown below.\n\n| | MS | FA | Conf. | DER |\n|-----------------------------------|:-----:|:----:|:-----:|:-----:|\n| Overlap-aware VB resegmentation | 9.84 | **2.06** | 9.60 | 21.50 |\n| Overlap-aware spectral clustering | 11.48 | 2.27 | 9.81 | 23.56 |\n| Region Proposal Network | **9.49** | 7.68 | 8.25 | 25.43 |\n| DOVER-Lap (Hungarian mapping) | 9.98 | 2.13 | 8.25 | 20.35 |\n| DOVER-Lap (Greedy mapping)* | 9.96 | 2.16 | **7.75** | **19.86** |\n\n_* The Greedy label mapping is exponential in number of inputs (see [this paper](https://arxiv.org/abs/2104.01954))._\n\n## Running time\n\nThe algorithm is implemented in pure Python with NumPy for tensor computations. \nThe time complexity is expected to increase exponentially with the number of \ninputs, but it should be reasonable for combining up to 10 input hypotheses. For \ncombining more than 10 inputs, we recommend setting `--label-mapping hungarian`.\n\nFor smaller number of inputs (up to 5), the algorithm should take only a few seconds\nto run on a laptop.\n\n## Combining 2 systems with DOVER-Lap\n\nDOVER-Lap is meant to be used to combine **more than 2 systems**, since\nblack-box voting between 2 systems does not make much sense. Still, if 2 systems\nare provided as input, we fall back on the Hungarian algorithm for label mapping,\nsince it is provably optimal for this case. Both the systems are assigned equal\nweights, and in case of voting conflicts, the region is assigned to both\nlabels. This is not the intended use case and will almost certainly lead\nto performance degradation.\n\n## Citation\n\n```\n@article{Raj2021Doverlap,\n title={{DOVER-Lap}: A Method for Combining Overlap-aware Diarization Outputs},\n author={D.Raj and P.Garcia and Z.Huang and S.Watanabe and D.Povey and A.Stolcke and S.Khudanpur},\n journal={2021 IEEE Spoken Language Technology Workshop (SLT)},\n year={2021}\n}\n\n@article{Raj2021ReformulatingDL,\n title={Reformulating {DOVER-Lap} Label Mapping as a Graph Partitioning Problem},\n author={Desh Raj and S. Khudanpur},\n journal={INTERSPEECH},\n year={2021},\n}\n```\n\n## Contact\n\nFor issues/bug reports, please raise an Issue in this repository, or reach out to me at `draj@cs.jhu.edu`.\n",
"bugtrack_url": null,
"license": "Apache-2.0 License",
"summary": "Combine overlap-aware diarization output RTTMs",
"version": "1.3.1",
"project_urls": {
"Homepage": "https://github.com/desh2608/dover-lap"
},
"split_keywords": [
"diarization",
"dover"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2973b8ebce0f84a212e1cb53f6a9f8119653718e26146b7382ba723a238b324b",
"md5": "b33e1b4354ca4ca4c757592b8e76e3b8",
"sha256": "0fff21e8e8211b45db5ae5d9b7df03512aef143c852ad978696e66d448309f10"
},
"downloads": -1,
"filename": "dover_lap-1.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b33e1b4354ca4ca4c757592b8e76e3b8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 23781,
"upload_time": "2023-09-24T00:30:52",
"upload_time_iso_8601": "2023-09-24T00:30:52.201569Z",
"url": "https://files.pythonhosted.org/packages/29/73/b8ebce0f84a212e1cb53f6a9f8119653718e26146b7382ba723a238b324b/dover_lap-1.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a49ecf07cf46dee1117bc913396ba2f5f714f934e2bfa37738325d5f44a91f47",
"md5": "67a9363429cd2a773bb5c8759000817c",
"sha256": "6bb0c27eea333795eea15e11775d4519cc52a00834865e2eeab9cc76a859cab6"
},
"downloads": -1,
"filename": "dover-lap-1.3.1.tar.gz",
"has_sig": false,
"md5_digest": "67a9363429cd2a773bb5c8759000817c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 20576,
"upload_time": "2023-09-24T00:30:53",
"upload_time_iso_8601": "2023-09-24T00:30:53.845618Z",
"url": "https://files.pythonhosted.org/packages/a4/9e/cf07cf46dee1117bc913396ba2f5f714f934e2bfa37738325d5f44a91f47/dover-lap-1.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-24 00:30:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "desh2608",
"github_project": "dover-lap",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "dover-lap"
}
JSON
