# ECoLaF
This package offers a user-friendly toolkit to build an adaptive multimodal late fusion pipeline (*ECoLaF*) based on the Dempster-Shafer Theory from multiple single-modality neural networks.
The proposed pipeline can be used for semantic segmentation and classification tasks.
<!-- Mettre les équations et des exemples plus visuels pourrait pimper la description !!! -->
## INSTALLATION
To install the package, use the following command: `pip install ecolaf`.
## USAGE
Firstly, a single-modality neural network needs to be defined for each modality. Please make sure that each neural network has a *forward* method. If the dataset you are working with has *K* classes, please make sure that your models output *K+1* values to fit the Dempster-Shafer framework.
Then you can build an *ECoLaF* pipeline as follows:
`MyModel = ECOLAF(list_of_single_modality_networks, num_classes)`.
Given a list of multimodal images, the inference can be done as follows:
`output = MyModel(list_of_images, **kwargs)`.
The optional arguments of the forward method are listed bellow:
```
output_unimodal (Boolean): returns the list of outputs for each model
output_conflict (Boolean): returns a conflict map of shape BxMxHxW
output_discounting_coef (Boolean): returns the discounting coefficients map of shape BxMxHxW
interpolation (Boolean): interpolates the output to the original image size
output_keyword (String): key to access to the output tensor if the model's output is a dictionary
(for example, the deeplabV3_resnet50 model from torchvision requires output_keyword='out')
B: batch_size, M: number of modalities, H: images height, W: images width
```
An example is provided in `test/test_code.py`.
## CITATIONS
If you use ECoLaF, please cite the following work:
```
@InProceedings{deregnaucourt_2025_WACV,
author = {Deregnaucourt, Lucas and Laghmara, Hind and Lechervy, Alexis and Ainouz, Samia},
title = {A Conflict-guided Evidential Fusion for Multimodal Semantic Segmentation},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
month = {February},
year = {2025},
}
```
# CHANGELOG
## Version 0.1.0 12.01.2024
Repository creation
## Version 0.1.1 12.03.2024
Minor changes
## Version 0.1.2 12.03.2024
Minor changes
## Version 0.1.3 12.03.2024
Minor changes
Raw data
{
"_id": null,
"home_page": "https://github.com/deregnaucourtlucas/ECOLAF",
"name": "ecolaf",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "Dempster-Shafer Theory, Deep Learning, Multimodal, Semantic Segmentation, Classification",
"author": "Lucas Deregnaucourt",
"author_email": "lucas.deregnaucourt@insa-rouen.fr",
"download_url": "https://files.pythonhosted.org/packages/3a/eb/6cb370c524fa3e2d8e1caf4edc53e7de4c43226bcb197c86df28e2f43503/ecolaf-0.1.3.tar.gz",
"platform": null,
"description": "# ECoLaF\n\nThis package offers a user-friendly toolkit to build an adaptive multimodal late fusion pipeline (*ECoLaF*) based on the Dempster-Shafer Theory from multiple single-modality neural networks.\nThe proposed pipeline can be used for semantic segmentation and classification tasks.\n\n<!-- Mettre les \u00e9quations et des exemples plus visuels pourrait pimper la description !!! -->\n\n## INSTALLATION\nTo install the package, use the following command: `pip install ecolaf`.\n\n## USAGE\nFirstly, a single-modality neural network needs to be defined for each modality. Please make sure that each neural network has a *forward* method. If the dataset you are working with has *K* classes, please make sure that your models output *K+1* values to fit the Dempster-Shafer framework. \n\nThen you can build an *ECoLaF* pipeline as follows: \n`MyModel = ECOLAF(list_of_single_modality_networks, num_classes)`.\n\nGiven a list of multimodal images, the inference can be done as follows: \n`output = MyModel(list_of_images, **kwargs)`.\n\nThe optional arguments of the forward method are listed bellow:\n\n```\noutput_unimodal (Boolean): returns the list of outputs for each model\n\noutput_conflict (Boolean): returns a conflict map of shape BxMxHxW\n\noutput_discounting_coef (Boolean): returns the discounting coefficients map of shape BxMxHxW\n\ninterpolation (Boolean): interpolates the output to the original image size\n\noutput_keyword (String): key to access to the output tensor if the model's output is a dictionary \n(for example, the deeplabV3_resnet50 model from torchvision requires output_keyword='out')\n\nB: batch_size, M: number of modalities, H: images height, W: images width\n```\n\nAn example is provided in `test/test_code.py`.\n\n## CITATIONS\n\nIf you use ECoLaF, please cite the following work:\n\n```\n@InProceedings{deregnaucourt_2025_WACV,\n author = {Deregnaucourt, Lucas and Laghmara, Hind and Lechervy, Alexis and Ainouz, Samia},\n title = {A Conflict-guided Evidential Fusion for Multimodal Semantic Segmentation},\n booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},\n month = {February},\n year = {2025},\n}\n```\n\n\n# CHANGELOG\n\n## Version 0.1.0 12.01.2024\nRepository creation\n\n## Version 0.1.1 12.03.2024\nMinor changes\n\n## Version 0.1.2 12.03.2024\nMinor changes\n\n## Version 0.1.3 12.03.2024\nMinor changes\n",
"bugtrack_url": null,
"license": null,
"summary": "A toolkit python package to build an adaptive multimodal late fusion pipeline based on Dempster-Shafer Theory for semantic segmentation and classification tasks.",
"version": "0.1.3",
"project_urls": {
"Homepage": "https://github.com/deregnaucourtlucas/ECOLAF"
},
"split_keywords": [
"dempster-shafer theory",
" deep learning",
" multimodal",
" semantic segmentation",
" classification"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "54cf35f82a9282034504a85cea7a7ff658f4b03801eaa421ee5dca94f3b530ff",
"md5": "af61294a3de662207b94a926024af564",
"sha256": "e04881c79d63f7c60af410676c8b8245625be1826d2ce63a2747191f3f27d3e8"
},
"downloads": -1,
"filename": "ecolaf-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "af61294a3de662207b94a926024af564",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7049,
"upload_time": "2024-12-03T14:36:23",
"upload_time_iso_8601": "2024-12-03T14:36:23.663952Z",
"url": "https://files.pythonhosted.org/packages/54/cf/35f82a9282034504a85cea7a7ff658f4b03801eaa421ee5dca94f3b530ff/ecolaf-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3aeb6cb370c524fa3e2d8e1caf4edc53e7de4c43226bcb197c86df28e2f43503",
"md5": "d3de386335f78d582400683421a9e352",
"sha256": "10d1a2f28957ab9ec1d429b78165e1e138909cace20cd6fc3086f0864d4f5a82"
},
"downloads": -1,
"filename": "ecolaf-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "d3de386335f78d582400683421a9e352",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5977,
"upload_time": "2024-12-03T14:36:24",
"upload_time_iso_8601": "2024-12-03T14:36:24.792896Z",
"url": "https://files.pythonhosted.org/packages/3a/eb/6cb370c524fa3e2d8e1caf4edc53e7de4c43226bcb197c86df28e2f43503/ecolaf-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-03 14:36:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "deregnaucourtlucas",
"github_project": "ECOLAF",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "ecolaf"
}
JSON
