# Multimodal Emotion Expression Capture Amsterdam
[](https://github.com/mexca/mexca)
[](https://research-software-directory.org/software/mexca)
[](https://mexca.readthedocs.io/en/latest/index.html)
[](https://fair-software.eu)
[](https://sonarcloud.io/dashboard?id=mexca_mexca)
[](https://sonarcloud.io/dashboard?id=mexca_mexca)
[](https://github.com/mexca/mexca/actions/workflows/build.yml)
[](https://github.com/mexca/mexca/actions/workflows/cffconvert.yml)
[](https://github.com/mexca/mexca/actions/workflows/markdown-link-check.yml)
[](https://zenodo.org/badge/latestdoi/500818250)
[](https://hub.docker.com/u/mexca)
[](https://github.com/mexca/mexca/actions/workflows/docker.yml)
[](https://github.com/psf/black)
<div align="center">
<img src="mexca_logo.png" width="400">
</div>
mexca is an open-source Python package which aims to capture human emotion expressions from videos in a single pipeline.
## How To Use Mexca
mexca implements the customizable yet easy-to-use Multimodal Emotion eXpression Capture Amsterdam (MEXCA) pipeline for extracting emotion expression features from videos.
It contains building blocks that can be used to extract features for individual modalities (i.e., facial expressions, voice, and dialogue/spoken text).
The blocks can also be integrated into a single pipeline to extract the features from all modalities at once.
Next to extracting features, mexca can also identify the speakers shown in the video by clustering speaker and face representations.
This allows users to compare emotion expressions across speakers, time, and contexts.
Please cite mexca if you use it for scientific or commercial purposes.
<div align="center">
<img src="docs/mexca_flowchart.png" width="600">
</div>
## Installation
mexca can be installed on Windows, macOS and Linux. We recommend Windows 10, macOS 12.6.x, or Ubuntu. The base package can be installed from PyPI via `pip`:
```console
pip install mexca
```
The dependencies for the additional components can be installed via:
```console
pip install mexca[vid,spe,voi,tra,sen]
```
or:
```console
pip install mexca[all]
```
The abbreviations indicate:
* `vid`: FaceExtractor
* `spe`: SpeakerIdentifier
* `voi`: VoiceExtractor
* `tra`: AudioTranscriber
* `sen`: SentimentExtractor
For details on the requirements and installation procedure, see the [Quick Installation](https://mexca.readthedocs.io/en/latest/quick_installation.html) and [Installation Details](https://mexca.readthedocs.io/en/latest/installation_details.html) sections of our documentation.
## Getting Started
If you would like to learn how to use mexca, take a look at our [demo](https://github.com/mexca/mexca/tree/main/examples) notebook and the [Getting Started](https://mexca.readthedocs.io/en/latest/getting_started.html) section of our documentation.
## Examples and Recipes
In the `examples/` folder, we currently provide two Jupyter notebooks (and a short demo):
- [example_custom_pipeline_components](https://github.com/mexca/mexca/blob/main/examples/example_custom_pipeline_components.ipynb) shows how the standard MEXCA pipeline can be customized and extended
- [example_emotion_feature_extraction](https://github.com/mexca/mexca/blob/main/examples/example_emotion_feature_extraction.ipynb) shows how to apply the MEXCA pipeline to a video and conduct a basic analysis of the extracted features
The `recipes/` folder contains two Python scripts that can easily be reused in a new project:
- [recipe_postprocess_features](https://github.com/mexca/mexca/blob/main/recipes/recipe_postprocess_features.py) applies a standard postprocessing routine to extracted features
- [recipe_standard_pipeline](https://github.com/mexca/mexca/blob/main/recipes/recipe_standard_pipeline.py) applies the standard MEXCA pipeline to a list of videos
## Components
The pipeline components are described [here](https://mexca.readthedocs.io/en/latest/components.html).
## Documentation
The documentation of mexca can be found on [Read the Docs](https://mexca.readthedocs.io/en/latest/index.html).
## Contributing
If you want to contribute to the development of mexca,
have a look at the [contribution guidelines](CONTRIBUTING.md).
## License
The code is licensed under the Apache 2.0 License. This means that mexca can be used, modified and redistributed for free, even for commercial purposes.
## Credits
Mexca is being developed by the [Netherlands eScience Center](https://www.esciencecenter.nl/) in collaboration with the [Hot Politics Lab](http://www.hotpolitics.eu/) at the University of Amsterdam.
This package was created with [Cookiecutter](https://github.com/audreyr/cookiecutter) and the [NLeSC/python-template](https://github.com/NLeSC/python-template).
[^1]: We explain the rationale for this setup in the [Docker](https://mexca.readthedocs.io/en/latest/docker.html) section.
Raw data
{
"_id": null,
"home_page": "https://github.com/mexca/mexca",
"name": "mexca",
"maintainer": "",
"docs_url": null,
"requires_python": "<3.11,>3.7",
"maintainer_email": "",
"keywords": "emotion,multimodal,expression",
"author": "Malte Luken",
"author_email": "m.luken@esciencecenter.nl",
"download_url": "https://files.pythonhosted.org/packages/71/90/0b15986f249b105427d29c0413cea5a18df5597804fb48fcc9b6b82980c7/mexca-1.0.1.tar.gz",
"platform": null,
"description": "\n# Multimodal Emotion Expression Capture Amsterdam\n\n[](https://github.com/mexca/mexca)\n[](https://research-software-directory.org/software/mexca)\n[](https://mexca.readthedocs.io/en/latest/index.html)\n[](https://fair-software.eu)\n[](https://sonarcloud.io/dashboard?id=mexca_mexca)\n[](https://sonarcloud.io/dashboard?id=mexca_mexca)\n[](https://github.com/mexca/mexca/actions/workflows/build.yml)\n[](https://github.com/mexca/mexca/actions/workflows/cffconvert.yml)\n[](https://github.com/mexca/mexca/actions/workflows/markdown-link-check.yml)\n[](https://zenodo.org/badge/latestdoi/500818250)\n[](https://hub.docker.com/u/mexca)\n[](https://github.com/mexca/mexca/actions/workflows/docker.yml)\n[](https://github.com/psf/black)\n\n<div align=\"center\">\n<img src=\"mexca_logo.png\" width=\"400\">\n</div>\n\nmexca is an open-source Python package which aims to capture human emotion expressions from videos in a single pipeline.\n\n## How To Use Mexca\n\nmexca implements the customizable yet easy-to-use Multimodal Emotion eXpression Capture Amsterdam (MEXCA) pipeline for extracting emotion expression features from videos.\nIt contains building blocks that can be used to extract features for individual modalities (i.e., facial expressions, voice, and dialogue/spoken text).\nThe blocks can also be integrated into a single pipeline to extract the features from all modalities at once.\nNext to extracting features, mexca can also identify the speakers shown in the video by clustering speaker and face representations.\nThis allows users to compare emotion expressions across speakers, time, and contexts.\n\nPlease cite mexca if you use it for scientific or commercial purposes.\n\n<div align=\"center\">\n<img src=\"docs/mexca_flowchart.png\" width=\"600\">\n</div>\n\n## Installation\n\nmexca can be installed on Windows, macOS and Linux. We recommend Windows 10, macOS 12.6.x, or Ubuntu. The base package can be installed from PyPI via `pip`:\n\n```console\npip install mexca\n```\n\nThe dependencies for the additional components can be installed via:\n\n```console\npip install mexca[vid,spe,voi,tra,sen]\n```\n\nor:\n\n```console\npip install mexca[all]\n```\n\nThe abbreviations indicate:\n\n* `vid`: FaceExtractor\n* `spe`: SpeakerIdentifier\n* `voi`: VoiceExtractor\n* `tra`: AudioTranscriber\n* `sen`: SentimentExtractor\n\nFor details on the requirements and installation procedure, see the [Quick Installation](https://mexca.readthedocs.io/en/latest/quick_installation.html) and [Installation Details](https://mexca.readthedocs.io/en/latest/installation_details.html) sections of our documentation.\n\n## Getting Started\n\nIf you would like to learn how to use mexca, take a look at our [demo](https://github.com/mexca/mexca/tree/main/examples) notebook and the [Getting Started](https://mexca.readthedocs.io/en/latest/getting_started.html) section of our documentation.\n\n## Examples and Recipes\n\nIn the `examples/` folder, we currently provide two Jupyter notebooks (and a short demo):\n\n- [example_custom_pipeline_components](https://github.com/mexca/mexca/blob/main/examples/example_custom_pipeline_components.ipynb) shows how the standard MEXCA pipeline can be customized and extended\n- [example_emotion_feature_extraction](https://github.com/mexca/mexca/blob/main/examples/example_emotion_feature_extraction.ipynb) shows how to apply the MEXCA pipeline to a video and conduct a basic analysis of the extracted features\n\nThe `recipes/` folder contains two Python scripts that can easily be reused in a new project:\n\n- [recipe_postprocess_features](https://github.com/mexca/mexca/blob/main/recipes/recipe_postprocess_features.py) applies a standard postprocessing routine to extracted features\n- [recipe_standard_pipeline](https://github.com/mexca/mexca/blob/main/recipes/recipe_standard_pipeline.py) applies the standard MEXCA pipeline to a list of videos\n\n## Components\n\nThe pipeline components are described [here](https://mexca.readthedocs.io/en/latest/components.html).\n\n## Documentation\n\nThe documentation of mexca can be found on [Read the Docs](https://mexca.readthedocs.io/en/latest/index.html).\n\n## Contributing\n\nIf you want to contribute to the development of mexca,\nhave a look at the [contribution guidelines](CONTRIBUTING.md).\n\n## License\n\nThe code is licensed under the Apache 2.0 License. This means that mexca can be used, modified and redistributed for free, even for commercial purposes.\n\n## Credits\n\nMexca is being developed by the [Netherlands eScience Center](https://www.esciencecenter.nl/) in collaboration with the [Hot Politics Lab](http://www.hotpolitics.eu/) at the University of Amsterdam.\n\nThis package was created with [Cookiecutter](https://github.com/audreyr/cookiecutter) and the [NLeSC/python-template](https://github.com/NLeSC/python-template).\n\n[^1]: We explain the rationale for this setup in the [Docker](https://mexca.readthedocs.io/en/latest/docker.html) section.\n",
"bugtrack_url": null,
"license": "",
"summary": "Emotion expression capture from multiple modalities.",
"version": "1.0.1",
"project_urls": {
"Bug Tracker": "https://github.com/mexca/mexca/issues",
"Homepage": "https://github.com/mexca/mexca"
},
"split_keywords": [
"emotion",
"multimodal",
"expression"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "858035d198645e1e50a8ffa16ddea734a632d99b5cf0b172eae88b7f87283b85",
"md5": "2437f0a078f97dd9dadc63c3dbb8efa4",
"sha256": "fdb9866da0e25e419a578a1c3b7c4b1574291152c88ad54bf4acb73b68671492"
},
"downloads": -1,
"filename": "mexca-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2437f0a078f97dd9dadc63c3dbb8efa4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.11,>3.7",
"size": 57839,
"upload_time": "2024-01-17T15:35:57",
"upload_time_iso_8601": "2024-01-17T15:35:57.975224Z",
"url": "https://files.pythonhosted.org/packages/85/80/35d198645e1e50a8ffa16ddea734a632d99b5cf0b172eae88b7f87283b85/mexca-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "71900b15986f249b105427d29c0413cea5a18df5597804fb48fcc9b6b82980c7",
"md5": "de4d2343dbe99223808974c29b499382",
"sha256": "1306b0b8f8fbfcb4d9f3e7fdce77d4bbee21d238b2628db13ed518973c14a810"
},
"downloads": -1,
"filename": "mexca-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "de4d2343dbe99223808974c29b499382",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.11,>3.7",
"size": 64415,
"upload_time": "2024-01-17T15:35:59",
"upload_time_iso_8601": "2024-01-17T15:35:59.315523Z",
"url": "https://files.pythonhosted.org/packages/71/90/0b15986f249b105427d29c0413cea5a18df5597804fb48fcc9b6b82980c7/mexca-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-17 15:35:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mexca",
"github_project": "mexca",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "accelerate",
"specs": [
[
"==",
"0.18.0"
]
]
},
{
"name": "av",
"specs": [
[
"==",
"10.0"
]
]
},
{
"name": "bitsandbytes",
"specs": [
[
"==",
"0.38.1"
]
]
},
{
"name": "docker",
"specs": [
[
">=",
"6.0.1"
]
]
},
{
"name": "facenet-pytorch",
"specs": [
[
"==",
"2.5.2"
]
]
},
{
"name": "gdown",
"specs": [
[
"==",
"4.6.0"
]
]
},
{
"name": "intervaltree",
"specs": [
[
">=",
"3.1.0"
]
]
},
{
"name": "moviepy",
"specs": [
[
">=",
"1.0.3"
]
]
},
{
"name": "numpy",
"specs": [
[
"<",
"1.24"
],
[
">=",
"1.21"
]
]
},
{
"name": "openai-whisper",
"specs": []
},
{
"name": "polars",
"specs": [
[
"==",
"0.19.8"
]
]
},
{
"name": "protobuf",
"specs": [
[
"==",
"3.20"
]
]
},
{
"name": "pyannote.audio",
"specs": [
[
"==",
"3.1.1"
]
]
},
{
"name": "pyannote.core",
"specs": [
[
">=",
"5.0"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.4.2"
]
]
},
{
"name": "pyyaml",
"specs": [
[
">=",
"6.0"
]
]
},
{
"name": "scikit-learn",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "scipy",
"specs": [
[
">=",
"1.7.3"
]
]
},
{
"name": "sentencepiece",
"specs": []
},
{
"name": "spectralcluster",
"specs": [
[
"==",
"0.2.16"
]
]
},
{
"name": "srt",
"specs": [
[
">=",
"3.5.2"
]
]
},
{
"name": "torch",
"specs": [
[
">=",
"2.0"
]
]
},
{
"name": "tqdm",
"specs": [
[
">=",
"4.64.0"
]
]
},
{
"name": "transformers",
"specs": [
[
"==",
"4.25.1"
]
]
}
],
"lcname": "mexca"
}
JSON
