# exordium
[](LICENSE)
[](https://www.python.org)
[](https://pytorch.org)
[](https://www.mypy-lang.org/)
Collection of preprocessing functions and deep learning methods.
# Supported features
## Audio
* frequently used io for audio files
* openSMILE feature extraction
* spectrogram calculation
* Wav2Vec2 feature extraction
* CLAP feature extraction
* WavLM feature extraction
## Video
* frequently used io for videos and frames
* bounding box manipulation methods
* face detection with RetinaFace
* face landmarks and head pose with 3DDFA_V2
* body pose estimation with max-human-pose-estimator
* categorical and dimensional emotion estimation with EmoNet
* iris and pupil landmark estimation with MediaPipe Iris
* fine eye landmark estimation with MediaPipe FaceMesh
* eye gaze vector estimation with L2CS-Net
* tracking using IoU and DeepFace
* FAb-Net feature extraction
* OpenFace feature extraction
* R2+1D feature extraction
* Robust Video Matting background removal
* SWIN transformer feature extraction
* FaceMesh landmark estimation
* CLIP feature extraction
* OpenGraphAU action unit estimation
## Text
* BERT feature extraction
* RoBERTa feature extraction
* XML-RoBERTa feature extraction
* Whisper fast speech-to-text
## Utils
* parallel processing
* io decorators
* loss functions
* normalization
* padding/truncating
## Visualization
* graphs
* 3D headpose
* 2D landmarks
* gaze
* saliency maps
* dataframes to images
# Setup
### Install package with all base and optional dependencies from PyPI
```
pip install exordium[all]
```
### Install package with base dependencies from PyPI
```
pip install exordium
```
### Install optional dependencies for specific modules
The following extras will install the base and specific dependencies for using TDDFA_V2.
```
pip install exordium[tddfa]
```
You can install multiple optional dependencies as well.
```
pip install exordium[tddfa,audio]
```
#### Supported extras definitions:
| extras tag | description |
| --- | --- |
| audio | dependencies to process audio data |
| text | dependency to process textual data |
| tddfa | dependencies of TDDFA_V2 for landmark and headpose estimation, or related transformations |
| detection | dependencies for automatic face detection and tracking in videos |
| video | dependencies for various video feature extraction methods |
| all | all previously described extras will be installed |
Note: If you are not sure which tag should be used, just go with the all-mighty "all".
### Install package for development
```
git clone https://github.com/fodorad/exordium
cd exordium
pip install -e .[all]
pip install -U -r requirements.txt
python -m unittest discover -s test
```
# Projects using exordium
### (2023) BlinkLinMulT
LinMulT is trained for blink presence detection and eye state recognition tasks.
Our results demonstrate comparable or superior performance compared to state-of-the-art models on 2 tasks, using 7 public benchmark databases.
* paper: BlinkLinMulT: Transformer-based Eye Blink Detection (accepted, available soon)
* code: https://github.com/fodorad/BlinkLinMulT
### (2022) PersonalityLinMulT
LinMulT is trained for Big Five personality trait estimation using the First Impressions V2 dataset and sentiment estimation using the MOSI and MOSEI datasets.
* paper: Multimodal Sentiment and Personality Perception Under Speech: A Comparison of Transformer-based Architectures ([pdf](https://proceedings.mlr.press/v173/fodor22a/fodor22a.pdf), [website](https://proceedings.mlr.press/v173/fodor22a.html))
* code: https://github.com/fodorad/PersonalityLinMulT
# What's next
* Add support for Blink estimation (DenseNet121, LinT, BlinkLinMulT)
* Add support for Personality trait estimation (PersonalityLinMulT)
# Updates
* 1.3.0: Add support for OpenGraphAU, FaceMesh, SWIM, RVM, XML-RoBERTa, CLIP, CLAP.
* 1.2.0: Add support for L2CS-Net gaze estimation.
* 1.1.0: PyPI publish.
* 1.0.0: Release version.
# Contact
* Ádám Fodor (foauaai@inf.elte.hu)
Raw data
{
"_id": null,
"home_page": null,
"name": "exordium",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "audio, detection, multimodal, preprocess, text, tracking, video",
"author": null,
"author_email": "fodorad <foauaai@inf.elte.hu>",
"download_url": "https://files.pythonhosted.org/packages/b4/a9/7054191990579fad0db6c74a04c085c17dd01622424eb3b6fd2582dbc981/exordium-1.4.1.tar.gz",
"platform": null,
"description": "# exordium\n[](LICENSE)\n[](https://www.python.org)\n[](https://pytorch.org)\n[](https://www.mypy-lang.org/)\n\nCollection of preprocessing functions and deep learning methods.\n\n# Supported features\n## Audio\n* frequently used io for audio files\n* openSMILE feature extraction\n* spectrogram calculation\n* Wav2Vec2 feature extraction\n* CLAP feature extraction\n* WavLM feature extraction\n\n## Video\n* frequently used io for videos and frames\n* bounding box manipulation methods\n* face detection with RetinaFace\n* face landmarks and head pose with 3DDFA_V2\n* body pose estimation with max-human-pose-estimator\n* categorical and dimensional emotion estimation with EmoNet\n* iris and pupil landmark estimation with MediaPipe Iris\n* fine eye landmark estimation with MediaPipe FaceMesh\n* eye gaze vector estimation with L2CS-Net\n* tracking using IoU and DeepFace\n* FAb-Net feature extraction\n* OpenFace feature extraction\n* R2+1D feature extraction\n* Robust Video Matting background removal\n* SWIN transformer feature extraction\n* FaceMesh landmark estimation\n* CLIP feature extraction\n* OpenGraphAU action unit estimation\n\n## Text\n* BERT feature extraction\n* RoBERTa feature extraction\n* XML-RoBERTa feature extraction\n* Whisper fast speech-to-text\n\n## Utils\n* parallel processing\n* io decorators\n* loss functions\n* normalization\n* padding/truncating\n\n## Visualization\n* graphs\n* 3D headpose\n* 2D landmarks\n* gaze\n* saliency maps\n* dataframes to images\n\n# Setup\n### Install package with all base and optional dependencies from PyPI\n```\npip install exordium[all]\n```\n### Install package with base dependencies from PyPI\n```\npip install exordium\n```\n### Install optional dependencies for specific modules\nThe following extras will install the base and specific dependencies for using TDDFA_V2.\n```\npip install exordium[tddfa]\n```\nYou can install multiple optional dependencies as well.\n```\npip install exordium[tddfa,audio]\n```\n\n#### Supported extras definitions:\n| extras tag | description |\n| --- | --- |\n| audio | dependencies to process audio data |\n| text | dependency to process textual data |\n| tddfa | dependencies of TDDFA_V2 for landmark and headpose estimation, or related transformations |\n| detection | dependencies for automatic face detection and tracking in videos |\n| video | dependencies for various video feature extraction methods |\n| all | all previously described extras will be installed |\n\nNote: If you are not sure which tag should be used, just go with the all-mighty \"all\".\n\n### Install package for development\n```\ngit clone https://github.com/fodorad/exordium\ncd exordium\npip install -e .[all]\npip install -U -r requirements.txt\npython -m unittest discover -s test\n```\n\n# Projects using exordium\n\n### (2023) BlinkLinMulT\nLinMulT is trained for blink presence detection and eye state recognition tasks.\nOur results demonstrate comparable or superior performance compared to state-of-the-art models on 2 tasks, using 7 public benchmark databases.\n* paper: BlinkLinMulT: Transformer-based Eye Blink Detection (accepted, available soon)\n* code: https://github.com/fodorad/BlinkLinMulT\n\n### (2022) PersonalityLinMulT\nLinMulT is trained for Big Five personality trait estimation using the First Impressions V2 dataset and sentiment estimation using the MOSI and MOSEI datasets.\n* paper: Multimodal Sentiment and Personality Perception Under Speech: A Comparison of Transformer-based Architectures ([pdf](https://proceedings.mlr.press/v173/fodor22a/fodor22a.pdf), [website](https://proceedings.mlr.press/v173/fodor22a.html))\n* code: https://github.com/fodorad/PersonalityLinMulT\n\n# What's next\n* Add support for Blink estimation (DenseNet121, LinT, BlinkLinMulT)\n* Add support for Personality trait estimation (PersonalityLinMulT)\n\n# Updates\n* 1.3.0: Add support for OpenGraphAU, FaceMesh, SWIM, RVM, XML-RoBERTa, CLIP, CLAP.\n* 1.2.0: Add support for L2CS-Net gaze estimation.\n* 1.1.0: PyPI publish.\n* 1.0.0: Release version.\n\n# Contact\n* \u00c1d\u00e1m Fodor (foauaai@inf.elte.hu)",
"bugtrack_url": null,
"license": null,
"summary": "Collection of utility tools and deep learning methods.",
"version": "1.4.1",
"project_urls": {
"Documentation": "https://github.com/fodorad/exordium#readme",
"Issues": "https://github.com/fodorad/exordium/issues",
"Source": "https://github.com/fodorad/exordium"
},
"split_keywords": [
"audio",
" detection",
" multimodal",
" preprocess",
" text",
" tracking",
" video"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "5e37c8665c3c2cf2607b1bdfe7fcb4d9d806dfe889f134856eefa49bc8ea9b20",
"md5": "7ecc4cf96851f6c786e8d79db15cc550",
"sha256": "21fb53c7252d351d77a8aac99741eb0d0031cb69384536f4a3734d47d2175779"
},
"downloads": -1,
"filename": "exordium-1.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7ecc4cf96851f6c786e8d79db15cc550",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 97690,
"upload_time": "2025-01-10T09:27:49",
"upload_time_iso_8601": "2025-01-10T09:27:49.208803Z",
"url": "https://files.pythonhosted.org/packages/5e/37/c8665c3c2cf2607b1bdfe7fcb4d9d806dfe889f134856eefa49bc8ea9b20/exordium-1.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b4a97054191990579fad0db6c74a04c085c17dd01622424eb3b6fd2582dbc981",
"md5": "61b7467370b8ca20c1df65ba2201147b",
"sha256": "f955a9a3cf08197fc56f8e421020bfe2d7b7e18b30d1ec68858a6ae6fcbc2b5b"
},
"downloads": -1,
"filename": "exordium-1.4.1.tar.gz",
"has_sig": false,
"md5_digest": "61b7467370b8ca20c1df65ba2201147b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 20278360,
"upload_time": "2025-01-10T09:27:44",
"upload_time_iso_8601": "2025-01-10T09:27:44.505242Z",
"url": "https://files.pythonhosted.org/packages/b4/a9/7054191990579fad0db6c74a04c085c17dd01622424eb3b6fd2582dbc981/exordium-1.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-10 09:27:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fodorad",
"github_project": "exordium#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "toml",
"specs": []
},
{
"name": "Pillow",
"specs": []
},
{
"name": "opencv-python",
"specs": []
},
{
"name": "opencv-contrib-python",
"specs": []
},
{
"name": "scipy",
"specs": []
},
{
"name": "torch",
"specs": []
},
{
"name": "decord",
"specs": []
},
{
"name": "tqdm",
"specs": []
},
{
"name": "matplotlib",
"specs": []
}
],
"lcname": "exordium"
}
JSON
