# <div align="center"> **NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction (Accepted at WACV24)** </div>
<p align="center">
<img src="https://github.com/thohemp/archive/blob/main/nitec.gif" alt="animated" />
</p>
## **Citing**
If you find our work useful, please cite the paper:
```BibTeX
@misc{hempel2023nitec,
title={NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction},
author={Thorsten Hempel and Magnus Jung and Ahmed A. Abdelrahman and Ayoub Al-Hamadi},
year={2023},
eprint={2311.04505},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## <div align="center"> **Paper**</div>
> [Thorsten Hempel, Magnus Jung, Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction", *accepted at WACV 2024*.](https://arxiv.org/abs/2311.04505)
## <div align="center"> **Abstract**</div>
>Eye contact is a crucial non-verbal interaction modality and plays an important role in our everyday social life. While humans are very sensitive to eye contact, the capabilities of machines to capture a person's gaze are still mediocre. We tackle this challenge and present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions, making it a valuable resource for advancing ego-vision-based eye contact research. Our extensive evaluations on NITEC demonstrate strong cross-dataset performance, emphasizing its effectiveness and adaptability in various scenarios, that allows seamless utilization to the fields of computer vision, human-computer interaction, and social robotics. We make our NITEC dataset publicly available to foster reproducibility and further exploration in the field of ego-vision interaction.
# <div align="center"> Quick Usage: </div>
```sh
pip install face_detection@git+https://github.com/elliottzheng/face-detection
pip install nitec
```
Example usage:
```py
from nitec import NITEC_Classifier, visualize
import cv2
nitec_pipeline = NITEC_Classifier(
weights= CWD / 'models' / 'nitec_rs18_e20.pth',
device=torch.device('cuda') # or 'cpu'
)
cap = cv2.VideoCapture(0)
_, frame = cap.read()
# Process frame and visualize
results = nitec_pipeline.predict(frame)
frame = visualize(frame, results, confidence=0.5)
```
# <div align="center"> Train / Test </div>
## NITEC Dataset
Prepare the dataset as explained [ here](data/README.MD).
## Snapshots
Download from here: https://drive.google.com/drive/folders/1zc6NZZ6yA4NJ52Nn0bgky1XpZs9Z0hSJ?usp=sharing
## Train
```py
python train.py \
--gpu 0 \
--num_epochs 50 \
--batch_size 64 \
--lr 0.0001 \
```
## Test
```py
python test.py \
--snapshot models/nitec_rs18_20.pth \
--gpu 0 \
```
Raw data
{
"_id": null,
"home_page": "",
"name": "nitec",
"maintainer": "",
"docs_url": null,
"requires_python": ">3.8",
"maintainer_email": "",
"keywords": "eye contact,estimation,human-robot interact,hri,deep-learning,pytorch,engagement,wacv",
"author": "Thorsten Hempel, Magnus Jung, Ahmed Abdelrahman",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/3c/fb/5b8d662d2ce9f3028a2313af7e8476f58cdd9b4f356efd4a27a546e6d367/nitec-0.0.2.tar.gz",
"platform": null,
"description": "# <div align=\"center\"> **NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction (Accepted at WACV24)** </div>\n\n<p align=\"center\">\n <img src=\"https://github.com/thohemp/archive/blob/main/nitec.gif\" alt=\"animated\" />\n</p>\n\n## **Citing**\n\nIf you find our work useful, please cite the paper:\n\n```BibTeX\n@misc{hempel2023nitec,\n title={NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction}, \n author={Thorsten Hempel and Magnus Jung and Ahmed A. Abdelrahman and Ayoub Al-Hamadi},\n year={2023},\n eprint={2311.04505},\n archivePrefix={arXiv},\n primaryClass={cs.CV}\n}\n```\n## <div align=\"center\"> **Paper**</div>\n> [Thorsten Hempel, Magnus Jung, Ahmed A. Abdelrahman and Ayoub Al-Hamadi, \"NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction\", *accepted at WACV 2024*.](https://arxiv.org/abs/2311.04505)\n\n## <div align=\"center\"> **Abstract**</div>\n>Eye contact is a crucial non-verbal interaction modality and plays an important role in our everyday social life. While humans are very sensitive to eye contact, the capabilities of machines to capture a person's gaze are still mediocre. We tackle this challenge and present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions, making it a valuable resource for advancing ego-vision-based eye contact research. Our extensive evaluations on NITEC demonstrate strong cross-dataset performance, emphasizing its effectiveness and adaptability in various scenarios, that allows seamless utilization to the fields of computer vision, human-computer interaction, and social robotics. We make our NITEC dataset publicly available to foster reproducibility and further exploration in the field of ego-vision interaction.\n\n\n# <div align=\"center\"> Quick Usage: </div>\n\n```sh\npip install face_detection@git+https://github.com/elliottzheng/face-detection\npip install nitec\n```\n\nExample usage:\n\n```py\nfrom nitec import NITEC_Classifier, visualize\nimport cv2\n\nnitec_pipeline = NITEC_Classifier(\n weights= CWD / 'models' / 'nitec_rs18_e20.pth',\n device=torch.device('cuda') # or 'cpu'\n)\n\ncap = cv2.VideoCapture(0)\n\n_, frame = cap.read() \n# Process frame and visualize\nresults = nitec_pipeline.predict(frame)\nframe = visualize(frame, results, confidence=0.5)\n\n```\n\n\n\n# <div align=\"center\"> Train / Test </div>\n\n## NITEC Dataset\nPrepare the dataset as explained [ here](data/README.MD).\n\n## Snapshots\n\nDownload from here: https://drive.google.com/drive/folders/1zc6NZZ6yA4NJ52Nn0bgky1XpZs9Z0hSJ?usp=sharing\n\n## Train\n```py\n python train.py \\\n --gpu 0 \\\n --num_epochs 50 \\\n --batch_size 64 \\\n --lr 0.0001 \\\n```\n\n\n## Test\n\n```py\n python test.py \\\n --snapshot models/nitec_rs18_20.pth \\\n --gpu 0 \\\n```\n\n",
"bugtrack_url": null,
"license": "",
"summary": "The official PyTorch implementation of NITEC for eye contact detection.",
"version": "0.0.2",
"project_urls": {
"homepath": "https://github.com/thohemp/nitec",
"repository": "https://github.com/thohemp/nitec"
},
"split_keywords": [
"eye contact",
"estimation",
"human-robot interact",
"hri",
"deep-learning",
"pytorch",
"engagement",
"wacv"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "92be7bc7264384fb33eb23b1b2e419431578d87fdf1b25776b00893ba8e32361",
"md5": "7f220f7c6d0c1e1929cd6b1a4fabc625",
"sha256": "76b7a44b8bcc8dd7862c4d9e95edddf827cb05f4573b761f6033711b3da934ec"
},
"downloads": -1,
"filename": "nitec-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7f220f7c6d0c1e1929cd6b1a4fabc625",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">3.8",
"size": 7328,
"upload_time": "2023-11-20T10:00:46",
"upload_time_iso_8601": "2023-11-20T10:00:46.964702Z",
"url": "https://files.pythonhosted.org/packages/92/be/7bc7264384fb33eb23b1b2e419431578d87fdf1b25776b00893ba8e32361/nitec-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3cfb5b8d662d2ce9f3028a2313af7e8476f58cdd9b4f356efd4a27a546e6d367",
"md5": "9fb928b7d0aa892beab8f654be660d11",
"sha256": "e62ed6f1556aaecdd15476dc1948c73f83a502ebfe110d67e7c7922ebd8b254a"
},
"downloads": -1,
"filename": "nitec-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "9fb928b7d0aa892beab8f654be660d11",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">3.8",
"size": 6721,
"upload_time": "2023-11-20T10:00:48",
"upload_time_iso_8601": "2023-11-20T10:00:48.737685Z",
"url": "https://files.pythonhosted.org/packages/3c/fb/5b8d662d2ce9f3028a2313af7e8476f58cdd9b4f356efd4a27a546e6d367/nitec-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-20 10:00:48",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "thohemp",
"github_project": "nitec",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "nitec"
}
JSON
