# ATHENA Toolbox
ATHENA (Automatically Tracking Hands Expertly with No Annotations) is a Python-based toolbox designed to process multi-camera video recordings, extract 2D and 3D body and hand landmarks using MediaPipe, and perform triangulation and refinement of these landmarks. The toolbox provides a user-friendly GUI for selecting videos and configuring processing options.
<table>
<tr>
<td><img src="athena/logo.png" alt="logo" width="240"/></td>
<td><img src="athena/gui.png" alt="GUI screenshot" width="400"/></td>
</tr>
</table>
## Features
- Multi-Camera Processing: Handles multiple camera inputs for comprehensive analysis.
- MediaPipe Integration: Utilizes Google’s MediaPipe for extracting 2D landmarks of the body and hands.
- 3D Triangulation: Triangulates 2D landmarks from multiple cameras to reconstruct 3D landmarks.
- Smoothing and Refinement: Applies smoothing algorithms to refine the 3D landmarks.
- Parallel Processing: Supports multiprocessing to speed up the processing of large datasets.
- GUI for Easy Configuration: Provides a graphical user interface to select folders, recordings, and set processing options.
- Visualization: Offers options to save images and videos of the processed landmarks for visualization.
## Installation
### Prerequisites
- Operating System: Windows, macOS, or Linux.
- Python Version: Python 3.12
- Hardware Requirements:
- CPU: Multi-core processor recommended for parallel processing.
- GPU (Optional): NVIDIA GPU for accelerated processing (if GPU processing is enabled).
### Installation Steps
Use your package manager of choice to create an environment with Python 3.12. For example, using conda:
```console
conda create -n athena python=3.12
```
Activate the environment:
```console
conda activate athena
```
Then install the package:
```console
pip install athena-tracking
```
Or to get the latest version directly from GitHub:
```console
pip install git+https://github.com/neural-control-and-computation-lab/athena.git
```
## Usage
### 1. Organize Your Videos
Place your synchronized video recordings in a main folder, structured as follows:
```console
main_folder/
├── videos/
│ ├── recording1/
│ │ ├── cam0.avi
│ │ ├── cam1.avi
│ │ └── ...
│ ├── recording2/
│ │ ├── cam0.avi
│ │ ├── cam1.avi
│ │ └── ...
└── calibration/
├── cam0.yaml
├── cam1.yaml
└── ...
```
- Each recording folder should contain video files from multiple cameras (e.g., cam0.avi, cam1.avi).
- The calibration folder should contain calibration files (.yaml) for each camera.
For recording and calibrating your cameras, we highly recommend the [JARVIS Toolbox](https://jarvis-mocap.github.io/jarvis-docs/).
### 2. Ensure Calibration Files are Correct
- Calibration files should be labelled with camera names that match recorded videos.
- Calibration files should contain intrinsic and extrinsic parameters for each camera.
- The calibration files are essential for accurate triangulation of 3D landmarks.
### 3. Running the Toolbox
Launch the GUI:
```console
athena
```
1. Select Main Folder and Recordings
- Click on the “Select Folder” button to choose your main folder containing the videos and calibration directories.
- A new window will appear, allowing you to select specific recordings to process. Select the desired recordings and click “Select”.
2. Configure Processing Options
- General Settings:
- Fraction of Frames to Process: Slide to select the fraction of frames you want to process (from 0 to 1). Default is 1.0 (process all frames).
- Number of Parallel Processes: Choose the number of processes for parallel processing. Default is the number of available CPU cores.
- GPU Processing: Check this option to enable GPU acceleration (requires compatible NVIDIA GPU).
- MediaPipe Processing:
- Run Mediapipe: Check this option to run MediaPipe for extracting 2D landmarks.
- Save Images: Save images with landmarks drawn (can consume significant storage).
- Save Video: Save videos with landmarks overlaid (requires “Save Images” to be enabled).
- Minimum Hand Detection & Tracking Confidence: Adjust the confidence threshold for hand detection (default is 0.9).
- Minimum Pose Detection & Tracking Confidence: Adjust the confidence threshold for pose detection (default is 0.9).
- Run Triangulation and Refinement:
- Run Triangulation and Refinement: Check this option to perform 3D triangulation of landmarks and filtering.
- All points: low-freq cutoff (Hz): Set the low-frequency cutoff for smoothing of all points (default is 10 Hz).
- Save 3D Images: Save images of the 3D landmarks.
- Save 3D Video: Save videos of the 3D landmarks.
- Save Refined 2D Images: Save images of the refined 2D landmarks after triangulation.
- Save Refined 2D Video: Save videos of the refined 2D landmarks
3. Start Processing
- Click the “GO” button to start processing.
- A progress window will appear, showing the processing progress and average FPS.
### 4. Output Folder Structure
After processing, the toolbox will create additional directories within your main folder:
```console
main_folder/
├── images/ # Contains images with landmarks drawn
├── imagesrefined/ # Contains refined images after triangulation
├── landmarks/ # Contains 2D and 3D landmark data
├── videos_processed/ # Contains processed videos with landmarks overlaid
└── ... # Original videos and calibration files
```
To create a single video that contains all videos (2D and 3D), you can use the 'montage' script and select the recording folder using the popup window:
```console
python -m athena.montage
```
### Troubleshooting
- MediaPipe Errors: Ensure that MediaPipe is correctly installed and compatible with your Python version. MediaPipe may have specific requirements, especially for GPU support.
- Calibration Mismatch: Verify that the number of calibration files matches the number of camera videos.
- High Memory Usage: Processing large videos or saving images and videos can consume significant memory and storage.
- Permission Issues: Ensure that you have read/write permissions for the directories where data is stored and processed.
- Conda Environment Activation: Always make sure that the Conda environment is activated before running the scripts.
### Contributing
Contributions are welcome! If you encounter issues or have suggestions for improvements, please create an issue or submit a pull request on GitHub.
### Frequently Asked Questions (FAQ)
Q: Do I need a GPU to run the ATHENA Toolbox?\
A: No, a GPU is not required and does not generally speed up processing, which is already highly parallelized. If you have a compatible NVIDIA GPU, you can enable GPU processing in the options. Ensure that your GPU drivers and CUDA toolkit are correctly installed.
Q: Can I process videos from only one camera?\
A: Yes, you can process videos from a single camera to extract 2D landmarks. However, triangulation to 3D landmarks requires synchronized videos from at least two cameras and their corresponding calibration files.
Q: How do I obtain the calibration files?\
A: Calibration files are generated using camera calibration techniques, often involving capturing images of a known pattern (like a chessboard) from different angles. You can use OpenCV’s calibration tools or other software to create these .yaml files.
ATHENA accepts calibration files created by JARVIS or Anipose without any modification.
Q: The processing is slow. How can I speed it up?\
A: You can increase the number of parallel processes if your CPU has more cores. Enabling GPU processing can also significantly speed up the landmark detection step. Additionally, processing a smaller fraction of frames can reduce computation time.
Q: Where can I find the output data after processing?\
A: Processed data, images, and videos are saved in the images/, imagesrefined/, landmarks/, and videos_processed/ directories within your main folder.
Raw data
{
"_id": null,
"home_page": null,
"name": "athena-tracking",
"maintainer": null,
"docs_url": null,
"requires_python": "==3.12.*",
"maintainer_email": null,
"keywords": "athena, hand tracking, pose estimation, machine learning",
"author": null,
"author_email": "Jonathan Michaels <jmichae@yorku.ca>, Daanish Mulla <daanish.mulla@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/b3/90/0e69c9162aa06121d2fd25f6b6892169eea8222ac5d45004c9fb28abcf02/athena_tracking-0.9.tar.gz",
"platform": null,
"description": "# ATHENA Toolbox\nATHENA (Automatically Tracking Hands Expertly with No Annotations) is a Python-based toolbox designed to process multi-camera video recordings, extract 2D and 3D body and hand landmarks using MediaPipe, and perform triangulation and refinement of these landmarks. The toolbox provides a user-friendly GUI for selecting videos and configuring processing options.\n\n<table>\n <tr>\n <td><img src=\"athena/logo.png\" alt=\"logo\" width=\"240\"/></td>\n <td><img src=\"athena/gui.png\" alt=\"GUI screenshot\" width=\"400\"/></td>\n </tr>\n</table>\n\n## Features\n- Multi-Camera Processing: Handles multiple camera inputs for comprehensive analysis.\n- MediaPipe Integration: Utilizes Google\u2019s MediaPipe for extracting 2D landmarks of the body and hands.\n- 3D Triangulation: Triangulates 2D landmarks from multiple cameras to reconstruct 3D landmarks.\n- Smoothing and Refinement: Applies smoothing algorithms to refine the 3D landmarks.\n- Parallel Processing: Supports multiprocessing to speed up the processing of large datasets.\n- GUI for Easy Configuration: Provides a graphical user interface to select folders, recordings, and set processing options.\n- Visualization: Offers options to save images and videos of the processed landmarks for visualization.\n\n## Installation\n### Prerequisites\n- Operating System: Windows, macOS, or Linux.\n- Python Version: Python 3.12\n- Hardware Requirements:\n- CPU: Multi-core processor recommended for parallel processing.\n- GPU (Optional): NVIDIA GPU for accelerated processing (if GPU processing is enabled).\n\n### Installation Steps\n\nUse your package manager of choice to create an environment with Python 3.12. For example, using conda:\n```console\nconda create -n athena python=3.12\n```\nActivate the environment:\n```console\nconda activate athena\n```\nThen install the package:\n```console\npip install athena-tracking\n```\n\nOr to get the latest version directly from GitHub:\n```console\npip install git+https://github.com/neural-control-and-computation-lab/athena.git\n```\n\n## Usage\n### 1.\tOrganize Your Videos\nPlace your synchronized video recordings in a main folder, structured as follows:\n```console\nmain_folder/\n\u251c\u2500\u2500 videos/\n\u2502 \u251c\u2500\u2500 recording1/\n\u2502 \u2502 \u251c\u2500\u2500 cam0.avi\n\u2502 \u2502 \u251c\u2500\u2500 cam1.avi\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2502 \u251c\u2500\u2500 recording2/\n\u2502 \u2502 \u251c\u2500\u2500 cam0.avi\n\u2502 \u2502 \u251c\u2500\u2500 cam1.avi\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2514\u2500\u2500 calibration/\n \u251c\u2500\u2500 cam0.yaml\n \u251c\u2500\u2500 cam1.yaml\n \u2514\u2500\u2500 ...\n```\n\n- Each recording folder should contain video files from multiple cameras (e.g., cam0.avi, cam1.avi).\n- The calibration folder should contain calibration files (.yaml) for each camera.\n\nFor recording and calibrating your cameras, we highly recommend the [JARVIS Toolbox](https://jarvis-mocap.github.io/jarvis-docs/).\n\n### 2.\tEnsure Calibration Files are Correct\n- Calibration files should be labelled with camera names that match recorded videos.\n- Calibration files should contain intrinsic and extrinsic parameters for each camera.\n- The calibration files are essential for accurate triangulation of 3D landmarks.\n\n### 3. Running the Toolbox\nLaunch the GUI:\n```console\nathena\n```\n\n1. Select Main Folder and Recordings \n - Click on the \u201cSelect Folder\u201d button to choose your main folder containing the videos and calibration directories.\n - A new window will appear, allowing you to select specific recordings to process. Select the desired recordings and click \u201cSelect\u201d.\n2. Configure Processing Options\n - General Settings:\n - Fraction of Frames to Process: Slide to select the fraction of frames you want to process (from 0 to 1). Default is 1.0 (process all frames).\n - Number of Parallel Processes: Choose the number of processes for parallel processing. Default is the number of available CPU cores.\n - GPU Processing: Check this option to enable GPU acceleration (requires compatible NVIDIA GPU).\n - MediaPipe Processing:\n - Run Mediapipe: Check this option to run MediaPipe for extracting 2D landmarks.\n - Save Images: Save images with landmarks drawn (can consume significant storage).\n - Save Video: Save videos with landmarks overlaid (requires \u201cSave Images\u201d to be enabled).\n - Minimum Hand Detection & Tracking Confidence: Adjust the confidence threshold for hand detection (default is 0.9).\n - Minimum Pose Detection & Tracking Confidence: Adjust the confidence threshold for pose detection (default is 0.9).\n - Run Triangulation and Refinement:\n - Run Triangulation and Refinement: Check this option to perform 3D triangulation of landmarks and filtering.\n - All points: low-freq cutoff (Hz): Set the low-frequency cutoff for smoothing of all points (default is 10 Hz).\n - Save 3D Images: Save images of the 3D landmarks.\n - Save 3D Video: Save videos of the 3D landmarks.\n - Save Refined 2D Images: Save images of the refined 2D landmarks after triangulation.\n - Save Refined 2D Video: Save videos of the refined 2D landmarks\n3. Start Processing\n - Click the \u201cGO\u201d button to start processing.\n - A progress window will appear, showing the processing progress and average FPS.\n\n### 4. Output Folder Structure\nAfter processing, the toolbox will create additional directories within your main folder:\n```console\nmain_folder/\n\u251c\u2500\u2500 images/ # Contains images with landmarks drawn\n\u251c\u2500\u2500 imagesrefined/ # Contains refined images after triangulation\n\u251c\u2500\u2500 landmarks/ # Contains 2D and 3D landmark data\n\u251c\u2500\u2500 videos_processed/ # Contains processed videos with landmarks overlaid\n\u2514\u2500\u2500 ... # Original videos and calibration files\n```\n\nTo create a single video that contains all videos (2D and 3D), you can use the 'montage' script and select the recording folder using the popup window:\n```console\npython -m athena.montage\n```\n\n### Troubleshooting\n- MediaPipe Errors: Ensure that MediaPipe is correctly installed and compatible with your Python version. MediaPipe may have specific requirements, especially for GPU support.\n- Calibration Mismatch: Verify that the number of calibration files matches the number of camera videos.\n- High Memory Usage: Processing large videos or saving images and videos can consume significant memory and storage.\n- Permission Issues: Ensure that you have read/write permissions for the directories where data is stored and processed.\n- Conda Environment Activation: Always make sure that the Conda environment is activated before running the scripts.\n\n### Contributing\nContributions are welcome! If you encounter issues or have suggestions for improvements, please create an issue or submit a pull request on GitHub.\n\n### Frequently Asked Questions (FAQ)\n\nQ: Do I need a GPU to run the ATHENA Toolbox?\\\nA: No, a GPU is not required and does not generally speed up processing, which is already highly parallelized. If you have a compatible NVIDIA GPU, you can enable GPU processing in the options. Ensure that your GPU drivers and CUDA toolkit are correctly installed.\n\nQ: Can I process videos from only one camera?\\\nA: Yes, you can process videos from a single camera to extract 2D landmarks. However, triangulation to 3D landmarks requires synchronized videos from at least two cameras and their corresponding calibration files.\n\nQ: How do I obtain the calibration files?\\\nA: Calibration files are generated using camera calibration techniques, often involving capturing images of a known pattern (like a chessboard) from different angles. You can use OpenCV\u2019s calibration tools or other software to create these .yaml files.\nATHENA accepts calibration files created by JARVIS or Anipose without any modification.\n\nQ: The processing is slow. How can I speed it up?\\\nA: You can increase the number of parallel processes if your CPU has more cores. Enabling GPU processing can also significantly speed up the landmark detection step. Additionally, processing a smaller fraction of frames can reduce computation time.\n\nQ: Where can I find the output data after processing?\\\nA: Processed data, images, and videos are saved in the images/, imagesrefined/, landmarks/, and videos_processed/ directories within your main folder.\n",
"bugtrack_url": null,
"license": null,
"summary": "ATHENA: Automatically Tracking Hands Expertly with No Annotations",
"version": "0.9",
"project_urls": {
"GitHub": "https://github.com/neural-control-and-computation-lab/athena"
},
"split_keywords": [
"athena",
" hand tracking",
" pose estimation",
" machine learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e363087f1e53178e2f0116a7339105fe1e136b73105951f2560ae6f89889c6fd",
"md5": "cbd2a3fcbe912b5e81b696ad1b1735fe",
"sha256": "e6abaae430332959e2742b327b36f856027240024dead0c268c3e7f103af1d7d"
},
"downloads": -1,
"filename": "athena_tracking-0.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cbd2a3fcbe912b5e81b696ad1b1735fe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "==3.12.*",
"size": 14293141,
"upload_time": "2025-08-11T20:49:36",
"upload_time_iso_8601": "2025-08-11T20:49:36.280973Z",
"url": "https://files.pythonhosted.org/packages/e3/63/087f1e53178e2f0116a7339105fe1e136b73105951f2560ae6f89889c6fd/athena_tracking-0.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b3900e69c9162aa06121d2fd25f6b6892169eea8222ac5d45004c9fb28abcf02",
"md5": "81c62e4b630c81821914ea10f655c5ef",
"sha256": "58303c734f09b3942b2dad7409dd62b8500e7b0ee692e11edae57262c6cbc5d2"
},
"downloads": -1,
"filename": "athena_tracking-0.9.tar.gz",
"has_sig": false,
"md5_digest": "81c62e4b630c81821914ea10f655c5ef",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "==3.12.*",
"size": 14256245,
"upload_time": "2025-08-11T20:49:39",
"upload_time_iso_8601": "2025-08-11T20:49:39.274252Z",
"url": "https://files.pythonhosted.org/packages/b3/90/0e69c9162aa06121d2fd25f6b6892169eea8222ac5d45004c9fb28abcf02/athena_tracking-0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-11 20:49:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "neural-control-and-computation-lab",
"github_project": "athena",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "athena-tracking"
}
JSON
