# ATHENA Toolbox
ATHENA (Automatically Tracking Hands Expertly with No Annotations) is a Python-based toolbox designed to process multi-camera video recordings, extract 2D and 3D body and hand landmarks using MediaPipe, and perform triangulation and refinement of these landmarks. The toolbox provides a user-friendly GUI for selecting videos and configuring processing options.
## Features
- Multi-Camera Processing: Handles multiple camera inputs for comprehensive analysis.
- MediaPipe Integration: Utilizes Google’s MediaPipe for extracting 2D landmarks of the body and hands.
- 3D Triangulation: Triangulates 2D landmarks from multiple cameras to reconstruct 3D landmarks.
- Smoothing and Refinement: Applies smoothing algorithms to refine the 3D landmarks.
- Parallel Processing: Supports multiprocessing to speed up the processing of large datasets.
- GUI for Easy Configuration: Provides a graphical user interface to select folders, recordings, and set processing options.
- Visualization: Offers options to save images and videos of the processed landmarks for visualization.
## Installation
### Prerequisites
- Operating System: Windows, macOS, or Linux.
- Python Version: Python 3.9 or higher.
- Hardware Requirements:
- CPU: Multi-core processor recommended for parallel processing.
- GPU (Optional): NVIDIA GPU for accelerated processing (if GPU processing is enabled).
### Installation Steps
```console
git clone https://github.com/neural-control-and-computation-lab/athena.git
cd athena
pip install .
```
It is recommended to create a conda environment before installing to manage dependencies and avoid conflicts.
## Usage
### 1. Organize Your Videos
Place your video recordings in a main folder, structured as follows:
```console
main_folder/
├── videos/
│ ├── recording1/
│ │ ├── cam0.avi
│ │ ├── cam1.avi
│ │ └── ...
│ ├── recording2/
│ │ ├── cam0.avi
│ │ ├── cam1.avi
│ │ └── ...
└── calibration/
├── cam0.yaml
├── cam1.yaml
└── ...
```
- Each recording folder should contain video files from multiple cameras (e.g., cam0.avi, cam1.avi).
- The calibration folder should contain calibration files (.yaml) for each camera.
### 2. Ensure Calibration Files are Correct
- Calibration files should be labelled with camera names that match recorded videos.
- Calibration files should contain intrinsic and extrinsic parameters for each camera.
- The calibration files are essential for accurate triangulation of 3D landmarks.
### 3. Running the Toolbox
Launch the GUI:
```console
athena
```
1. Select Main Folder and Recordings
- Click on the “Select Folder” button to choose your main folder containing the videos and calibration directories.
- A new window will appear, allowing you to select specific recordings to process. Select the desired recordings and click “Select”.
2. Configure Processing Options
- General Settings:
- Fraction of Frames to Process: Slide to select the fraction of frames you want to process (from 0 to 1). Default is 1.0 (process all frames).
- Number of Parallel Processes: Choose the number of processes for parallel processing. Default is the number of available CPU cores.
- GPU Processing: Check this option to enable GPU acceleration (requires compatible NVIDIA GPU).
- MediaPipe Processing:
- Run Mediapipe: Check this option to run MediaPipe for extracting 2D landmarks.
- Save Images: Save images with landmarks drawn (can consume significant storage).
- Save Video: Save videos with landmarks overlaid (requires “Save Images” to be enabled).
- Minimum Hand Detection & Tracking Confidence: Adjust the confidence threshold for hand detection (default is 0.9).
- Minimum Pose Detection & Tracking Confidence: Adjust the confidence threshold for pose detection (default is 0.9).
- Triangulation:
- Triangulation: Check this option to perform 3D triangulation of landmarks.
- Save Images: Save images of the 3D landmarks.
- Save Video: Save videos of the 3D landmarks.
3. Start Processing
- Click the “GO” button to start processing.
- A progress window will appear, showing the processing progress and average FPS.
### 4. Output Folder Structure
After processing, the toolbox will create additional directories within your main folder:
```console
main_folder/
├── images/ # Contains images with landmarks drawn
├── imagesrefined/ # Contains refined images after triangulation
├── landmarks/ # Contains 2D and 3D landmark data
├── videos_processed/ # Contains processed videos with landmarks overlaid
└── ... # Original videos and calibration files
```
### Troubleshooting
- MediaPipe Errors: Ensure that MediaPipe is correctly installed and compatible with your Python version. MediaPipe may have specific requirements, especially for GPU support.
- Calibration Mismatch: Verify that the number of calibration files matches the number of camera videos.
- High Memory Usage: Processing large videos or saving images and videos can consume significant memory and storage.
- Permission Issues: Ensure that you have read/write permissions for the directories where data is stored and processed.
- Conda Environment Activation: Always make sure that the Conda environment is activated before running the scripts.
### Contributing
Contributions are welcome! If you encounter issues or have suggestions for improvements, please create an issue or submit a pull request on GitHub.
### Frequently Asked Questions (FAQ)
Q: Do I need a GPU to run the ATHENA Toolbox?\
A: No, a GPU is not required and does not generally speed up processing, which is already highly parallelized. If you have a compatible NVIDIA GPU, you can enable GPU processing in the options. Ensure that your GPU drivers and CUDA toolkit are correctly installed.
Q: Can I process videos from only one camera?\
A: Yes, you can process videos from a single camera to extract 2D landmarks. However, triangulation to 3D landmarks requires synchronized videos from at least two cameras and their corresponding calibration files.
Q: How do I obtain the calibration files?\
A: Calibration files are generated using camera calibration techniques, often involving capturing images of a known pattern (like a chessboard) from different angles. You can use OpenCV’s calibration tools or other software to create these .yaml files.
Q: The processing is slow. How can I speed it up?\
A: You can increase the number of parallel processes if your CPU has more cores. Enabling GPU processing can also significantly speed up the landmark detection step. Additionally, processing a smaller fraction of frames can reduce computation time.
Q: Where can I find the output data after processing?\
A: Processed data, images, and videos are saved in the images/, imagesrefined/, landmarks/, and videos_processed/ directories within your main folder.
Q: I encounter an error related to av or PyAV.\
A: Ensure that av is installed via Conda using the conda install av -c conda-forge command, as it may have dependencies that are better handled by Conda.
Raw data
{
"_id": null,
"home_page": null,
"name": "athena-tracking",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "athena, hand tracking, pose estimation, machine learning",
"author": null,
"author_email": "Jonathan Michaels <jmichae@yorku.ca>, Daanish Mulla <daanish.mulla@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/59/9a/568e0db1e338974e5e7a7adf5533ad7fc4c9e5c258ea4973fa3c3271e122/athena_tracking-0.3.tar.gz",
"platform": null,
"description": "# ATHENA Toolbox\nATHENA (Automatically Tracking Hands Expertly with No Annotations) is a Python-based toolbox designed to process multi-camera video recordings, extract 2D and 3D body and hand landmarks using MediaPipe, and perform triangulation and refinement of these landmarks. The toolbox provides a user-friendly GUI for selecting videos and configuring processing options.\n\n## Features\n- Multi-Camera Processing: Handles multiple camera inputs for comprehensive analysis.\n- MediaPipe Integration: Utilizes Google\u2019s MediaPipe for extracting 2D landmarks of the body and hands.\n- 3D Triangulation: Triangulates 2D landmarks from multiple cameras to reconstruct 3D landmarks.\n- Smoothing and Refinement: Applies smoothing algorithms to refine the 3D landmarks.\n- Parallel Processing: Supports multiprocessing to speed up the processing of large datasets.\n- GUI for Easy Configuration: Provides a graphical user interface to select folders, recordings, and set processing options.\n- Visualization: Offers options to save images and videos of the processed landmarks for visualization.\n\n## Installation\n### Prerequisites\n- Operating System: Windows, macOS, or Linux.\n- Python Version: Python 3.9 or higher.\n- Hardware Requirements:\n- CPU: Multi-core processor recommended for parallel processing.\n- GPU (Optional): NVIDIA GPU for accelerated processing (if GPU processing is enabled).\n\n### Installation Steps\n\n```console\ngit clone https://github.com/neural-control-and-computation-lab/athena.git\ncd athena\npip install .\n```\n\nIt is recommended to create a conda environment before installing to manage dependencies and avoid conflicts.\n\n## Usage\n### 1.\tOrganize Your Videos\nPlace your video recordings in a main folder, structured as follows:\n```console\nmain_folder/\n\u251c\u2500\u2500 videos/\n\u2502 \u251c\u2500\u2500 recording1/\n\u2502 \u2502 \u251c\u2500\u2500 cam0.avi\n\u2502 \u2502 \u251c\u2500\u2500 cam1.avi\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2502 \u251c\u2500\u2500 recording2/\n\u2502 \u2502 \u251c\u2500\u2500 cam0.avi\n\u2502 \u2502 \u251c\u2500\u2500 cam1.avi\n\u2502 \u2502 \u2514\u2500\u2500 ...\n\u2514\u2500\u2500 calibration/\n \u251c\u2500\u2500 cam0.yaml\n \u251c\u2500\u2500 cam1.yaml\n \u2514\u2500\u2500 ...\n```\n\n- Each recording folder should contain video files from multiple cameras (e.g., cam0.avi, cam1.avi).\n- The calibration folder should contain calibration files (.yaml) for each camera.\n\n### 2.\tEnsure Calibration Files are Correct\n- Calibration files should be labelled with camera names that match recorded videos.\n- Calibration files should contain intrinsic and extrinsic parameters for each camera.\n- The calibration files are essential for accurate triangulation of 3D landmarks.\n\n### 3. Running the Toolbox\nLaunch the GUI:\n```console\nathena\n```\n\n1. Select Main Folder and Recordings \n - Click on the \u201cSelect Folder\u201d button to choose your main folder containing the videos and calibration directories.\n - A new window will appear, allowing you to select specific recordings to process. Select the desired recordings and click \u201cSelect\u201d.\n2. Configure Processing Options\n - General Settings:\n - Fraction of Frames to Process: Slide to select the fraction of frames you want to process (from 0 to 1). Default is 1.0 (process all frames).\n - Number of Parallel Processes: Choose the number of processes for parallel processing. Default is the number of available CPU cores.\n - GPU Processing: Check this option to enable GPU acceleration (requires compatible NVIDIA GPU).\n - MediaPipe Processing:\n - Run Mediapipe: Check this option to run MediaPipe for extracting 2D landmarks.\n - Save Images: Save images with landmarks drawn (can consume significant storage).\n - Save Video: Save videos with landmarks overlaid (requires \u201cSave Images\u201d to be enabled).\n - Minimum Hand Detection & Tracking Confidence: Adjust the confidence threshold for hand detection (default is 0.9).\n - Minimum Pose Detection & Tracking Confidence: Adjust the confidence threshold for pose detection (default is 0.9).\n - Triangulation:\n - Triangulation: Check this option to perform 3D triangulation of landmarks.\n - Save Images: Save images of the 3D landmarks.\n - Save Video: Save videos of the 3D landmarks.\n3. Start Processing\n - Click the \u201cGO\u201d button to start processing.\n - A progress window will appear, showing the processing progress and average FPS.\n\n### 4. Output Folder Structure\nAfter processing, the toolbox will create additional directories within your main folder:\n```console\nmain_folder/\n\u251c\u2500\u2500 images/ # Contains images with landmarks drawn\n\u251c\u2500\u2500 imagesrefined/ # Contains refined images after triangulation\n\u251c\u2500\u2500 landmarks/ # Contains 2D and 3D landmark data\n\u251c\u2500\u2500 videos_processed/ # Contains processed videos with landmarks overlaid\n\u2514\u2500\u2500 ... # Original videos and calibration files\n```\n\n### Troubleshooting\n- MediaPipe Errors: Ensure that MediaPipe is correctly installed and compatible with your Python version. MediaPipe may have specific requirements, especially for GPU support.\n- Calibration Mismatch: Verify that the number of calibration files matches the number of camera videos.\n- High Memory Usage: Processing large videos or saving images and videos can consume significant memory and storage.\n- Permission Issues: Ensure that you have read/write permissions for the directories where data is stored and processed.\n- Conda Environment Activation: Always make sure that the Conda environment is activated before running the scripts.\n\n### Contributing\nContributions are welcome! If you encounter issues or have suggestions for improvements, please create an issue or submit a pull request on GitHub.\n\n### Frequently Asked Questions (FAQ)\n\nQ: Do I need a GPU to run the ATHENA Toolbox?\\\nA: No, a GPU is not required and does not generally speed up processing, which is already highly parallelized. If you have a compatible NVIDIA GPU, you can enable GPU processing in the options. Ensure that your GPU drivers and CUDA toolkit are correctly installed.\n\nQ: Can I process videos from only one camera?\\\nA: Yes, you can process videos from a single camera to extract 2D landmarks. However, triangulation to 3D landmarks requires synchronized videos from at least two cameras and their corresponding calibration files.\n\nQ: How do I obtain the calibration files?\\\nA: Calibration files are generated using camera calibration techniques, often involving capturing images of a known pattern (like a chessboard) from different angles. You can use OpenCV\u2019s calibration tools or other software to create these .yaml files.\n\nQ: The processing is slow. How can I speed it up?\\\nA: You can increase the number of parallel processes if your CPU has more cores. Enabling GPU processing can also significantly speed up the landmark detection step. Additionally, processing a smaller fraction of frames can reduce computation time.\n\nQ: Where can I find the output data after processing?\\\nA: Processed data, images, and videos are saved in the images/, imagesrefined/, landmarks/, and videos_processed/ directories within your main folder.\n\nQ: I encounter an error related to av or PyAV.\\\nA: Ensure that av is installed via Conda using the conda install av -c conda-forge command, as it may have dependencies that are better handled by Conda.\n",
"bugtrack_url": null,
"license": null,
"summary": "ATHENA: Automatically Tracking Hands Expertly with No Annotations",
"version": "0.3",
"project_urls": {
"GitHub": "https://github.com/neural-control-and-computation-lab/athena"
},
"split_keywords": [
"athena",
" hand tracking",
" pose estimation",
" machine learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8dc332d5681e5c73349752979d70cb43176727bf865402ee5e533d1c575c5b19",
"md5": "f9dfcbd1d2c62f2e9c12d34df2b03f6e",
"sha256": "9f03c0c638785f9bbc6b7664fd3f21b17bcf569c78950cf94228e9ed20269a95"
},
"downloads": -1,
"filename": "athena_tracking-0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f9dfcbd1d2c62f2e9c12d34df2b03f6e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 14291920,
"upload_time": "2024-11-10T22:24:50",
"upload_time_iso_8601": "2024-11-10T22:24:50.746701Z",
"url": "https://files.pythonhosted.org/packages/8d/c3/32d5681e5c73349752979d70cb43176727bf865402ee5e533d1c575c5b19/athena_tracking-0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "599a568e0db1e338974e5e7a7adf5533ad7fc4c9e5c258ea4973fa3c3271e122",
"md5": "af0be5780e790a36afd548bdeee8d901",
"sha256": "9e505003faf1916cbf6f1f4e14ce638b6f6b1af73a384c20bc71e31e8a01a377"
},
"downloads": -1,
"filename": "athena_tracking-0.3.tar.gz",
"has_sig": false,
"md5_digest": "af0be5780e790a36afd548bdeee8d901",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 14254950,
"upload_time": "2024-11-10T22:24:53",
"upload_time_iso_8601": "2024-11-10T22:24:53.654273Z",
"url": "https://files.pythonhosted.org/packages/59/9a/568e0db1e338974e5e7a7adf5533ad7fc4c9e5c258ea4973fa3c3271e122/athena_tracking-0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-10 22:24:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "neural-control-and-computation-lab",
"github_project": "athena",
"github_not_found": true,
"lcname": "athena-tracking"
}
JSON
