# `pose-format`
This repository helps developers interested in Sign Language Processing (SLP) by providing a complete toolkit for working with poses.
It includes a file format with Python and Javascript readers and writers, which hopefully makes its usage simple.
### File Format Structure
The file format is designed to accommodate any pose type, an arbitrary number of people, and an indefinite number of frames.
Therefore it is also very suitable for video data, and not only single frames.
At the core of the file format is `Header` and a `Body`.
* The header for example contains the following information:
- The total number of pose points. (How many points exist.)
- The exact positions of these points. (Where do they exist.)
- The connections between these points. (How are they connected.)
More about the header and the body details and their binary specifics can be found in [docs/specs/v0.1.md](specs_v01.rst#specs_v01).
### Python Usage Guide:
#### 1. Installation:
```bash
pip install pose-format
```
#### 2. Estimating Pose from Video:
```bash
video_to_pose --format mediapipe -i example.mp4 -o example.pose
# Or if you have a directory of videos
videos_to_poses --format mediapipe --directory /path/to/videos
# You can also specify additional arguments
video_to_pose --format mediapipe -i example.mp4 -o example.pose \
--additional-config="model_complexity=2,smooth_landmarks=false,refine_face_landmarks=true"
# Recursively search for videos within a directory, and process them 10 at a time
videos_to_poses --format mediapipe -num-workers 10 --recursive --directory /path/to/videos
```
#### 3. Reading `.pose` Files:
To load a `.pose` file, use the `Pose` class.
```python
from pose_format import Pose
data_buffer = open("file.pose", "rb").read()
pose = Pose.read(data_buffer)
numpy_data = pose.body.data
confidence_measure = pose.body.confidence
```
By default, the library uses NumPy (`numpy`) for storing and manipulating pose data. However, integration with PyTorch (`torch`) and TensorFlow (`tensorflow`) is supported, just do the following:
```python
from pose_format.pose import Pose
data_buffer = open("file.pose", "rb").read()
# Load data as a PyTorch tensor:
from pose_format.torch import TorchPoseBody
pose = Pose.read(buffer, TorchPoseBody)
# Or as a TensorFlow tensor:
from pose_format.tensorflow.pose_body import TensorflowPoseBody
pose = Pose.read(buffer, TensorflowPoseBody)
```
If you initially loaded the data in a NumPy format and want to convert it to PyTorch or TensorFlow format, do the following:
```python
from pose_format.numpy import NumPyPoseBody
# Create a pose object that internally stores data as a NumPy array
pose = Pose.read(buffer, NumPyPoseBody)
# Convert to PyTorch:
pose.torch()
# Convert to TensorFlow:
pose.tensorflow()
```
#### 4. Data Manipulation:
Once poses are loaded, the library offers many ways to manipulate the created `Pose` objects.
##### Normalizing Data:
Maintaining data consistency is very important and data normalization is one method to do this. By normalizing the pose data, all pose information is brought to a consistent scale. This allows every pose to be normalized based on a constant feature of the body.
For instance, you can set the shoulder width to a consistent measurement across all data points. This is useful for comparing poses across different individuals.
* See this example for manually specifying a standard body feature, such as the shoulder width, for normalization:
```python
pose.normalize(p.header.normalization_info(
p1=("pose_keypoints_2d", "RShoulder"),
p2=("pose_keypoints_2d", "LShoulder")
))
```
* If normalization info is not specified, normalize() will automatically base normalization on shoulder joints.
```python
pose.normalize() # same result as above, but attempts to automatically select shoulder points based on format
```
* Keypoint values can be standardized to have a mean of zero and unit variance:
```python
# Normalize all keypoints:
pose.normalize_distribution()
```
The usual way to do this is to compute a separate mean and standard deviation for each keypoint and each dimension (usually x and y). This can be achieved with the `axis` argument of `normalize_distribution`.
```python
# Normalize each keypoint separately:
pose.normalize_distribution(axis=(0, 1, 2))
```
##### Augmentation:
Data augmentation is very important for improving the performance of machine learning models. We now provide a simple way to augment pose data.
* Apply 2D data augmentation:
```python
pose.augment2d(rotation_std=0.2, shear_std=0.2, scale_std=0.2)
```
##### Interpolation
If you're dealing with video data and need to adjust its frame rate, use the interpolation functions.
To change the frame rate of a video, using data interpolation, use the `interpolate_fps` method which gets a new `fps` and a interpolation `kind`.
```python
pose.interpolate_fps(24, kind='cubic')
pose.interpolate_fps(24, kind='linear')
```
#### 5. Visualization
You can visualize the poses stored in the `.pose` files.
Use the `PoseVisualizer` class for visualization tasks, such as generating videos or overlaying pose data on existing videos.
* To save as a video:
```python
from pose_format import Pose
from pose_format.pose_visualizer import PoseVisualizer
with open("example.pose", "rb") as f:
pose = Pose.read(f.read())
v = PoseVisualizer(pose)
v.save_video("example.mp4", v.draw())
```
* To overlay pose on an existing video:
```python
# Draws pose on top of video.
v.save_video("example.mp4", v.draw_on_video("background_video_path.mp4"))
```
* Convert to GIF:
For those using Google Colab, poses can be converted to GIFs for easy inspection.
```python
# In a Colab notebook
from IPython.display import Image
v.save_gif("test.gif", v.draw())
display(Image(open('test.gif','rb').read()))
```
#### 6. Integration with External Data Sources:
If you have pose data in OpenPose or MediaPipe Holistic format, you can easily import it.
##### Loading OpenPose and MediaPipe Holistic Data
* For OpenPose:
To load an OpenPose `directory`, use the `load_openpose_directory` utility:
```python
from pose_format.utils.openpose import load_openpose_directory
directory = "/path/to/openpose/directory"
pose = load_openpose_directory(directory, fps=24, width=1000, height=1000)
```
* For MediaPipe Holistic:
Similarly, to load a MediaPipe Holistic `directory`, use the `load_MediaPipe_directory` utility:
```python
from pose_format.utils.holistic import load_MediaPipe_directory
directory = "/path/to/holistic/directory"
pose = load_MediaPipe_directory(directory, fps=24, width=1000, height=1000)
```
### Running Tests:
To ensure the integrity of the toolkit, you can run tests using Bazel:
* Using bazel:
```bash
cd src/python/pose_format
bazel test ... --test_output=errors
```
Alternatively, use a different testing framework to run tests, such as pytest. To run an individual test file.
* Or employ pytest:
```bash
# From src/python directory
pytest .
# or for a single file
pytest pose_format/tensorflow/masked/tensor_test.py
```
### Acknowledging the Work
If you use our toolkit in your research or projects, please consider citing the work:
```bibtex
@misc{moryossef2021pose-format,
title={pose-format: Library for viewing, augmenting, and handling .pose files},
author={Moryossef, Amit and M\"{u}ller, Mathias and Fahrni, Rebecka},
howpublished={\url{https://github.com/sign-language-processing/pose}},
year={2021}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "pose-format",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Pose Files, Pose Interpolation, Pose Augmentation",
"author": null,
"author_email": "Amit Moryossef <amitmoryossef@gmail.com>, Mathias M\u00fcller <anon1@ymous.com>, Rebecka Fahrni <anon2@ymous.com>",
"download_url": "https://files.pythonhosted.org/packages/52/53/0414c59cb15f225c63f845bcad0315e9b859ec63d9276b44a54adc3e972b/pose_format-0.10.5.tar.gz",
"platform": null,
"description": "# `pose-format`\n\nThis repository helps developers interested in Sign Language Processing (SLP) by providing a complete toolkit for working with poses. \nIt includes a file format with Python and Javascript readers and writers, which hopefully makes its usage simple.\n\n### File Format Structure\nThe file format is designed to accommodate any pose type, an arbitrary number of people, and an indefinite number of frames. \nTherefore it is also very suitable for video data, and not only single frames.\n\nAt the core of the file format is `Header` and a `Body`.\n\n* The header for example contains the following information:\n\n - The total number of pose points. (How many points exist.)\n - The exact positions of these points. (Where do they exist.)\n - The connections between these points. (How are they connected.)\n\nMore about the header and the body details and their binary specifics can be found in [docs/specs/v0.1.md](specs_v01.rst#specs_v01).\n\n### Python Usage Guide: \n\n#### 1. Installation: \n\n```bash\npip install pose-format\n```\n\n#### 2. Estimating Pose from Video:\n\n```bash\nvideo_to_pose --format mediapipe -i example.mp4 -o example.pose\n\n# Or if you have a directory of videos\nvideos_to_poses --format mediapipe --directory /path/to/videos\n\n# You can also specify additional arguments\nvideo_to_pose --format mediapipe -i example.mp4 -o example.pose \\\n --additional-config=\"model_complexity=2,smooth_landmarks=false,refine_face_landmarks=true\"\n\n# Recursively search for videos within a directory, and process them 10 at a time\nvideos_to_poses --format mediapipe -num-workers 10 --recursive --directory /path/to/videos \n\n```\n\n#### 3. Reading `.pose` Files: \n\nTo load a `.pose` file, use the `Pose` class.\n\n```python\nfrom pose_format import Pose\n\ndata_buffer = open(\"file.pose\", \"rb\").read()\npose = Pose.read(data_buffer)\n\nnumpy_data = pose.body.data\nconfidence_measure = pose.body.confidence\n```\n\nBy default, the library uses NumPy (`numpy`) for storing and manipulating pose data. However, integration with PyTorch (`torch`) and TensorFlow (`tensorflow`) is supported, just do the following: \n\n```python\nfrom pose_format.pose import Pose\n\ndata_buffer = open(\"file.pose\", \"rb\").read()\n\n# Load data as a PyTorch tensor:\nfrom pose_format.torch import TorchPoseBody\npose = Pose.read(buffer, TorchPoseBody)\n\n# Or as a TensorFlow tensor:\nfrom pose_format.tensorflow.pose_body import TensorflowPoseBody\npose = Pose.read(buffer, TensorflowPoseBody)\n```\n\nIf you initially loaded the data in a NumPy format and want to convert it to PyTorch or TensorFlow format, do the following:\n\n```python\nfrom pose_format.numpy import NumPyPoseBody\n\n# Create a pose object that internally stores data as a NumPy array\npose = Pose.read(buffer, NumPyPoseBody)\n\n# Convert to PyTorch:\npose.torch()\n\n# Convert to TensorFlow:\npose.tensorflow()\n```\n\n#### 4. Data Manipulation: \n\nOnce poses are loaded, the library offers many ways to manipulate the created `Pose` objects. \n\n##### Normalizing Data: \n\nMaintaining data consistency is very important and data normalization is one method to do this. By normalizing the pose data, all pose information is brought to a consistent scale. This allows every pose to be normalized based on a constant feature of the body.\n\nFor instance, you can set the shoulder width to a consistent measurement across all data points. This is useful for comparing poses across different individuals. \n\n* See this example for manually specifying a standard body feature, such as the shoulder width, for normalization:\n\n```python\npose.normalize(p.header.normalization_info(\n p1=(\"pose_keypoints_2d\", \"RShoulder\"),\n p2=(\"pose_keypoints_2d\", \"LShoulder\")\n))\n```\n\n* If normalization info is not specified, normalize() will automatically base normalization on shoulder joints.\n\n```python\npose.normalize() # same result as above, but attempts to automatically select shoulder points based on format\n```\n\n* Keypoint values can be standardized to have a mean of zero and unit variance:\n\n```python\n\n# Normalize all keypoints:\npose.normalize_distribution()\n```\n\nThe usual way to do this is to compute a separate mean and standard deviation for each keypoint and each dimension (usually x and y). This can be achieved with the `axis` argument of `normalize_distribution`. \n\n```python\n\n# Normalize each keypoint separately:\npose.normalize_distribution(axis=(0, 1, 2))\n```\n\n##### Augmentation: \nData augmentation is very important for improving the performance of machine learning models. We now provide a simple way to augment pose data.\n\n* Apply 2D data augmentation:\n\n```python\n\npose.augment2d(rotation_std=0.2, shear_std=0.2, scale_std=0.2)\n```\n\n##### Interpolation\nIf you're dealing with video data and need to adjust its frame rate, use the interpolation functions. \n\nTo change the frame rate of a video, using data interpolation, use the `interpolate_fps` method which gets a new `fps` and a interpolation `kind`.\n\n```python\npose.interpolate_fps(24, kind='cubic')\npose.interpolate_fps(24, kind='linear')\n```\n\n#### 5. Visualization\nYou can visualize the poses stored in the `.pose` files.\nUse the `PoseVisualizer` class for visualization tasks, such as generating videos or overlaying pose data on existing videos.\n\n* To save as a video: \n```python\nfrom pose_format import Pose\nfrom pose_format.pose_visualizer import PoseVisualizer\n\n\nwith open(\"example.pose\", \"rb\") as f:\n pose = Pose.read(f.read())\n\nv = PoseVisualizer(pose)\n\nv.save_video(\"example.mp4\", v.draw())\n```\n\n* To overlay pose on an existing video: \n\n\n```python\n# Draws pose on top of video. \nv.save_video(\"example.mp4\", v.draw_on_video(\"background_video_path.mp4\"))\n```\n\n* Convert to GIF: \nFor those using Google Colab, poses can be converted to GIFs for easy inspection. \n\n```python\n# In a Colab notebook\n\nfrom IPython.display import Image\n\nv.save_gif(\"test.gif\", v.draw())\n\ndisplay(Image(open('test.gif','rb').read()))\n```\n\n#### 6. Integration with External Data Sources:\nIf you have pose data in OpenPose or MediaPipe Holistic format, you can easily import it. \n\n##### Loading OpenPose and MediaPipe Holistic Data\n\n* For OpenPose: \n\nTo load an OpenPose `directory`, use the `load_openpose_directory` utility:\n\n\n```python\nfrom pose_format.utils.openpose import load_openpose_directory\n\ndirectory = \"/path/to/openpose/directory\"\npose = load_openpose_directory(directory, fps=24, width=1000, height=1000)\n```\n\n* For MediaPipe Holistic: \n\nSimilarly, to load a MediaPipe Holistic `directory`, use the `load_MediaPipe_directory` utility:\n\n```python\nfrom pose_format.utils.holistic import load_MediaPipe_directory\n\ndirectory = \"/path/to/holistic/directory\"\npose = load_MediaPipe_directory(directory, fps=24, width=1000, height=1000)\n```\n\n### Running Tests:\n\nTo ensure the integrity of the toolkit, you can run tests using Bazel:\n\n* Using bazel:\n\n```bash\ncd src/python/pose_format\nbazel test ... --test_output=errors\n```\n\nAlternatively, use a different testing framework to run tests, such as pytest. To run an individual test file.\n\n* Or employ pytest:\n\n```bash\n# From src/python directory\npytest .\n# or for a single file\npytest pose_format/tensorflow/masked/tensor_test.py\n```\n\n### Acknowledging the Work \n\nIf you use our toolkit in your research or projects, please consider citing the work:\n\n```bibtex\n@misc{moryossef2021pose-format, \n title={pose-format: Library for viewing, augmenting, and handling .pose files},\n author={Moryossef, Amit and M\\\"{u}ller, Mathias and Fahrni, Rebecka},\n howpublished={\\url{https://github.com/sign-language-processing/pose}},\n year={2021}\n}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Library for viewing, augmenting, and handling .pose files",
"version": "0.10.5",
"project_urls": null,
"split_keywords": [
"pose files",
" pose interpolation",
" pose augmentation"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "099cd0cca94a2c4d344127f4359f666073aa804098b60a45f1a58a6b668cf749",
"md5": "e6e3b7aa40bd3d9fea0418b93940ffc0",
"sha256": "5fea42d9b3b7ca335252992ebc4cc7e95fea8e803768f5bcf51e80899e818b27"
},
"downloads": -1,
"filename": "pose_format-0.10.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e6e3b7aa40bd3d9fea0418b93940ffc0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 100967,
"upload_time": "2025-08-18T14:45:01",
"upload_time_iso_8601": "2025-08-18T14:45:01.589408Z",
"url": "https://files.pythonhosted.org/packages/09/9c/d0cca94a2c4d344127f4359f666073aa804098b60a45f1a58a6b668cf749/pose_format-0.10.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "52530414c59cb15f225c63f845bcad0315e9b859ec63d9276b44a54adc3e972b",
"md5": "c121818de109e8c2331279d6eebe8599",
"sha256": "25a05d04c6dd2c60d9ae44add91f69593639a3618e570d225f5f7650db3f9b30"
},
"downloads": -1,
"filename": "pose_format-0.10.5.tar.gz",
"has_sig": false,
"md5_digest": "c121818de109e8c2331279d6eebe8599",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 78666,
"upload_time": "2025-08-18T14:45:02",
"upload_time_iso_8601": "2025-08-18T14:45:02.940247Z",
"url": "https://files.pythonhosted.org/packages/52/53/0414c59cb15f225c63f845bcad0315e9b859ec63d9276b44a54adc3e972b/pose_format-0.10.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-18 14:45:02",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pose-format"
}
JSON