# VISTAL: A visualization tool for temporal action localization
<p align="center" width="100%">
<img width="60%" src="https://raw.githubusercontent.com/x4Cx58x54/vistal/master/img/example_result.gif">
</p>
A lightweight tool for visualizing temporal action localization results. It generates .ass subtitle files containing timelines for videos.
## Installation
```
pip install vistal
```
## Tutorial
Import the library
```python
from vistal import vistal, ColourScheme, Colour
```
Pack the temporal labels into a list of `tuple(start, end, label_id)`, for example:
```python
prediction = [
# start, end, label_id
( 0, 2, 0),
( 2, 3, 1),
( 3, 5, 2),
( 5, 6, 3),
]
```
`start` and `end` are integers or floats in seconds, and `label_id` are integer IDs for each action. It is best that the whole video duration is covered by `(start, end)` sections.
And the actual temporal label, for example, is
```python
ground_truth = [
( 0, 1, 0),
( 1, 1.8, 3),
( 1.8, 5, 1),
( 4, 6, 2),
]
```
And another `dict` maps from label IDs to their names:
```python
label_names = {
0: 'foo',
1: 'bar',
2: 'baz',
3: 'background',
}
```
Now we create a colour scheme to determine what colour to represent each action:
```python
colour_scheme = ColourScheme(
colours=[
Colour(b=255, g=0, r=0),
Colour(b=0, g=255, r=0),
Colour(b=0, g=0, r=255),
Colour(alpha=255),
]
)
```
Or, we can generate some random colours. The last action is background, therefore it should be transparent.
```python
colour_scheme = ColourScheme(n_colours=4, transparent_id=3)
```
Suppose the video resolution is 1280x720, and it lasts for 6 seconds. By default, the display area of the subtitles is the same as the video frame area. Scale up the display resolution by 2 times, because a few thousand is normally enough:
```python
display_width = 1280 * 2
display_height = 720 * 2
video_duration = 6
```
The main function `vistal` creates a subtitle object:
```python
sub = vistal(
temporal_list_dict={
'gt ': ground_truth,
'pred': prediction
},
label_names=label_names,
colour_scheme=colour_scheme,
video_duration=video_duration,
display_width=display_width,
display_height=display_height,
timeline_height=72,
font_size=72,
font_name='Ubuntu Mono',
show_legend=True,
)
```
Save to an `.ass` file:
```python
sub.save('tutorial.ass')
```
Finally, play the video and load the subtitle to the player. Make sure your video player supports `.ass` subtitle, for example PotPlayer. Here is how it looks like on a blank video:
https://user-images.githubusercontent.com/41692486/196370592-8b7df8b9-d9a1-4004-9c8b-5df4107809e1.mp4
For another complete example, see [example.py](./example.py).
# FAQ
#### What video player supports the generated subtitles?
As far as I am aware of, VLC media player and PotPlayer on Windows works fine. [Comparison of video player software: Subtitle ability - Wikipidia](https://en.wikipedia.org/wiki/Comparison_of_video_player_software#Subtitle_ability)
#### Why the VLC media player sometimes fails to show some elements?
Try restart the video, without unloading the subtitles. For example, click "next media" while in "loop one" mode.
#### The moving cursor jumps rather than moves in PotPlayer.
Try right click video -> subtitles -> Enable ASS/SSA subtitle animations.
#### Why are everything looks like stretched?
`display_width` and `display_height` do not match the video aspect ratio.
#### How to burn the subtitles into the video?
FFmpeg is capable of doing this. For example: `ffmpeg -i {input_video_path} -vf scale={width}x{height},subtitles={subtitle_path} {output_path}`. [FFmpeg wiki: How To Burn Subtitles Into Video](https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo)
#### How to put the subtitles outside the video?
The solution is for PotPlayer. Aspect ratio of the display area (not video) leaving enough room for the subtitles needs to be determined beforehand, applied in right click -> Window Size -> Set Custom Window Size. Then `display_width` and `display_height` should match it too. Before playing the video, uncheck "Display text subs inside the video" in Preferences -> Subtitles.
#### There are small gaps between two rectangles in the timelines, while these two actions are exactly consecutive.
It is strongly recommended to set `display_width` and `display_height` to integer multiples of the display area dimensions. Normally the display area for subtitles is just the video frame area (except for the scenario in the section above: "How to put the subtitles outside the video"). Following these advices, one should be able to avoid this problem (that seems to be related to implementation of .ass subtitles).
Raw data
{
"_id": null,
"home_page": "https://github.com/x4Cx58x54/vistal",
"name": "vistal",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "visualization,temporal action localization,action segmentation,computer vision,video understanding",
"author": "x4Cx58x54",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/b4/1f/e8312e7596a76169322bb22341fb2e49643664109d7d34138a5e096d1f80/vistal-0.1.3.tar.gz",
"platform": null,
"description": "# VISTAL: A visualization tool for temporal action localization\n\n<p align=\"center\" width=\"100%\">\n <img width=\"60%\" src=\"https://raw.githubusercontent.com/x4Cx58x54/vistal/master/img/example_result.gif\">\n</p>\n\nA lightweight tool for visualizing temporal action localization results. It generates .ass subtitle files containing timelines for videos.\n\n\n## Installation\n\n```\npip install vistal\n```\n\n\n## Tutorial\n\nImport the library\n\n```python\nfrom vistal import vistal, ColourScheme, Colour\n```\n\nPack the temporal labels into a list of `tuple(start, end, label_id)`, for example:\n\n```python\nprediction = [\n # start, end, label_id\n ( 0, 2, 0),\n ( 2, 3, 1),\n ( 3, 5, 2),\n ( 5, 6, 3),\n]\n```\n\n`start` and `end` are integers or floats in seconds, and `label_id` are integer IDs for each action. It is best that the whole video duration is covered by `(start, end)` sections.\n\nAnd the actual temporal label, for example, is\n\n```python\nground_truth = [\n ( 0, 1, 0),\n ( 1, 1.8, 3),\n ( 1.8, 5, 1),\n ( 4, 6, 2),\n]\n```\n\nAnd another `dict` maps from label IDs to their names:\n\n```python\nlabel_names = {\n 0: 'foo',\n 1: 'bar',\n 2: 'baz',\n 3: 'background',\n}\n```\n\nNow we create a colour scheme to determine what colour to represent each action:\n\n```python\ncolour_scheme = ColourScheme(\n colours=[\n Colour(b=255, g=0, r=0),\n Colour(b=0, g=255, r=0),\n Colour(b=0, g=0, r=255),\n Colour(alpha=255),\n ]\n)\n```\n\nOr, we can generate some random colours. The last action is background, therefore it should be transparent.\n\n```python\ncolour_scheme = ColourScheme(n_colours=4, transparent_id=3)\n```\n\nSuppose the video resolution is 1280x720, and it lasts for 6 seconds. By default, the display area of the subtitles is the same as the video frame area. Scale up the display resolution by 2 times, because a few thousand is normally enough:\n\n```python\ndisplay_width = 1280 * 2\ndisplay_height = 720 * 2\nvideo_duration = 6\n```\n\nThe main function `vistal` creates a subtitle object:\n\n```python\nsub = vistal(\n temporal_list_dict={\n 'gt ': ground_truth,\n 'pred': prediction\n },\n label_names=label_names,\n colour_scheme=colour_scheme,\n video_duration=video_duration,\n display_width=display_width,\n display_height=display_height,\n timeline_height=72,\n font_size=72,\n font_name='Ubuntu Mono',\n show_legend=True,\n)\n```\n\nSave to an `.ass` file:\n\n```python\nsub.save('tutorial.ass')\n```\n\nFinally, play the video and load the subtitle to the player. Make sure your video player supports `.ass` subtitle, for example PotPlayer. Here is how it looks like on a blank video:\n\nhttps://user-images.githubusercontent.com/41692486/196370592-8b7df8b9-d9a1-4004-9c8b-5df4107809e1.mp4\n\nFor another complete example, see [example.py](./example.py).\n\n# FAQ\n\n#### What video player supports the generated subtitles?\n\nAs far as I am aware of, VLC media player and PotPlayer on Windows works fine. [Comparison of video player software: Subtitle ability - Wikipidia](https://en.wikipedia.org/wiki/Comparison_of_video_player_software#Subtitle_ability)\n\n\n#### Why the VLC media player sometimes fails to show some elements?\n\nTry restart the video, without unloading the subtitles. For example, click \"next media\" while in \"loop one\" mode.\n\n\n#### The moving cursor jumps rather than moves in PotPlayer.\n\nTry right click video -> subtitles -> Enable ASS/SSA subtitle animations.\n\n\n#### Why are everything looks like stretched?\n\n`display_width` and `display_height` do not match the video aspect ratio.\n\n\n#### How to burn the subtitles into the video?\n\nFFmpeg is capable of doing this. For example: `ffmpeg -i {input_video_path} -vf scale={width}x{height},subtitles={subtitle_path} {output_path}`. [FFmpeg wiki: How To Burn Subtitles Into Video](https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo)\n\n\n#### How to put the subtitles outside the video?\n\nThe solution is for PotPlayer. Aspect ratio of the display area (not video) leaving enough room for the subtitles needs to be determined beforehand, applied in right click -> Window Size -> Set Custom Window Size. Then `display_width` and `display_height` should match it too. Before playing the video, uncheck \"Display text subs inside the video\" in Preferences -> Subtitles.\n\n\n#### There are small gaps between two rectangles in the timelines, while these two actions are exactly consecutive.\n\nIt is strongly recommended to set `display_width` and `display_height` to integer multiples of the display area dimensions. Normally the display area for subtitles is just the video frame area (except for the scenario in the section above: \"How to put the subtitles outside the video\"). Following these advices, one should be able to avoid this problem (that seems to be related to implementation of .ass subtitles).\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "A visualization tool for temporal action localization",
"version": "0.1.3",
"split_keywords": [
"visualization",
"temporal action localization",
"action segmentation",
"computer vision",
"video understanding"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a17b214abc322a334bf83df487194d96d798b1e95810584d51ca565005e0f070",
"md5": "79ecc2b6217c3d9bff1a5525e252e4ef",
"sha256": "0001505dd45413c14a36043b5d700509d86c139b7668b33c3c0cabbec9ecb603"
},
"downloads": -1,
"filename": "vistal-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "79ecc2b6217c3d9bff1a5525e252e4ef",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 12390,
"upload_time": "2023-03-30T07:07:22",
"upload_time_iso_8601": "2023-03-30T07:07:22.390209Z",
"url": "https://files.pythonhosted.org/packages/a1/7b/214abc322a334bf83df487194d96d798b1e95810584d51ca565005e0f070/vistal-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b41fe8312e7596a76169322bb22341fb2e49643664109d7d34138a5e096d1f80",
"md5": "df74452a4fa392de2ca325f0dd6cc7a3",
"sha256": "622228bfd36978ad09c2b5aef7af1fbe5f88c7f9e0593ca108e6feaa32563171"
},
"downloads": -1,
"filename": "vistal-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "df74452a4fa392de2ca325f0dd6cc7a3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 13285,
"upload_time": "2023-03-30T07:07:24",
"upload_time_iso_8601": "2023-03-30T07:07:24.180836Z",
"url": "https://files.pythonhosted.org/packages/b4/1f/e8312e7596a76169322bb22341fb2e49643664109d7d34138a5e096d1f80/vistal-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-30 07:07:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "x4Cx58x54",
"github_project": "vistal",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "vistal"
}
JSON
