# PyToon
[](https://pepy.tech/project/pytoon)
## Overview
PyToon is a Python-based animation library for animating characters and their mouth movements. This tool uses machine learning-based audio analysis techniques to automatically lip-sync animated character mouth movements to a given audio recording of someone talking.
[Example Output Video](https://youtu.be/fX2loRnr7II)
## Features
- Automatically create cartoon animated lip-sync videos from just an audio file.
- Use a provided transcript or let PyToon automatically generate the transcript from the audio with built-in text-to-speech.
- Programmatically generate animated videos.
- OS Independent! PyToon works on Mac, Windows, and Linux.
- Optimized for both CPU and GPU.
- Fast Processing! A 60-second lip-sync animation clip takes ~39 seconds to generate.
## Getting Started
1. Install pytoon: `pip3 install pytoon`
2. Install ffmpeg:
- Mac: `brew install ffmpeg`
- Linux: `sudo apt install ffmpeg`
- Windows: Install from [ffmpeg.org](https://ffmpeg.org/download.html)
## Basic Usage
### Example 1: Generating Animation from an MP3 File (with transcript)
If you have a transcript of the audio, you can directly pass it to the `animate` function.
```python
from pytoon.animator import animate
from moviepy.editor import VideoFileClip
# Read audio transcript to a string.
transcript_path = "./.temp/speech.txt"
with open(transcript_path, "r") as file:
transcript = file.read()
# Create a PyToon animation
animation = animate(
audio_file="speech.mp3", # Input audio
transcript=transcript, # Audio transcript
)
# Overlay the animation on top of another video and save as an .mp4 file.
background_video = VideoFileClip("./path/to/background_video.mp4")
animation.export(path='video_with_transcript.mp4', background=background_video, scale=0.7)
```
### Example 2: Example 1: Generating Animation from an MP3 File (without transcript)
If you do not have a transcript, PyToon can automatically generate one using text-to-speech.
```python
from pytoon.animator import animate
from moviepy.editor import VideoFileClip
# Create a PyToon animation without providing a transcript
animation = animate(
audio_file="speech.mp3" # Input audio (transcript will be auto-generated)
)
# Overlay the animation on top of another video and save as an .mp4 file.
background_video = VideoFileClip("./path/to/background_video.mp4")
animation.export(path='video_auto_transcript.mp4', background=background_video, scale=0.7)
```
## Acknowledgements
This project uses character images created by [lazykh](https://github.com/carykh/lazykh).
Raw data
{
"_id": null,
"home_page": "https://github.com/lukerbs/apple",
"name": "pytoon",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "animation, forced alignment, lip-sync, audio forced alignment, phoneme, generate subtitles",
"author": "Luke Kerbs",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/75/31/d118c3d80cc1a4c0020a40cb21230eb670c3b7ffab30db56308ee06d40e3/pytoon-1.5.0.tar.gz",
"platform": null,
"description": "\n# PyToon\n[](https://pepy.tech/project/pytoon)\n\n## Overview \nPyToon is a Python-based animation library for animating characters and their mouth movements. This tool uses machine learning-based audio analysis techniques to automatically lip-sync animated character mouth movements to a given audio recording of someone talking.\n\n[Example Output Video](https://youtu.be/fX2loRnr7II)\n\n## Features\n- Automatically create cartoon animated lip-sync videos from just an audio file.\n- Use a provided transcript or let PyToon automatically generate the transcript from the audio with built-in text-to-speech.\n- Programmatically generate animated videos.\n- OS Independent! PyToon works on Mac, Windows, and Linux.\n- Optimized for both CPU and GPU.\n- Fast Processing! A 60-second lip-sync animation clip takes ~39 seconds to generate.\n\n## Getting Started \n1. Install pytoon: `pip3 install pytoon`\n2. Install ffmpeg:\n - Mac: `brew install ffmpeg`\n - Linux: `sudo apt install ffmpeg`\n - Windows: Install from [ffmpeg.org](https://ffmpeg.org/download.html)\n\n## Basic Usage\n\n### Example 1: Generating Animation from an MP3 File (with transcript)\nIf you have a transcript of the audio, you can directly pass it to the `animate` function.\n\n```python\nfrom pytoon.animator import animate\nfrom moviepy.editor import VideoFileClip\n\n# Read audio transcript to a string.\ntranscript_path = \"./.temp/speech.txt\"\nwith open(transcript_path, \"r\") as file:\n transcript = file.read()\n\n# Create a PyToon animation \nanimation = animate(\n audio_file=\"speech.mp3\", # Input audio\n transcript=transcript, # Audio transcript\n)\n\n# Overlay the animation on top of another video and save as an .mp4 file.\nbackground_video = VideoFileClip(\"./path/to/background_video.mp4\")\nanimation.export(path='video_with_transcript.mp4', background=background_video, scale=0.7)\n```\n\n### Example 2: Example 1: Generating Animation from an MP3 File (without transcript)\nIf you do not have a transcript, PyToon can automatically generate one using text-to-speech.\n\n```python\nfrom pytoon.animator import animate\nfrom moviepy.editor import VideoFileClip\n\n# Create a PyToon animation without providing a transcript\nanimation = animate(\n audio_file=\"speech.mp3\" # Input audio (transcript will be auto-generated)\n)\n\n# Overlay the animation on top of another video and save as an .mp4 file.\nbackground_video = VideoFileClip(\"./path/to/background_video.mp4\")\nanimation.export(path='video_auto_transcript.mp4', background=background_video, scale=0.7)\n```\n\n## Acknowledgements\nThis project uses character images created by [lazykh](https://github.com/carykh/lazykh).\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python library for lip-syncing cartoons to voice recordings.",
"version": "1.5.0",
"project_urls": {
"Homepage": "https://github.com/lukerbs/apple"
},
"split_keywords": [
"animation",
" forced alignment",
" lip-sync",
" audio forced alignment",
" phoneme",
" generate subtitles"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "eb6a1de5a2460113c5c7990d2b52e5860bf20aafc565b5780b5585aeeaab8d14",
"md5": "0baa641636aebe1a224b9bc8b3208ff5",
"sha256": "bb7913f23770f6e39019c6a177a515319e0f7fb77dcc916fc979a784cbae737f"
},
"downloads": -1,
"filename": "pytoon-1.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0baa641636aebe1a224b9bc8b3208ff5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 3831585,
"upload_time": "2024-12-04T07:25:07",
"upload_time_iso_8601": "2024-12-04T07:25:07.865843Z",
"url": "https://files.pythonhosted.org/packages/eb/6a/1de5a2460113c5c7990d2b52e5860bf20aafc565b5780b5585aeeaab8d14/pytoon-1.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7531d118c3d80cc1a4c0020a40cb21230eb670c3b7ffab30db56308ee06d40e3",
"md5": "9a0cd53af60e09fe6f2406891401236e",
"sha256": "a48afa9f23b32bc8c0085a68cd98f3debb6298c7b5ef60f0be28efbaa29d2d31"
},
"downloads": -1,
"filename": "pytoon-1.5.0.tar.gz",
"has_sig": false,
"md5_digest": "9a0cd53af60e09fe6f2406891401236e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3828586,
"upload_time": "2024-12-04T07:25:12",
"upload_time_iso_8601": "2024-12-04T07:25:12.777012Z",
"url": "https://files.pythonhosted.org/packages/75/31/d118c3d80cc1a4c0020a40cb21230eb670c3b7ffab30db56308ee06d40e3/pytoon-1.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-04 07:25:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lukerbs",
"github_project": "apple",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "annotated-types",
"specs": [
[
"==",
"0.6.0"
]
]
},
{
"name": "backports.tarfile",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2024.2.2"
]
]
},
{
"name": "cffi",
"specs": [
[
"==",
"1.17.1"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.3.2"
]
]
},
{
"name": "click",
"specs": [
[
"==",
"8.1.7"
]
]
},
{
"name": "decorator",
"specs": [
[
"==",
"4.4.2"
]
]
},
{
"name": "Distance",
"specs": [
[
"==",
"0.1.3"
]
]
},
{
"name": "docutils",
"specs": [
[
"==",
"0.21.2"
]
]
},
{
"name": "ffmpeg-python",
"specs": [
[
"==",
"0.2.0"
]
]
},
{
"name": "filelock",
"specs": [
[
"==",
"3.13.2"
]
]
},
{
"name": "forcealign",
"specs": [
[
"==",
"1.1.9"
]
]
},
{
"name": "fsspec",
"specs": [
[
"==",
"2024.3.1"
]
]
},
{
"name": "future",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "g2p-en",
"specs": [
[
"==",
"2.1.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.6"
]
]
},
{
"name": "imageio",
"specs": [
[
"==",
"2.34.0"
]
]
},
{
"name": "imageio-ffmpeg",
"specs": [
[
"==",
"0.4.9"
]
]
},
{
"name": "importlib_metadata",
"specs": [
[
"==",
"8.5.0"
]
]
},
{
"name": "inflect",
"specs": [
[
"==",
"7.0.0"
]
]
},
{
"name": "jaraco.classes",
"specs": [
[
"==",
"3.4.0"
]
]
},
{
"name": "jaraco.context",
"specs": [
[
"==",
"6.0.1"
]
]
},
{
"name": "jaraco.functools",
"specs": [
[
"==",
"4.1.0"
]
]
},
{
"name": "Jinja2",
"specs": [
[
"==",
"3.1.3"
]
]
},
{
"name": "joblib",
"specs": [
[
"==",
"1.3.2"
]
]
},
{
"name": "keyring",
"specs": [
[
"==",
"25.5.0"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"3.0.0"
]
]
},
{
"name": "MarkupSafe",
"specs": [
[
"==",
"2.1.5"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "more-itertools",
"specs": [
[
"==",
"10.5.0"
]
]
},
{
"name": "moviepy",
"specs": [
[
"==",
"1.0.3"
]
]
},
{
"name": "mpmath",
"specs": [
[
"==",
"1.3.0"
]
]
},
{
"name": "networkx",
"specs": [
[
"==",
"3.2.1"
]
]
},
{
"name": "nh3",
"specs": [
[
"==",
"0.2.19"
]
]
},
{
"name": "nltk",
"specs": [
[
"==",
"3.8.1"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"1.26.4"
]
]
},
{
"name": "opencv-python",
"specs": [
[
"==",
"4.9.0.80"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"24.2"
]
]
},
{
"name": "pillow",
"specs": [
[
"==",
"10.2.0"
]
]
},
{
"name": "pkginfo",
"specs": [
[
"==",
"1.12.0"
]
]
},
{
"name": "proglog",
"specs": [
[
"==",
"0.1.10"
]
]
},
{
"name": "pycparser",
"specs": [
[
"==",
"2.22"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.6.4"
]
]
},
{
"name": "pydantic_core",
"specs": [
[
"==",
"2.16.3"
]
]
},
{
"name": "pydub",
"specs": [
[
"==",
"0.25.1"
]
]
},
{
"name": "Pygments",
"specs": [
[
"==",
"2.18.0"
]
]
},
{
"name": "python-dotenv",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "readme_renderer",
"specs": [
[
"==",
"44.0"
]
]
},
{
"name": "regex",
"specs": [
[
"==",
"2023.12.25"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.31.0"
]
]
},
{
"name": "requests-toolbelt",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "rfc3986",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"13.9.4"
]
]
},
{
"name": "scipy",
"specs": [
[
"==",
"1.12.0"
]
]
},
{
"name": "soundfile",
"specs": [
[
"==",
"0.12.1"
]
]
},
{
"name": "sox",
"specs": [
[
"==",
"1.5.0"
]
]
},
{
"name": "sympy",
"specs": [
[
"==",
"1.12"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "torchaudio",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "tqdm",
"specs": [
[
"==",
"4.66.2"
]
]
},
{
"name": "twine",
"specs": [
[
"==",
"6.0.1"
]
]
},
{
"name": "typing_extensions",
"specs": [
[
"==",
"4.10.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "zipp",
"specs": [
[
"==",
"3.21.0"
]
]
}
],
"lcname": "pytoon"
}
