| Name | karaoke-video-generator JSON |
| Version |
0.4.0
JSON |
| download |
| home_page | |
| Summary | Fully automated creation of _acceptable_ karaoke music videos from any music on YouTube, using open source tools and AI (e.g. Whisper and MDX-Net) |
| upload_time | 2023-07-08 03:42:45 |
| maintainer | |
| docs_url | None |
| author | Andrew Beveridge |
| requires_python | >=3.9,<3.11 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# KaraokeHunt: Karaoke video generator
Fully automated creation of _acceptable_ karaoke music videos from any music on YouTube, using open source tools and AI (e.g. Whisper and MDX-Net)
## Context
This is one experimental tool as part of the journey towards implementing the [full vision](https://docs.google.com/document/d/19LS1aJI8YwSmkWmDdpCHpmTGiHL9l0VDJ1SxSl4l6Z8/edit#) for KaraokeHunt (https://karaokehunt.com).
Some of the other components include:
- [Lyrics from Genius](https://github.com/karaokenerds/lyrics-from-genius)
- [Karaoke Song Finder (Spreadsheet generator)](https://github.com/karaokenerds/music-data-karaoke-song-sheets)
- [KaraokeHunt Mobile App](https://github.com/karaokenerds/karaokehunt-app)
## Idea steps
- Fetch the requested YouTube video using [yt-dlp](https://github.com/yt-dlp/yt-dlp) and extract the audio to wav using ffmpeg
- Run that audio through an ML-based vocal isolation model tuned for karaoke (e.g. [UVR-MDX-NET Karaoke 2](https://github.com/Anjok07/ultimatevocalremovergui/blob/master/models/MDX_Net_Models/model_data/model_name_mapper.json#L12) to get high quality instrumental audio without lead vocals but retaining backing vocals
- Run the lead vocal track through [whisper-timestamped](https://github.com/linto-ai/whisper-timestamped) to generate a time-synced lyrics file
- Correct the detected lyrics by fetching lyrics from a human-input source (e.g. musicxmatch/spotify using [syrics](https://github.com/akashrchandran/syrics), genius using [lyrics-from-genius](https://github.com/karaokenerds/lyrics-from-genius) and attempting to match up segments with the whisper-heard lyrics whilst maintaining timestamps
- Potentially also consider splitting words by syllable (e.g. using [python-syllables](https://github.com/prosegrinder/python-syllables) and attempting to guess the sub-word timestamps
- Generate a new video file using the instrumental audio and a background image, with the synced lyrics “burned” into the video at the correct timestamps
- Lots of scope to make this really nice, e.g. adjusting kerning dynamically to fit longer lines on one screen, but also lots of gotchas e.g. super long lines needing to be split at a reasonable place
- Publish this video to YouTube
Raw data
{
"_id": null,
"home_page": "",
"name": "karaoke-video-generator",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<3.11",
"maintainer_email": "",
"keywords": "",
"author": "Andrew Beveridge",
"author_email": "andrew@beveridge.uk",
"download_url": "https://files.pythonhosted.org/packages/6e/85/74bf056ad2b4c30f081c8daca6098d783c4373dd3d096aaea880a1b36632/karaoke_video_generator-0.4.0.tar.gz",
"platform": null,
"description": "# KaraokeHunt: Karaoke video generator\nFully automated creation of _acceptable_ karaoke music videos from any music on YouTube, using open source tools and AI (e.g. Whisper and MDX-Net)\n\n## Context\nThis is one experimental tool as part of the journey towards implementing the [full vision](https://docs.google.com/document/d/19LS1aJI8YwSmkWmDdpCHpmTGiHL9l0VDJ1SxSl4l6Z8/edit#) for KaraokeHunt (https://karaokehunt.com).\n\nSome of the other components include:\n- [Lyrics from Genius](https://github.com/karaokenerds/lyrics-from-genius)\n- [Karaoke Song Finder (Spreadsheet generator)](https://github.com/karaokenerds/music-data-karaoke-song-sheets)\n- [KaraokeHunt Mobile App](https://github.com/karaokenerds/karaokehunt-app)\n\n## Idea steps\n- Fetch the requested YouTube video using [yt-dlp](https://github.com/yt-dlp/yt-dlp) and extract the audio to wav using ffmpeg\n- Run that audio through an ML-based vocal isolation model tuned for karaoke (e.g. [UVR-MDX-NET Karaoke 2](https://github.com/Anjok07/ultimatevocalremovergui/blob/master/models/MDX_Net_Models/model_data/model_name_mapper.json#L12) to get high quality instrumental audio without lead vocals but retaining backing vocals\n- Run the lead vocal track through [whisper-timestamped](https://github.com/linto-ai/whisper-timestamped) to generate a time-synced lyrics file\n- Correct the detected lyrics by fetching lyrics from a human-input source (e.g. musicxmatch/spotify using [syrics](https://github.com/akashrchandran/syrics), genius using [lyrics-from-genius](https://github.com/karaokenerds/lyrics-from-genius) and attempting to match up segments with the whisper-heard lyrics whilst maintaining timestamps\n - Potentially also consider splitting words by syllable (e.g. using [python-syllables](https://github.com/prosegrinder/python-syllables) and attempting to guess the sub-word timestamps \n- Generate a new video file using the instrumental audio and a background image, with the synced lyrics \u201cburned\u201d into the video at the correct timestamps\n - Lots of scope to make this really nice, e.g. adjusting kerning dynamically to fit longer lines on one screen, but also lots of gotchas e.g. super long lines needing to be split at a reasonable place\n- Publish this video to YouTube\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Fully automated creation of _acceptable_ karaoke music videos from any music on YouTube, using open source tools and AI (e.g. Whisper and MDX-Net)",
"version": "0.4.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "78b0e0221817339924fd900948a3bd76d31b15116d4c27092eb85609e57fa761",
"md5": "61cc363548e8e34a4dc106175fee862e",
"sha256": "51af86667d1acfa3587b19b9651638fe9f9a672e0635279d332465bc117ea648"
},
"downloads": -1,
"filename": "karaoke_video_generator-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "61cc363548e8e34a4dc106175fee862e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<3.11",
"size": 7815,
"upload_time": "2023-07-08T03:42:44",
"upload_time_iso_8601": "2023-07-08T03:42:44.738508Z",
"url": "https://files.pythonhosted.org/packages/78/b0/e0221817339924fd900948a3bd76d31b15116d4c27092eb85609e57fa761/karaoke_video_generator-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6e8574bf056ad2b4c30f081c8daca6098d783c4373dd3d096aaea880a1b36632",
"md5": "8c511517ba57bab5ed8aa617d199de84",
"sha256": "46588cc56596f46b314d9213e52e8a3c9efc743daddf3d96395fa578528357ca"
},
"downloads": -1,
"filename": "karaoke_video_generator-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "8c511517ba57bab5ed8aa617d199de84",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<3.11",
"size": 6616,
"upload_time": "2023-07-08T03:42:45",
"upload_time_iso_8601": "2023-07-08T03:42:45.717582Z",
"url": "https://files.pythonhosted.org/packages/6e/85/74bf056ad2b4c30f081c8daca6098d783c4373dd3d096aaea880a1b36632/karaoke_video_generator-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-08 03:42:45",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "karaoke-video-generator"
}
