# fc2-live-dl
> Tool to download FC2 live streams
[](https://pypi.org/project/fc2-live-dl/ "PyPI")
## Requirements
- Python 3.8
- ffmpeg
## Features
- Wait for a stream to start and automatically start recording
- Save comment/chat logs
- Authenticate with cookies (Netscape format, same one used with youtube-dl)
- Remux recordings to .mp4/.m4a after it's done
- Continuously monitor multiple streams in parallel and automatically start
downloading when any of them goes online
- Get notifications when streams come online via
[Apprise](https://github.com/caronc/apprise)
## Installation
### Using pip
To install the latest stable version:
```
pip install --upgrade fc2-live-dl
```
To install the latest development version:
```
pip install --upgrade git+https://github.com/HoloArchivists/fc2-live-dl.git#egg=fc2-live-dl
```
### Using docker
```
docker pull ghcr.io/holoarchivists/fc2-live-dl:latest
```
## Usage
```
fc2-live-dl https://live.fc2.com/<...>
```
```
usage: fc2-live-dl [-h] [-v]
[--quality {150Kbps,400Kbps,1.2Mbps,2Mbps,3Mbps,sound}]
[--latency {low,high,mid}] [--threads THREADS] [-o OUTPUT]
[--no-remux] [-k] [-x] [--cookies COOKIES] [--write-chat]
[--write-info-json] [--write-thumbnail] [--wait]
[--wait-for-quality-timeout WAIT_FOR_QUALITY_TIMEOUT]
[--poll-interval POLL_INTERVAL]
[--log-level {silent,error,warn,info,debug,trace}]
[--trust-env-proxy] [--dump-websocket]
url
positional arguments:
url A live.fc2.com URL.
options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
--quality {150Kbps,400Kbps,1.2Mbps,2Mbps,3Mbps,sound}
Quality of the stream to download. Default is 3Mbps.
--latency {low,high,mid}
Stream latency. Select a higher latency if
experiencing stability issues. Default is mid.
--threads THREADS The size of the thread pool used to download segments.
Default is 1.
-o OUTPUT, --output OUTPUT
Set the output filename format. Supports formatting
options similar to youtube-dl. Default is '%(date)s
%(title)s (%(channel_name)s).%(ext)s'
Available format options:
channel_id (string): ID of the broadcast
channel_name (string): broadcaster's profile name
date (string): local date YYYY-MM-DD
time (string): local time HHMMSS
ext (string): file extension
title (string): title of the live broadcast
--no-remux Do not remux recordings into mp4/m4a after it is
finished.
-k, --keep-intermediates
Keep the raw .ts recordings after it has been remuxed.
-x, --extract-audio Generate an audio-only copy of the stream.
--cookies COOKIES Path to a cookies file.
--write-chat Save live chat into a json file.
--write-info-json Dump output stream information into a json file.
--write-thumbnail Download thumbnail into a file
--wait Wait until the broadcast goes live, then start
recording.
--wait-for-quality-timeout WAIT_FOR_QUALITY_TIMEOUT
If the requested quality is not available, keep
retrying up to this many seconds before falling back
to the next best quality. Default is 15 seconds.
--poll-interval POLL_INTERVAL
How many seconds between checks to see if broadcast is
live. Default is 5.
--log-level {silent,error,warn,info,debug,trace}
Log level verbosity. Default is info.
--trust-env-proxy Trust environment variables for proxy settings.
--dump-websocket Dump all websocket communication to a file for
debugging
```
### Using proxies
To use a HTTP proxy, pass the `--trust-env-proxy` flag and set your proxy
settings in the `HTTP_PROXY`, `HTTPS_PROXY`, `WS_PROXY` or `WSS_PROXY`
environment variables. If not present, proxy settings are taken from the
[`~/.netrc` file](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html).
For more information, check
[aiohttp's documentation](https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support).
## autofc2
> Monitor multiple channels at the same time, and automatically start
> downloading when any of them goes online
```
autofc2 --config autofc2.json
```
Where the `autofc2.json` file looks like this:
```json
{
"autofc2": {
"log_level": "info"
},
"default_params": {
"quality": "3Mbps",
"latency": "mid",
"threads": 4,
"outtmpl": "%(channel_name)s %(_en_name)s/%(date)s %(title)s.%(ext)s",
"write_chat": false,
"write_info_json": false,
"write_thumbnail": false,
"wait_for_live": true,
"wait_for_quality_timeout": 15,
"wait_poll_interval": 5,
"cookies_file": null,
"remux": true,
"keep_intermediates": false,
"extract_audio": true,
"trust_env_proxy": false
},
"notifications": [
{
"url": "discord://{WebhookID}/{WebhookToken}",
"message": "%(channel_name)s is live!\nhttps://live.fc2.com/%(channel_id)s"
}
],
"channels": {
"91544481": {
"_en_name": "Necoma Karin",
"quality": "sound",
"write_thumbnail": true
},
"72364867": { "_en_name": "Uno Sakura" },
"40740626": { "_en_name": "Komae Nadeshiko" },
"81840800": { "_en_name": "Ronomiya Hinagiku" }
}
}
```
The `default_params` object will be the parameters applied to all of the
channels. Check the usage section above for more information on each parameter.
Note that `wait_for_live` needs to be set to `true` for the script to work
properly. You can also override the parameters per-channel.
Arbitrary parameters can be specified by prefixing them with `_`, and will be
accessible in `outtmpl`. This is useful for specifying custom filenames just
like in the example above. In the example I'm using `_en_name`, but you can use
anything as long as it starts with `_`.
For notifications, the URL follows the
[Apprise syntax](https://github.com/caronc/apprise#supported-notifications). For
example, if you want to use Discord webhooks, use the `discord://` like so:
- Original URL: `https://discord.com/api/webhooks/12341234/abcdabcd`
- Turns into: `discord://12341234/abcdabcd`
You can find out more about the different types of notifiers and how to
configure them on
[Apprise's GitHub](https://github.com/caronc/apprise#supported-notifications).
The `message` of the notifications follow the same syntax as `outtmpl`.
**NOTE Windows users**: When specifying a file path (e.g. for cookies) in the
json, double up your backslashes, for example:
`"cookies_file": "C:\\Documents\\cookies.txt"`.
Once configured, you can run the script:
```
autofc2 --config autofc2.json
```
If you need to change the config json, feel free to change it while the script
is running. It will reload the file if it detects any changes. Note that
parameters will not be updated for ongoing streams (i.e. if the script is
recording a stream and you change its settings, it will continue recording with
the old settings and will only apply the new configuration to future
recordings).
## Running autofc2 with Docker
You can run autofc2 using the Docker image by mounting your config json and your
output directory, as well as overriding the default `cmd` with `autofc2` like
so:
```bash
# The following mounts `./autofc2.json` into the correct location in the docker
# container, as well as an `/recordings` folder for the recordings. You'll need to
# set the `outtmpl` to something like `/recordings/%(channel_name)s ...`
docker run --rm \
-v $(pwd)/autofc2.json:/app/autofc2.json:ro \
-v $(pwd)/recordings:/recordings \
-e TZ=Asia/Tokyo \
ghcr.io/holoarchivists/fc2-live-dl:latest \
autofc2 --config /app/autofc2.json
```
The above command runs the container in the foreground. If you want it to keep
running in the background, you can replace the `--rm` flag with `-d`. The `TZ`
environment can be set to your local timezone, and will affect the timestamps in
the logs.
**⚠️ IMPORTANT NOTE**: Make sure you set your `outtmpl` properly to match the
bind mounts (`-v`), and test that the files are properly saved to your computer.
**You will lose your recordings** if you don't configure this properly!
You can also use docker-compose to keep your config in a single file:
- Download the
[`docker-compose.autofc2.yml`](https://raw.githubusercontent.com/HoloArchivists/fc2-live-dl/main/docker-compose.autofc2.yml)
file into some folder, and name it `docker-compose.yml`.
- Place your `autofc2.json` in the same folder and modify the `outtmpl` so it
starts with `/recordings/`:
```
"outtmpl": "/recordings/%(channel_name)s %(_en_name)s/%(date)s %(title)s.%(ext)s"
```
- Run it!
```bash
# Prepare the recordings directory with the right permissions
mkdir ./recordings && chown 1000:1000 ./recordings
# Run the thing
docker-compose up -d
# Check the logs
docker-compose logs -f
# If you wanna kill it
docker-compose down
```
## Notes
- FC2 does not allow multiple connections to the same stream, so you can't watch
in the browser while downloading. You can instead preview the file being
downloaded using `mpv` or `vlc`. Alternatively, log in with an account on your
browser.
- Recording only starts from when you start the tool. This tool cannot "seek
back" and record streams from the start.
- If you can't run `fc2-live-dl` or `autofc2`, try uninstalling and reinstalling
with `pip uninstall fc2-live-dl`.
## Known issues
- Tested to work under Linux. It should work on Windows, but no guarantees. If
you're facing any issues on Windows, please
[file an issue](https://github.com/HoloArchivists/fc2-live-dl/issues/new).
- autofc2 will freak out over a private/paid streams.
- `--wait` doesn't work sometimes because FC2 would announce that the stream is
live before the playlist is available. Use `autofc2` if you want to make sure
streams get saved.
- When monitoring many channels with `autofc2`, if you face any 5xx errors, try
increasing the `wait_poll_interval` to something higher.
Raw data
{
"_id": null,
"home_page": "https://github.com/HoloArchivists/fc2-live-dl",
"name": "fc2-live-dl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "Hizkia Felix",
"author_email": "felix@hizkifw.me",
"download_url": "https://files.pythonhosted.org/packages/11/4a/d8055e773b1b971575c2512003d9e92e813d966b8f0b0b64de7cdc6017aa/fc2-live-dl-2.2.0.tar.gz",
"platform": null,
"description": "# fc2-live-dl\n\n> Tool to download FC2 live streams\n\n[](https://pypi.org/project/fc2-live-dl/ \"PyPI\")\n\n## Requirements\n\n- Python 3.8\n- ffmpeg\n\n## Features\n\n- Wait for a stream to start and automatically start recording\n- Save comment/chat logs\n- Authenticate with cookies (Netscape format, same one used with youtube-dl)\n- Remux recordings to .mp4/.m4a after it's done\n- Continuously monitor multiple streams in parallel and automatically start\n downloading when any of them goes online\n- Get notifications when streams come online via\n [Apprise](https://github.com/caronc/apprise)\n\n## Installation\n\n### Using pip\n\nTo install the latest stable version:\n\n```\npip install --upgrade fc2-live-dl\n```\n\nTo install the latest development version:\n\n```\npip install --upgrade git+https://github.com/HoloArchivists/fc2-live-dl.git#egg=fc2-live-dl\n```\n\n### Using docker\n\n```\ndocker pull ghcr.io/holoarchivists/fc2-live-dl:latest\n```\n\n## Usage\n\n```\nfc2-live-dl https://live.fc2.com/<...>\n```\n\n```\nusage: fc2-live-dl [-h] [-v]\n [--quality {150Kbps,400Kbps,1.2Mbps,2Mbps,3Mbps,sound}]\n [--latency {low,high,mid}] [--threads THREADS] [-o OUTPUT]\n [--no-remux] [-k] [-x] [--cookies COOKIES] [--write-chat]\n [--write-info-json] [--write-thumbnail] [--wait]\n [--wait-for-quality-timeout WAIT_FOR_QUALITY_TIMEOUT]\n [--poll-interval POLL_INTERVAL]\n [--log-level {silent,error,warn,info,debug,trace}]\n [--trust-env-proxy] [--dump-websocket]\n url\n\npositional arguments:\n url A live.fc2.com URL.\n\noptions:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n --quality {150Kbps,400Kbps,1.2Mbps,2Mbps,3Mbps,sound}\n Quality of the stream to download. Default is 3Mbps.\n --latency {low,high,mid}\n Stream latency. Select a higher latency if\n experiencing stability issues. Default is mid.\n --threads THREADS The size of the thread pool used to download segments.\n Default is 1.\n -o OUTPUT, --output OUTPUT\n Set the output filename format. Supports formatting\n options similar to youtube-dl. Default is '%(date)s\n %(title)s (%(channel_name)s).%(ext)s'\n\n Available format options:\n channel_id (string): ID of the broadcast\n channel_name (string): broadcaster's profile name\n date (string): local date YYYY-MM-DD\n time (string): local time HHMMSS\n ext (string): file extension\n title (string): title of the live broadcast\n --no-remux Do not remux recordings into mp4/m4a after it is\n finished.\n -k, --keep-intermediates\n Keep the raw .ts recordings after it has been remuxed.\n -x, --extract-audio Generate an audio-only copy of the stream.\n --cookies COOKIES Path to a cookies file.\n --write-chat Save live chat into a json file.\n --write-info-json Dump output stream information into a json file.\n --write-thumbnail Download thumbnail into a file\n --wait Wait until the broadcast goes live, then start\n recording.\n --wait-for-quality-timeout WAIT_FOR_QUALITY_TIMEOUT\n If the requested quality is not available, keep\n retrying up to this many seconds before falling back\n to the next best quality. Default is 15 seconds.\n --poll-interval POLL_INTERVAL\n How many seconds between checks to see if broadcast is\n live. Default is 5.\n --log-level {silent,error,warn,info,debug,trace}\n Log level verbosity. Default is info.\n --trust-env-proxy Trust environment variables for proxy settings.\n --dump-websocket Dump all websocket communication to a file for\n debugging\n\n```\n\n### Using proxies\n\nTo use a HTTP proxy, pass the `--trust-env-proxy` flag and set your proxy\nsettings in the `HTTP_PROXY`, `HTTPS_PROXY`, `WS_PROXY` or `WSS_PROXY`\nenvironment variables. If not present, proxy settings are taken from the\n[`~/.netrc` file](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html).\n\nFor more information, check\n[aiohttp's documentation](https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support).\n\n## autofc2\n\n> Monitor multiple channels at the same time, and automatically start\n> downloading when any of them goes online\n\n```\nautofc2 --config autofc2.json\n```\n\nWhere the `autofc2.json` file looks like this:\n\n```json\n{\n \"autofc2\": {\n \"log_level\": \"info\"\n },\n \"default_params\": {\n \"quality\": \"3Mbps\",\n \"latency\": \"mid\",\n \"threads\": 4,\n \"outtmpl\": \"%(channel_name)s %(_en_name)s/%(date)s %(title)s.%(ext)s\",\n \"write_chat\": false,\n \"write_info_json\": false,\n \"write_thumbnail\": false,\n \"wait_for_live\": true,\n \"wait_for_quality_timeout\": 15,\n \"wait_poll_interval\": 5,\n \"cookies_file\": null,\n \"remux\": true,\n \"keep_intermediates\": false,\n \"extract_audio\": true,\n \"trust_env_proxy\": false\n },\n \"notifications\": [\n {\n \"url\": \"discord://{WebhookID}/{WebhookToken}\",\n \"message\": \"%(channel_name)s is live!\\nhttps://live.fc2.com/%(channel_id)s\"\n }\n ],\n \"channels\": {\n \"91544481\": {\n \"_en_name\": \"Necoma Karin\",\n \"quality\": \"sound\",\n \"write_thumbnail\": true\n },\n \"72364867\": { \"_en_name\": \"Uno Sakura\" },\n \"40740626\": { \"_en_name\": \"Komae Nadeshiko\" },\n \"81840800\": { \"_en_name\": \"Ronomiya Hinagiku\" }\n }\n}\n```\n\nThe `default_params` object will be the parameters applied to all of the\nchannels. Check the usage section above for more information on each parameter.\nNote that `wait_for_live` needs to be set to `true` for the script to work\nproperly. You can also override the parameters per-channel.\n\nArbitrary parameters can be specified by prefixing them with `_`, and will be\naccessible in `outtmpl`. This is useful for specifying custom filenames just\nlike in the example above. In the example I'm using `_en_name`, but you can use\nanything as long as it starts with `_`.\n\nFor notifications, the URL follows the\n[Apprise syntax](https://github.com/caronc/apprise#supported-notifications). For\nexample, if you want to use Discord webhooks, use the `discord://` like so:\n\n- Original URL: `https://discord.com/api/webhooks/12341234/abcdabcd`\n- Turns into: `discord://12341234/abcdabcd`\n\nYou can find out more about the different types of notifiers and how to\nconfigure them on\n[Apprise's GitHub](https://github.com/caronc/apprise#supported-notifications).\n\nThe `message` of the notifications follow the same syntax as `outtmpl`.\n\n**NOTE Windows users**: When specifying a file path (e.g. for cookies) in the\njson, double up your backslashes, for example:\n`\"cookies_file\": \"C:\\\\Documents\\\\cookies.txt\"`.\n\nOnce configured, you can run the script:\n\n```\nautofc2 --config autofc2.json\n```\n\nIf you need to change the config json, feel free to change it while the script\nis running. It will reload the file if it detects any changes. Note that\nparameters will not be updated for ongoing streams (i.e. if the script is\nrecording a stream and you change its settings, it will continue recording with\nthe old settings and will only apply the new configuration to future\nrecordings).\n\n## Running autofc2 with Docker\n\nYou can run autofc2 using the Docker image by mounting your config json and your\noutput directory, as well as overriding the default `cmd` with `autofc2` like\nso:\n\n```bash\n# The following mounts `./autofc2.json` into the correct location in the docker\n# container, as well as an `/recordings` folder for the recordings. You'll need to\n# set the `outtmpl` to something like `/recordings/%(channel_name)s ...`\ndocker run --rm \\\n -v $(pwd)/autofc2.json:/app/autofc2.json:ro \\\n -v $(pwd)/recordings:/recordings \\\n -e TZ=Asia/Tokyo \\\n ghcr.io/holoarchivists/fc2-live-dl:latest \\\n autofc2 --config /app/autofc2.json\n```\n\nThe above command runs the container in the foreground. If you want it to keep\nrunning in the background, you can replace the `--rm` flag with `-d`. The `TZ`\nenvironment can be set to your local timezone, and will affect the timestamps in\nthe logs.\n\n**\u26a0\ufe0f IMPORTANT NOTE**: Make sure you set your `outtmpl` properly to match the\nbind mounts (`-v`), and test that the files are properly saved to your computer.\n**You will lose your recordings** if you don't configure this properly!\n\nYou can also use docker-compose to keep your config in a single file:\n\n- Download the\n [`docker-compose.autofc2.yml`](https://raw.githubusercontent.com/HoloArchivists/fc2-live-dl/main/docker-compose.autofc2.yml)\n file into some folder, and name it `docker-compose.yml`.\n- Place your `autofc2.json` in the same folder and modify the `outtmpl` so it\n starts with `/recordings/`:\n\n ```\n \"outtmpl\": \"/recordings/%(channel_name)s %(_en_name)s/%(date)s %(title)s.%(ext)s\"\n ```\n\n- Run it!\n\n ```bash\n # Prepare the recordings directory with the right permissions\n mkdir ./recordings && chown 1000:1000 ./recordings\n\n # Run the thing\n docker-compose up -d\n\n # Check the logs\n docker-compose logs -f\n\n # If you wanna kill it\n docker-compose down\n ```\n\n## Notes\n\n- FC2 does not allow multiple connections to the same stream, so you can't watch\n in the browser while downloading. You can instead preview the file being\n downloaded using `mpv` or `vlc`. Alternatively, log in with an account on your\n browser.\n- Recording only starts from when you start the tool. This tool cannot \"seek\n back\" and record streams from the start.\n- If you can't run `fc2-live-dl` or `autofc2`, try uninstalling and reinstalling\n with `pip uninstall fc2-live-dl`.\n\n## Known issues\n\n- Tested to work under Linux. It should work on Windows, but no guarantees. If\n you're facing any issues on Windows, please\n [file an issue](https://github.com/HoloArchivists/fc2-live-dl/issues/new).\n- autofc2 will freak out over a private/paid streams.\n- `--wait` doesn't work sometimes because FC2 would announce that the stream is\n live before the playlist is available. Use `autofc2` if you want to make sure\n streams get saved.\n- When monitoring many channels with `autofc2`, if you face any 5xx errors, try\n increasing the `wait_poll_interval` to something higher.\n",
"bugtrack_url": null,
"license": "",
"summary": "Download live streams from FC2",
"version": "2.2.0",
"project_urls": {
"Bug Tracker": "https://github.com/HoloArchivists/fc2-live-dl/issues",
"Homepage": "https://github.com/HoloArchivists/fc2-live-dl"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9c26b0a718a06ce8cd42bb4876b1fe7343c52f4f07ad9c6bc8856dd18c6674ee",
"md5": "cbcd20f90afcf987cd0b70f0c112889c",
"sha256": "252db0ee1a4aad24ccce81f8368e526ba08b67944e6f7779078b97af2173d945"
},
"downloads": -1,
"filename": "fc2_live_dl-2.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cbcd20f90afcf987cd0b70f0c112889c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 22440,
"upload_time": "2023-07-17T16:50:25",
"upload_time_iso_8601": "2023-07-17T16:50:25.512631Z",
"url": "https://files.pythonhosted.org/packages/9c/26/b0a718a06ce8cd42bb4876b1fe7343c52f4f07ad9c6bc8856dd18c6674ee/fc2_live_dl-2.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "114ad8055e773b1b971575c2512003d9e92e813d966b8f0b0b64de7cdc6017aa",
"md5": "217185f7ee6fabdd08d18487599fe8bc",
"sha256": "1558802323704a1caf1e94bcc04bdb3ccb6374a4102ff86b9c415d6a79e00552"
},
"downloads": -1,
"filename": "fc2-live-dl-2.2.0.tar.gz",
"has_sig": false,
"md5_digest": "217185f7ee6fabdd08d18487599fe8bc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 23611,
"upload_time": "2023-07-17T16:50:28",
"upload_time_iso_8601": "2023-07-17T16:50:28.418754Z",
"url": "https://files.pythonhosted.org/packages/11/4a/d8055e773b1b971575c2512003d9e92e813d966b8f0b0b64de7cdc6017aa/fc2-live-dl-2.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-17 16:50:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "HoloArchivists",
"github_project": "fc2-live-dl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "aiohttp",
"specs": [
[
">=",
"3.7.4.post0"
]
]
}
],
"lcname": "fc2-live-dl"
}
JSON
