# Redis Record
This lets you record redis streams and commands so that you can replay them easily at a later time.
Under the hood, it uses [mcap](https://mcap.dev/docs/python/) or zip files as a storage format.
## Getting Started - Using Python
To install the redis-record package:
```bash
pip install redis_record
```
### Iterating over a recording
```python
from redis_record.storage import get_player
# open your recording
recording_dir = 'my-recordings'
name = 'my-favorite-recording'
with get_player(name, recording_dir) as player:
# you can read messages like this
stream_id, timestamp, data_bytes = player.next_message()
# or just iterate over them like this
for stream_id, timestamp, data_bytes in player:
...
```
### Configuring your environment
By default, we assume that redis is accessible via localhost:6379, but if that's not the case you can change that quite easily.
```bash
export REDIS_HOST=localhost
export REDIS_PORT=6379
```
Each command can also accept `--host localhost --port 6379` arguments as well.
### Record
To start an on-demand recording:
```bash
python -m redis_record.record my_recording
```
To stop, just Interrupt the script (Ctrl-C).
This will create a file at `./recordings/my_recording.mcap`.
### Replay
To replay the file into the system, do:
```bash
python -m redis_record.replay my_recording
```
### What did I record last week?
List recording names.
```bash
python -m redis_record list
```
Get info about a recording.
```bash
python -m redis_record info my_recording
```
## Getting Started - Using Docker
The recorder is designed to be a long-running process, meaning that you can deploy it as a docker container and just control it using redis commands.
This is useful if you want to be able to control the recording remotely and always have the data save to the same place.
```bash
git clone https://github.com/beasteers/redis-record.git
cd redis-record
docker-compose up -d
# to observe the recording process
docker logs redis-record
```
### Record
To start a recording, do:
```bash
python -m redis_record start my_second_recording
```
To stop a recording, do:
```bash
python -m redis_record stop my_second_recording
```
### Replay
Currently, the replay container isn't a long-running container so you still need to invoke it like above:
```bash
python -m redis_record.replay my_recording
```
## Recording more than streams
The previous method is designed to capture XADD commands (data added to Redis streams). If you want to capture
other redis commands, we can leverage Redis's MONITOR command to capture all commands.
### Record
```bash
python redis_record.record.monitor my_other_recording
```
By default, it will capture any of the SET command variants (`xadd, set, hmset, hset, hsetnx, lset, mset, msetnx, psetex, setbit, setrange, setex, setnx, getset, json.set, json.mset`), but it's easy enough to change!
```bash
python redis_record.record.monitor my_other_recording --record-cmds '[xadd,set]'
```
I was initially going to just do `[xadd,set]` but figured trying to cover a more general use case as a default would be better.
### Replay
To replay:
```bash
python redis_record.replay.monitor my_other_recording
```
## Zip Format
The directory structure is as follows:
```
recordings/
my_recording/
data_stream_1/
12345678-0_12456789-0.zip
12456789-0_12567890-0.zip
...
my_other_stream/
...
```
Inside the zipped files have the redis timestamp as the filename and the data is the serialized bytes of the `'d'` key in the stream.
## TODOs
- recording expiration (auto-stop a recording after e.g. 1 minute of inactivity)
- s3 recording file storage
- alternative exporters - e.g. mp4 - but would need consistent/general format.
Raw data
{
"_id": null,
"home_page": "https://github.com/beasteers/redis-record",
"name": "redis-record",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "redis record streams video streaming",
"author": "Bea Steers",
"author_email": "bea.steers@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c7/ac/7e039736eefd1389adfac8fd32bbf453adefe82a7ae0b0b6a63a84dc5f5d/redis_record-0.0.6.tar.gz",
"platform": null,
"description": "# Redis Record\n\nThis lets you record redis streams and commands so that you can replay them easily at a later time.\n\nUnder the hood, it uses [mcap](https://mcap.dev/docs/python/) or zip files as a storage format.\n\n## Getting Started - Using Python\n\nTo install the redis-record package:\n```bash\npip install redis_record\n```\n\n### Iterating over a recording\n\n```python\nfrom redis_record.storage import get_player\n\n# open your recording\nrecording_dir = 'my-recordings'\nname = 'my-favorite-recording'\nwith get_player(name, recording_dir) as player:\n # you can read messages like this\n stream_id, timestamp, data_bytes = player.next_message()\n\n # or just iterate over them like this\n for stream_id, timestamp, data_bytes in player:\n ...\n```\n\n### Configuring your environment\n\nBy default, we assume that redis is accessible via localhost:6379, but if that's not the case you can change that quite easily.\n```bash\nexport REDIS_HOST=localhost\nexport REDIS_PORT=6379\n```\nEach command can also accept `--host localhost --port 6379` arguments as well.\n\n### Record\n\nTo start an on-demand recording:\n```bash\npython -m redis_record.record my_recording\n```\nTo stop, just Interrupt the script (Ctrl-C).\n\nThis will create a file at `./recordings/my_recording.mcap`.\n\n### Replay\n\nTo replay the file into the system, do:\n```bash\npython -m redis_record.replay my_recording\n```\n\n### What did I record last week?\n\nList recording names.\n```bash\npython -m redis_record list\n```\n\nGet info about a recording.\n```bash\npython -m redis_record info my_recording\n```\n\n## Getting Started - Using Docker\n\nThe recorder is designed to be a long-running process, meaning that you can deploy it as a docker container and just control it using redis commands.\n\nThis is useful if you want to be able to control the recording remotely and always have the data save to the same place.\n\n```bash\ngit clone https://github.com/beasteers/redis-record.git\ncd redis-record\n\ndocker-compose up -d\n# to observe the recording process\ndocker logs redis-record\n```\n\n### Record\n\nTo start a recording, do:\n```bash\npython -m redis_record start my_second_recording\n```\n\nTo stop a recording, do:\n```bash\npython -m redis_record stop my_second_recording\n```\n\n### Replay\n\nCurrently, the replay container isn't a long-running container so you still need to invoke it like above:\n```bash\npython -m redis_record.replay my_recording\n```\n\n## Recording more than streams\n\nThe previous method is designed to capture XADD commands (data added to Redis streams). If you want to capture \nother redis commands, we can leverage Redis's MONITOR command to capture all commands.\n\n### Record\n\n```bash\npython redis_record.record.monitor my_other_recording\n```\n\nBy default, it will capture any of the SET command variants (`xadd, set, hmset, hset, hsetnx, lset, mset, msetnx, psetex, setbit, setrange, setex, setnx, getset, json.set, json.mset`), but it's easy enough to change! \n\n\n```bash\npython redis_record.record.monitor my_other_recording --record-cmds '[xadd,set]'\n```\nI was initially going to just do `[xadd,set]` but figured trying to cover a more general use case as a default would be better.\n\n\n### Replay\n\nTo replay:\n\n```bash\npython redis_record.replay.monitor my_other_recording\n```\n\n## Zip Format\nThe directory structure is as follows:\n```\nrecordings/\n my_recording/\n data_stream_1/\n 12345678-0_12456789-0.zip\n 12456789-0_12567890-0.zip\n ...\n my_other_stream/\n ...\n```\nInside the zipped files have the redis timestamp as the filename and the data is the serialized bytes of the `'d'` key in the stream.\n\n## TODOs\n - recording expiration (auto-stop a recording after e.g. 1 minute of inactivity)\n - s3 recording file storage\n - alternative exporters - e.g. mp4 - but would need consistent/general format.\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "redis streams recordings",
"version": "0.0.6",
"project_urls": {
"Homepage": "https://github.com/beasteers/redis-record"
},
"split_keywords": [
"redis",
"record",
"streams",
"video",
"streaming"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c7ac7e039736eefd1389adfac8fd32bbf453adefe82a7ae0b0b6a63a84dc5f5d",
"md5": "9c132e8c5e34780dc4f4019c24f37b49",
"sha256": "db1b5ebec210a43bd7a32288d700370f2e52315868b1e2bf2a8cf285404eca91"
},
"downloads": -1,
"filename": "redis_record-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "9c132e8c5e34780dc4f4019c24f37b49",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 18482,
"upload_time": "2023-11-07T06:21:41",
"upload_time_iso_8601": "2023-11-07T06:21:41.405585Z",
"url": "https://files.pythonhosted.org/packages/c7/ac/7e039736eefd1389adfac8fd32bbf453adefe82a7ae0b0b6a63a84dc5f5d/redis_record-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-07 06:21:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "beasteers",
"github_project": "redis-record",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "redis-record"
}
JSON
