# vitallens-python
[](https://github.com/Rouast-Labs/vitallens-python/actions/workflows/main.yml)
[](https://pypi.org/project/vitallens/)
[](https://www.rouast.com/api/)
[](https://doi.org/10.48550/arXiv.2312.06892)
Estimate vital signs such as heart rate and respiratory rate from video.
`vitallens-python` is a Python client for the [**VitalLens API**](https://www.rouast.com/vitallens/), using the same neural net for inference as our [free iOS app VitalLens](https://apps.apple.com/us/app/vitallens/id6472757649).
Furthermore, it includes fast implementations of several other heart rate estimation methods from video such as `G`, `CHROM`, and `POS`.
- Accepts as input either a video filepath or an in-memory video as `np.ndarray`
- Performs fast face detection if required - you can also pass existing detections
- `vitallens.Method.VITALLENS` supports *heart rate*, *respiratory rate*, *pulse waveform*, and *respiratory waveform* estimation. In addition, it returns an estimation confidence for each vital. We are working to support more vital signs in the future.
- `vitallens.Method.{G/CHROM/POS}` support faster, but less accurate *heart rate* and *pulse waveform* estimation.
- While `VITALLENS` requires an API Key, `G`, `CHROM`, and `POS` do not. [Register on our website to get a free API Key.](https://www.rouast.com/api/)
Estimate vitals in a few lines of code:
```python
from vitallens import VitalLens, Method
vl = VitalLens(method=Method.VITALLENS, api_key="YOUR_API_KEY")
result = vl("video.mp4")
print(result)
```
### Disclaimer
`vitallens-python` provides vital sign estimates for general wellness purposes only. It is not intended for medical use. Always consult with your doctor for any health concerns or for medically precise measurement.
See also our [Terms of Service for the VitalLens API](https://www.rouast.com/api/terms) and our [Privacy Policy](https://www.rouast.com/privacy).
## Installation
General prerequisites are `python>=3.8` and `ffmpeg` installed and accessible via the `$PATH` environment variable.
The easiest way to install the latest version of `vitallens-python` and its Python dependencies:
```
pip install vitallens
```
Alternatively, it can be done by cloning the source:
```
git clone https://github.com/Rouast-Labs/vitallens-python.git
pip install ./vitallens-python
```
## How to use
To start using `vitallens-python`, first create an instance of `vitallens.VitalLens`.
It can be configured using the following parameters:
| Parameter | Description | Default |
|-------------------------|------------------------------------------------------------------------------------|--------------------|
| method | Inference method. {`Method.VITALLENS`, `Method.POS`, `Method.CHROM` or `Method.G`} | `Method.VITALLENS` |
| mode | Operation mode. {`Mode.BATCH` for indep. videos or `Mode.BURST` for video stream} | `Mode.BATCH` |
| api_key | Usage key for the VitalLens API (required for `Method.VITALLENS`) | `None` |
| detect_faces | `True` if faces need to be detected, otherwise `False`. | `True` |
| estimate_running_vitals | Set `True` to compute running vitals (e.g., `running_heart_rate`). | `True` |
| fdet_max_faces | The maximum number of faces to detect (if necessary). | `1` |
| fdet_fs | Frequency [Hz] at which faces should be scanned - otherwise linearly interpolated. | `1.0` |
| export_to_json | If `True`, write results to a json file. | `True` |
| export_dir | The directory to which json files are written. | `.` |
Once instantiated, `vitallens.VitalLens` can be called to estimate vitals.
In `Mode.BATCH` calls are assumed to be working on independent videos, whereas in `Mode.BURST` we expect the subsequent calls to pass the next frames of the same video (stream) as `np.ndarray`.
Calls are configured using the following parameters:
| Parameter | Description | Default |
|---------------------|---------------------------------------------------------------------------------------|---------|
| video | The video to analyze. Either a path to a video file or `np.ndarray`. [More info here.](https://github.com/Rouast-Labs/vitallens-python/raw/main/vitallens/client.py#L114) | |
| faces | Face detections. Ignored unless `detect_faces=False`. [More info here.](https://github.com/Rouast-Labs/vitallens-python/raw/main/vitallens/client.py#L117) | `None` |
| fps | Sampling frequency of the input video. Required if video is `np.ndarray`. | `None` |
| override_fps_target | Target frequency for inference (optional - use methods's default otherwise). | `None` |
| export_filename | Filename for json export if applicable. | `None` |
The estimation results are returned as a `list`. It contains a `dict` for each distinct face, with the following structure:
```
[
{
'face': {
'coordinates': <Face coordinates for each frame as np.ndarray of shape (n_frames, 4)>,
'confidence': <Face live confidence for each frame as np.ndarray of shape (n_frames,)>,
'note': <Explanatory note>
},
'vital_signs': {
'heart_rate': {
'value': <Estimated global value as float scalar>,
'unit': <Value unit>,
'confidence': <Estimation confidence as float scalar>,
'note': <Explanatory note>
},
'respiratory_rate': {
'value': <Estimated global value as float scalar>,
'unit': <Value unit>,
'confidence': <Estimation confidence as float scalar>,
'note': <Explanatory note>
},
'ppg_waveform': {
'data': <Estimated waveform value for each frame as np.ndarray of shape (n_frames,)>,
'unit': <Data unit>,
'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,
'note': <Explanatory note>
},
'respiratory_waveform': {
'data': <Estimated waveform value for each frame as np.ndarray of shape (n_frames,)>,
'unit': <Data unit>,
'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,
'note': <Explanatory note>
},
},
"message": <Message about estimates>
},
{
<same structure for face 2 if present>
},
...
]
```
If the video is long enough and `estimate_running_vitals=True`, the results additionally contain running vitals:
```
[
{
...
'vital_signs': {
...
'running_heart_rate': {
'data': <Estimated value for each frame as np.ndarray of shape (n_frames,)>,
'unit': <Value unit>,
'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,
'note': <Explanatory note>
},
'running_respiratory_rate': {
'data': <Estimated value for each frame as np.ndarray of shape (n_frames,)>,
'unit': <Value unit>,
'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,
'note': <Explanatory note>
}
}
...
},
...
]
```
## Examples to get started
### Live test with webcam in real-time
Test `vitallens` in real-time with your webcam using the script `examples/live.py`.
This uses `Mode.BURST` to update results continuously (approx. every 2 seconds for `Method.VITALLENS`).
Some options are available:
- `method`: Choose from [`VITALLENS`, `POS`, `G`, `CHROM`] (Default: `VITALLENS`)
- `api_key`: Pass your API Key. Required if using `method=VITALLENS`.
May need to install requirements first: `pip install opencv-python`
```
python examples/live.py --method=VITALLENS --api_key=YOUR_API_KEY
```
### Compare results with gold-standard labels using our example script
There is an example Python script in `examples/test.py` which uses `Mode.BATCH` to run vitals estimation and plot the predictions against ground truth labels recorded with gold-standard medical equipment.
Some options are available:
- `method`: Choose from [`VITALLENS`, `POS`, `G`, `CHROM`] (Default: `VITALLENS`)
- `video_path`: Path to video (Default: `examples/sample_video_1.mp4`)
- `vitals_path`: Path to gold-standard vitals (Default: `examples/sample_vitals_1.csv`)
- `api_key`: Pass your API Key. Required if using `method=VITALLENS`.
May need to install requirements first: `pip install matplotlib pandas`
For example, to reproduce the results from the banner image on the [VitalLens API Webpage](https://www.rouast.com/api/):
```
python examples/test.py --method=VITALLENS --video_path=examples/sample_video_2.mp4 --vitals_path=examples/sample_vitals_2.csv --api_key=YOUR_API_KEY
```
This sample is kindly provided by the [VitalVideos](http://vitalvideos.org) dataset.
### Use VitalLens API to estimate vitals from a video file
```python
from vitallens import VitalLens, Method
vl = VitalLens(method=Method.VITALLENS, api_key="YOUR_API_KEY")
result = vl("video.mp4")
```
### Use POS method on an `np.ndarray` of video frames
```python
from vitallens import VitalLens, Method
my_video_arr = ...
my_video_fps = 30
vl = VitalLens(method=Method.POS)
result = vl(my_video_arr, fps=my_video_fps)
```
### Run example script with Docker
If you encounter issues installing `vitallens-python` dependencies directly, you can use our Docker image, which contains all necessary tools and libraries.
This docker image is set up to execute the example Python script in `examples/test.py` for you.
#### Prerequisites
- [Docker](https://docs.docker.com/engine/install/) installed on your system.
#### Usage
1. Clone the repository
```
git clone https://github.com/Rouast-Labs/vitallens-python.git && cd vitallens-python
```
2. Build the Docker image
```
docker build -t vitallens .
```
3. Run the Docker container
To run the example script on the sample video:
```
docker run vitallens \
--api_key "your_api_key_here" \
--vitals_path "examples/sample_vitals_2.csv" \
--video_path "examples/sample_video_2.mp4" \
--method "VITALLENS"
```
You can also run it on your own video:
```
docker run vitallens \
--api_key "your_api_key_here" \
--video_path "path/to/your/video.mp4" \
--method "VITALLENS"
```
4. View the results
The results will print to the console in text form.
Please note that the example script plots won't work when running them through Docker. To to get the plot as an image file, run:
```
docker cp <container_id>:/app/results.png .
```
## Linting and tests
Before running tests, please make sure that you have an environment variable `VITALLENS_DEV_API_KEY` set to a valid API Key.
To lint and run tests:
```
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
pytest
```
## Build
To build:
```
python -m build
```
Raw data
{
"_id": null,
"home_page": null,
"name": "vitallens",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "python, rppg, vital signs monitoring, heart rate, pulse, respiration",
"author": null,
"author_email": "Philipp Rouast <philipp@rouast.com>",
"download_url": "https://files.pythonhosted.org/packages/76/58/607c03e84d161b02790241ed95c1f5cf9c44181420ca652550f030a74a09/vitallens-0.4.tar.gz",
"platform": null,
"description": "# vitallens-python\n\n[](https://github.com/Rouast-Labs/vitallens-python/actions/workflows/main.yml)\n[](https://pypi.org/project/vitallens/)\n[](https://www.rouast.com/api/)\n[](https://doi.org/10.48550/arXiv.2312.06892)\n\nEstimate vital signs such as heart rate and respiratory rate from video.\n\n`vitallens-python` is a Python client for the [**VitalLens API**](https://www.rouast.com/vitallens/), using the same neural net for inference as our [free iOS app VitalLens](https://apps.apple.com/us/app/vitallens/id6472757649).\nFurthermore, it includes fast implementations of several other heart rate estimation methods from video such as `G`, `CHROM`, and `POS`.\n\n- Accepts as input either a video filepath or an in-memory video as `np.ndarray`\n- Performs fast face detection if required - you can also pass existing detections\n- `vitallens.Method.VITALLENS` supports *heart rate*, *respiratory rate*, *pulse waveform*, and *respiratory waveform* estimation. In addition, it returns an estimation confidence for each vital. We are working to support more vital signs in the future.\n- `vitallens.Method.{G/CHROM/POS}` support faster, but less accurate *heart rate* and *pulse waveform* estimation.\n- While `VITALLENS` requires an API Key, `G`, `CHROM`, and `POS` do not. [Register on our website to get a free API Key.](https://www.rouast.com/api/)\n\nEstimate vitals in a few lines of code:\n\n```python\nfrom vitallens import VitalLens, Method\n\nvl = VitalLens(method=Method.VITALLENS, api_key=\"YOUR_API_KEY\")\nresult = vl(\"video.mp4\")\nprint(result)\n```\n\n### Disclaimer\n\n`vitallens-python` provides vital sign estimates for general wellness purposes only. It is not intended for medical use. Always consult with your doctor for any health concerns or for medically precise measurement.\n\nSee also our [Terms of Service for the VitalLens API](https://www.rouast.com/api/terms) and our [Privacy Policy](https://www.rouast.com/privacy).\n\n## Installation\n\nGeneral prerequisites are `python>=3.8` and `ffmpeg` installed and accessible via the `$PATH` environment variable.\n\nThe easiest way to install the latest version of `vitallens-python` and its Python dependencies:\n\n```\npip install vitallens\n```\n\nAlternatively, it can be done by cloning the source:\n\n```\ngit clone https://github.com/Rouast-Labs/vitallens-python.git\npip install ./vitallens-python\n```\n\n## How to use\n\nTo start using `vitallens-python`, first create an instance of `vitallens.VitalLens`. \nIt can be configured using the following parameters:\n\n| Parameter | Description | Default |\n|-------------------------|------------------------------------------------------------------------------------|--------------------|\n| method | Inference method. {`Method.VITALLENS`, `Method.POS`, `Method.CHROM` or `Method.G`} | `Method.VITALLENS` |\n| mode | Operation mode. {`Mode.BATCH` for indep. videos or `Mode.BURST` for video stream} | `Mode.BATCH` |\n| api_key | Usage key for the VitalLens API (required for `Method.VITALLENS`) | `None` |\n| detect_faces | `True` if faces need to be detected, otherwise `False`. | `True` |\n| estimate_running_vitals | Set `True` to compute running vitals (e.g., `running_heart_rate`). | `True` |\n| fdet_max_faces | The maximum number of faces to detect (if necessary). | `1` |\n| fdet_fs | Frequency [Hz] at which faces should be scanned - otherwise linearly interpolated. | `1.0` |\n| export_to_json | If `True`, write results to a json file. | `True` |\n| export_dir | The directory to which json files are written. | `.` |\n\nOnce instantiated, `vitallens.VitalLens` can be called to estimate vitals.\nIn `Mode.BATCH` calls are assumed to be working on independent videos, whereas in `Mode.BURST` we expect the subsequent calls to pass the next frames of the same video (stream) as `np.ndarray`.\nCalls are configured using the following parameters:\n\n| Parameter | Description | Default |\n|---------------------|---------------------------------------------------------------------------------------|---------|\n| video | The video to analyze. Either a path to a video file or `np.ndarray`. [More info here.](https://github.com/Rouast-Labs/vitallens-python/raw/main/vitallens/client.py#L114) | |\n| faces | Face detections. Ignored unless `detect_faces=False`. [More info here.](https://github.com/Rouast-Labs/vitallens-python/raw/main/vitallens/client.py#L117) | `None` |\n| fps | Sampling frequency of the input video. Required if video is `np.ndarray`. | `None` |\n| override_fps_target | Target frequency for inference (optional - use methods's default otherwise). | `None` |\n| export_filename | Filename for json export if applicable. | `None` |\n\nThe estimation results are returned as a `list`. It contains a `dict` for each distinct face, with the following structure:\n\n```\n[\n {\n 'face': {\n 'coordinates': <Face coordinates for each frame as np.ndarray of shape (n_frames, 4)>,\n 'confidence': <Face live confidence for each frame as np.ndarray of shape (n_frames,)>,\n 'note': <Explanatory note>\n },\n 'vital_signs': {\n 'heart_rate': {\n 'value': <Estimated global value as float scalar>,\n 'unit': <Value unit>,\n 'confidence': <Estimation confidence as float scalar>,\n 'note': <Explanatory note>\n },\n 'respiratory_rate': {\n 'value': <Estimated global value as float scalar>,\n 'unit': <Value unit>,\n 'confidence': <Estimation confidence as float scalar>,\n 'note': <Explanatory note>\n },\n 'ppg_waveform': {\n 'data': <Estimated waveform value for each frame as np.ndarray of shape (n_frames,)>,\n 'unit': <Data unit>,\n 'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,\n 'note': <Explanatory note>\n },\n 'respiratory_waveform': {\n 'data': <Estimated waveform value for each frame as np.ndarray of shape (n_frames,)>,\n 'unit': <Data unit>,\n 'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,\n 'note': <Explanatory note>\n },\n },\n \"message\": <Message about estimates>\n },\n { \n <same structure for face 2 if present>\n },\n ...\n]\n```\n\nIf the video is long enough and `estimate_running_vitals=True`, the results additionally contain running vitals:\n\n```\n[\n {\n ...\n 'vital_signs': {\n ...\n 'running_heart_rate': {\n 'data': <Estimated value for each frame as np.ndarray of shape (n_frames,)>,\n 'unit': <Value unit>,\n 'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,\n 'note': <Explanatory note>\n },\n 'running_respiratory_rate': {\n 'data': <Estimated value for each frame as np.ndarray of shape (n_frames,)>,\n 'unit': <Value unit>,\n 'confidence': <Estimation confidence for each frame as np.ndarray of shape (n_frames,)>,\n 'note': <Explanatory note>\n }\n }\n ...\n },\n ...\n]\n```\n\n## Examples to get started\n\n### Live test with webcam in real-time\n\nTest `vitallens` in real-time with your webcam using the script `examples/live.py`.\nThis uses `Mode.BURST` to update results continuously (approx. every 2 seconds for `Method.VITALLENS`).\nSome options are available:\n\n- `method`: Choose from [`VITALLENS`, `POS`, `G`, `CHROM`] (Default: `VITALLENS`)\n- `api_key`: Pass your API Key. Required if using `method=VITALLENS`.\n\nMay need to install requirements first: `pip install opencv-python`\n\n```\npython examples/live.py --method=VITALLENS --api_key=YOUR_API_KEY\n```\n\n### Compare results with gold-standard labels using our example script\n\nThere is an example Python script in `examples/test.py` which uses `Mode.BATCH` to run vitals estimation and plot the predictions against ground truth labels recorded with gold-standard medical equipment.\nSome options are available:\n\n- `method`: Choose from [`VITALLENS`, `POS`, `G`, `CHROM`] (Default: `VITALLENS`)\n- `video_path`: Path to video (Default: `examples/sample_video_1.mp4`)\n- `vitals_path`: Path to gold-standard vitals (Default: `examples/sample_vitals_1.csv`)\n- `api_key`: Pass your API Key. Required if using `method=VITALLENS`.\n\nMay need to install requirements first: `pip install matplotlib pandas`\n\nFor example, to reproduce the results from the banner image on the [VitalLens API Webpage](https://www.rouast.com/api/):\n\n```\npython examples/test.py --method=VITALLENS --video_path=examples/sample_video_2.mp4 --vitals_path=examples/sample_vitals_2.csv --api_key=YOUR_API_KEY\n```\n\nThis sample is kindly provided by the [VitalVideos](http://vitalvideos.org) dataset.\n\n### Use VitalLens API to estimate vitals from a video file\n\n```python\nfrom vitallens import VitalLens, Method\n\nvl = VitalLens(method=Method.VITALLENS, api_key=\"YOUR_API_KEY\")\nresult = vl(\"video.mp4\")\n```\n\n### Use POS method on an `np.ndarray` of video frames\n\n```python\nfrom vitallens import VitalLens, Method\n\nmy_video_arr = ...\nmy_video_fps = 30\nvl = VitalLens(method=Method.POS)\nresult = vl(my_video_arr, fps=my_video_fps)\n```\n\n### Run example script with Docker\n\nIf you encounter issues installing `vitallens-python` dependencies directly, you can use our Docker image, which contains all necessary tools and libraries.\nThis docker image is set up to execute the example Python script in `examples/test.py` for you. \n\n#### Prerequisites\n\n- [Docker](https://docs.docker.com/engine/install/) installed on your system.\n\n#### Usage\n\n1. Clone the repository\n\n```\ngit clone https://github.com/Rouast-Labs/vitallens-python.git && cd vitallens-python\n```\n\n2. Build the Docker image\n\n```\ndocker build -t vitallens .\n```\n\n3. Run the Docker container\n\nTo run the example script on the sample video:\n\n```\ndocker run vitallens \\ \n --api_key \"your_api_key_here\" \\\n --vitals_path \"examples/sample_vitals_2.csv\" \\\n --video_path \"examples/sample_video_2.mp4\" \\\n --method \"VITALLENS\"\n```\n\nYou can also run it on your own video:\n\n```\ndocker run vitallens \\ \n --api_key \"your_api_key_here\" \\\n --video_path \"path/to/your/video.mp4\" \\\n --method \"VITALLENS\"\n```\n\n4. View the results\n\nThe results will print to the console in text form.\n\nPlease note that the example script plots won't work when running them through Docker. To to get the plot as an image file, run:\n\n```\ndocker cp <container_id>:/app/results.png .\n```\n\n## Linting and tests\n\nBefore running tests, please make sure that you have an environment variable `VITALLENS_DEV_API_KEY` set to a valid API Key. \nTo lint and run tests:\n\n```\nflake8 . --count --select=E9,F63,F7,F82 --show-source --statistics\npytest\n```\n\n## Build\n\nTo build:\n\n```\npython -m build\n```\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Vital sign estimation from facial video",
"version": "0.4",
"project_urls": {
"Issues": "https://github.com/Rouast-Labs/vitallens-python/issues",
"Repository": "https://github.com/Rouast-Labs/vitallens-python.git"
},
"split_keywords": [
"python",
" rppg",
" vital signs monitoring",
" heart rate",
" pulse",
" respiration"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3f536f4ef3b813f79d855721e2bb9f14f5a139ef23c4cdb25e620d44821f3e0c",
"md5": "81fadfdfc6c60e6ee3d36e41542251ae",
"sha256": "91ac42a3c47855d73f76a5eafdcb0b36fa8972b130ddf81768061cddf0838701"
},
"downloads": -1,
"filename": "vitallens-0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "81fadfdfc6c60e6ee3d36e41542251ae",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 1089151,
"upload_time": "2024-11-14T02:17:22",
"upload_time_iso_8601": "2024-11-14T02:17:22.419401Z",
"url": "https://files.pythonhosted.org/packages/3f/53/6f4ef3b813f79d855721e2bb9f14f5a139ef23c4cdb25e620d44821f3e0c/vitallens-0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7658607c03e84d161b02790241ed95c1f5cf9c44181420ca652550f030a74a09",
"md5": "f2596359eff3b4a0c5a5a021123859f4",
"sha256": "8fae718ead5c4e7fa70fea5e57f07276f0a69a457e3871fb231479aebeb23545"
},
"downloads": -1,
"filename": "vitallens-0.4.tar.gz",
"has_sig": false,
"md5_digest": "f2596359eff3b4a0c5a5a021123859f4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1092234,
"upload_time": "2024-11-14T02:17:25",
"upload_time_iso_8601": "2024-11-14T02:17:25.394057Z",
"url": "https://files.pythonhosted.org/packages/76/58/607c03e84d161b02790241ed95c1f5cf9c44181420ca652550f030a74a09/vitallens-0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-14 02:17:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Rouast-Labs",
"github_project": "vitallens-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "vitallens"
}
JSON
