| Name | audioseal JSON |
| Version |
0.1.4
JSON |
| download |
| home_page | None |
| Summary | Watermarking and detection for speech audios |
| upload_time | 2024-06-24 12:03:07 |
| maintainer | None |
| docs_url | None |
| author | Facebook AI Research |
| requires_python | >=3.8 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# :loud_sound: AudioSeal: Proactive Localized Watermarking
<a href="https://www.python.org/"><img alt="Python" src="https://img.shields.io/badge/-Python 3.8+-blue?style=for-the-badge&logo=python&logoColor=white"></a>
<a href="https://black.readthedocs.io/en/stable/"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-black.svg?style=for-the-badge&labelColor=gray"></a>
Inference code for AudioSeal, a method for speech localized watermarking, with state-of-the-art robustness and detector speed (training code coming soon).
More details can be found in the [paper](https://arxiv.org/abs/2401.17264).
[[`arXiv`](https://arxiv.org/abs/2401.17264)]
[[`Colab notebook`](https://colab.research.google.com/github/facebookresearch/audioseal/blob/master/examples/colab.ipynb)][[🤗`Hugging Face`](https://huggingface.co/facebook/audioseal)]
# Updates:
- 2024-06-17: Training code is now available. Check the [instruction](./docs/TRAINING.md) !!!
- 2024-05-31: Our paper gets accepted at ICML'24 :)
- 2024-04-02: We have updated our license to full MIT license (including the license for the model weights) ! Now you can use AudioSeal in commercial application too !
- 2024-02-29: AudioSeal 0.1.2 is out, with more bug fixes for resampled audios and updated notebooks
# Abtract
We introduce AudioSeal, a method for speech localized watermarking, with state-of-the-art robustness and detector speed. It jointly trains a generator that embeds a watermark in the audio, and a detector that detects the watermarked fragments in longer audios, even in the presence of editing.
Audioseal achieves state-of-the-art detection performance of both natural and synthetic speech at the sample level (1/16k second resolution), it generates limited alteration of signal quality and is robust to many types of audio editing.
Audioseal is designed with a fast, single-pass detector, that significantly surpasses existing models in speed — achieving detection up to two orders of magnitude faster, making it ideal for large-scale and real-time applications.
# :mate: Installation
AudioSeal requires Python >=3.8, Pytorch >= 1.13.0, [omegaconf](https://omegaconf.readthedocs.io/), [julius](https://pypi.org/project/julius/), and numpy. To install from PyPI:
```
pip install audioseal
```
To install from source: Clone this repo and install in editable mode:
```
git clone https://github.com/facebookresearch/audioseal
cd audioseal
pip install -e .
```
# :gear: Models
You can find all the model checkpoints on the [Hugging Face Hub](https://huggingface.co/facebook/audioseal). We provide the checkpoints for the following models:
- [AudioSeal Generator](src/audioseal/cards/audioseal_wm_16bits.yaml).
It takes as input an audio signal (as a waveform), and outputs a watermark of the same size as the input, that can be added to the input to watermark it.
Optionally, it can also take as input a secret message of 16-bits that will be encoded in the watermark.
- [AudioSeal Detector](src/audioseal/cards/audioseal_detector_16bits.yaml).
It takes as input an audio signal (as a waveform), and outputs a probability that the input contains a watermark at each sample of the audio (every 1/16k s).
Optionally, it may also output the secret message encoded in the watermark.
Note that the message is optional and has no influence on the detection output. It may be used to identify a model version for instance (up to $2**16=65536$ possible choices).
**Note**: We are working to release the training code for anyone wants to build their own watermarker. Stay tuned !
# :abacus: Usage
Audioseal provides a simple API to watermark and detect the watermarks from an audio sample. Example usage:
```python
from audioseal import AudioSeal
# model name corresponds to the YAML card file name found in audioseal/cards
model = AudioSeal.load_generator("audioseal_wm_16bits")
# Other way is to load directly from the checkpoint
# model = Watermarker.from_pretrained(checkpoint_path, device = wav.device)
# a torch tensor of shape (batch, channels, samples) and a sample rate
# It is important to process the audio to the same sample rate as the model
# expectes. In our case, we support 16khz audio
wav, sr = ..., 16000
watermark = model.get_watermark(wav, sr)
# Optional: you can add a 16-bit message to embed in the watermark
# msg = torch.randint(0, 2, (wav.shape(0), model.msg_processor.nbits), device=wav.device)
# watermark = model.get_watermark(wav, message = msg)
watermarked_audio = wav + watermark
detector = AudioSeal.load_detector("audioseal_detector_16bits")
# To detect the messages in the high-level.
result, message = detector.detect_watermark(watermarked_audio, sr)
print(result) # result is a float number indicating the probability of the audio being watermarked,
print(message) # message is a binary vector of 16 bits
# To detect the messages in the low-level.
result, message = detector(watermarked_audio, sr)
# result is a tensor of size batch x 2 x frames, indicating the probability (positive and negative) of watermarking for each frame
# A watermarked audio should have result[:, 1, :] > 0.5
print(result[:, 1 , :])
# Message is a tensor of size batch x 16, indicating of the probability of each bit to be 1.
# message will be a random tensor if the detector detects no watermarking from the audio
print(message)
```
# Train your own watermarking model
See [here](./docs/TRAINING.md) for details on how to train your own Watermarking model.
# Want to contribute?
We welcome Pull Requests with improvements or suggestions.
If you want to flag an issue or propose an improvement, but dont' know how to realize it, create a GitHub Issue.
# Troubleshooting
- If you encounter the error `ValueError: not enough values to unpack (expected 3, got 2)`, this is because we expect a batch of audio tensors as inputs. Add one
dummy batch dimension to your input (e.g. `wav.unsqueeze(0)`, see [example notebook for getting started](examples/Getting_started.ipynb)).
- In Windows machines, if you encounter the error `KeyError raised while resolving interpolation: "Environmen variable 'USER' not found"`: This is due to an old checkpoint
uploaded to the model hub, which is not compatible in Windows. Try to invalidate the cache by removing the files in `C:\Users\<USER>\.cache\audioseal`
and re-run again.
- If you use torchaudio to handle your audios and encounter the error `Couldn't find appropriate backend to handle uri ...`, this is due to newer version of
torchaudio does not handle the default backend well. Either downgrade your torchaudio to `2.1.0` or earlier, or install `soundfile` as your audio backend.
# License
- The code in this repository is released under the MIT license as found in the [LICENSE file](LICENSE).
# Maintainers:
- [Tuan Tran](https://github.com/antoine-tran)
- [Hady Elsahar](https://github.com/hadyelsahar)
- [Pierre Fernandez](https://github.com/pierrefdz)
- [Robin San Roman](https://github.com/robinsrm)
# Citation
If you find this repository useful, please consider giving a star :star: and please cite as:
```
@article{sanroman2024proactive,
title={Proactive Detection of Voice Cloning with Localized Watermarking},
author={San Roman, Robin and Fernandez, Pierre and Elsahar, Hady and D´efossez, Alexandre and Furon, Teddy and Tran, Tuan},
journal={ICML},
year={2024}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "audioseal",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Facebook AI Research",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/ca/e1/5a4548b076fe54250a0678c09b8d0c868d2d9aeff8354c8b752be74ec001/audioseal-0.1.4.tar.gz",
"platform": null,
"description": "# :loud_sound: AudioSeal: Proactive Localized Watermarking\n\n<a href=\"https://www.python.org/\"><img alt=\"Python\" src=\"https://img.shields.io/badge/-Python 3.8+-blue?style=for-the-badge&logo=python&logoColor=white\"></a>\n<a href=\"https://black.readthedocs.io/en/stable/\"><img alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-black.svg?style=for-the-badge&labelColor=gray\"></a>\n\nInference code for AudioSeal, a method for speech localized watermarking, with state-of-the-art robustness and detector speed (training code coming soon).\nMore details can be found in the [paper](https://arxiv.org/abs/2401.17264).\n\n[[`arXiv`](https://arxiv.org/abs/2401.17264)]\n[[`Colab notebook`](https://colab.research.google.com/github/facebookresearch/audioseal/blob/master/examples/colab.ipynb)][[\ud83e\udd17`Hugging Face`](https://huggingface.co/facebook/audioseal)]\n\n\n\n# Updates:\n\n- 2024-06-17: Training code is now available. Check the [instruction](./docs/TRAINING.md) !!!\n- 2024-05-31: Our paper gets accepted at ICML'24 :)\n- 2024-04-02: We have updated our license to full MIT license (including the license for the model weights) ! Now you can use AudioSeal in commercial application too !\n- 2024-02-29: AudioSeal 0.1.2 is out, with more bug fixes for resampled audios and updated notebooks\n\n# Abtract\n\nWe introduce AudioSeal, a method for speech localized watermarking, with state-of-the-art robustness and detector speed. It jointly trains a generator that embeds a watermark in the audio, and a detector that detects the watermarked fragments in longer audios, even in the presence of editing.\nAudioseal achieves state-of-the-art detection performance of both natural and synthetic speech at the sample level (1/16k second resolution), it generates limited alteration of signal quality and is robust to many types of audio editing. \nAudioseal is designed with a fast, single-pass detector, that significantly surpasses existing models in speed \u2014 achieving detection up to two orders of magnitude faster, making it ideal for large-scale and real-time applications.\n\n# :mate: Installation\n\nAudioSeal requires Python >=3.8, Pytorch >= 1.13.0, [omegaconf](https://omegaconf.readthedocs.io/), [julius](https://pypi.org/project/julius/), and numpy. To install from PyPI:\n\n```\npip install audioseal\n```\n\nTo install from source: Clone this repo and install in editable mode:\n\n```\ngit clone https://github.com/facebookresearch/audioseal\ncd audioseal\npip install -e .\n```\n\n# :gear: Models\n\nYou can find all the model checkpoints on the [Hugging Face Hub](https://huggingface.co/facebook/audioseal). We provide the checkpoints for the following models:\n\n- [AudioSeal Generator](src/audioseal/cards/audioseal_wm_16bits.yaml).\n It takes as input an audio signal (as a waveform), and outputs a watermark of the same size as the input, that can be added to the input to watermark it.\n Optionally, it can also take as input a secret message of 16-bits that will be encoded in the watermark.\n- [AudioSeal Detector](src/audioseal/cards/audioseal_detector_16bits.yaml).\n It takes as input an audio signal (as a waveform), and outputs a probability that the input contains a watermark at each sample of the audio (every 1/16k s).\n Optionally, it may also output the secret message encoded in the watermark.\n\nNote that the message is optional and has no influence on the detection output. It may be used to identify a model version for instance (up to $2**16=65536$ possible choices).\n\n**Note**: We are working to release the training code for anyone wants to build their own watermarker. Stay tuned !\n\n# :abacus: Usage\n\nAudioseal provides a simple API to watermark and detect the watermarks from an audio sample. Example usage:\n\n```python\n\nfrom audioseal import AudioSeal\n\n# model name corresponds to the YAML card file name found in audioseal/cards\nmodel = AudioSeal.load_generator(\"audioseal_wm_16bits\")\n\n# Other way is to load directly from the checkpoint\n# model = Watermarker.from_pretrained(checkpoint_path, device = wav.device)\n\n# a torch tensor of shape (batch, channels, samples) and a sample rate\n# It is important to process the audio to the same sample rate as the model\n# expectes. In our case, we support 16khz audio \nwav, sr = ..., 16000\n\nwatermark = model.get_watermark(wav, sr)\n\n# Optional: you can add a 16-bit message to embed in the watermark\n# msg = torch.randint(0, 2, (wav.shape(0), model.msg_processor.nbits), device=wav.device)\n# watermark = model.get_watermark(wav, message = msg)\n\nwatermarked_audio = wav + watermark\n\ndetector = AudioSeal.load_detector(\"audioseal_detector_16bits\")\n\n# To detect the messages in the high-level.\nresult, message = detector.detect_watermark(watermarked_audio, sr)\n\nprint(result) # result is a float number indicating the probability of the audio being watermarked,\nprint(message) # message is a binary vector of 16 bits\n\n\n# To detect the messages in the low-level.\nresult, message = detector(watermarked_audio, sr)\n\n# result is a tensor of size batch x 2 x frames, indicating the probability (positive and negative) of watermarking for each frame\n# A watermarked audio should have result[:, 1, :] > 0.5\nprint(result[:, 1 , :]) \n\n# Message is a tensor of size batch x 16, indicating of the probability of each bit to be 1.\n# message will be a random tensor if the detector detects no watermarking from the audio\nprint(message) \n```\n\n# Train your own watermarking model\n\nSee [here](./docs/TRAINING.md) for details on how to train your own Watermarking model.\n\n# Want to contribute?\n\n We welcome Pull Requests with improvements or suggestions.\n If you want to flag an issue or propose an improvement, but dont' know how to realize it, create a GitHub Issue.\n\n# Troubleshooting\n\n- If you encounter the error `ValueError: not enough values to unpack (expected 3, got 2)`, this is because we expect a batch of audio tensors as inputs. Add one\ndummy batch dimension to your input (e.g. `wav.unsqueeze(0)`, see [example notebook for getting started](examples/Getting_started.ipynb)).\n\n- In Windows machines, if you encounter the error `KeyError raised while resolving interpolation: \"Environmen variable 'USER' not found\"`: This is due to an old checkpoint\nuploaded to the model hub, which is not compatible in Windows. Try to invalidate the cache by removing the files in `C:\\Users\\<USER>\\.cache\\audioseal`\nand re-run again.\n\n- If you use torchaudio to handle your audios and encounter the error `Couldn't find appropriate backend to handle uri ...`, this is due to newer version of\ntorchaudio does not handle the default backend well. Either downgrade your torchaudio to `2.1.0` or earlier, or install `soundfile` as your audio backend.\n\n# License\n\n- The code in this repository is released under the MIT license as found in the [LICENSE file](LICENSE).\n\n# Maintainers:\n- [Tuan Tran](https://github.com/antoine-tran)\n- [Hady Elsahar](https://github.com/hadyelsahar)\n- [Pierre Fernandez](https://github.com/pierrefdz)\n- [Robin San Roman](https://github.com/robinsrm)\n\n# Citation\n\nIf you find this repository useful, please consider giving a star :star: and please cite as:\n\n```\n@article{sanroman2024proactive,\n title={Proactive Detection of Voice Cloning with Localized Watermarking},\n author={San Roman, Robin and Fernandez, Pierre and Elsahar, Hady and D\u00b4efossez, Alexandre and Furon, Teddy and Tran, Tuan},\n journal={ICML},\n year={2024}\n}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Watermarking and detection for speech audios",
"version": "0.1.4",
"project_urls": {
"Source": "https://github.com/facebookresearch/audioseal",
"Tracker": "https://github.com/facebookresearch/audioseal/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "9120b8bf24be673e159e99b2c3e2df128a19f2f217862118becb14db7bb559ba",
"md5": "b4b0bb477ae8b3e745e776695c1226ad",
"sha256": "e1a36741fadeae3b71d2891ed0639a262f0924b6615be737e005dc629f470fae"
},
"downloads": -1,
"filename": "audioseal-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b4b0bb477ae8b3e745e776695c1226ad",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 21220,
"upload_time": "2024-06-24T12:03:01",
"upload_time_iso_8601": "2024-06-24T12:03:01.440726Z",
"url": "https://files.pythonhosted.org/packages/91/20/b8bf24be673e159e99b2c3e2df128a19f2f217862118becb14db7bb559ba/audioseal-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "cae15a4548b076fe54250a0678c09b8d0c868d2d9aeff8354c8b752be74ec001",
"md5": "3211ceabd08326cd1e0ec054f87d1518",
"sha256": "28580a640a47f445347a76ce9f02e639fae781535e5bfbde24baa4767ce58317"
},
"downloads": -1,
"filename": "audioseal-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "3211ceabd08326cd1e0ec054f87d1518",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1866433,
"upload_time": "2024-06-24T12:03:07",
"upload_time_iso_8601": "2024-06-24T12:03:07.613717Z",
"url": "https://files.pythonhosted.org/packages/ca/e1/5a4548b076fe54250a0678c09b8d0c868d2d9aeff8354c8b752be74ec001/audioseal-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-24 12:03:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "facebookresearch",
"github_project": "audioseal",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "audioseal"
}
