# RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning
[![build][tests-badge]][tests]
[![docs][docs-badge]][docs]
[![PyPI Python Version][pypi-versions-badge]][pypi]
[![PyPI version][pypi-badge]][pypi]
[tests-badge]: https://github.com/google-research/robopianist/actions/workflows/ci.yml/badge.svg
[docs-badge]: https://github.com/google-research/robopianist/actions/workflows/docs.yml/badge.svg
[tests]: https://github.com/google-research/robopianist/actions/workflows/ci.yml
[docs]: https://google-research.github.io/robopianist/
[pypi-versions-badge]: https://img.shields.io/pypi/pyversions/robopianist
[pypi-badge]: https://badge.fury.io/py/robopianist.svg
[pypi]: https://pypi.org/project/robopianist/
[](https://youtu.be/VBFn_Gg0yD8)
RoboPianist is a new benchmarking suite for high-dimensional control, targeted at testing high spatial and temporal precision, coordination, and planning, all with an underactuated system frequently making-and-breaking contacts. The proposed challenge is *mastering the piano* through bi-manual dexterity, using a pair of simulated anthropomorphic robot hands.
This codebase contains software and tasks for the benchmark, and is powered by [MuJoCo](https://mujoco.org/).
- [Latest Updates](#latest-updates)
- [Getting Started](#getting-started)
- [Installation](#installation)
- [Install from source](#install-from-source)
- [Install from PyPI](#install-from-pypi)
- [Optional: Download additional soundfonts](#optional-download-additional-soundfonts)
- [MIDI Dataset](#midi-dataset)
- [CLI](#cli)
- [Contributing](#contributing)
- [FAQ](#faq)
- [Citing RoboPianist](#citing-robopianist)
- [Acknowledgements](#acknowledgements)
- [License and Disclaimer](#license-and-disclaimer)
-------
## Latest Updates
- [24/12/2023] Updated install script so that it checks out the correct Menagerie commit. Please re-run `bash scripts/install_deps.sh` to update your installation.
- [17/08/2023] Added a [pixel wrapper](robopianist/wrappers/pixels.py) for augmenting the observation space with RGB images.
- [11/08/2023] Code to train the model-free RL policies is now public, see [robopianist-rl](https://github.com/kevinzakka/robopianist-rl).
-------
## Getting Started
We've created an introductory [Colab](https://colab.research.google.com/github/google-research/robopianist/blob/main/tutorial.ipynb) notebook that demonstrates how to use RoboPianist. It includes code for loading and customizing a piano playing task, and a demonstration of a pretrained policy playing a short snippet of *Twinkle Twinkle Little Star*. Click the button below to get started!
[](https://colab.research.google.com/github/google-research/robopianist/blob/main/tutorial.ipynb)
## Installation
RoboPianist is supported on both Linux and macOS and can be installed with Python >= 3.8. We recommend using [Miniconda](https://docs.conda.io/en/latest/miniconda.html) to manage your Python environment.
### Install from source
The recommended way to install this package is from source. Start by cloning the repository:
```bash
git clone https://github.com/google-research/robopianist.git && cd robopianist
```
Next, install the prerequisite dependencies:
```bash
git submodule init && git submodule update
bash scripts/install_deps.sh
```
Finally, create a new conda environment and install RoboPianist in editable mode:
```bash
conda create -n pianist python=3.10
conda activate pianist
pip install -e ".[dev]"
```
To test your installation, run `make test` and verify that all tests pass.
### Install from PyPI
First, install the prerequisite dependencies:
```bash
bash <(curl -s https://raw.githubusercontent.com/google-research/robopianist/main/scripts/install_deps.sh) --no-soundfonts
```
Next, create a new conda environment and install RoboPianist:
```bash
conda create -n pianist python=3.10
conda activate pianist
pip install --upgrade robopianist
```
### Optional: Download additional soundfonts
We recommend installing additional soundfonts to improve the quality of the synthesized audio. You can easily do this using the RoboPianist CLI:
```bash
robopianist soundfont --download
```
For more soundfont-related commands, see [docs/soundfonts.md](docs/soundfonts.md).
## MIDI Dataset
The PIG dataset cannot be redistributed on GitHub due to licensing restrictions. See [docs/dataset](docs/dataset.md) for instructions on where to download it and how to preprocess it.
## CLI
RoboPianist comes with a command line interface (CLI) that can be used to download additional soundfonts, play MIDI files, preprocess the PIG dataset, and more. For more information, see [docs/cli.md](docs/cli.md).
## Contributing
We welcome contributions to RoboPianist. Please see [docs/contributing.md](docs/contributing.md) for more information.
## FAQ
See [docs/faq.md](docs/faq.md) for a list of frequently asked questions.
## Citing RoboPianist
If you use RoboPianist in your work, please use the following citation:
```bibtex
@inproceedings{robopianist2023,
author = {Zakka, Kevin and Wu, Philipp and Smith, Laura and Gileadi, Nimrod and Howell, Taylor and Peng, Xue Bin and Singh, Sumeet and Tassa, Yuval and Florence, Pete and Zeng, Andy and Abbeel, Pieter},
title = {RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning},
booktitle = {Conference on Robot Learning (CoRL)},
year = {2023},
}
```
## Acknowledgements
We would like to thank the following people for making this project possible:
- [Philipp Wu](https://www.linkedin.com/in/wuphilipp/) and [Mohit Shridhar](https://mohitshridhar.com/) for being a constant source of inspiration and support.
- [Ilya Kostrikov](https://www.kostrikov.xyz/) for constantly raising the bar for RL engineering and for invaluable debugging help.
- The [Magenta](https://magenta.tensorflow.org/) team for helpful pointers and feedback.
- The [MuJoCo](https://mujoco.org/) team for the development of the MuJoCo physics engine and their support throughout the project.
## License and Disclaimer
[MuJoco Menagerie](https://github.com/deepmind/mujoco_menagerie)'s license can be found [here](https://github.com/deepmind/mujoco_menagerie/blob/main/LICENSE). Soundfont licensing information can be found [here](docs/soundfonts.md). MIDI licensing information can be found [here](docs/dataset.md). All other code is licensed under an [Apache-2.0 License](LICENSE).
This is not an officially supported Google product.
Raw data
{
"_id": null,
"home_page": "https://github.com/google-research/robopianist",
"name": "robopianist",
"maintainer": "Kevin Zakka",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "kevinarmandzakka@gmail.com",
"keywords": "reinforcement-learning mujoco bimanual dexterous-manipulation piano",
"author": "Kevin Zakka",
"author_email": "kevinarmandzakka@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/eb/d0/05ca4146debfd0df530b6f1ba78295ad2b61868caa2540b9965578b82d89/robopianist-1.0.10.tar.gz",
"platform": null,
"description": "# RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning\n\n[![build][tests-badge]][tests]\n[![docs][docs-badge]][docs]\n[![PyPI Python Version][pypi-versions-badge]][pypi]\n[![PyPI version][pypi-badge]][pypi]\n\n[tests-badge]: https://github.com/google-research/robopianist/actions/workflows/ci.yml/badge.svg\n[docs-badge]: https://github.com/google-research/robopianist/actions/workflows/docs.yml/badge.svg\n[tests]: https://github.com/google-research/robopianist/actions/workflows/ci.yml\n[docs]: https://google-research.github.io/robopianist/\n[pypi-versions-badge]: https://img.shields.io/pypi/pyversions/robopianist\n[pypi-badge]: https://badge.fury.io/py/robopianist.svg\n[pypi]: https://pypi.org/project/robopianist/\n\n[](https://youtu.be/VBFn_Gg0yD8)\n\nRoboPianist is a new benchmarking suite for high-dimensional control, targeted at testing high spatial and temporal precision, coordination, and planning, all with an underactuated system frequently making-and-breaking contacts. The proposed challenge is *mastering the piano* through bi-manual dexterity, using a pair of simulated anthropomorphic robot hands.\n\nThis codebase contains software and tasks for the benchmark, and is powered by [MuJoCo](https://mujoco.org/).\n\n- [Latest Updates](#latest-updates)\n- [Getting Started](#getting-started)\n- [Installation](#installation)\n - [Install from source](#install-from-source)\n - [Install from PyPI](#install-from-pypi)\n - [Optional: Download additional soundfonts](#optional-download-additional-soundfonts)\n- [MIDI Dataset](#midi-dataset)\n- [CLI](#cli)\n- [Contributing](#contributing)\n- [FAQ](#faq)\n- [Citing RoboPianist](#citing-robopianist)\n- [Acknowledgements](#acknowledgements)\n- [License and Disclaimer](#license-and-disclaimer)\n\n-------\n\n## Latest Updates\n\n- [24/12/2023] Updated install script so that it checks out the correct Menagerie commit. Please re-run `bash scripts/install_deps.sh` to update your installation.\n- [17/08/2023] Added a [pixel wrapper](robopianist/wrappers/pixels.py) for augmenting the observation space with RGB images.\n- [11/08/2023] Code to train the model-free RL policies is now public, see [robopianist-rl](https://github.com/kevinzakka/robopianist-rl).\n\n-------\n\n## Getting Started\n\nWe've created an introductory [Colab](https://colab.research.google.com/github/google-research/robopianist/blob/main/tutorial.ipynb) notebook that demonstrates how to use RoboPianist. It includes code for loading and customizing a piano playing task, and a demonstration of a pretrained policy playing a short snippet of *Twinkle Twinkle Little Star*. Click the button below to get started!\n\n[](https://colab.research.google.com/github/google-research/robopianist/blob/main/tutorial.ipynb)\n\n## Installation\n\nRoboPianist is supported on both Linux and macOS and can be installed with Python >= 3.8. We recommend using [Miniconda](https://docs.conda.io/en/latest/miniconda.html) to manage your Python environment.\n\n### Install from source\n\nThe recommended way to install this package is from source. Start by cloning the repository:\n\n```bash\ngit clone https://github.com/google-research/robopianist.git && cd robopianist\n```\n\nNext, install the prerequisite dependencies:\n\n```bash\ngit submodule init && git submodule update\nbash scripts/install_deps.sh\n```\n\nFinally, create a new conda environment and install RoboPianist in editable mode:\n\n```bash\nconda create -n pianist python=3.10\nconda activate pianist\n\npip install -e \".[dev]\"\n```\n\nTo test your installation, run `make test` and verify that all tests pass.\n\n### Install from PyPI\n\nFirst, install the prerequisite dependencies:\n\n```bash\nbash <(curl -s https://raw.githubusercontent.com/google-research/robopianist/main/scripts/install_deps.sh) --no-soundfonts\n```\n\nNext, create a new conda environment and install RoboPianist:\n\n```bash\nconda create -n pianist python=3.10\nconda activate pianist\n\npip install --upgrade robopianist\n```\n\n### Optional: Download additional soundfonts\n\nWe recommend installing additional soundfonts to improve the quality of the synthesized audio. You can easily do this using the RoboPianist CLI:\n\n```bash\nrobopianist soundfont --download\n```\n\nFor more soundfont-related commands, see [docs/soundfonts.md](docs/soundfonts.md).\n\n## MIDI Dataset\n\nThe PIG dataset cannot be redistributed on GitHub due to licensing restrictions. See [docs/dataset](docs/dataset.md) for instructions on where to download it and how to preprocess it.\n\n## CLI\n\nRoboPianist comes with a command line interface (CLI) that can be used to download additional soundfonts, play MIDI files, preprocess the PIG dataset, and more. For more information, see [docs/cli.md](docs/cli.md).\n\n## Contributing\n\nWe welcome contributions to RoboPianist. Please see [docs/contributing.md](docs/contributing.md) for more information.\n\n## FAQ\n\nSee [docs/faq.md](docs/faq.md) for a list of frequently asked questions.\n\n## Citing RoboPianist\n\nIf you use RoboPianist in your work, please use the following citation:\n\n```bibtex\n@inproceedings{robopianist2023,\n author = {Zakka, Kevin and Wu, Philipp and Smith, Laura and Gileadi, Nimrod and Howell, Taylor and Peng, Xue Bin and Singh, Sumeet and Tassa, Yuval and Florence, Pete and Zeng, Andy and Abbeel, Pieter},\n title = {RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning},\n booktitle = {Conference on Robot Learning (CoRL)},\n year = {2023},\n}\n```\n\n## Acknowledgements\n\nWe would like to thank the following people for making this project possible:\n\n- [Philipp Wu](https://www.linkedin.com/in/wuphilipp/) and [Mohit Shridhar](https://mohitshridhar.com/) for being a constant source of inspiration and support.\n- [Ilya Kostrikov](https://www.kostrikov.xyz/) for constantly raising the bar for RL engineering and for invaluable debugging help.\n- The [Magenta](https://magenta.tensorflow.org/) team for helpful pointers and feedback.\n- The [MuJoCo](https://mujoco.org/) team for the development of the MuJoCo physics engine and their support throughout the project.\n\n## License and Disclaimer\n\n[MuJoco Menagerie](https://github.com/deepmind/mujoco_menagerie)'s license can be found [here](https://github.com/deepmind/mujoco_menagerie/blob/main/LICENSE). Soundfont licensing information can be found [here](docs/soundfonts.md). MIDI licensing information can be found [here](docs/dataset.md). All other code is licensed under an [Apache-2.0 License](LICENSE).\n\nThis is not an officially supported Google product.\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "A benchmark for high-dimensional robot control",
"version": "1.0.10",
"project_urls": {
"Homepage": "https://github.com/google-research/robopianist"
},
"split_keywords": [
"reinforcement-learning",
"mujoco",
"bimanual",
"dexterous-manipulation",
"piano"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0c87db3bbca95c335d52f0be39908f6b5b63aaf8ca35a468f530fd59ba7f7a99",
"md5": "49fff65764acc29a600ee1b78a772105",
"sha256": "785bb78cc50a4a0e2215732ab0726dcf09ffccd24d09287a67d5cbd722750861"
},
"downloads": -1,
"filename": "robopianist-1.0.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "49fff65764acc29a600ee1b78a772105",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 7806952,
"upload_time": "2023-12-24T09:13:26",
"upload_time_iso_8601": "2023-12-24T09:13:26.621095Z",
"url": "https://files.pythonhosted.org/packages/0c/87/db3bbca95c335d52f0be39908f6b5b63aaf8ca35a468f530fd59ba7f7a99/robopianist-1.0.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ebd005ca4146debfd0df530b6f1ba78295ad2b61868caa2540b9965578b82d89",
"md5": "02352e53e53ad4fdae0508e3b55e5f58",
"sha256": "c60aa885ead3c8bf74f204d79694af19430ee7500bb2a8a75dc30e66b1bcf1c4"
},
"downloads": -1,
"filename": "robopianist-1.0.10.tar.gz",
"has_sig": false,
"md5_digest": "02352e53e53ad4fdae0508e3b55e5f58",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 7761595,
"upload_time": "2023-12-24T09:13:29",
"upload_time_iso_8601": "2023-12-24T09:13:29.068209Z",
"url": "https://files.pythonhosted.org/packages/eb/d0/05ca4146debfd0df530b6f1ba78295ad2b61868caa2540b9965578b82d89/robopianist-1.0.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-24 09:13:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "google-research",
"github_project": "robopianist",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "robopianist"
}
JSON
