# SAPIEN: A SimulAted Part-based Interactive ENvironment
SAPIEN is a realistic and physics-rich simulated environment that hosts a
large-scale set for articulated objects. It enables various robotic vision and
interaction tasks that require detailed part-level understanding. SAPIEN is a
collaborative effort between researchers at UCSD, Stanford and SFU. The dataset
is a continuation of ShapeNet and PartNet.
## Change Log
<details open>
<summary>2.2</summary>
- Rename `VulkanRenderer` to `SapienRenderer` (VulkanRenderer is still an alias)
- Support **ray tracing** in `SapienRenderer`
- Deprecate `KuafuRenderer`, use the rt shader in `SapienRenderer` instead
- **GPU-accelerated stereo depth sensor simulation**
- **Render server**
- Python 3.11
- bug fixes
- Fix inverse kinematics default active joint mask (now defaults to all 1s)
- Fix incorrectly exported memory in Vulkan-Cuda interop
- Fix joint `get_global_pose`
</details>
<details>
<summary>2.1</summary>
- Python 3.10
- Bug fixes
- crash when not using renderer
- joint force limit (was impulse limit)
- incorrect inertia computation in scaled URDF
- incorrect point-light shadow
- incorrect collision when loaded from dae
- Utility improvements
- set_material
- active light
- flat shading
- dynamic point rendering
- envmap generation
- multi-thread envs
</details>
<details>
<summary>2.1</summary>
- Refactor light system
- Remove light functions on scene.renderer_scene
- Refactor camera system
- Cameras no longer require mounts
- Camera can change its mount and mounted pose by `camera.set_parent` and
`camera.set_local_pose`.
- When camera is not mounted, setting local pose is setting its global pose.
- Add functions `scene.add_camera` and `scene.remove_camera`
- `add_mounted_camera` can be replaced with `add_camera` followed by
`camera.set_parent` and `camera.set_local_pose`. `add_mounted_camera` is
still provided but fovx should not longer be provided.
- Remove functions related to mount, including `find_camera_by_mount`.
- Cameras now support full camera parameters through `camera.near`,
`camera.far`, `camera.set_fovx`, `camera.set_fovy`,
`camera.set_focal_lengths`, `camera.set_principal_point`, `camera.skew`, and
the all-in-one method `camera.set_perspective_parameters`.
- Refactor render shape system
- Originally, after `actor.get_visual_bodies()` and
`visual_body.get_render_shapes()`, users typically do `shape.scale` and
`shape.pose`. These are no longer valid. It is required to check
`visual_body.type`. When `type` is `mesh`, `shape.scale` is replaced with
`visual_body.scale` and `shape.pose` is replaced by
`visual_body.local_pose`. These changes are made to match `add_visual_shape`
functions when building the actor.
</details>
<details>
<summary>pre2.0</summary>
- Shader change: 4th component in default camera shader now gives the 0-1 depth value.
- Add "critical" and "off" log levels.
- Add support for pointcloud and line rendering (for visualizing camera and point cloud)
- Performance: the same shader only compile once per process
- Bug fix
- Articulation setDriveTarget was now correctly reversed for prismatic joint (joint setDriveTarget is not affected)
- Fix kinematic articulation loader
</details>
<details>
<summary>1 to 2 migration</summary>
- replace `scene.renderer_scene.add_xxx_light` with `scene.add_xxx_light`
- replace `scene.remove_mounted_camera` with `scene.remove_camera`
- optionally, remove `fovx` from `scene.add_mounted_camera`.
</details>
<details>
<summary>1.1</summary>
- Support nonconvex static/kinematic collision shape
- Add warning for small mass/inertia
- Introduce Entity as the base class of Actors
- Add Light classes inherited from entity, allowing manipulate light objects in sapien scene
- Updates to the viewer
- rename actor to entity when appropriate
- Partial support the material tag in URDF loader (primitive shape, single color)
- Bug fixes for the renderer
- Support inner and outer FOV for spotlight
</details>
<details>
<summary>1.0</summary>
- Replace the old Vulkan based renderer completely
- See `sapien.core.renderer` for details
- Expose GUI functionalities to Python
- Reimplement Vulkan viewer in Python
- Expose PhysX shape wrapper to Python. For example,
- Collision shapes can be retrieved through `actor.get_collision_shapes`
- Collision groups on a shape can be set by `CollisionShape.set_collision_groups`
- Shapes are now also available in `Contact`.
- API changes
- Render material creation is now `renderer.create_material()`
- in actor builder: `add_xxx_shape` is replaced with `add_xxx_collision`.
- move light functions from scene to `scene.renderer_scene`
- Add centrifugal and Coriolis force.
- Change default physical parameters for better stability.
</details>
## SAPIEN Engine
SAPIEN Engine provides physical simulation for articulated objects. It powers
reinforcement learning and robotics with its pure Python interface.
## SAPIEN Renderer
SAPIEN provides rasterized and ray traced rendering with Vulkan.
## PartNet-Mobility
SAPIEN releases PartNet-Mobility dataset, which is a collection of 2K
articulated objects with motion annotations and rendernig material. The dataset
powers research for generalizable computer vision and manipulation.
## Website and Documentation
SAPIEN Website: [https://sapien.ucsd.edu/](https://sapien.ucsd.edu/). SAPIEN
Documentation:
[https://sapien.ucsd.edu/docs/latest/index.html](https://sapien.ucsd.edu/docs/latest/index.html).
## Build from source
### Before build
Make sure all submodules are initialized `git submodule update --init --recursive`.
### Build with Docker
To build SAPIEN, simply run `./docker_build_wheels.sh`. It is not recommended to
build outside of our provided docker.
For reference, the Dockerfile is provided [here](/docker/Dockerfile). Note that
PhysX needs to be compiled with clang-9 into static libraries before building
the Docker image.
### Build without Docker
It can be tricky to setup all dependencies outside of a Docker environment. You
need to install all dependencies according to the [Docker
environment](/docker/Dockerfile). If all dependencies set up correctly, run
`python setup.py bdist_wheel` to build the wheel.
## Cite SAPIEN
If you use SAPIEN and its assets, please cite the following works:
```
@InProceedings{Xiang_2020_SAPIEN,
author = {Xiang, Fanbo and Qin, Yuzhe and Mo, Kaichun and Xia, Yikuan and Zhu, Hao and Liu, Fangchen and Liu, Minghua and Jiang, Hanxiao and Yuan, Yifu and Wang, He and Yi, Li and Chang, Angel X. and Guibas, Leonidas J. and Su, Hao},
title = {{SAPIEN}: A SimulAted Part-based Interactive ENvironment},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}}
```
```
@InProceedings{Mo_2019_CVPR,
author = {Mo, Kaichun and Zhu, Shilin and Chang, Angel X. and Yi, Li and Tripathi, Subarna and Guibas, Leonidas J. and Su, Hao},
title = {{PartNet}: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level {3D} Object Understanding},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}
```
```
@article{chang2015shapenet,
title={Shapenet: An information-rich 3d model repository},
author={Chang, Angel X and Funkhouser, Thomas and Guibas, Leonidas and Hanrahan, Pat and Huang, Qixing and Li, Zimo and Savarese, Silvio and Savva, Manolis and Song, Shuran and Su, Hao and others},
journal={arXiv preprint arXiv:1512.03012},
year={2015}
}
```
If you use SAPIEN Realistic Depth generated by SAPIEN's simulated depth sensor, please cite the following work:
```
@ARTICLE{10027470,
author={Zhang, Xiaoshuai and Chen, Rui and Li, Ang and Xiang, Fanbo and Qin, Yuzhe and Gu, Jiayuan and Ling, Zhan and Liu, Minghua and Zeng, Peiyu and Han, Songfang and Huang, Zhiao and Mu, Tongzhou and Xu, Jing and Su, Hao},
journal={IEEE Transactions on Robotics},
title={Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation},
year={2023},
volume={},
number={},
pages={1-19},
doi={10.1109/TRO.2023.3235591}}
```
Raw data
{
"_id": null,
"home_page": "https://sapien.ucsd.edu",
"name": "sapien",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "robotics simulator dataset articulation partnet",
"author": "Sapien",
"author_email": "sapienaicontact@gmail.com",
"download_url": "",
"platform": null,
"description": "# SAPIEN: A SimulAted Part-based Interactive ENvironment\nSAPIEN is a realistic and physics-rich simulated environment that hosts a\nlarge-scale set for articulated objects. It enables various robotic vision and\ninteraction tasks that require detailed part-level understanding. SAPIEN is a\ncollaborative effort between researchers at UCSD, Stanford and SFU. The dataset\nis a continuation of ShapeNet and PartNet.\n\n## Change Log\n\n<details open>\n<summary>2.2</summary>\n\n- Rename `VulkanRenderer` to `SapienRenderer` (VulkanRenderer is still an alias)\n- Support **ray tracing** in `SapienRenderer`\n- Deprecate `KuafuRenderer`, use the rt shader in `SapienRenderer` instead\n- **GPU-accelerated stereo depth sensor simulation**\n- **Render server**\n- Python 3.11\n- bug fixes\n - Fix inverse kinematics default active joint mask (now defaults to all 1s)\n - Fix incorrectly exported memory in Vulkan-Cuda interop\n - Fix joint `get_global_pose`\n</details>\n\n<details>\n<summary>2.1</summary>\n\n- Python 3.10\n- Bug fixes\n - crash when not using renderer\n - joint force limit (was impulse limit)\n - incorrect inertia computation in scaled URDF\n - incorrect point-light shadow\n - incorrect collision when loaded from dae\n- Utility improvements\n - set_material\n - active light\n - flat shading\n - dynamic point rendering\n - envmap generation\n - multi-thread envs\n\n</details>\n\n<details>\n<summary>2.1</summary>\n\n- Refactor light system\n - Remove light functions on scene.renderer_scene\n- Refactor camera system\n - Cameras no longer require mounts\n - Camera can change its mount and mounted pose by `camera.set_parent` and\n `camera.set_local_pose`.\n - When camera is not mounted, setting local pose is setting its global pose.\n - Add functions `scene.add_camera` and `scene.remove_camera`\n - `add_mounted_camera` can be replaced with `add_camera` followed by\n `camera.set_parent` and `camera.set_local_pose`. `add_mounted_camera` is\n still provided but fovx should not longer be provided.\n - Remove functions related to mount, including `find_camera_by_mount`.\n - Cameras now support full camera parameters through `camera.near`,\n `camera.far`, `camera.set_fovx`, `camera.set_fovy`,\n `camera.set_focal_lengths`, `camera.set_principal_point`, `camera.skew`, and\n the all-in-one method `camera.set_perspective_parameters`.\n- Refactor render shape system\n - Originally, after `actor.get_visual_bodies()` and\n `visual_body.get_render_shapes()`, users typically do `shape.scale` and\n `shape.pose`. These are no longer valid. It is required to check\n `visual_body.type`. When `type` is `mesh`, `shape.scale` is replaced with\n `visual_body.scale` and `shape.pose` is replaced by\n `visual_body.local_pose`. These changes are made to match `add_visual_shape`\n functions when building the actor.\n</details>\n\n<details>\n<summary>pre2.0</summary>\n\n- Shader change: 4th component in default camera shader now gives the 0-1 depth value.\n- Add \"critical\" and \"off\" log levels.\n- Add support for pointcloud and line rendering (for visualizing camera and point cloud)\n- Performance: the same shader only compile once per process\n- Bug fix\n - Articulation setDriveTarget was now correctly reversed for prismatic joint (joint setDriveTarget is not affected)\n - Fix kinematic articulation loader\n</details>\n\n<details>\n<summary>1 to 2 migration</summary>\n\n- replace `scene.renderer_scene.add_xxx_light` with `scene.add_xxx_light`\n- replace `scene.remove_mounted_camera` with `scene.remove_camera`\n- optionally, remove `fovx` from `scene.add_mounted_camera`.\n</details>\n\n\n<details>\n<summary>1.1</summary>\n\n- Support nonconvex static/kinematic collision shape\n- Add warning for small mass/inertia\n- Introduce Entity as the base class of Actors\n- Add Light classes inherited from entity, allowing manipulate light objects in sapien scene\n- Updates to the viewer\n - rename actor to entity when appropriate\n- Partial support the material tag in URDF loader (primitive shape, single color)\n- Bug fixes for the renderer\n- Support inner and outer FOV for spotlight\n</details>\n\n<details>\n<summary>1.0</summary>\n\n- Replace the old Vulkan based renderer completely\n - See `sapien.core.renderer` for details\n- Expose GUI functionalities to Python\n- Reimplement Vulkan viewer in Python \n- Expose PhysX shape wrapper to Python. For example,\n - Collision shapes can be retrieved through `actor.get_collision_shapes`\n - Collision groups on a shape can be set by `CollisionShape.set_collision_groups`\n - Shapes are now also available in `Contact`.\n- API changes\n - Render material creation is now `renderer.create_material()`\n - in actor builder: `add_xxx_shape` is replaced with `add_xxx_collision`.\n - move light functions from scene to `scene.renderer_scene`\n- Add centrifugal and Coriolis force.\n- Change default physical parameters for better stability.\n</details>\n\n## SAPIEN Engine\nSAPIEN Engine provides physical simulation for articulated objects. It powers\nreinforcement learning and robotics with its pure Python interface.\n\n## SAPIEN Renderer\nSAPIEN provides rasterized and ray traced rendering with Vulkan.\n\n## PartNet-Mobility\nSAPIEN releases PartNet-Mobility dataset, which is a collection of 2K\narticulated objects with motion annotations and rendernig material. The dataset\npowers research for generalizable computer vision and manipulation.\n\n## Website and Documentation\nSAPIEN Website: [https://sapien.ucsd.edu/](https://sapien.ucsd.edu/). SAPIEN\nDocumentation:\n[https://sapien.ucsd.edu/docs/latest/index.html](https://sapien.ucsd.edu/docs/latest/index.html).\n\n## Build from source\n### Before build\nMake sure all submodules are initialized `git submodule update --init --recursive`.\n\n### Build with Docker\nTo build SAPIEN, simply run `./docker_build_wheels.sh`. It is not recommended to\nbuild outside of our provided docker.\n\nFor reference, the Dockerfile is provided [here](/docker/Dockerfile). Note that\nPhysX needs to be compiled with clang-9 into static libraries before building\nthe Docker image.\n\n### Build without Docker\nIt can be tricky to setup all dependencies outside of a Docker environment. You\nneed to install all dependencies according to the [Docker\nenvironment](/docker/Dockerfile). If all dependencies set up correctly, run\n`python setup.py bdist_wheel` to build the wheel.\n\n## Cite SAPIEN\nIf you use SAPIEN and its assets, please cite the following works:\n```\n@InProceedings{Xiang_2020_SAPIEN,\nauthor = {Xiang, Fanbo and Qin, Yuzhe and Mo, Kaichun and Xia, Yikuan and Zhu, Hao and Liu, Fangchen and Liu, Minghua and Jiang, Hanxiao and Yuan, Yifu and Wang, He and Yi, Li and Chang, Angel X. and Guibas, Leonidas J. and Su, Hao},\ntitle = {{SAPIEN}: A SimulAted Part-based Interactive ENvironment},\nbooktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},\nmonth = {June},\nyear = {2020}}\n```\n```\n@InProceedings{Mo_2019_CVPR,\nauthor = {Mo, Kaichun and Zhu, Shilin and Chang, Angel X. and Yi, Li and Tripathi, Subarna and Guibas, Leonidas J. and Su, Hao},\ntitle = {{PartNet}: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level {3D} Object Understanding},\nbooktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},\nmonth = {June},\nyear = {2019}\n}\n```\n```\n@article{chang2015shapenet,\ntitle={Shapenet: An information-rich 3d model repository},\nauthor={Chang, Angel X and Funkhouser, Thomas and Guibas, Leonidas and Hanrahan, Pat and Huang, Qixing and Li, Zimo and Savarese, Silvio and Savva, Manolis and Song, Shuran and Su, Hao and others},\njournal={arXiv preprint arXiv:1512.03012},\nyear={2015}\n}\n```\nIf you use SAPIEN Realistic Depth generated by SAPIEN's simulated depth sensor, please cite the following work:\n```\n@ARTICLE{10027470,\n author={Zhang, Xiaoshuai and Chen, Rui and Li, Ang and Xiang, Fanbo and Qin, Yuzhe and Gu, Jiayuan and Ling, Zhan and Liu, Minghua and Zeng, Peiyu and Han, Songfang and Huang, Zhiao and Mu, Tongzhou and Xu, Jing and Su, Hao},\n journal={IEEE Transactions on Robotics}, \n title={Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation}, \n year={2023},\n volume={},\n number={},\n pages={1-19},\n doi={10.1109/TRO.2023.3235591}}\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "['SAPIEN: A SimulAted Parted based Interactive ENvironment']",
"version": "2.2.2",
"project_urls": {
"Documentation": "https://sapien.ucsd.edu/docs",
"Homepage": "https://sapien.ucsd.edu"
},
"split_keywords": [
"robotics",
"simulator",
"dataset",
"articulation",
"partnet"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5475747bd51446fb1f9ece4a8ac3b22963c86ab0239e8e96f037fce774151158",
"md5": "ddd7093cc863424be758c80d64d136e2",
"sha256": "9c69e88938bb79f79d7922c0c7886c51141efb607abb500fe67ce4575b445db0"
},
"downloads": -1,
"filename": "sapien-2.2.2-cp310-cp310-manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "ddd7093cc863424be758c80d64d136e2",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.7",
"size": 39093824,
"upload_time": "2023-08-03T20:42:04",
"upload_time_iso_8601": "2023-08-03T20:42:04.870776Z",
"url": "https://files.pythonhosted.org/packages/54/75/747bd51446fb1f9ece4a8ac3b22963c86ab0239e8e96f037fce774151158/sapien-2.2.2-cp310-cp310-manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e3301549a98d8a2cb3c7c8d02c463680d980d36dd23f3dc52cb90e7ff9f9920f",
"md5": "6517cc41ed99f00dc509e238061cbfbc",
"sha256": "575f7eda16136b6cfee7d0519df622a037e90f4e164573a8ffa278645c7c16df"
},
"downloads": -1,
"filename": "sapien-2.2.2-cp311-cp311-manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "6517cc41ed99f00dc509e238061cbfbc",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.7",
"size": 39093330,
"upload_time": "2023-08-03T20:42:09",
"upload_time_iso_8601": "2023-08-03T20:42:09.015804Z",
"url": "https://files.pythonhosted.org/packages/e3/30/1549a98d8a2cb3c7c8d02c463680d980d36dd23f3dc52cb90e7ff9f9920f/sapien-2.2.2-cp311-cp311-manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e6355b07765421d96cca3bc36b6b1da463081a33eb15ccf146f5c6e52b848d30",
"md5": "7027a8422953d2f232b94903e0098c0c",
"sha256": "c8568851e1384263c62af87412b3f70b350b468e9bcf7e7a7203c9869b31d64b"
},
"downloads": -1,
"filename": "sapien-2.2.2-cp37-cp37m-manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "7027a8422953d2f232b94903e0098c0c",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": ">=3.7",
"size": 39113229,
"upload_time": "2023-08-03T20:42:12",
"upload_time_iso_8601": "2023-08-03T20:42:12.885207Z",
"url": "https://files.pythonhosted.org/packages/e6/35/5b07765421d96cca3bc36b6b1da463081a33eb15ccf146f5c6e52b848d30/sapien-2.2.2-cp37-cp37m-manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fa8d0dd721d197bba53f40f005008bd994a65f1a6e97ef351c2ab6c119d0ef1b",
"md5": "0f51dae11542e90603bbed21506c4786",
"sha256": "096e80c4da7a3377185c9c7763238af50cb5c9baba74a78a4e08ae585b364026"
},
"downloads": -1,
"filename": "sapien-2.2.2-cp38-cp38-manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "0f51dae11542e90603bbed21506c4786",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.7",
"size": 39093623,
"upload_time": "2023-08-03T20:42:17",
"upload_time_iso_8601": "2023-08-03T20:42:17.731486Z",
"url": "https://files.pythonhosted.org/packages/fa/8d/0dd721d197bba53f40f005008bd994a65f1a6e97ef351c2ab6c119d0ef1b/sapien-2.2.2-cp38-cp38-manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e0a15e8e2eb886309de0c27ddaffaaa5e0e577d1d81dad2dacfb538a79e0ae3a",
"md5": "0c603b2c5a55d5bcb908d1df2473904f",
"sha256": "2e18640ba5c70003884e1169b407d3c2363e9211a91937b6cc00c07b465f848f"
},
"downloads": -1,
"filename": "sapien-2.2.2-cp39-cp39-manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "0c603b2c5a55d5bcb908d1df2473904f",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.7",
"size": 39094054,
"upload_time": "2023-08-03T20:42:23",
"upload_time_iso_8601": "2023-08-03T20:42:23.138994Z",
"url": "https://files.pythonhosted.org/packages/e0/a1/5e8e2eb886309de0c27ddaffaaa5e0e577d1d81dad2dacfb538a79e0ae3a/sapien-2.2.2-cp39-cp39-manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-03 20:42:04",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "sapien"
}
JSON