## PyPose: A Library for Robot Learning with Physics-based Optimization
-----
Deep learning has had remarkable success in robotic perception, but its data-centric nature suffers when it comes to generalizing to ever-changing environments. By contrast, physics-based optimization generalizes better, but it does not perform as well in complicated tasks due to the lack of high-level semantic information and the reliance on manual parametric tuning. To take advantage of these two complementary worlds, we present PyPose: a **robotics-oriented**, **PyTorch-based** library that combines **deep perceptual models** with **physics-based optimization techniques**. Our design goal for PyPose is to make it **user-friendly**, **efficient**, and **interpretable** with a tidy and well-organized architecture. Using an **imperative style interface**, it can be easily integrated into **real-world robotic applications**.
-----
### Current Features
##### [LieTensor](https://pypose.org/docs/main/modules/)
- Lie group: [`SO3`](https://pypose.org/docs/main/generated/pypose.SO3/), [`SE3`](https://pypose.org/docs/main/generated/pypose.SE3/), [`Sim3`](https://pypose.org/docs/main/generated/pypose.Sim3/), [`RxSO3`](https://pypose.org/docs/main/generated/pypose.RxSO3/)
- Lie algebra: [`so3`](https://pypose.org/docs/main/generated/pypose.so3/), [`se3`](https://pypose.org/docs/main/generated/pypose.se3/), [`sim3`](https://pypose.org/docs/main/generated/pypose.sim3/), [`rxso3`](https://pypose.org/docs/main/generated/pypose.rxso3/)
##### [Modules](https://pypose.org/docs/main/modules/)
- System: [`LTI`](https://pypose.org/docs/main/generated/pypose.module.LTI), [`LTV`](https://pypose.org/docs/main/generated/pypose.module.LTV), [`NLS`](https://pypose.org/docs/main/generated/pypose.module.NLS)
- Filter: [`EKF`](https://pypose.org/docs/main/generated/pypose.module.EKF/), [`UKF`](https://pypose.org/docs/main/generated/pypose.module.UKF/), [`PF`](https://pypose.org/docs/main/generated/pypose.module.PF/)
- PnP Solver: [`EPnP`](https://pypose.org/docs/main/generated/pypose.module.EPnP/)
- Linear Quadratic Regulator: [`LQR`](https://pypose.org/docs/main/generated/pypose.module.LQR/)
- IMU Preintegration: [`IMUPreintegrator`](https://pypose.org/docs/main/generated/pypose.module.IMUPreintegrator/)
- ......
##### [Second-order Optimizers](https://pypose.org/docs/main/optim/)
- [`GaussNewton`](https://pypose.org/docs/main/generated/pypose.optim.GaussNewton)
- [`LevenbergMarquardt`](https://pypose.org/docs/main/generated/pypose.optim.LevenbergMarquardt/)
- ......
Want more features? [Create an issue here](https://github.com/pypose/pypose/issues) to request new features.
##### PyPose is highly efficient and supports parallel computing for Jacobian of Lie group and Lie algebra. See following comparison.
<img width="1167" alt="image" src="https://user-images.githubusercontent.com/8695500/203210668-1a90224a-ae08-4d31-b9d1-e293be75ef3e.png">
Efficiency and memory comparison of batched Lie group operations (we take Theseus performance as 1×).
More information about efficiency comparison goes to [our paper for PyPose](https://arxiv.org/abs/2209.15428).
## Getting Started
### Installation
#### Install from **pypi**
```bash
pip install pypose
```
#### Install from source
1. Requirement:
On Ubuntu, macOS, or Windows, install [PyTorch](https://pytorch.org/), then run:
```bash
pip install -r requirements/runtime.txt
```
2. Install locally:
```bash
git clone https://github.com/pypose/pypose.git
cd pypose && python setup.py develop
```
3. Run tests
```bash
pytest
```
#### For contributors
1. Make sure the above installation is correct.
2. Go to [CONTRIBUTING.md](CONTRIBUTING.md)
#### Examples
1. The following code sample shows how to rotate random points and compute the gradient of batched rotation.
```python
>>> import torch, pypose as pp
>>> # A random so(3) LieTensor
>>> r = pp.randn_so3(2, requires_grad=True)
so3Type LieTensor:
tensor([[ 0.1606, 0.0232, -1.5516],
[-0.0807, -0.7184, -0.1102]], requires_grad=True)
>>> R = r.Exp() # Equivalent to: R = pp.Exp(r)
SO3Type LieTensor:
tensor([[ 0.0724, 0.0104, -0.6995, 0.7109],
[-0.0395, -0.3513, -0.0539, 0.9339]], grad_fn=<AliasBackward0>)
>>> p = R @ torch.randn(3) # Rotate random point
tensor([[ 0.8045, -0.8555, 0.5260],
[ 0.3502, 0.8337, 0.9154]], grad_fn=<ViewBackward0>)
>>> p.sum().backward() # Compute gradient
>>> r.grad # Print gradient
tensor([[-0.7920, -0.9510, 1.7110],
[-0.2659, 0.5709, -0.3855]])
```
2. This example shows how to estimate batched inverse of transform by a second-order optimizer. Two usage options for a `scheduler` are provided, each of which can work independently.
```python
>>> from torch import nn
>>> import torch, pypose as pp
>>> from pypose.optim import LM
>>> from pypose.optim.strategy import Constant
>>> from pypose.optim.scheduler import StopOnPlateau
>>> class InvNet(nn.Module):
def __init__(self, *dim):
super().__init__()
init = pp.randn_SE3(*dim)
self.pose = pp.Parameter(init)
def forward(self, input):
error = (self.pose @ input).Log()
return error.tensor()
>>> device = torch.device("cuda")
>>> input = pp.randn_SE3(2, 2, device=device)
>>> invnet = InvNet(2, 2).to(device)
>>> strategy = Constant(damping=1e-4)
>>> optimizer = LM(invnet, strategy=strategy)
>>> scheduler = StopOnPlateau(optimizer, steps=10, patience=3, decreasing=1e-3, verbose=True)
>>> # 1st option, full optimization
>>> scheduler.optimize(input=input)
>>> # 2nd option, step optimization
>>> while scheduler.continual():
loss = optimizer.step(input)
scheduler.step(loss)
>>> # Note: remove one of the above options for usage!
```
For more usage, see [Documentation](https://pypose.org/docs). For more applications, see [Examples](https://github.com/pypose/pypose/tree/main/examples).
## Citing PyPose
If you use PyPose, please cite the paper below. You may also [download it here](https://arxiv.org/abs/2209.15428).
```bibtex
@inproceedings{wang2023pypose,
title = {{PyPose}: A Library for Robot Learning with Physics-based Optimization},
author = {Wang, Chen and Gao, Dasong and Xu, Kuan and Geng, Junyi and Hu, Yaoyu and Qiu, Yuheng and Li, Bowen and Yang, Fan and Moon, Brady and Pandey, Abhinav and Aryan and Xu, Jiahe and Wu, Tianhao and He, Haonan and Huang, Daning and Ren, Zhongqiang and Zhao, Shibo and Fu, Taimeng and Reddy, Pranay and Lin, Xiao and Wang, Wenshan and Shi, Jingnan and Talak, Rajat and Cao, Kun and Du, Yi and Wang, Han and Yu, Huai and Wang, Shanzhao and Chen, Siyu and Kashyap, Ananth and Bandaru, Rohan and Dantu, Karthik and Wu, Jiajun and Xie, Lihua and Carlone, Luca and Hutter, Marco and Scherer, Sebastian},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2023}
}
```
More papers describing PyPose:
```bibtex
@inproceedings{zhan2023pypose,
title = {{PyPose} v0.6: The Imperative Programming Interface for Robotics},
author = {Zitong Zhan and Xiangfu Li and Qihang Li and Haonan He and Abhinav Pandey and Haitao Xiao and Yangmengfei Xu and Xiangyu Chen and Kuan Xu and Kun Cao and Zhipeng Zhao and Zihan Wang and Huan Xu and Zihang Fang and Yutian Chen and Wentao Wang and Xu Fang and Yi Du and Tianhao Wu and Xiao Lin and Yuheng Qiu and Fan Yang and Jingnan Shi and Shaoshu Su and Yiren Lu and Taimeng Fu and Karthik Dantu and Jiajun Wu and Lihua Xie and Marco Hutter and Luca Carlone and Sebastian Scherer and Daning Huang and Yaoyu Hu and Junyi Geng and Chen Wang},
year = {2023},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop},
}
```
Raw data
{
"_id": null,
"home_page": "https://pypose.org",
"name": "pypose",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "robotics, deep learning, pytorch",
"author": "Chen Wang and PyPose Team",
"author_email": "admin@pypose.org",
"download_url": "https://files.pythonhosted.org/packages/97/3b/ae898565c08456ae12a0dcf5c6c25b8e58b596f322d4a59f48fe01a3e87a/pypose-0.6.9.tar.gz",
"platform": null,
"description": "## PyPose: A Library for Robot Learning with Physics-based Optimization\n\n\n\n-----\n\nDeep learning has had remarkable success in robotic perception, but its data-centric nature suffers when it comes to generalizing to ever-changing environments. By contrast, physics-based optimization generalizes better, but it does not perform as well in complicated tasks due to the lack of high-level semantic information and the reliance on manual parametric tuning. To take advantage of these two complementary worlds, we present PyPose: a **robotics-oriented**, **PyTorch-based** library that combines **deep perceptual models** with **physics-based optimization techniques**. Our design goal for PyPose is to make it **user-friendly**, **efficient**, and **interpretable** with a tidy and well-organized architecture. Using an **imperative style interface**, it can be easily integrated into **real-world robotic applications**. \n\n\n-----\n\n### Current Features\n\n##### [LieTensor](https://pypose.org/docs/main/modules/)\n\n- Lie group: [`SO3`](https://pypose.org/docs/main/generated/pypose.SO3/), [`SE3`](https://pypose.org/docs/main/generated/pypose.SE3/), [`Sim3`](https://pypose.org/docs/main/generated/pypose.Sim3/), [`RxSO3`](https://pypose.org/docs/main/generated/pypose.RxSO3/)\n- Lie algebra: [`so3`](https://pypose.org/docs/main/generated/pypose.so3/), [`se3`](https://pypose.org/docs/main/generated/pypose.se3/), [`sim3`](https://pypose.org/docs/main/generated/pypose.sim3/), [`rxso3`](https://pypose.org/docs/main/generated/pypose.rxso3/)\n\n##### [Modules](https://pypose.org/docs/main/modules/)\n\n- System: [`LTI`](https://pypose.org/docs/main/generated/pypose.module.LTI), [`LTV`](https://pypose.org/docs/main/generated/pypose.module.LTV), [`NLS`](https://pypose.org/docs/main/generated/pypose.module.NLS)\n- Filter: [`EKF`](https://pypose.org/docs/main/generated/pypose.module.EKF/), [`UKF`](https://pypose.org/docs/main/generated/pypose.module.UKF/), [`PF`](https://pypose.org/docs/main/generated/pypose.module.PF/)\n- PnP Solver: [`EPnP`](https://pypose.org/docs/main/generated/pypose.module.EPnP/)\n- Linear Quadratic Regulator: [`LQR`](https://pypose.org/docs/main/generated/pypose.module.LQR/)\n- IMU Preintegration: [`IMUPreintegrator`](https://pypose.org/docs/main/generated/pypose.module.IMUPreintegrator/)\n- ......\n\n##### [Second-order Optimizers](https://pypose.org/docs/main/optim/)\n\n- [`GaussNewton`](https://pypose.org/docs/main/generated/pypose.optim.GaussNewton)\n- [`LevenbergMarquardt`](https://pypose.org/docs/main/generated/pypose.optim.LevenbergMarquardt/)\n- ......\n\nWant more features? [Create an issue here](https://github.com/pypose/pypose/issues) to request new features.\n\n##### PyPose is highly efficient and supports parallel computing for Jacobian of Lie group and Lie algebra. See following comparison.\n\n<img width=\"1167\" alt=\"image\" src=\"https://user-images.githubusercontent.com/8695500/203210668-1a90224a-ae08-4d31-b9d1-e293be75ef3e.png\">\n\nEfficiency and memory comparison of batched Lie group operations (we take Theseus performance as 1\u00d7).\n\nMore information about efficiency comparison goes to [our paper for PyPose](https://arxiv.org/abs/2209.15428).\n\n## Getting Started\n \n### Installation\n\n#### Install from **pypi**\n```bash\npip install pypose\n```\n\n#### Install from source\n\n1. Requirement:\n\nOn Ubuntu, macOS, or Windows, install [PyTorch](https://pytorch.org/), then run:\n\n```bash\npip install -r requirements/runtime.txt\n```\n\n2. Install locally:\n\n```bash\ngit clone https://github.com/pypose/pypose.git\ncd pypose && python setup.py develop\n```\n\n3. Run tests\n\n```bash\npytest\n```\n\n#### For contributors\n\n1. Make sure the above installation is correct. \n\n2. Go to [CONTRIBUTING.md](CONTRIBUTING.md)\n\n\n#### Examples\n\n1. The following code sample shows how to rotate random points and compute the gradient of batched rotation.\n\n```python\n>>> import torch, pypose as pp\n\n>>> # A random so(3) LieTensor\n>>> r = pp.randn_so3(2, requires_grad=True)\n so3Type LieTensor:\n tensor([[ 0.1606, 0.0232, -1.5516],\n [-0.0807, -0.7184, -0.1102]], requires_grad=True)\n\n>>> R = r.Exp() # Equivalent to: R = pp.Exp(r)\n SO3Type LieTensor:\n tensor([[ 0.0724, 0.0104, -0.6995, 0.7109],\n [-0.0395, -0.3513, -0.0539, 0.9339]], grad_fn=<AliasBackward0>)\n\n>>> p = R @ torch.randn(3) # Rotate random point\n tensor([[ 0.8045, -0.8555, 0.5260],\n [ 0.3502, 0.8337, 0.9154]], grad_fn=<ViewBackward0>)\n\n>>> p.sum().backward() # Compute gradient\n>>> r.grad # Print gradient\n tensor([[-0.7920, -0.9510, 1.7110],\n [-0.2659, 0.5709, -0.3855]])\n```\n\n2. This example shows how to estimate batched inverse of transform by a second-order optimizer. Two usage options for a `scheduler` are provided, each of which can work independently.\n\n```python\n>>> from torch import nn\n>>> import torch, pypose as pp\n>>> from pypose.optim import LM\n>>> from pypose.optim.strategy import Constant\n>>> from pypose.optim.scheduler import StopOnPlateau\n\n>>> class InvNet(nn.Module):\n\n def __init__(self, *dim):\n super().__init__()\n init = pp.randn_SE3(*dim)\n self.pose = pp.Parameter(init)\n\n def forward(self, input):\n error = (self.pose @ input).Log()\n return error.tensor()\n\n>>> device = torch.device(\"cuda\")\n>>> input = pp.randn_SE3(2, 2, device=device)\n>>> invnet = InvNet(2, 2).to(device)\n>>> strategy = Constant(damping=1e-4)\n>>> optimizer = LM(invnet, strategy=strategy)\n>>> scheduler = StopOnPlateau(optimizer, steps=10, patience=3, decreasing=1e-3, verbose=True)\n\n>>> # 1st option, full optimization\n>>> scheduler.optimize(input=input)\n\n>>> # 2nd option, step optimization\n>>> while scheduler.continual():\n loss = optimizer.step(input)\n scheduler.step(loss)\n\n>>> # Note: remove one of the above options for usage!\n```\n\nFor more usage, see [Documentation](https://pypose.org/docs). For more applications, see [Examples](https://github.com/pypose/pypose/tree/main/examples).\n\n## Citing PyPose\n\nIf you use PyPose, please cite the paper below. You may also [download it here](https://arxiv.org/abs/2209.15428).\n\n```bibtex\n@inproceedings{wang2023pypose,\n title = {{PyPose}: A Library for Robot Learning with Physics-based Optimization},\n author = {Wang, Chen and Gao, Dasong and Xu, Kuan and Geng, Junyi and Hu, Yaoyu and Qiu, Yuheng and Li, Bowen and Yang, Fan and Moon, Brady and Pandey, Abhinav and Aryan and Xu, Jiahe and Wu, Tianhao and He, Haonan and Huang, Daning and Ren, Zhongqiang and Zhao, Shibo and Fu, Taimeng and Reddy, Pranay and Lin, Xiao and Wang, Wenshan and Shi, Jingnan and Talak, Rajat and Cao, Kun and Du, Yi and Wang, Han and Yu, Huai and Wang, Shanzhao and Chen, Siyu and Kashyap, Ananth and Bandaru, Rohan and Dantu, Karthik and Wu, Jiajun and Xie, Lihua and Carlone, Luca and Hutter, Marco and Scherer, Sebastian},\n booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n year = {2023}\n}\n```\n\nMore papers describing PyPose:\n\n```bibtex\n@inproceedings{zhan2023pypose,\n title = {{PyPose} v0.6: The Imperative Programming Interface for Robotics},\n author = {Zitong Zhan and Xiangfu Li and Qihang Li and Haonan He and Abhinav Pandey and Haitao Xiao and Yangmengfei Xu and Xiangyu Chen and Kuan Xu and Kun Cao and Zhipeng Zhao and Zihan Wang and Huan Xu and Zihang Fang and Yutian Chen and Wentao Wang and Xu Fang and Yi Du and Tianhao Wu and Xiao Lin and Yuheng Qiu and Fan Yang and Jingnan Shi and Shaoshu Su and Yiren Lu and Taimeng Fu and Karthik Dantu and Jiajun Wu and Lihua Xie and Marco Hutter and Luca Carlone and Sebastian Scherer and Daning Huang and Yaoyu Hu and Junyi Geng and Chen Wang},\n year = {2023},\n booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop},\n}\n```\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "To connect classic robotics with modern learning methods.",
"version": "0.6.9",
"project_urls": {
"Bug Tracker": "https://github.com/pypose/pypose/issues",
"Documentation": "https://pypose.org/docs",
"Download": "https://github.com/pypose/pypose",
"Homepage": "https://pypose.org",
"Source Code": "https://github.com/pypose/pypose"
},
"split_keywords": [
"robotics",
" deep learning",
" pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bb19762602c0970d3779a2a500fca9a92c5f6a18347c8e54765aef7394cf9750",
"md5": "aba581a9802de741149adea27bd07dfe",
"sha256": "2393fda54a8e20f8ca91bfca00e44ab6a32216100370d194150bf2b6e0e71f85"
},
"downloads": -1,
"filename": "pypose-0.6.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "aba581a9802de741149adea27bd07dfe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 128165,
"upload_time": "2024-12-01T17:07:01",
"upload_time_iso_8601": "2024-12-01T17:07:01.903311Z",
"url": "https://files.pythonhosted.org/packages/bb/19/762602c0970d3779a2a500fca9a92c5f6a18347c8e54765aef7394cf9750/pypose-0.6.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "973bae898565c08456ae12a0dcf5c6c25b8e58b596f322d4a59f48fe01a3e87a",
"md5": "f8c75a5c815b448aaa120488d703e2b7",
"sha256": "f1bc29b11e72448b08251cd54638db621d2d61123aa59bbccdd8a9b730e4eb6d"
},
"downloads": -1,
"filename": "pypose-0.6.9.tar.gz",
"has_sig": false,
"md5_digest": "f8c75a5c815b448aaa120488d703e2b7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 115582,
"upload_time": "2024-12-01T17:07:03",
"upload_time_iso_8601": "2024-12-01T17:07:03.669768Z",
"url": "https://files.pythonhosted.org/packages/97/3b/ae898565c08456ae12a0dcf5c6c25b8e58b596f322d4a59f48fe01a3e87a/pypose-0.6.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-01 17:07:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pypose",
"github_project": "pypose",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"circle": true,
"lcname": "pypose"
}
JSON
