<img width="936" height="431" alt="Screenshot 2025-07-10 at 5 12 49 PM" src="https://github.com/user-attachments/assets/8bcd69d5-c02c-4f17-bb33-25c6363f1935" />
## LBM - TRI (wip)
Implementation of the Large Behavioral Model architecture for [dexterous manipulation](https://arxiv.org/abs/2507.05331) from Toyota Research Institute.
[Project Page](https://toyotaresearchinstitute.github.io/lbm1/)
[Data Normalization Takeaway](https://github.com/tuul-ai/robotbuilder/blob/main/timestep_norm.md)
## Install
```shell
$ pip install TRI-LBM
```
## Usage
```python
import torch
from TRI_LBM.lbm import LBM
lbm = LBM(
action_dim = 20,
dim_pose = 10
)
commands = ['pick up the apple and place in the blue tray']
images = torch.randn(1, 3, 3, 224, 224)
actions = torch.randn(1, 16, 20)
pose = torch.randn(1, 10)
loss = lbm(
text = commands,
images = images,
pose = pose,
actions = actions,
)
loss.backward()
# after much training
sampled_actions = lbm.sample(
text = commands,
images = images,
pose = pose,
) # (1, 16, 20)
```
## Citations
```bibtex
@misc{trilbmteam2025carefulexaminationlargebehavior,
title = {A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation},
author = {TRI LBM Team and Jose Barreiros and Andrew Beaulieu and Aditya Bhat and Rick Cory and Eric Cousineau and Hongkai Dai and Ching-Hsin Fang and Kunimatsu Hashimoto and Muhammad Zubair Irshad and Masha Itkina and Naveen Kuppuswamy and Kuan-Hui Lee and Katherine Liu and Dale McConachie and Ian McMahon and Haruki Nishimura and Calder Phillips-Grafflin and Charles Richter and Paarth Shah and Krishnan Srinivasan and Blake Wulfe and Chen Xu and Mengchao Zhang and Alex Alspach and Maya Angeles and Kushal Arora and Vitor Campagnolo Guizilini and Alejandro Castro and Dian Chen and Ting-Sheng Chu and Sam Creasey and Sean Curtis and Richard Denitto and Emma Dixon and Eric Dusel and Matthew Ferreira and Aimee Goncalves and Grant Gould and Damrong Guoy and Swati Gupta and Xuchen Han and Kyle Hatch and Brendan Hathaway and Allison Henry and Hillel Hochsztein and Phoebe Horgan and Shun Iwase and Donovon Jackson and Siddharth Karamcheti and Sedrick Keh and Joseph Masterjohn and Jean Mercat and Patrick Miller and Paul Mitiguy and Tony Nguyen and Jeremy Nimmer and Yuki Noguchi and Reko Ong and Aykut Onol and Owen Pfannenstiehl and Richard Poyner and Leticia Priebe Mendes Rocha and Gordon Richardson and Christopher Rodriguez and Derick Seale and Michael Sherman and Mariah Smith-Jones and David Tago and Pavel Tokmakov and Matthew Tran and Basile Van Hoorick and Igor Vasiljevic and Sergey Zakharov and Mark Zolotas and Rares Ambrus and Kerri Fetzer-Borelli and Benjamin Burchfiel and Hadas Kress-Gazit and Siyuan Feng and Stacie Ford and Russ Tedrake},
year = {2025},
eprint = {2507.05331},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2507.05331},
}
```
```bibtex
@inproceedings{Wagenmaker2025SteeringYD,
title = {Steering Your Diffusion Policy with Latent Space Reinforcement Learning},
author = {Andrew Wagenmaker and Mitsuhiko Nakamoto and Yunchu Zhang and Seohong Park and Waleed Yagoub and Anusha Nagabandi and Abhishek Gupta and Sergey Levine},
year = {2025},
url = {https://api.semanticscholar.org/CorpusID:279464702}
}
```
Raw data
{
"_id": null,
"home_page": null,
"name": "TRI-LBM",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "artificial intelligence, deep learning, transformers, attention mechanism, diffusion, robotic dexterity",
"author": null,
"author_email": "Phil Wang <lucidrains@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/3c/45/373d22d13bcbdf447a3a9cfb01838d3f997b224f0ca94ef9b87841abf457/tri_lbm-0.0.6.tar.gz",
"platform": null,
"description": "<img width=\"936\" height=\"431\" alt=\"Screenshot 2025-07-10 at 5 12 49\u202fPM\" src=\"https://github.com/user-attachments/assets/8bcd69d5-c02c-4f17-bb33-25c6363f1935\" />\n\n## LBM - TRI (wip)\n\nImplementation of the Large Behavioral Model architecture for [dexterous manipulation](https://arxiv.org/abs/2507.05331) from Toyota Research Institute. \n\n[Project Page](https://toyotaresearchinstitute.github.io/lbm1/)\n\n[Data Normalization Takeaway](https://github.com/tuul-ai/robotbuilder/blob/main/timestep_norm.md)\n\n\n## Install\n\n```shell\n$ pip install TRI-LBM\n```\n\n## Usage\n\n```python\nimport torch\nfrom TRI_LBM.lbm import LBM\n\nlbm = LBM(\n action_dim = 20,\n dim_pose = 10\n)\n\ncommands = ['pick up the apple and place in the blue tray']\nimages = torch.randn(1, 3, 3, 224, 224)\nactions = torch.randn(1, 16, 20)\npose = torch.randn(1, 10)\n\nloss = lbm(\n text = commands,\n images = images,\n pose = pose,\n actions = actions,\n)\n\nloss.backward()\n\n# after much training\n\nsampled_actions = lbm.sample(\n text = commands,\n images = images,\n pose = pose,\n) # (1, 16, 20)\n```\n\n## Citations\n\n```bibtex\n@misc{trilbmteam2025carefulexaminationlargebehavior,\n title = {A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation}, \n author = {TRI LBM Team and Jose Barreiros and Andrew Beaulieu and Aditya Bhat and Rick Cory and Eric Cousineau and Hongkai Dai and Ching-Hsin Fang and Kunimatsu Hashimoto and Muhammad Zubair Irshad and Masha Itkina and Naveen Kuppuswamy and Kuan-Hui Lee and Katherine Liu and Dale McConachie and Ian McMahon and Haruki Nishimura and Calder Phillips-Grafflin and Charles Richter and Paarth Shah and Krishnan Srinivasan and Blake Wulfe and Chen Xu and Mengchao Zhang and Alex Alspach and Maya Angeles and Kushal Arora and Vitor Campagnolo Guizilini and Alejandro Castro and Dian Chen and Ting-Sheng Chu and Sam Creasey and Sean Curtis and Richard Denitto and Emma Dixon and Eric Dusel and Matthew Ferreira and Aimee Goncalves and Grant Gould and Damrong Guoy and Swati Gupta and Xuchen Han and Kyle Hatch and Brendan Hathaway and Allison Henry and Hillel Hochsztein and Phoebe Horgan and Shun Iwase and Donovon Jackson and Siddharth Karamcheti and Sedrick Keh and Joseph Masterjohn and Jean Mercat and Patrick Miller and Paul Mitiguy and Tony Nguyen and Jeremy Nimmer and Yuki Noguchi and Reko Ong and Aykut Onol and Owen Pfannenstiehl and Richard Poyner and Leticia Priebe Mendes Rocha and Gordon Richardson and Christopher Rodriguez and Derick Seale and Michael Sherman and Mariah Smith-Jones and David Tago and Pavel Tokmakov and Matthew Tran and Basile Van Hoorick and Igor Vasiljevic and Sergey Zakharov and Mark Zolotas and Rares Ambrus and Kerri Fetzer-Borelli and Benjamin Burchfiel and Hadas Kress-Gazit and Siyuan Feng and Stacie Ford and Russ Tedrake},\n year = {2025},\n eprint = {2507.05331},\n archivePrefix = {arXiv},\n primaryClass = {cs.RO},\n url = {https://arxiv.org/abs/2507.05331}, \n}\n```\n\n```bibtex\n@inproceedings{Wagenmaker2025SteeringYD,\n title = {Steering Your Diffusion Policy with Latent Space Reinforcement Learning},\n author = {Andrew Wagenmaker and Mitsuhiko Nakamoto and Yunchu Zhang and Seohong Park and Waleed Yagoub and Anusha Nagabandi and Abhishek Gupta and Sergey Levine},\n year = {2025},\n url = {https://api.semanticscholar.org/CorpusID:279464702}\n}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Large Behavioral Model from Toyota Research",
"version": "0.0.6",
"project_urls": {
"Homepage": "https://pypi.org/project/TRI-LBM/",
"Repository": "https://github.com/lucidrains/TRI-LBM"
},
"split_keywords": [
"artificial intelligence",
" deep learning",
" transformers",
" attention mechanism",
" diffusion",
" robotic dexterity"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "2e384526838fe9fe8519a2ac017242c88c06e181219e053c9f177ee1ad26f9a8",
"md5": "ccc1630414daa5bab5860c0e9f96eaf7",
"sha256": "1f306e9547a72b57732fd1d07d01ed501ed137d73d11267aa1f7b19caa9c3fc4"
},
"downloads": -1,
"filename": "tri_lbm-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ccc1630414daa5bab5860c0e9f96eaf7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 6745,
"upload_time": "2025-07-27T15:41:41",
"upload_time_iso_8601": "2025-07-27T15:41:41.415360Z",
"url": "https://files.pythonhosted.org/packages/2e/38/4526838fe9fe8519a2ac017242c88c06e181219e053c9f177ee1ad26f9a8/tri_lbm-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "3c45373d22d13bcbdf447a3a9cfb01838d3f997b224f0ca94ef9b87841abf457",
"md5": "3f504063644bd1ed8cbc7d75c81d2e6c",
"sha256": "82040fb390c9f24cd914795a9fcdeeb731419aa9620b71122f164138753d71ec"
},
"downloads": -1,
"filename": "tri_lbm-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "3f504063644bd1ed8cbc7d75c81d2e6c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 6875,
"upload_time": "2025-07-27T15:41:42",
"upload_time_iso_8601": "2025-07-27T15:41:42.642334Z",
"url": "https://files.pythonhosted.org/packages/3c/45/373d22d13bcbdf447a3a9cfb01838d3f997b224f0ca94ef9b87841abf457/tri_lbm-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-27 15:41:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lucidrains",
"github_project": "TRI-LBM",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tri-lbm"
}
JSON
