[](https://discord.gg/qUtxnK2NMf)
# AutoRT
Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents". This repo will implement the multi agent system that transforms a scene into a list of ranked and priortized tasks for an robotic action model to execute. This is an very effective setup that I personally believe is the future for swarming robotic foundation models!
This project will be implemented using Swarms, for the various llms and use the official RT-1 as the robotic action model.
[PAPER LINK](https://auto-rt.github.io/static/pdf/AutoRT.pdf)
## Install
`$ pip3 install autort-swarms`
## Usage
### AutoRTAgent
A single AutoRT agent that: analyzes a scene using visual COT -> generate tasks -> filter tasks -> execute it with a robotic transformer.
```python
# Import necessary modules
import os
from autort import AutoRTSwarm, AutoRTAgent
# Set the OpenAI API key
openai_api_key = os.getenv("OPENAI_API_KEY")
# Define a list of AutoRTAgent instances
agents = [
AutoRTAgent(openai_api_key, max_tokens=1000),
AutoRTAgent(openai_api_key, max_tokens=1000),
]
# Create an instance of AutoRTSwarm with the agents and datastore
autort_swarm = AutoRTSwarm(agents)
# Run the AutoRTSwarm with the given inputs
autort_swarm.run(
"There is a bottle on the table.",
"https://i.imgur.com/2qY9f8U.png",
)
```
### AutoRTSwarm
A team of AutoRT agents where you can plug in and play any number of `AutoRTAgents` with customization. First, the task will be routed to each agent and then all of their outputs will be saved.
```python
# Import necessary modules
import os
from autort import AutoRTSwarm, AutoRTAgent
# Set the OpenAI API key
openai_api_key = os.getenv("OPENAI_API_KEY")
# Define a list of AutoRTAgent instances
agents = [
AutoRTAgent(openai_api_key, max_tokens=1000),
AutoRTAgent(openai_api_key, max_tokens=1000),
]
# Create an instance of AutoRTSwarm with the agents and datastore
autort_swarm = AutoRTSwarm(agents)
# Run the AutoRTSwarm with the given inputs
autort_swarm.run(
"There is a bottle on the table.",
"https://i.imgur.com/2qY9f8U.png",
)
```
## Citation
```bibtex
@inproceedings{
anonymous2023autort,
title={Auto{RT}: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents},
author={Anonymous},
booktitle={Submitted to The Twelfth International Conference on Learning Representations},
year={2023},
url={https://openreview.net/forum?id=xVlcbh0poD},
note={under review}
}
```
# License
MIT
# Todo
- [ ] Implement a run method into `AutoRTSwarm` that runs all the agents with APIs.
- [ ] Make it able to send commands to a certain agent using the swarm network.
- [ ] Send a task to all agents in the swarm network
- [ ]
Raw data
{
"_id": null,
"home_page": "https://github.com/kyegomez/AutoRT",
"name": "autort-swarms",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6,<4.0",
"maintainer_email": "",
"keywords": "artificial intelligence,deep learning,optimizers,Prompt Engineering",
"author": "Kye Gomez",
"author_email": "kye@apac.ai",
"download_url": "https://files.pythonhosted.org/packages/6f/e6/dbe95e3e424a024f94a4e68fe1d11b036075d0ea9fa182ebd207fd9f9179/autort_swarms-0.0.6.tar.gz",
"platform": null,
"description": "[](https://discord.gg/qUtxnK2NMf)\n\n# AutoRT\n\nImplementation of AutoRT: \"AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents\". This repo will implement the multi agent system that transforms a scene into a list of ranked and priortized tasks for an robotic action model to execute. This is an very effective setup that I personally believe is the future for swarming robotic foundation models!\n\nThis project will be implemented using Swarms, for the various llms and use the official RT-1 as the robotic action model.\n\n[PAPER LINK](https://auto-rt.github.io/static/pdf/AutoRT.pdf)\n\n## Install\n`$ pip3 install autort-swarms`\n\n\n## Usage\n\n\n### AutoRTAgent\nA single AutoRT agent that: analyzes a scene using visual COT -> generate tasks -> filter tasks -> execute it with a robotic transformer.\n```python\n# Import necessary modules\nimport os\nfrom autort import AutoRTSwarm, AutoRTAgent\n\n# Set the OpenAI API key\nopenai_api_key = os.getenv(\"OPENAI_API_KEY\")\n\n# Define a list of AutoRTAgent instances\nagents = [\n AutoRTAgent(openai_api_key, max_tokens=1000),\n AutoRTAgent(openai_api_key, max_tokens=1000),\n]\n\n# Create an instance of AutoRTSwarm with the agents and datastore\nautort_swarm = AutoRTSwarm(agents)\n\n# Run the AutoRTSwarm with the given inputs\nautort_swarm.run(\n \"There is a bottle on the table.\",\n \"https://i.imgur.com/2qY9f8U.png\",\n)\n```\n\n\n### AutoRTSwarm\nA team of AutoRT agents where you can plug in and play any number of `AutoRTAgents` with customization. First, the task will be routed to each agent and then all of their outputs will be saved.\n```python\n# Import necessary modules\nimport os\nfrom autort import AutoRTSwarm, AutoRTAgent\n\n# Set the OpenAI API key\nopenai_api_key = os.getenv(\"OPENAI_API_KEY\")\n\n# Define a list of AutoRTAgent instances\nagents = [\n AutoRTAgent(openai_api_key, max_tokens=1000),\n AutoRTAgent(openai_api_key, max_tokens=1000),\n]\n\n# Create an instance of AutoRTSwarm with the agents and datastore\nautort_swarm = AutoRTSwarm(agents)\n\n# Run the AutoRTSwarm with the given inputs\nautort_swarm.run(\n \"There is a bottle on the table.\",\n \"https://i.imgur.com/2qY9f8U.png\",\n)\n```\n\n## Citation\n```bibtex\n@inproceedings{\n anonymous2023autort,\n title={Auto{RT}: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents},\n author={Anonymous},\n booktitle={Submitted to The Twelfth International Conference on Learning Representations},\n year={2023},\n url={https://openreview.net/forum?id=xVlcbh0poD},\n note={under review}\n}\n\n```\n\n\n# License\nMIT\n\n\n\n# Todo\n- [ ] Implement a run method into `AutoRTSwarm` that runs all the agents with APIs.\n- [ ] Make it able to send commands to a certain agent using the swarm network.\n- [ ] Send a task to all agents in the swarm network\n- [ ] ",
"bugtrack_url": null,
"license": "MIT",
"summary": "AutoRT - Pytorch",
"version": "0.0.6",
"project_urls": {
"Documentation": "https://github.com/kyegomez/AutoRT",
"Homepage": "https://github.com/kyegomez/AutoRT",
"Repository": "https://github.com/kyegomez/AutoRT"
},
"split_keywords": [
"artificial intelligence",
"deep learning",
"optimizers",
"prompt engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "858a904d3b6ad1789b845d4b9e5b5fc4fbff79ce9156239c7b5dc0792091edae",
"md5": "06bc0b75ca7a7459504effee2f52edca",
"sha256": "73cc54b30201ab228008da1befa048b3b15d3b3f3c432374d7cf076c293e94c0"
},
"downloads": -1,
"filename": "autort_swarms-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "06bc0b75ca7a7459504effee2f52edca",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6,<4.0",
"size": 9323,
"upload_time": "2024-02-14T18:48:01",
"upload_time_iso_8601": "2024-02-14T18:48:01.689797Z",
"url": "https://files.pythonhosted.org/packages/85/8a/904d3b6ad1789b845d4b9e5b5fc4fbff79ce9156239c7b5dc0792091edae/autort_swarms-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6fe6dbe95e3e424a024f94a4e68fe1d11b036075d0ea9fa182ebd207fd9f9179",
"md5": "1238b668d040ba9b85d3b1b051e82352",
"sha256": "d35f1fbfe83d63a6bea378448e29d8488ee8fde15a72ca607cd737564767588a"
},
"downloads": -1,
"filename": "autort_swarms-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "1238b668d040ba9b85d3b1b051e82352",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6,<4.0",
"size": 9294,
"upload_time": "2024-02-14T18:48:03",
"upload_time_iso_8601": "2024-02-14T18:48:03.075463Z",
"url": "https://files.pythonhosted.org/packages/6f/e6/dbe95e3e424a024f94a4e68fe1d11b036075d0ea9fa182ebd207fd9f9179/autort_swarms-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-14 18:48:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kyegomez",
"github_project": "AutoRT",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "autort-swarms"
}
JSON
