<h4 align="center">
<img src="docs/_static/logo.png" alt="RA-Agent logo" style="width:70%; ">
<a href="https://rdagent.azurewebsites.net" target="_blank">🖥️ Live Demo</a> | <a href="https://rdagent.azurewebsites.net/factor_loop" target="_blank">🎥 Demo Video</a> <a href="https://www.youtube.com/watch?v=JJ4JYO3HscM&list=PLALmKB0_N3_i52fhUmPQiL4jsO354uopR" target="_blank">▶️YouTube</a> | <a href="https://rdagent.readthedocs.io/en/latest/index.html" target="_blank">📖 Documentation</a> | <a href="#-paperwork-list"> 📃 Papers </a>
</h3>
[](https://github.com/microsoft/RD-Agent/actions/workflows/ci.yml)
[](https://github.com/microsoft/RD-Agent/actions/workflows/github-code-scanning/codeql)
[](https://github.com/microsoft/RD-Agent/actions/workflows/dependabot/dependabot-updates)
[](https://github.com/microsoft/RD-Agent/actions/workflows/pr.yml)
[](https://github.com/microsoft/RD-Agent/actions/workflows/release.yml)
[](https://pypi.org/project/rdagent/#files)
[](https://pypi.org/project/rdagent/)
[](https://pypi.org/project/rdagent/)
[](https://github.com/microsoft/RD-Agent/releases)
[](https://github.com/microsoft/RD-Agent/blob/main/LICENSE)
[](https://github.com/pre-commit/pre-commit)
[](http://mypy-lang.org/)
[](https://github.com/astral-sh/ruff)
[](https://discord.gg/ybQ97B6Jjy)
[](https://github.com/microsoft/RD-Agent/actions/workflows/readthedocs-preview.yml) <!-- this badge is too long, please place it in the last one to make it pretty -->
# 📰 News
| 🗞️ News | 📝 Description |
| -- | ------ |
| Official WeChat group release | We created a WeChat group, welcome to join! (🗪[QR Code](docs/WeChat_QR_code.jpg)) |
| Official Discord release | We launch our first chatting channel in Discord (🗪[](https://discord.gg/ybQ97B6Jjy)) |
| First release | **RDAgent** is released on GitHub |
# 🌟 Introduction
<div align="center">
<img src="docs/_static/scen.png" alt="Our focused scenario" style="width:80%; ">
</div>
RDAgent aims to automate the most critical and valuable aspects of the industrial R&D process, and we begin with focusing on the data-driven scenarios to streamline the development of models and data.
Methodologically, we have identified a framework with two key components: 'R' for proposing new ideas and 'D' for implementing them.
We believe that the automatic evolution of R&D will lead to solutions of significant industrial value.
<!-- Tag Cloud -->
R&D is a very general scenario. The advent of RDAgent can be your
- 💰 **Automatic Quant Factory** ([🎥Demo Video](https://rdagent.azurewebsites.net/factor_loop)|[▶️YouTube](https://www.youtube.com/watch?v=X4DK2QZKaKY&t=6s))
- 🤖 **Data Mining Agent:** Iteratively proposing data & models ([🎥Demo Video 1](https://rdagent.azurewebsites.net/model_loop)|[▶️YouTube](https://www.youtube.com/watch?v=dm0dWL49Bc0&t=104s)) ([🎥Demo Video 2](https://rdagent.azurewebsites.net/dmm)|[▶️YouTube](https://www.youtube.com/watch?v=VIaSTZuoZg4)) and implementing them by gaining knowledge from data.
- 🦾 **Research Copilot:** Auto read research papers ([🎥Demo Video](https://rdagent.azurewebsites.net/report_model)|[▶️YouTube](https://www.youtube.com/watch?v=BiA2SfdKQ7o)) / financial reports ([🎥Demo Video](https://rdagent.azurewebsites.net/report_factor)|[▶️YouTube](https://www.youtube.com/watch?v=ECLTXVcSx-c)) and implement model structures or building datasets.
- ...
You can click the links above to view the demo. We're continuously adding more methods and scenarios to the project to enhance your R&D processes and boost productivity.
Additionally, you can take a closer look at the examples in our **[🖥️ Live Demo](https://rdagent.azurewebsites.net/)**.
<div align="center">
<a href="https://rdagent.azurewebsites.net/" target="_blank">
<img src="docs/_static/demo.png" alt="Watch the demo" width="80%">
</a>
</div>
# ⚡ Quick start
You can try above demos by running the following command:
### 🐳 Docker installation.
Users must ensure Docker is installed before attempting most scenarios. Please refer to the [official 🐳Docker page](https://docs.docker.com/engine/install/) for installation instructions.
### 🐍 Create a Conda Environment
- Create a new conda environment with Python (3.10 and 3.11 are well-tested in our CI):
```sh
conda create -n rdagent python=3.10
```
- Activate the environment:
```sh
conda activate rdagent
```
### 🛠️ Install the RDAgent
- You can directly install the RDAgent package from PyPI:
```sh
pip install rdagent
```
### ⚙️ Configuration
- You have to config your GPT model in the `.env`
```bash
cat << EOF > .env
OPENAI_API_KEY=<your_api_key>
# EMBEDDING_MODEL=text-embedding-3-small
CHAT_MODEL=gpt-4-turbo
EOF
```
### 🚀 Run the Application
The **[🖥️ Live Demo](https://rdagent.azurewebsites.net/)** is implemented by the following commands(each item represents one demo, you can select the one you prefer):
- Run the **Automated Quantitative Trading & Iterative Factors Evolution**: [Qlib](http://github.com/microsoft/qlib) self-loop factor proposal and implementation application
```sh
rdagent fin_factor
```
- Run the **Automated Quantitative Trading & Iterative Model Evolution**: [Qlib](http://github.com/microsoft/qlib) self-loop model proposal and implementation application
```sh
rdagent fin_model
```
- Run the **Automated Medical Prediction Model Evolution**: Medical self-loop model proposal and implementation application
>(1) Apply for an account at [PhysioNet](https://physionet.org/). <br /> (2) Request access to FIDDLE preprocessed data: [FIDDLE Dataset](https://physionet.org/content/mimic-eicu-fiddle-feature/1.0.0/). <br />
(3) Place your username and password in `.env`.
```bash
cat << EOF >> .env
DM_USERNAME=<your_username>
DM_PASSWORD=<your_password>
EOF
```
```sh
rdagent med_model
```
- Run the **Automated Quantitative Trading & Factors Extraction from Financial Reports**: Run the [Qlib](http://github.com/microsoft/qlib) factor extraction and implementation application based on financial reports
```sh
# 1. Generally, you can run this scenario using the following command:
rdagent fin_factor_report --report_folder=<Your financial reports folder path>
# 2. Specifically, you need to prepare some financial reports first. You can follow this concrete example:
wget https://github.com/SunsetWolf/rdagent_resource/releases/download/reports/all_reports.zip
unzip all_reports.zip -d git_ignore_folder/reports
rdagent fin_factor_report --report_folder=git_ignore_folder/reports
```
- Run the **Automated Model Research & Development Copilot**: model extraction and implementation application
```sh
# 1. Generally, you can run your own papers/reports with the following command:
rdagent general_model <Your paper URL>
# 2. Specifically, you can do it like this. For more details and additional paper examples, use `rdagent general_model -h`:
rdagent general_model "https://arxiv.org/pdf/2210.09789"
```
### 🖥️ Monitor the Application Results
- You can serve our demo app to monitor the RD loop by running the following command:
```sh
rdagent ui --port 80 --log_dir <your log folder like "log/">
```
# 🏭 Scenarios
We have applied RD-Agent to multiple valuable data-driven industrial scenarios.
## 🎯 Goal: Agent for Data-driven R&D
In this project, we are aiming to build an Agent to automate Data-Driven R\&D that can
+ 📄 Read real-world material (reports, papers, etc.) and **extract** key formulas, descriptions of interested **features** and **models**, which are the key components of data-driven R&D .
+ 🛠️ **Implement** the extracted formulas (e.g., features, factors, and models) in runnable codes.
+ Due to the limited ability of LLM in implementing at once, build an evolving process for the agent to improve performance by learning from feedback and knowledge.
+ 💡 Propose **new ideas** based on current knowledge and observations.
<!--  -->
## 📈 Scenarios/Demos
In the two key areas of data-driven scenarios, model implementation and data building, our system aims to serve two main roles: 🦾Copilot and 🤖Agent.
- The 🦾Copilot follows human instructions to automate repetitive tasks.
- The 🤖Agent, being more autonomous, actively proposes ideas for better results in the future.
The supported scenarios are listed below:
| Scenario/Target | Model Implementation | Data Building |
| -- | -- | -- |
| **💹 Finance** | 🤖 [Iteratively Proposing Ideas & Evolving](https://rdagent.azurewebsites.net/model_loop)[▶️YouTube](https://www.youtube.com/watch?v=dm0dWL49Bc0&t=104s) | 🤖 [Iteratively Proposing Ideas & Evolving](https://rdagent.azurewebsites.net/factor_loop) [▶️YouTube](https://www.youtube.com/watch?v=X4DK2QZKaKY&t=6s) <br/> 🦾 [Auto reports reading & implementation](https://rdagent.azurewebsites.net/report_factor)[▶️YouTube](https://www.youtube.com/watch?v=ECLTXVcSx-c) |
| **🩺 Medical** | 🤖 [Iteratively Proposing Ideas & Evolving](https://rdagent.azurewebsites.net/dmm)[▶️YouTube](https://www.youtube.com/watch?v=VIaSTZuoZg4) | - |
| **🏭 General** | 🦾 [Auto paper reading & implementation](https://rdagent.azurewebsites.net/report_model)[▶️YouTube](https://www.youtube.com/watch?v=BiA2SfdKQ7o) | - |
Different scenarios vary in entrance and configuration. Please check the detailed setup tutorial in the scenarios documents.
Here is a gallery of [successful explorations](https://github.com/SunsetWolf/rdagent_resource/releases/download/demo_traces/demo_traces.zip) (5 traces showed in **[🖥️ Live Demo](https://rdagent.azurewebsites.net/)**). You can download and view the execution trace using the command below:
```bash
rdagent ui --port 80 --log_dir ./demo_traces
```
Please refer to **[📖readthedocs_scen](https://rdagent.readthedocs.io/en/latest/scens/catalog.html)** for more details of the scenarios.
# ⚙️ Framework
<div align="center">
<img src="docs/_static/Framework-RDAgent.png" alt="Framework-RDAgent" width="85%">
</div>
Automating the R&D process in data science is a highly valuable yet underexplored area in industry. We propose a framework to push the boundaries of this important research field.
The research questions within this framework can be divided into three main categories:
| Research Area | Paper/Work List |
|--------------------|-----------------|
| **Benchmark the R&D abilities** | [Benchmark](#benchmark) |
| **Idea proposal:** Explore new ideas or refine existing ones | [Research](#research) |
| **Ability to realize ideas:** Implement and execute ideas | [Development](#development) |
We believe that the key to delivering high-quality solutions lies in the ability to evolve R&D capabilities. Agents should learn like human experts, continuously improving their R&D skills.
More documents can be found in the **[📖 readthedocs](https://rdagent.readthedocs.io/)**.
# 📃 Paper/Work list
## 📊 Benchmark
- [Towards Data-Centric Automatic R&D](https://arxiv.org/abs/2404.11276)
```BibTeX
@misc{chen2024datacentric,
title={Towards Data-Centric Automatic R&D},
author={Haotian Chen and Xinjie Shen and Zeqi Ye and Wenjun Feng and Haoxue Wang and Xiao Yang and Xu Yang and Weiqing Liu and Jiang Bian},
year={2024},
eprint={2404.11276},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
```
## 🔍 Research
In a data mining expert's daily research and development process, they propose a hypothesis (e.g., a model structure like RNN can capture patterns in time-series data), design experiments (e.g., finance data contains time-series and we can verify the hypothesis in this scenario), implement the experiment as code (e.g., Pytorch model structure), and then execute the code to get feedback (e.g., metrics, loss curve, etc.). The experts learn from the feedback and improve in the next iteration.
Based on the principles above, we have established a basic method framework that continuously proposes hypotheses, verifies them, and gets feedback from the real-world practice. This is the first scientific research automation framework that supports linking with real-world verification.
For more detail, please refer to our **[🖥️ Live Demo page](https://rdagent.azurewebsites.net)**.
## 🛠️ Development
- [Collaborative Evolving Strategy for Automatic Data-Centric Development](https://arxiv.org/abs/2407.18690)
```BibTeX
@misc{yang2024collaborative,
title={Collaborative Evolving Strategy for Automatic Data-Centric Development},
author={Xu Yang and Haotian Chen and Wenjun Feng and Haoxue Wang and Zeqi Ye and Xinjie Shen and Xiao Yang and Shizhao Sun and Weiqing Liu and Jiang Bian},
year={2024},
eprint={2407.18690},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
```
# 🤝 Contributing
## 📝 Guidelines
This project welcomes contributions and suggestions.
Contributing to this project is straightforward and rewarding. Whether it's solving an issue, addressing a bug, enhancing documentation, or even correcting a typo, every contribution is valuable and helps improve RDAgent.
To get started, you can explore the issues list, or search for `TODO:` comments in the codebase by running the command `grep -r "TODO:"`.
<img src="https://img.shields.io/github/contributors-anon/microsoft/RD-Agent"/>
<a href="https://github.com/microsoft/RD-Agent/graphs/contributors">
<img src="https://contrib.rocks/image?repo=microsoft/RD-Agent&max=100&columns=15" />
</a>
Before we released RD-Agent as an open-source project on GitHub, it was an internal project within our group. Unfortunately, the internal commit history was not preserved when we removed some confidential code. As a result, some contributions from our group members, including Haotian Chen, Wenjun Feng, Haoxue Wang, Zeqi Ye, Xinjie Shen, and Jinhui Li, were not included in the public commits.
# ⚖️ Legal disclaimer
<p style="line-height: 1; font-style: italic;">The RD-agent is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. The RD-agent is aimed to facilitate research and development process in the financial industry and not ready-to-use for any financial investment or advice. Users shall independently assess and test the risks of the RD-agent in a specific use scenario, ensure the responsible use of AI technology, including but not limited to developing and integrating risk mitigation measures, and comply with all applicable laws and regulations in all applicable jurisdictions. The RD-agent does not provide financial opinions or reflect the opinions of Microsoft, nor is it designed to replace the role of qualified financial professionals in formulating, assessing, and approving finance products. The inputs and outputs of the RD-agent belong to the users and users shall assume all liability under any theory of liability, whether in contract, torts, regulatory, negligence, products liability, or otherwise, associated with use of the RD-agent and any inputs and outputs thereof.</p>
Raw data
{
"_id": null,
"home_page": null,
"name": "rdagent",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "Autonomous Agents, Large Language Models, Research and Development",
"author": null,
"author_email": "MSRA-MIIC <xuyang1@microsoft.com>",
"download_url": "https://files.pythonhosted.org/packages/ce/8c/34cce77d0e68577cb559de1757211f670f6fe8d84caa7096dc45bfd9e845/rdagent-0.3.0.tar.gz",
"platform": null,
"description": "<h4 align=\"center\">\n <img src=\"docs/_static/logo.png\" alt=\"RA-Agent logo\" style=\"width:70%; \">\n \n <a href=\"https://rdagent.azurewebsites.net\" target=\"_blank\">\ud83d\udda5\ufe0f Live Demo</a> | <a href=\"https://rdagent.azurewebsites.net/factor_loop\" target=\"_blank\">\ud83c\udfa5 Demo Video</a> <a href=\"https://www.youtube.com/watch?v=JJ4JYO3HscM&list=PLALmKB0_N3_i52fhUmPQiL4jsO354uopR\" target=\"_blank\">\u25b6\ufe0fYouTube</a> | <a href=\"https://rdagent.readthedocs.io/en/latest/index.html\" target=\"_blank\">\ud83d\udcd6 Documentation</a> | <a href=\"#-paperwork-list\"> \ud83d\udcc3 Papers </a>\n</h3>\n\n\n[](https://github.com/microsoft/RD-Agent/actions/workflows/ci.yml)\n[](https://github.com/microsoft/RD-Agent/actions/workflows/github-code-scanning/codeql)\n[](https://github.com/microsoft/RD-Agent/actions/workflows/dependabot/dependabot-updates)\n[](https://github.com/microsoft/RD-Agent/actions/workflows/pr.yml)\n[](https://github.com/microsoft/RD-Agent/actions/workflows/release.yml)\n[](https://pypi.org/project/rdagent/#files)\n[](https://pypi.org/project/rdagent/)\n[](https://pypi.org/project/rdagent/)\n[](https://github.com/microsoft/RD-Agent/releases)\n[](https://github.com/microsoft/RD-Agent/blob/main/LICENSE)\n[](https://github.com/pre-commit/pre-commit)\n[](http://mypy-lang.org/)\n[](https://github.com/astral-sh/ruff)\n[](https://discord.gg/ybQ97B6Jjy)\n[](https://github.com/microsoft/RD-Agent/actions/workflows/readthedocs-preview.yml) <!-- this badge is too long, please place it in the last one to make it pretty --> \n\n# \ud83d\udcf0 News\n| \ud83d\uddde\ufe0f News | \ud83d\udcdd Description |\n| -- | ------ |\n| Official WeChat group release | We created a WeChat group, welcome to join! (\ud83d\uddea[QR Code](docs/WeChat_QR_code.jpg)) |\n| Official Discord release | We launch our first chatting channel in Discord (\ud83d\uddea[](https://discord.gg/ybQ97B6Jjy)) |\n| First release | **RDAgent** is released on GitHub |\n\n\n# \ud83c\udf1f Introduction\n<div align=\"center\">\n <img src=\"docs/_static/scen.png\" alt=\"Our focused scenario\" style=\"width:80%; \">\n</div>\n\nRDAgent aims to automate the most critical and valuable aspects of the industrial R&D process, and we begin with focusing on the data-driven scenarios to streamline the development of models and data. \nMethodologically, we have identified a framework with two key components: 'R' for proposing new ideas and 'D' for implementing them.\nWe believe that the automatic evolution of R&D will lead to solutions of significant industrial value.\n\n\n<!-- Tag Cloud -->\nR&D is a very general scenario. The advent of RDAgent can be your\n- \ud83d\udcb0 **Automatic Quant Factory** ([\ud83c\udfa5Demo Video](https://rdagent.azurewebsites.net/factor_loop)|[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=X4DK2QZKaKY&t=6s))\n- \ud83e\udd16 **Data Mining Agent:** Iteratively proposing data & models ([\ud83c\udfa5Demo Video 1](https://rdagent.azurewebsites.net/model_loop)|[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=dm0dWL49Bc0&t=104s)) ([\ud83c\udfa5Demo Video 2](https://rdagent.azurewebsites.net/dmm)|[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=VIaSTZuoZg4)) and implementing them by gaining knowledge from data.\n- \ud83e\uddbe **Research Copilot:** Auto read research papers ([\ud83c\udfa5Demo Video](https://rdagent.azurewebsites.net/report_model)|[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=BiA2SfdKQ7o)) / financial reports ([\ud83c\udfa5Demo Video](https://rdagent.azurewebsites.net/report_factor)|[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=ECLTXVcSx-c)) and implement model structures or building datasets.\n- ...\n\nYou can click the links above to view the demo. We're continuously adding more methods and scenarios to the project to enhance your R&D processes and boost productivity. \n\nAdditionally, you can take a closer look at the examples in our **[\ud83d\udda5\ufe0f Live Demo](https://rdagent.azurewebsites.net/)**.\n\n<div align=\"center\">\n <a href=\"https://rdagent.azurewebsites.net/\" target=\"_blank\">\n <img src=\"docs/_static/demo.png\" alt=\"Watch the demo\" width=\"80%\">\n </a>\n</div>\n\n\n# \u26a1 Quick start\n\nYou can try above demos by running the following command:\n\n### \ud83d\udc33 Docker installation.\nUsers must ensure Docker is installed before attempting most scenarios. Please refer to the [official \ud83d\udc33Docker page](https://docs.docker.com/engine/install/) for installation instructions.\n\n### \ud83d\udc0d Create a Conda Environment\n- Create a new conda environment with Python (3.10 and 3.11 are well-tested in our CI):\n ```sh\n conda create -n rdagent python=3.10\n ```\n- Activate the environment:\n ```sh\n conda activate rdagent\n ```\n\n### \ud83d\udee0\ufe0f Install the RDAgent\n- You can directly install the RDAgent package from PyPI:\n ```sh\n pip install rdagent\n ```\n\n### \u2699\ufe0f Configuration\n- You have to config your GPT model in the `.env`\n ```bash\n cat << EOF > .env\n OPENAI_API_KEY=<your_api_key>\n # EMBEDDING_MODEL=text-embedding-3-small\n CHAT_MODEL=gpt-4-turbo\n EOF\n ```\n\n### \ud83d\ude80 Run the Application\n\nThe **[\ud83d\udda5\ufe0f Live Demo](https://rdagent.azurewebsites.net/)** is implemented by the following commands(each item represents one demo, you can select the one you prefer):\n\n- Run the **Automated Quantitative Trading & Iterative Factors Evolution**: [Qlib](http://github.com/microsoft/qlib) self-loop factor proposal and implementation application\n ```sh\n rdagent fin_factor\n ```\n\n- Run the **Automated Quantitative Trading & Iterative Model Evolution**: [Qlib](http://github.com/microsoft/qlib) self-loop model proposal and implementation application\n ```sh\n rdagent fin_model\n ```\n\n- Run the **Automated Medical Prediction Model Evolution**: Medical self-loop model proposal and implementation application\n >(1) Apply for an account at [PhysioNet](https://physionet.org/). <br /> (2) Request access to FIDDLE preprocessed data: [FIDDLE Dataset](https://physionet.org/content/mimic-eicu-fiddle-feature/1.0.0/). <br />\n (3) Place your username and password in `.env`.\n ```bash\n cat << EOF >> .env\n DM_USERNAME=<your_username>\n DM_PASSWORD=<your_password>\n EOF\n ```\n ```sh\n rdagent med_model\n ```\n\n- Run the **Automated Quantitative Trading & Factors Extraction from Financial Reports**: Run the [Qlib](http://github.com/microsoft/qlib) factor extraction and implementation application based on financial reports\n ```sh\n # 1. Generally, you can run this scenario using the following command:\n rdagent fin_factor_report --report_folder=<Your financial reports folder path>\n\n # 2. Specifically, you need to prepare some financial reports first. You can follow this concrete example:\n wget https://github.com/SunsetWolf/rdagent_resource/releases/download/reports/all_reports.zip\n unzip all_reports.zip -d git_ignore_folder/reports\n rdagent fin_factor_report --report_folder=git_ignore_folder/reports\n ```\n\n- Run the **Automated Model Research & Development Copilot**: model extraction and implementation application\n ```sh\n # 1. Generally, you can run your own papers/reports with the following command:\n rdagent general_model <Your paper URL>\n\n # 2. Specifically, you can do it like this. For more details and additional paper examples, use `rdagent general_model -h`:\n rdagent general_model \"https://arxiv.org/pdf/2210.09789\"\n ```\n\n### \ud83d\udda5\ufe0f Monitor the Application Results\n- You can serve our demo app to monitor the RD loop by running the following command:\n ```sh\n rdagent ui --port 80 --log_dir <your log folder like \"log/\">\n ```\n\n# \ud83c\udfed Scenarios\n\nWe have applied RD-Agent to multiple valuable data-driven industrial scenarios.\n\n\n## \ud83c\udfaf Goal: Agent for Data-driven R&D\n\nIn this project, we are aiming to build an Agent to automate Data-Driven R\\&D that can\n+ \ud83d\udcc4 Read real-world material (reports, papers, etc.) and **extract** key formulas, descriptions of interested **features** and **models**, which are the key components of data-driven R&D .\n+ \ud83d\udee0\ufe0f **Implement** the extracted formulas (e.g., features, factors, and models) in runnable codes.\n + Due to the limited ability of LLM in implementing at once, build an evolving process for the agent to improve performance by learning from feedback and knowledge.\n+ \ud83d\udca1 Propose **new ideas** based on current knowledge and observations.\n\n<!--  -->\n\n## \ud83d\udcc8 Scenarios/Demos\n\nIn the two key areas of data-driven scenarios, model implementation and data building, our system aims to serve two main roles: \ud83e\uddbeCopilot and \ud83e\udd16Agent. \n- The \ud83e\uddbeCopilot follows human instructions to automate repetitive tasks. \n- The \ud83e\udd16Agent, being more autonomous, actively proposes ideas for better results in the future.\n\nThe supported scenarios are listed below:\n\n| Scenario/Target | Model Implementation | Data Building |\n| -- | -- | -- |\n| **\ud83d\udcb9 Finance** | \ud83e\udd16 [Iteratively Proposing Ideas & Evolving](https://rdagent.azurewebsites.net/model_loop)[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=dm0dWL49Bc0&t=104s) | \ud83e\udd16 [Iteratively Proposing Ideas & Evolving](https://rdagent.azurewebsites.net/factor_loop) [\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=X4DK2QZKaKY&t=6s) <br/> \ud83e\uddbe [Auto reports reading & implementation](https://rdagent.azurewebsites.net/report_factor)[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=ECLTXVcSx-c) |\n| **\ud83e\ude7a Medical** | \ud83e\udd16 [Iteratively Proposing Ideas & Evolving](https://rdagent.azurewebsites.net/dmm)[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=VIaSTZuoZg4) | - |\n| **\ud83c\udfed General** | \ud83e\uddbe [Auto paper reading & implementation](https://rdagent.azurewebsites.net/report_model)[\u25b6\ufe0fYouTube](https://www.youtube.com/watch?v=BiA2SfdKQ7o) | - |\n\nDifferent scenarios vary in entrance and configuration. Please check the detailed setup tutorial in the scenarios documents.\n\nHere is a gallery of [successful explorations](https://github.com/SunsetWolf/rdagent_resource/releases/download/demo_traces/demo_traces.zip) (5 traces showed in **[\ud83d\udda5\ufe0f Live Demo](https://rdagent.azurewebsites.net/)**). You can download and view the execution trace using the command below:\n\n```bash\nrdagent ui --port 80 --log_dir ./demo_traces\n```\n\nPlease refer to **[\ud83d\udcd6readthedocs_scen](https://rdagent.readthedocs.io/en/latest/scens/catalog.html)** for more details of the scenarios.\n\n# \u2699\ufe0f Framework\n\n<div align=\"center\">\n <img src=\"docs/_static/Framework-RDAgent.png\" alt=\"Framework-RDAgent\" width=\"85%\">\n</div>\n\n\nAutomating the R&D process in data science is a highly valuable yet underexplored area in industry. We propose a framework to push the boundaries of this important research field.\n\nThe research questions within this framework can be divided into three main categories:\n| Research Area | Paper/Work List |\n|--------------------|-----------------|\n| **Benchmark the R&D abilities** | [Benchmark](#benchmark) |\n| **Idea proposal:** Explore new ideas or refine existing ones | [Research](#research) |\n| **Ability to realize ideas:** Implement and execute ideas | [Development](#development) |\n\nWe believe that the key to delivering high-quality solutions lies in the ability to evolve R&D capabilities. Agents should learn like human experts, continuously improving their R&D skills.\n\nMore documents can be found in the **[\ud83d\udcd6 readthedocs](https://rdagent.readthedocs.io/)**.\n\n# \ud83d\udcc3 Paper/Work list\n\n## \ud83d\udcca Benchmark\n- [Towards Data-Centric Automatic R&D](https://arxiv.org/abs/2404.11276)\n```BibTeX\n@misc{chen2024datacentric,\n title={Towards Data-Centric Automatic R&D},\n author={Haotian Chen and Xinjie Shen and Zeqi Ye and Wenjun Feng and Haoxue Wang and Xiao Yang and Xu Yang and Weiqing Liu and Jiang Bian},\n year={2024},\n eprint={2404.11276},\n archivePrefix={arXiv},\n primaryClass={cs.AI}\n}\n```\n\n\n## \ud83d\udd0d Research\n\nIn a data mining expert's daily research and development process, they propose a hypothesis (e.g., a model structure like RNN can capture patterns in time-series data), design experiments (e.g., finance data contains time-series and we can verify the hypothesis in this scenario), implement the experiment as code (e.g., Pytorch model structure), and then execute the code to get feedback (e.g., metrics, loss curve, etc.). The experts learn from the feedback and improve in the next iteration.\n\nBased on the principles above, we have established a basic method framework that continuously proposes hypotheses, verifies them, and gets feedback from the real-world practice. This is the first scientific research automation framework that supports linking with real-world verification.\n\nFor more detail, please refer to our **[\ud83d\udda5\ufe0f Live Demo page](https://rdagent.azurewebsites.net)**.\n\n## \ud83d\udee0\ufe0f Development\n\n- [Collaborative Evolving Strategy for Automatic Data-Centric Development](https://arxiv.org/abs/2407.18690)\n```BibTeX\n@misc{yang2024collaborative,\n title={Collaborative Evolving Strategy for Automatic Data-Centric Development},\n author={Xu Yang and Haotian Chen and Wenjun Feng and Haoxue Wang and Zeqi Ye and Xinjie Shen and Xiao Yang and Shizhao Sun and Weiqing Liu and Jiang Bian},\n year={2024},\n eprint={2407.18690},\n archivePrefix={arXiv},\n primaryClass={cs.AI}\n}\n```\n\n\n\n# \ud83e\udd1d Contributing\n\n## \ud83d\udcdd Guidelines\nThis project welcomes contributions and suggestions.\nContributing to this project is straightforward and rewarding. Whether it's solving an issue, addressing a bug, enhancing documentation, or even correcting a typo, every contribution is valuable and helps improve RDAgent.\n\nTo get started, you can explore the issues list, or search for `TODO:` comments in the codebase by running the command `grep -r \"TODO:\"`.\n\n<img src=\"https://img.shields.io/github/contributors-anon/microsoft/RD-Agent\"/>\n\n<a href=\"https://github.com/microsoft/RD-Agent/graphs/contributors\">\n <img src=\"https://contrib.rocks/image?repo=microsoft/RD-Agent&max=100&columns=15\" />\n</a>\n\nBefore we released RD-Agent as an open-source project on GitHub, it was an internal project within our group. Unfortunately, the internal commit history was not preserved when we removed some confidential code. As a result, some contributions from our group members, including Haotian Chen, Wenjun Feng, Haoxue Wang, Zeqi Ye, Xinjie Shen, and Jinhui Li, were not included in the public commits.\n\n# \u2696\ufe0f Legal disclaimer\n<p style=\"line-height: 1; font-style: italic;\">The RD-agent is provided \u201cas is\u201d, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. The RD-agent is aimed to facilitate research and development process in the financial industry and not ready-to-use for any financial investment or advice. Users shall independently assess and test the risks of the RD-agent in a specific use scenario, ensure the responsible use of AI technology, including but not limited to developing and integrating risk mitigation measures, and comply with all applicable laws and regulations in all applicable jurisdictions. The RD-agent does not provide financial opinions or reflect the opinions of Microsoft, nor is it designed to replace the role of qualified financial professionals in formulating, assessing, and approving finance products. The inputs and outputs of the RD-agent belong to the users and users shall assume all liability under any theory of liability, whether in contract, torts, regulatory, negligence, products liability, or otherwise, associated with use of the RD-agent and any inputs and outputs thereof.</p>\n",
"bugtrack_url": null,
"license": null,
"summary": "Research & Development Agent",
"version": "0.3.0",
"project_urls": {
"homepage": "https://github.com/microsoft/RD-Agent/",
"issue": "https://github.com/microsoft/RD-Agent/issues"
},
"split_keywords": [
"autonomous agents",
" large language models",
" research and development"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "25c1d0595f0d62081685508a484d236fa0e394347ef50b9e91ec017209eb9b98",
"md5": "51f49426990403c5214fc1e97827f0de",
"sha256": "10adff7508d5be97c37faac0e0c23be3b142525bd34a2e893158f072286766db"
},
"downloads": -1,
"filename": "rdagent-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "51f49426990403c5214fc1e97827f0de",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 508537,
"upload_time": "2024-10-21T09:39:43",
"upload_time_iso_8601": "2024-10-21T09:39:43.758695Z",
"url": "https://files.pythonhosted.org/packages/25/c1/d0595f0d62081685508a484d236fa0e394347ef50b9e91ec017209eb9b98/rdagent-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ce8c34cce77d0e68577cb559de1757211f670f6fe8d84caa7096dc45bfd9e845",
"md5": "fce75ff926b77785ec6947278f843e83",
"sha256": "4841763442a6736c8fcb2c99d585cb916a60d1faa11de0c9fd42a75281946dda"
},
"downloads": -1,
"filename": "rdagent-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "fce75ff926b77785ec6947278f843e83",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 2436810,
"upload_time": "2024-10-21T09:39:45",
"upload_time_iso_8601": "2024-10-21T09:39:45.746723Z",
"url": "https://files.pythonhosted.org/packages/ce/8c/34cce77d0e68577cb559de1757211f670f6fe8d84caa7096dc45bfd9e845/rdagent-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-21 09:39:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "microsoft",
"github_project": "RD-Agent",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "rdagent"
}
JSON
