# Installation and Usage Guide for `behavior-eval`
## Installation
### Step 1: Create a Conda Virtual Environment for `behavior-eval`
```
conda create -n behavior-eval python=3.8 -y
conda activate behavior-eval
```
### Step 2: Install `behavior-eval`
You can install from pip:
```
pip install behavior-eval
```
You can also install from source and use editable mode if you want to modify the source files:
```
git clone https://github.com/embodied-agent-eval/behavior-eval.git
cd behavior-eval
pip install -e .
```
### Step 3: Install `iGibson`
There might be issues during the installation of `iGibson`.
To minimize and identify potential issues, we recommend:
1. Review the system requirements section of the [iGibson installation guide](https://stanfordvl.github.io/iGibson/installation.html).
2. **Install CMake Using Conda (do not use pip)**:
```
conda install cmake
```
3. **Install `iGibson`**:
We provided a script for automatically installing `iGibson`:
```
python -m behavior_eval.utils.install_igibson_utils
```
You can also do it on your own:
```
git clone https://github.com/embodied-agent-eval/iGibson.git --recursive
cd iGibson
pip install -e . # If you want to use editable mode
# or
pip install . # Recommended
```
We've successfully tested the installation on Linux servers, Windows 10+, and Mac OS X.
### Step 4: Download Assets for `iGibson`
```
python -m behavior_eval.utils.download_utils
```
## Usage
To run `behavior-eval`, use the following command:
```
python -m behavior_eval.main
```
(By default, this will generate the prompts for action sequencing.)
### Parameters:
- `module`: Specifies the module to use. Options are:
- `goal_interpretation`
- `action_sequence`
- `subgoal_decomposition`
- `transition_modeling`
- `func`: Specifies the function to execute. Options are:
- `evaluate_results`
- `generate_prompts`
- `worker_num`: Number of workers for multiprocessing.
- `llm_response_dir`: Directory containing LLM responses (HELM outputs).
- `result_dir`: Directory to store results.
### Example Usage:
1. To generate prompts using the `action_sequence` module:
```
python -m behavior_eval.main --module=action_sequence --func=generate_prompts
```
2. To evaluate results using the `action_sequence` module:
```
python -m behavior_eval.main --module=action_sequence --func=evaluate_results --llm_response_dir=<your_llm_response_dir>
```
Replace `<your_llm_response_dir>` with the path to your LLM response directory.
Raw data
{
"_id": null,
"home_page": "https://github.com/embodied-agent-eval/behavior-eval",
"name": "behavior-eval",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "stanford",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/98/45/b2129bacaa2da0d99ef19ba7da7e0939bfcf1f0d35eada3b2dbaea62ee50/behavior_eval-1.0.7.tar.gz",
"platform": null,
"description": "# Installation and Usage Guide for `behavior-eval`\r\n\r\n## Installation\r\n\r\n### Step 1: Create a Conda Virtual Environment for `behavior-eval`\r\n```\r\nconda create -n behavior-eval python=3.8 -y\r\nconda activate behavior-eval\r\n```\r\n\r\n### Step 2: Install `behavior-eval`\r\n\r\nYou can install from pip:\r\n```\r\npip install behavior-eval\r\n```\r\n\r\nYou can also install from source and use editable mode if you want to modify the source files:\r\n```\r\ngit clone https://github.com/embodied-agent-eval/behavior-eval.git\r\ncd behavior-eval\r\npip install -e .\r\n```\r\n\r\n### Step 3: Install `iGibson`\r\n\r\nThere might be issues during the installation of `iGibson`. \r\n\r\nTo minimize and identify potential issues, we recommend:\r\n\r\n1. Review the system requirements section of the [iGibson installation guide](https://stanfordvl.github.io/iGibson/installation.html).\r\n\r\n2. **Install CMake Using Conda (do not use pip)**: \r\n ```\r\n conda install cmake\r\n ```\r\n\r\n3. **Install `iGibson`**: \r\n We provided a script for automatically installing `iGibson`:\r\n ```\r\n python -m behavior_eval.utils.install_igibson_utils\r\n ```\r\n \r\n You can also do it on your own:\r\n ```\r\n git clone https://github.com/embodied-agent-eval/iGibson.git --recursive\r\n cd iGibson\r\n pip install -e . # If you want to use editable mode\r\n # or\r\n pip install . # Recommended\r\n ```\r\n\r\nWe've successfully tested the installation on Linux servers, Windows 10+, and Mac OS X.\r\n\r\n### Step 4: Download Assets for `iGibson`\r\n```\r\npython -m behavior_eval.utils.download_utils\r\n```\r\n\r\n## Usage\r\n\r\nTo run `behavior-eval`, use the following command:\r\n```\r\npython -m behavior_eval.main\r\n```\r\n\r\n(By default, this will generate the prompts for action sequencing.)\r\n\r\n### Parameters:\r\n\r\n- `module`: Specifies the module to use. Options are:\r\n - `goal_interpretation`\r\n - `action_sequence`\r\n - `subgoal_decomposition`\r\n - `transition_modeling`\r\n- `func`: Specifies the function to execute. Options are:\r\n - `evaluate_results`\r\n - `generate_prompts`\r\n- `worker_num`: Number of workers for multiprocessing.\r\n- `llm_response_dir`: Directory containing LLM responses (HELM outputs).\r\n- `result_dir`: Directory to store results.\r\n\r\n### Example Usage:\r\n\r\n1. To generate prompts using the `action_sequence` module:\r\n ```\r\n python -m behavior_eval.main --module=action_sequence --func=generate_prompts\r\n ```\r\n\r\n2. To evaluate results using the `action_sequence` module:\r\n ```\r\n python -m behavior_eval.main --module=action_sequence --func=evaluate_results --llm_response_dir=<your_llm_response_dir>\r\n ```\r\n\r\nReplace `<your_llm_response_dir>` with the path to your LLM response directory.\r\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "1.0.7",
"project_urls": {
"Homepage": "https://github.com/embodied-agent-eval/behavior-eval"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e97dad63ae82c1e23f3eeac755c8a659da4318807e3ba0c18a44a90446860746",
"md5": "b8db3e81f5455b6cce16350216fd7ce9",
"sha256": "401224771e2eb4d6d8e4c1d77fac593bcab812f1b1f256b50ac3e6b7fd6a600b"
},
"downloads": -1,
"filename": "behavior_eval-1.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b8db3e81f5455b6cce16350216fd7ce9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 850800,
"upload_time": "2024-08-06T04:35:00",
"upload_time_iso_8601": "2024-08-06T04:35:00.773286Z",
"url": "https://files.pythonhosted.org/packages/e9/7d/ad63ae82c1e23f3eeac755c8a659da4318807e3ba0c18a44a90446860746/behavior_eval-1.0.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9845b2129bacaa2da0d99ef19ba7da7e0939bfcf1f0d35eada3b2dbaea62ee50",
"md5": "9a2c19198e4ae4f7c220094a08e2e699",
"sha256": "65f6db5bb63320c2ae99632a9b05e638bf914ed9022587133c6ef0fc829b50ba"
},
"downloads": -1,
"filename": "behavior_eval-1.0.7.tar.gz",
"has_sig": false,
"md5_digest": "9a2c19198e4ae4f7c220094a08e2e699",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 437658,
"upload_time": "2024-08-06T04:35:02",
"upload_time_iso_8601": "2024-08-06T04:35:02.712651Z",
"url": "https://files.pythonhosted.org/packages/98/45/b2129bacaa2da0d99ef19ba7da7e0939bfcf1f0d35eada3b2dbaea62ee50/behavior_eval-1.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-06 04:35:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "embodied-agent-eval",
"github_project": "behavior-eval",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "behavior-eval"
}
JSON
