# Hugging Face 🤗 x Stable-baselines3 v3.0
A library to load and upload Stable-baselines3 models from the Hub with Gymnasium and Gymnasium compatible environments.
⚠️ If you use Gym, you need to install `huggingface_sb3==2.3.1`
## Installation
### With pip
```
pip install huggingface-sb3
```
## Examples
We wrote a tutorial on how to use 🤗 Hub and Stable-Baselines3 [here](https://colab.research.google.com/github/huggingface/deep-rl-class/blob/master/notebooks/unit1/unit1.ipynb)
If you use **Colab or a Virtual/Screenless Machine**, you can check Case 3 and Case 4.
### Case 1: I want to download a model from the Hub
```python
import gymnasium as gym
from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
# Retrieve the model from the hub
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename = name of the model zip file from the repository
checkpoint = load_from_hub(
repo_id="sb3/demo-hf-CartPole-v1",
filename="ppo-CartPole-v1.zip",
)
model = PPO.load(checkpoint)
# Evaluate the agent and watch it
eval_env = gym.make("CartPole-v1")
mean_reward, std_reward = evaluate_policy(
model, eval_env, render=False, n_eval_episodes=5, deterministic=True, warn=False
)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")
```
### Case 2: I trained an agent and want to upload it to the Hub
With `package_to_hub()` **we'll save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub**.
It currently **works for Gym and Atari environments**. If you use another environment, you should use `push_to_hub()` instead.
First you need to be logged in to Hugging Face:
- If you're using Colab/Jupyter Notebooks:
```python
from huggingface_hub import notebook_login
notebook_login()
```
- Else:
```
huggingface-cli login
```
Then
**With `package_to_hub()`**:
```python
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import package_to_hub
# Create the environment
env_id = "LunarLander-v2"
env = make_vec_env(env_id, n_envs=1)
# Create the evaluation env
eval_env = make_vec_env(env_id, n_envs=1)
# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)
# Train the agent
model.learn(total_timesteps=int(5000))
# This method save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub
package_to_hub(model=model,
model_name="ppo-LunarLander-v2",
model_architecture="PPO",
env_id=env_id,
eval_env=eval_env,
repo_id="ThomasSimonini/ppo-LunarLander-v2",
commit_message="Test commit")
```
**With `push_to_hub()`**:
Push to hub only **push a file to the Hub**, if you want to save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub, use `package_to_hub()`
```python
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import push_to_hub
# Create the environment
env_id = "LunarLander-v2"
env = make_vec_env(env_id, n_envs=1)
# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)
# Train it for 10000 timesteps
model.learn(total_timesteps=10_000)
# Save the model
model.save("ppo-LunarLander-v2")
# Push this saved model .zip file to the hf repo
# If this repo does not exists it will be created
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename: the name of the file == "name" inside model.save("ppo-LunarLander-v2")
push_to_hub(
repo_id="ThomasSimonini/ppo-LunarLander-v2",
filename="ppo-LunarLander-v2.zip",
commit_message="Added LunarLander-v2 model trained with PPO",
)
```
### Case 3: I use Google Colab with Classic Control/Box2D Gym Environments
- You can use xvbf (virtual screen)
```
!apt-get install -y xvfb python-opengl > /dev/null 2>&1
```
- Just put your code inside a python file and run
```
!xvfb-run -s "-screen 0 1400x900x24" <your_python_file>
```
### Case 4: I use a Virtual/Remote Machine
- You can use xvbf (virtual screen)
```
xvfb-run -s "-screen 0 1400x900x24" <your_python_file>
```
### Case 5: I want to automate upload/download from the Hub
If you want to upload or download models for many environments, you might want to
automate this process.
It makes sense to adhere to a fixed naming scheme for models and repositories.
You will run into trouble when your environment names contain slashes.
Therefore, we provide some helper classes:
```python
import gymnasium as gym
from huggingface_sb3.naming_schemes import EnvironmentName, ModelName, ModelRepoId
env_name = EnvironmentName("seals/Walker2d-v0")
model_name = ModelName("ppo", env_name)
repo_id = ModelRepoId("YourOrganization", model_name)
# prints 'seals-Walker2d-v0'. Notice how the slash is removed so you can use it to
# construct file paths if you like.
print(env_name)
# you can still access the original gym id if needed
env = gym.make(env_name.gym_id)
# prints `ppo-seals-Walker2d-v0`
print(model_name)
# prints: `ppo-seals-Walker2d-v0.zip`.
# This is where `model.save(model_name)` will place the model file
print(model_name.filename)
# prints: `YourOrganization/ppo-seals-Walker2d-v0`
print(repo_id)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/huggingface/huggingface_sb3",
"name": "huggingface-sb3",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "reinforcement learning deep reinforcement learning RL",
"author": "Thomas Simonini, Omar Sanseviero and Hugging Face Team",
"author_email": "thomas.simonini@huggingface.co",
"download_url": "https://files.pythonhosted.org/packages/4b/df/db480915a4f86bd253eb0ee56b47bece40d8fcf2926d58a2b9891299aedc/huggingface_sb3-3.0.tar.gz",
"platform": null,
"description": "# Hugging Face \ud83e\udd17 x Stable-baselines3 v3.0\n\nA library to load and upload Stable-baselines3 models from the Hub with Gymnasium and Gymnasium compatible environments.\n\n\u26a0\ufe0f If you use Gym, you need to install `huggingface_sb3==2.3.1`\n\n## Installation\n### With pip\n```\npip install huggingface-sb3\n```\n\n## Examples\nWe wrote a tutorial on how to use \ud83e\udd17 Hub and Stable-Baselines3 [here](https://colab.research.google.com/github/huggingface/deep-rl-class/blob/master/notebooks/unit1/unit1.ipynb)\n\nIf you use **Colab or a Virtual/Screenless Machine**, you can check Case 3 and Case 4.\n\n### Case 1: I want to download a model from the Hub\n```python\nimport gymnasium as gym\n\nfrom huggingface_sb3 import load_from_hub\nfrom stable_baselines3 import PPO\nfrom stable_baselines3.common.evaluation import evaluate_policy\n\n# Retrieve the model from the hub\n## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})\n## filename = name of the model zip file from the repository\ncheckpoint = load_from_hub(\n repo_id=\"sb3/demo-hf-CartPole-v1\",\n filename=\"ppo-CartPole-v1.zip\",\n)\nmodel = PPO.load(checkpoint)\n\n# Evaluate the agent and watch it\neval_env = gym.make(\"CartPole-v1\")\nmean_reward, std_reward = evaluate_policy(\n model, eval_env, render=False, n_eval_episodes=5, deterministic=True, warn=False\n)\nprint(f\"mean_reward={mean_reward:.2f} +/- {std_reward}\")\n```\n\n### Case 2: I trained an agent and want to upload it to the Hub\nWith `package_to_hub()` **we'll save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub**.\nIt currently **works for Gym and Atari environments**. If you use another environment, you should use `push_to_hub()` instead.\n\nFirst you need to be logged in to Hugging Face:\n- If you're using Colab/Jupyter Notebooks:\n```python\nfrom huggingface_hub import notebook_login\nnotebook_login()\n```\n- Else:\n```\nhuggingface-cli login\n```\nThen\n\n**With `package_to_hub()`**:\n\n```python\nimport gymnasium as gym\n\nfrom stable_baselines3 import PPO\nfrom stable_baselines3.common.env_util import make_vec_env\nfrom huggingface_sb3 import package_to_hub\n\n# Create the environment\nenv_id = \"LunarLander-v2\"\nenv = make_vec_env(env_id, n_envs=1)\n\n# Create the evaluation env\neval_env = make_vec_env(env_id, n_envs=1)\n\n# Instantiate the agent\nmodel = PPO(\"MlpPolicy\", env, verbose=1)\n\n# Train the agent\nmodel.learn(total_timesteps=int(5000))\n\n# This method save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub\npackage_to_hub(model=model, \n model_name=\"ppo-LunarLander-v2\",\n model_architecture=\"PPO\",\n env_id=env_id,\n eval_env=eval_env,\n repo_id=\"ThomasSimonini/ppo-LunarLander-v2\",\n commit_message=\"Test commit\")\n```\n\n\n**With `push_to_hub()`**:\nPush to hub only **push a file to the Hub**, if you want to save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub, use `package_to_hub()`\n\n```python\nimport gymnasium as gym\n\nfrom stable_baselines3 import PPO\nfrom stable_baselines3.common.env_util import make_vec_env\nfrom huggingface_sb3 import push_to_hub\n\n# Create the environment\nenv_id = \"LunarLander-v2\"\nenv = make_vec_env(env_id, n_envs=1)\n\n# Instantiate the agent\nmodel = PPO(\"MlpPolicy\", env, verbose=1)\n\n# Train it for 10000 timesteps\nmodel.learn(total_timesteps=10_000)\n\n# Save the model\nmodel.save(\"ppo-LunarLander-v2\")\n\n# Push this saved model .zip file to the hf repo\n# If this repo does not exists it will be created\n## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})\n## filename: the name of the file == \"name\" inside model.save(\"ppo-LunarLander-v2\")\npush_to_hub(\n repo_id=\"ThomasSimonini/ppo-LunarLander-v2\",\n filename=\"ppo-LunarLander-v2.zip\",\n commit_message=\"Added LunarLander-v2 model trained with PPO\",\n)\n```\n### Case 3: I use Google Colab with Classic Control/Box2D Gym Environments\n- You can use xvbf (virtual screen)\n```\n!apt-get install -y xvfb python-opengl > /dev/null 2>&1\n```\n- Just put your code inside a python file and run\n```\n!xvfb-run -s \"-screen 0 1400x900x24\" <your_python_file>\n```\n\n### Case 4: I use a Virtual/Remote Machine\n- You can use xvbf (virtual screen)\n\n```\nxvfb-run -s \"-screen 0 1400x900x24\" <your_python_file>\n```\n\n### Case 5: I want to automate upload/download from the Hub\nIf you want to upload or download models for many environments, you might want to \nautomate this process. \nIt makes sense to adhere to a fixed naming scheme for models and repositories.\nYou will run into trouble when your environment names contain slashes.\nTherefore, we provide some helper classes:\n\n```python\nimport gymnasium as gym\nfrom huggingface_sb3.naming_schemes import EnvironmentName, ModelName, ModelRepoId\n\nenv_name = EnvironmentName(\"seals/Walker2d-v0\")\nmodel_name = ModelName(\"ppo\", env_name)\nrepo_id = ModelRepoId(\"YourOrganization\", model_name)\n\n# prints 'seals-Walker2d-v0'. Notice how the slash is removed so you can use it to \n# construct file paths if you like.\nprint(env_name)\n\n# you can still access the original gym id if needed\nenv = gym.make(env_name.gym_id) \n\n# prints `ppo-seals-Walker2d-v0`\nprint(model_name) \n\n# prints: `ppo-seals-Walker2d-v0.zip`. \n# This is where `model.save(model_name)` will place the model file\nprint(model_name.filename) \n\n# prints: `YourOrganization/ppo-seals-Walker2d-v0`\nprint(repo_id)\n```\n\n\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "Additional code for Stable-baselines3 to load and upload models from the Hub.",
"version": "3.0",
"project_urls": {
"Homepage": "https://github.com/huggingface/huggingface_sb3"
},
"split_keywords": [
"reinforcement",
"learning",
"deep",
"reinforcement",
"learning",
"rl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b52ce8c25087c6e43e5e68659f2a4651988831d4b3e70791c433537c90d147ce",
"md5": "e466eb9252e7006254662cc61eda9caf",
"sha256": "f543f1e5d840425044b9f83cd6915da77cc308c1a7dd4585208f17efad1e8321"
},
"downloads": -1,
"filename": "huggingface_sb3-3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e466eb9252e7006254662cc61eda9caf",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 9676,
"upload_time": "2023-09-07T14:39:41",
"upload_time_iso_8601": "2023-09-07T14:39:41.585152Z",
"url": "https://files.pythonhosted.org/packages/b5/2c/e8c25087c6e43e5e68659f2a4651988831d4b3e70791c433537c90d147ce/huggingface_sb3-3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4bdfdb480915a4f86bd253eb0ee56b47bece40d8fcf2926d58a2b9891299aedc",
"md5": "b8ff8c441bc85de04ae325ca9558534c",
"sha256": "abaf901808dc5827976e3029646e463da65536068d910cd8ef4a8e985087d515"
},
"downloads": -1,
"filename": "huggingface_sb3-3.0.tar.gz",
"has_sig": false,
"md5_digest": "b8ff8c441bc85de04ae325ca9558534c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10411,
"upload_time": "2023-09-07T14:39:42",
"upload_time_iso_8601": "2023-09-07T14:39:42.789747Z",
"url": "https://files.pythonhosted.org/packages/4b/df/db480915a4f86bd253eb0ee56b47bece40d8fcf2926d58a2b9891299aedc/huggingface_sb3-3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-07 14:39:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "huggingface",
"github_project": "huggingface_sb3",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "huggingface-sb3"
}
JSON
