# gym-super-mario-bros
[![BuildStatus][build-status]][ci-server]
[![PackageVersion][pypi-version]][pypi-home]
[![PythonVersion][python-version]][python-home]
[![Stable][pypi-status]][pypi-home]
[![Format][pypi-format]][pypi-home]
[![License][pypi-license]](LICENSE)
[build-status]: https://app.travis-ci.com/Kautenja/gym-super-mario-bros.svg?branch=master
[ci-server]: https://app.travis-ci.com/Kautenja/gym-super-mario-bros
[pypi-version]: https://badge.fury.io/py/gym-super-mario-bros.svg
[pypi-license]: https://img.shields.io/pypi/l/gym-super-mario-bros.svg
[pypi-status]: https://img.shields.io/pypi/status/gym-super-mario-bros.svg
[pypi-format]: https://img.shields.io/pypi/format/gym-super-mario-bros.svg
[pypi-home]: https://badge.fury.io/py/gym-super-mario-bros
[python-version]: https://img.shields.io/pypi/pyversions/gym-super-mario-bros.svg
[python-home]: https://python.org
An [OpenAI Gym](https://github.com/openai/gym) environment for
Super Mario Bros. & Super Mario Bros. 2 (Lost Levels) on The Nintendo
Entertainment System (NES) using
[the nes-py emulator](https://github.com/Kautenja/nes-py).
## Installation
The preferred installation of `gym-super-mario-bros` is from `pip`:
```shell
pip install gym-super-mario-bros
```
## Usage
### Python
You must import `gym_super_mario_bros` before trying to make an environment.
This is because gym environments are registered at runtime. By default,
`gym_super_mario_bros` environments use the full NES action space of 256
discrete actions. To contstrain this, `gym_super_mario_bros.actions` provides
three actions lists (`RIGHT_ONLY`, `SIMPLE_MOVEMENT`, and `COMPLEX_MOVEMENT`)
for the `nes_py.wrappers.JoypadSpace` wrapper. See
[gym_super_mario_bros/actions.py](https://github.com/Kautenja/gym-super-mario-bros/blob/master/gym_super_mario_bros/actions.py) for a
breakdown of the legal actions in each of these three lists.
```python
from nes_py.wrappers import JoypadSpace
import gym_super_mario_bros
from gym_super_mario_bros.actions import SIMPLE_MOVEMENT
env = gym_super_mario_bros.make('SuperMarioBros-v0')
env = JoypadSpace(env, SIMPLE_MOVEMENT)
done = True
for step in range(5000):
if done:
state = env.reset()
state, reward, done, info = env.step(env.action_space.sample())
env.render()
env.close()
```
**NOTE:** `gym_super_mario_bros.make` is just an alias to `gym.make` for
convenience.
**NOTE:** remove calls to `render` in training code for a nontrivial
speedup.
### Command Line
`gym_super_mario_bros` features a command line interface for playing
environments using either the keyboard, or uniform random movement.
```shell
gym_super_mario_bros -e <the environment ID to play> -m <`human` or `random`>
```
**NOTE:** by default, `-e` is set to `SuperMarioBros-v0` and `-m` is set to
`human`.
**NOTE:** `SuperMarioBrosRandomStages-*` support the `--stages/-S` flag for
supplying the set of stages to sample from like `-S 1-4 2-4 3-4 4-4`.
## Environments
These environments allow 3 attempts (lives) to make it through the 32 stages
in the game. The environments only send reward-able game-play frames to
agents; No cut-scenes, loading screens, etc. are sent from the NES emulator
to an agent nor can an agent perform actions during these instances. If a
cut-scene is not able to be skipped by hacking the NES's RAM, the environment
will lock the Python process until the emulator is ready for the next action.
| Environment | Game | ROM | Screenshot |
|:--------------------------------|:-----|:--------------|:-----------|
| `SuperMarioBros-v0` | SMB | standard | ![][v0] |
| `SuperMarioBros-v1` | SMB | downsample | ![][v1] |
| `SuperMarioBros-v2` | SMB | pixel | ![][v2] |
| `SuperMarioBros-v3` | SMB | rectangle | ![][v3] |
| `SuperMarioBros2-v0` | SMB2 | standard | ![][2-v0] |
| `SuperMarioBros2-v1` | SMB2 | downsample | ![][2-v1] |
[v0]: https://user-images.githubusercontent.com/2184469/40948820-3d15e5c2-6830-11e8-81d4-ecfaffee0a14.png
[v1]: https://user-images.githubusercontent.com/2184469/40948819-3cff6c48-6830-11e8-8373-8fad1665ac72.png
[v2]: https://user-images.githubusercontent.com/2184469/40948818-3cea09d4-6830-11e8-8efa-8f34d8b05b11.png
[v3]: https://user-images.githubusercontent.com/2184469/40948817-3cd6600a-6830-11e8-8abb-9cee6a31d377.png
[2-v0]: https://user-images.githubusercontent.com/2184469/40948822-3d3b8412-6830-11e8-860b-af3802f5373f.png
[2-v1]: https://user-images.githubusercontent.com/2184469/40948821-3d2d61a2-6830-11e8-8789-a92e750aa9a8.png
### Individual Stages
These environments allow a single attempt (life) to make it through a single
stage of the game.
Use the template
SuperMarioBros-<world>-<stage>-v<version>
where:
- `<world>` is a number in {1, 2, 3, 4, 5, 6, 7, 8} indicating the world
- `<stage>` is a number in {1, 2, 3, 4} indicating the stage within a world
- `<version>` is a number in {0, 1, 2, 3} specifying the ROM mode to use
- 0: standard ROM
- 1: downsampled ROM
- 2: pixel ROM
- 3: rectangle ROM
For example, to play 4-2 on the downsampled ROM, you would use the environment
id `SuperMarioBros-4-2-v1`.
### Random Stage Selection
The random stage selection environment randomly selects a stage and allows a
single attempt to clear it. Upon a death and subsequent call to `reset` the
environment randomly selects a new stage. This is only available for the
standard Super Mario Bros. game, _not_ Lost Levels (at the moment). To use
these environments, append `RandomStages` to the `SuperMarioBros` id. For
example, to use the standard ROM with random stage selection use
`SuperMarioBrosRandomStages-v0`. To seed the random stage selection use the
`seed` method of the env, i.e., `env.seed(222)`, before any calls to `reset`.
Alternatively pass the `seed` keyword argument to the `reset` method directly
like `reset(seed=222)`.
In addition to randomly selecting any of the 32 original stages, a subset of
user-defined stages can be specified to limit the random choice of stages to a
specific subset. For example, the stage selector could be limited to only
sample castle stages, water levels, underground, and more.
To specify a subset of stages to randomly sample from, create a list of each
stage to allow to be sampled and pass that list to the `gym.make()` function.
For example:
```python
gym.make('SuperMarioBrosRandomStages-v0', stages=['1-4', '2-4', '3-4', '4-4'])
```
The example above will sample a random stage from 1-4, 2-4, 3-4, and 4-4 upon
every call to `reset`.
## Step
Info about the rewards and info returned by the `step` method.
### Reward Function
The reward function assumes the objective of the game is to move as far right
as possible (increase the agent's _x_ value), as fast as possible, without
dying. To model this game, three separate variables compose the reward:
1. _v_: the difference in agent _x_ values between states
- in this case this is instantaneous velocity for the given step
- _v = x1 - x0_
- _x0_ is the x position before the step
- _x1_ is the x position after the step
- moving right ⇔ _v > 0_
- moving left ⇔ _v < 0_
- not moving ⇔ _v = 0_
2. _c_: the difference in the game clock between frames
- the penalty prevents the agent from standing still
- _c = c0 - c1_
- _c0_ is the clock reading before the step
- _c1_ is the clock reading after the step
- no clock tick ⇔ _c = 0_
- clock tick ⇔ _c < 0_
3. _d_: a death penalty that penalizes the agent for dying in a state
- this penalty encourages the agent to avoid death
- alive ⇔ _d = 0_
- dead ⇔ _d = -15_
_r = v + c + d_
The reward is clipped into the range _(-15, 15)_.
### `info` dictionary
The `info` dictionary returned by the `step` method contains the following
keys:
| Key | Type | Description
|:-----------|:-------|:------------------------------------------------------|
| `coins ` | `int` | The number of collected coins
| `flag_get` | `bool` | True if Mario reached a flag or ax
| `life` | `int` | The number of lives left, i.e., _{3, 2, 1}_
| `score` | `int` | The cumulative in-game score
| `stage` | `int` | The current stage, i.e., _{1, ..., 4}_
| `status` | `str` | Mario's status, i.e., _{'small', 'tall', 'fireball'}_
| `time` | `int` | The time left on the clock
| `world` | `int` | The current world, i.e., _{1, ..., 8}_
| `x_pos` | `int` | Mario's _x_ position in the stage (from the left)
| `y_pos` | `int` | Mario's _y_ position in the stage (from the bottom)
## Citation
Please cite `gym-super-mario-bros` if you use it in your research.
```tex
@misc{gym-super-mario-bros,
author = {Christian Kauten},
howpublished = {GitHub},
title = {{S}uper {M}ario {B}ros for {O}pen{AI} {G}ym},
URL = {https://github.com/Kautenja/gym-super-mario-bros},
year = {2018},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/Kautenja/gym-super-mario-bros",
"name": "gym-super-mario-bros",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "OpenAI-Gym NES Super-Mario-Bros Lost-Levels Reinforcement-Learning-Environment",
"author": "Christian Kauten",
"author_email": "kautencreations@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/0d/d0/af1263150596ff8cce3709c9883c1492de67b87b45322875cd6b03b8c967/gym_super_mario_bros-7.4.0.tar.gz",
"platform": null,
"description": "# gym-super-mario-bros\n\n[![BuildStatus][build-status]][ci-server]\n[![PackageVersion][pypi-version]][pypi-home]\n[![PythonVersion][python-version]][python-home]\n[![Stable][pypi-status]][pypi-home]\n[![Format][pypi-format]][pypi-home]\n[![License][pypi-license]](LICENSE)\n\n[build-status]: https://app.travis-ci.com/Kautenja/gym-super-mario-bros.svg?branch=master\n[ci-server]: https://app.travis-ci.com/Kautenja/gym-super-mario-bros\n[pypi-version]: https://badge.fury.io/py/gym-super-mario-bros.svg\n[pypi-license]: https://img.shields.io/pypi/l/gym-super-mario-bros.svg\n[pypi-status]: https://img.shields.io/pypi/status/gym-super-mario-bros.svg\n[pypi-format]: https://img.shields.io/pypi/format/gym-super-mario-bros.svg\n[pypi-home]: https://badge.fury.io/py/gym-super-mario-bros\n[python-version]: https://img.shields.io/pypi/pyversions/gym-super-mario-bros.svg\n[python-home]: https://python.org\n\n\n\nAn [OpenAI Gym](https://github.com/openai/gym) environment for\nSuper Mario Bros. & Super Mario Bros. 2 (Lost Levels) on The Nintendo\nEntertainment System (NES) using\n[the nes-py emulator](https://github.com/Kautenja/nes-py).\n\n## Installation\n\nThe preferred installation of `gym-super-mario-bros` is from `pip`:\n\n```shell\npip install gym-super-mario-bros\n```\n\n## Usage\n\n### Python\n\nYou must import `gym_super_mario_bros` before trying to make an environment.\nThis is because gym environments are registered at runtime. By default,\n`gym_super_mario_bros` environments use the full NES action space of 256\ndiscrete actions. To contstrain this, `gym_super_mario_bros.actions` provides\nthree actions lists (`RIGHT_ONLY`, `SIMPLE_MOVEMENT`, and `COMPLEX_MOVEMENT`)\nfor the `nes_py.wrappers.JoypadSpace` wrapper. See\n[gym_super_mario_bros/actions.py](https://github.com/Kautenja/gym-super-mario-bros/blob/master/gym_super_mario_bros/actions.py) for a\nbreakdown of the legal actions in each of these three lists.\n\n```python\nfrom nes_py.wrappers import JoypadSpace\nimport gym_super_mario_bros\nfrom gym_super_mario_bros.actions import SIMPLE_MOVEMENT\nenv = gym_super_mario_bros.make('SuperMarioBros-v0')\nenv = JoypadSpace(env, SIMPLE_MOVEMENT)\n\ndone = True\nfor step in range(5000):\n if done:\n state = env.reset()\n state, reward, done, info = env.step(env.action_space.sample())\n env.render()\n\nenv.close()\n```\n\n**NOTE:** `gym_super_mario_bros.make` is just an alias to `gym.make` for\nconvenience.\n\n**NOTE:** remove calls to `render` in training code for a nontrivial\nspeedup.\n\n### Command Line\n\n`gym_super_mario_bros` features a command line interface for playing\nenvironments using either the keyboard, or uniform random movement.\n\n```shell\ngym_super_mario_bros -e <the environment ID to play> -m <`human` or `random`>\n```\n\n**NOTE:** by default, `-e` is set to `SuperMarioBros-v0` and `-m` is set to\n`human`.\n\n**NOTE:** `SuperMarioBrosRandomStages-*` support the `--stages/-S` flag for\nsupplying the set of stages to sample from like `-S 1-4 2-4 3-4 4-4`.\n\n## Environments\n\nThese environments allow 3 attempts (lives) to make it through the 32 stages\nin the game. The environments only send reward-able game-play frames to\nagents; No cut-scenes, loading screens, etc. are sent from the NES emulator\nto an agent nor can an agent perform actions during these instances. If a\ncut-scene is not able to be skipped by hacking the NES's RAM, the environment\nwill lock the Python process until the emulator is ready for the next action.\n\n| Environment | Game | ROM | Screenshot |\n|:--------------------------------|:-----|:--------------|:-----------|\n| `SuperMarioBros-v0` | SMB | standard | ![][v0] |\n| `SuperMarioBros-v1` | SMB | downsample | ![][v1] |\n| `SuperMarioBros-v2` | SMB | pixel | ![][v2] |\n| `SuperMarioBros-v3` | SMB | rectangle | ![][v3] |\n| `SuperMarioBros2-v0` | SMB2 | standard | ![][2-v0] |\n| `SuperMarioBros2-v1` | SMB2 | downsample | ![][2-v1] |\n\n[v0]: https://user-images.githubusercontent.com/2184469/40948820-3d15e5c2-6830-11e8-81d4-ecfaffee0a14.png\n[v1]: https://user-images.githubusercontent.com/2184469/40948819-3cff6c48-6830-11e8-8373-8fad1665ac72.png\n[v2]: https://user-images.githubusercontent.com/2184469/40948818-3cea09d4-6830-11e8-8efa-8f34d8b05b11.png\n[v3]: https://user-images.githubusercontent.com/2184469/40948817-3cd6600a-6830-11e8-8abb-9cee6a31d377.png\n[2-v0]: https://user-images.githubusercontent.com/2184469/40948822-3d3b8412-6830-11e8-860b-af3802f5373f.png\n[2-v1]: https://user-images.githubusercontent.com/2184469/40948821-3d2d61a2-6830-11e8-8789-a92e750aa9a8.png\n\n### Individual Stages\n\nThese environments allow a single attempt (life) to make it through a single\nstage of the game.\n\nUse the template\n\n SuperMarioBros-<world>-<stage>-v<version>\n\nwhere:\n\n- `<world>` is a number in {1, 2, 3, 4, 5, 6, 7, 8} indicating the world\n- `<stage>` is a number in {1, 2, 3, 4} indicating the stage within a world\n- `<version>` is a number in {0, 1, 2, 3} specifying the ROM mode to use\n - 0: standard ROM\n - 1: downsampled ROM\n - 2: pixel ROM\n - 3: rectangle ROM\n\nFor example, to play 4-2 on the downsampled ROM, you would use the environment\nid `SuperMarioBros-4-2-v1`.\n\n### Random Stage Selection\n\nThe random stage selection environment randomly selects a stage and allows a\nsingle attempt to clear it. Upon a death and subsequent call to `reset` the\nenvironment randomly selects a new stage. This is only available for the\nstandard Super Mario Bros. game, _not_ Lost Levels (at the moment). To use\nthese environments, append `RandomStages` to the `SuperMarioBros` id. For\nexample, to use the standard ROM with random stage selection use\n`SuperMarioBrosRandomStages-v0`. To seed the random stage selection use the\n`seed` method of the env, i.e., `env.seed(222)`, before any calls to `reset`.\nAlternatively pass the `seed` keyword argument to the `reset` method directly\nlike `reset(seed=222)`.\n\nIn addition to randomly selecting any of the 32 original stages, a subset of\nuser-defined stages can be specified to limit the random choice of stages to a\nspecific subset. For example, the stage selector could be limited to only\nsample castle stages, water levels, underground, and more.\n\nTo specify a subset of stages to randomly sample from, create a list of each\nstage to allow to be sampled and pass that list to the `gym.make()` function.\nFor example:\n\n```python\ngym.make('SuperMarioBrosRandomStages-v0', stages=['1-4', '2-4', '3-4', '4-4'])\n```\n\nThe example above will sample a random stage from 1-4, 2-4, 3-4, and 4-4 upon\nevery call to `reset`.\n\n## Step\n\nInfo about the rewards and info returned by the `step` method.\n\n### Reward Function\n\nThe reward function assumes the objective of the game is to move as far right\nas possible (increase the agent's _x_ value), as fast as possible, without\ndying. To model this game, three separate variables compose the reward:\n\n1. _v_: the difference in agent _x_ values between states\n - in this case this is instantaneous velocity for the given step\n - _v = x1 - x0_\n - _x0_ is the x position before the step\n - _x1_ is the x position after the step\n - moving right \u21d4 _v > 0_\n - moving left \u21d4 _v < 0_\n - not moving \u21d4 _v = 0_\n2. _c_: the difference in the game clock between frames\n - the penalty prevents the agent from standing still\n - _c = c0 - c1_\n - _c0_ is the clock reading before the step\n - _c1_ is the clock reading after the step\n - no clock tick \u21d4 _c = 0_\n - clock tick \u21d4 _c < 0_\n3. _d_: a death penalty that penalizes the agent for dying in a state\n - this penalty encourages the agent to avoid death\n - alive \u21d4 _d = 0_\n - dead \u21d4 _d = -15_\n\n_r = v + c + d_\n\nThe reward is clipped into the range _(-15, 15)_.\n\n### `info` dictionary\n\nThe `info` dictionary returned by the `step` method contains the following\nkeys:\n\n| Key | Type | Description\n|:-----------|:-------|:------------------------------------------------------|\n| `coins ` | `int` | The number of collected coins\n| `flag_get` | `bool` | True if Mario reached a flag or ax\n| `life` | `int` | The number of lives left, i.e., _{3, 2, 1}_\n| `score` | `int` | The cumulative in-game score\n| `stage` | `int` | The current stage, i.e., _{1, ..., 4}_\n| `status` | `str` | Mario's status, i.e., _{'small', 'tall', 'fireball'}_\n| `time` | `int` | The time left on the clock\n| `world` | `int` | The current world, i.e., _{1, ..., 8}_\n| `x_pos` | `int` | Mario's _x_ position in the stage (from the left)\n| `y_pos` | `int` | Mario's _y_ position in the stage (from the bottom)\n\n## Citation\n\nPlease cite `gym-super-mario-bros` if you use it in your research.\n\n```tex\n@misc{gym-super-mario-bros,\n author = {Christian Kauten},\n howpublished = {GitHub},\n title = {{S}uper {M}ario {B}ros for {O}pen{AI} {G}ym},\n URL = {https://github.com/Kautenja/gym-super-mario-bros},\n year = {2018},\n}\n```\n\n\n",
"bugtrack_url": null,
"license": "Proprietary",
"summary": "Super Mario Bros. for OpenAI Gym",
"version": "7.4.0",
"project_urls": {
"Homepage": "https://github.com/Kautenja/gym-super-mario-bros"
},
"split_keywords": [
"openai-gym",
"nes",
"super-mario-bros",
"lost-levels",
"reinforcement-learning-environment"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0c4f0951ba1480c6f874a5e3b043f20d06fc1951cc03844e8bcfa6208291d712",
"md5": "288d8d1bff7cd86802603f4ee9b6340c",
"sha256": "5aa094115a32ed3d7f2eb044bc24486888af4f878695c9f26c7383a80c744602"
},
"downloads": -1,
"filename": "gym_super_mario_bros-7.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "288d8d1bff7cd86802603f4ee9b6340c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 199075,
"upload_time": "2022-06-21T04:40:04",
"upload_time_iso_8601": "2022-06-21T04:40:04.601412Z",
"url": "https://files.pythonhosted.org/packages/0c/4f/0951ba1480c6f874a5e3b043f20d06fc1951cc03844e8bcfa6208291d712/gym_super_mario_bros-7.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0dd0af1263150596ff8cce3709c9883c1492de67b87b45322875cd6b03b8c967",
"md5": "1b38dde6f870b322c685b2ebd9de3f1d",
"sha256": "9330e6e4f75b4405125abffb35bf1338b9027d2d87e7d4504e2cf05269541ac7"
},
"downloads": -1,
"filename": "gym_super_mario_bros-7.4.0.tar.gz",
"has_sig": false,
"md5_digest": "1b38dde6f870b322c685b2ebd9de3f1d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 200831,
"upload_time": "2022-06-21T04:40:07",
"upload_time_iso_8601": "2022-06-21T04:40:07.536170Z",
"url": "https://files.pythonhosted.org/packages/0d/d0/af1263150596ff8cce3709c9883c1492de67b87b45322875cd6b03b8c967/gym_super_mario_bros-7.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-06-21 04:40:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Kautenja",
"github_project": "gym-super-mario-bros",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "gym",
"specs": [
[
">=",
"0.10.9"
]
]
},
{
"name": "matplotlib",
"specs": [
[
">=",
"2.0.2"
]
]
},
{
"name": "nes-py",
"specs": [
[
">=",
"4.0.0"
]
]
},
{
"name": "numpy",
"specs": [
[
">=",
"1.14.2"
]
]
},
{
"name": "opencv-python",
"specs": [
[
">=",
"3.4.0.12"
]
]
},
{
"name": "pygame",
"specs": [
[
">=",
"1.9.3"
]
]
},
{
"name": "pyglet",
"specs": [
[
">=",
"1.3.2"
]
]
},
{
"name": "setuptools",
"specs": [
[
">=",
"39.0.1"
]
]
},
{
"name": "tqdm",
"specs": [
[
">=",
"4.19.5"
]
]
},
{
"name": "twine",
"specs": [
[
">=",
"1.11.0"
]
]
}
],
"lcname": "gym-super-mario-bros"
}
JSON
