<div align="center">
<!-- Header with logo to the left of the title -->
<table style="margin: 0 auto;">
<tr>
<td style="vertical-align: middle;">
<img src="https://understanding-search.github.io/maze-dataset/resources/logo.svg" alt="Maze Dataset Logo" width="50">
</td>
<td style="vertical-align: middle; padding-left: 10px;">
<h1 style="margin: 0;">maze-dataset</h1>
</td>
</tr>
</table>
<!-- Top prominent badges -->
<p>
<a href="https://pypi.org/project/maze-dataset/">
<img src="https://img.shields.io/pypi/v/maze-dataset" alt="PyPI">
</a>
<a href="https://understanding-search.github.io/maze-dataset/">
<img src="https://img.shields.io/badge/docs-available-brightgreen" alt="Docs">
</a>
<a href="https://understanding-search.github.io/maze-dataset/examples/maze_examples.html">
<img src="https://img.shields.io/badge/examples-view-blue" alt="Examples">
</a>
<a href="http://arxiv.org/abs/2309.10498">
<img src="https://img.shields.io/badge/arXiv-2309.10498-b31b1b.svg" alt="arXiv">
</a>
</p>
<!-- Diagram -->
<p>
<img src="https://understanding-search.github.io/maze-dataset/resources/diagram.svg" alt="Diagram" width="95%">
</p>
<!-- Additional badges -->
<p>
<a href="https://pypi.org/project/maze-dataset/">
<img src="https://img.shields.io/pypi/v/maze-dataset" alt="PyPI">
</a>
<img src="https://img.shields.io/badge/python-3.10%2B-blue?logo=python&logoColor=white" alt="Python Version">
<a href="https://github.com/understanding-search/maze-dataset/actions/workflows/checks.yml">
<img src="https://github.com/understanding-search/maze-dataset/actions/workflows/checks.yml/badge.svg" alt="Checks">
</a>
<a href="https://understanding-search.github.io/maze-dataset/coverage/coverage.txt">
<img src="https://understanding-search.github.io/maze-dataset/coverage/coverage.svg" alt="Coverage">
</a>
<img src="https://img.shields.io/github/languages/code-size/understanding-search/maze-dataset" alt="code size, bytes">
<img src="https://img.shields.io/github/commit-activity/t/understanding-search/maze-dataset" alt="GitHub commit activity">
<img src="https://img.shields.io/github/issues-closed/understanding-search/maze-dataset" alt="GitHub closed issues">
<img src="https://img.shields.io/github/issues-pr-closed/understanding-search/maze-dataset" alt="GitHub closed pull requests">
<img src="https://img.shields.io/pypi/dm/maze-dataset" alt="PyPI - Downloads">
<a href="https://juleskreuer.eu/projects/citation-badge">
<img alt="Citation Badge" src="https://api.juleskreuer.eu/citation-badge.php?doi=10.48550/arxiv.2309.10498">
</a>
</p>
</div>
<!-- [](https://pypi.org/project/maze-dataset/)
[](https://github.com/understanding-search/maze-dataset/actions/workflows/checks.yml)
[](docs/coverage/coverage.txt)
 -->
# `maze-dataset`
This package provides utilities for generation, filtering, solving, visualizing, and processing of mazes for training or evaluating ML systems. Primarily built for the [maze-transformer interpretability](https://github.com/understanding-search/maze-transformer) project. You can find our paper on it here: http://arxiv.org/abs/2309.10498
This package includes a variety of maze generation algorithms, including randomized depth first search, Wilson's algorithm for uniform spanning trees, and percolation. Datasets can be filtered to select mazes of a certain length or complexity, remove duplicates, and satisfy custom properties. A variety of output formats for visualization and training ML models are provided.
| | | | |
|---|---|---|---|
| <img src="https://understanding-search.github.io/maze-dataset/assets/maze_perc.png" alt="Maze generated via percolation" width="95%"/> | <img src="https://understanding-search.github.io/maze-dataset/assets/maze_dfs_constrained.png" alt="Maze generated via constrained randomized depth first search" width="95%"/> | <img src="https://understanding-search.github.io/maze-dataset/assets/mazeplot_heatmap.png" alt="Maze with random heatmap" width="95%"/> | <img src="https://understanding-search.github.io/maze-dataset/assets/mazeplot_path.png" alt="MazePlot with solution" width="95%"/> |
You can view and search through a wide variety of example mazes here: [`understanding-search.github.io/maze-dataset/examples/maze_examples`](https://understanding-search.github.io/maze-dataset/examples/maze_examples.html)
# Citing
If you use this code in your research, please cite [our paper](http://arxiv.org/abs/2309.10498):
```
@misc{maze-dataset,
title={A Configurable Library for Generating and Manipulating Maze Datasets},
author={Michael Igorevich Ivanitskiy and Rusheb Shah and Alex F. Spies and Tilman Räuker and Dan Valentine and Can Rager and Lucia Quirke and Chris Mathwin and Guillaume Corlouer and Cecilia Diniz Behn and Samy Wu Fung},
year={2023},
eprint={2309.10498},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={http://arxiv.org/abs/2309.10498}
}
```
# Installation
This package is [available on PyPI](https://pypi.org/project/maze-dataset/), and can be installed via
```
pip install maze-dataset
```
> Please note that due to an issue with the `rust-fst` package, some tokenization features are not available on macOS. Please see [#57](https://github.com/understanding-search/maze-dataset/issues/57)
# Docs
The full hosted documentation is available at [https://understanding-search.github.io/maze-dataset/](https://understanding-search.github.io/maze-dataset/).
Additionally, our [notebooks](https://understanding-search.github.io/maze-dataset/notebooks) serve as a good starting point for understanding the package.
# Usage
## Creating a dataset
To create a `MazeDataset`, you first create a `MazeDatasetConfig`:
```python
from maze_dataset import MazeDataset, MazeDatasetConfig
from maze_dataset.generation import LatticeMazeGenerators
cfg: MazeDatasetConfig = MazeDatasetConfig(
name="test", # name is only for you to keep track of things
grid_n=5, # number of rows/columns in the lattice
n_mazes=4, # number of mazes to generate
maze_ctor=LatticeMazeGenerators.gen_dfs, # algorithm to generate the maze
maze_ctor_kwargs=dict(do_forks=False), # additional parameters to pass to the maze generation algorithm
)
```
and then pass this config to the `MazeDataset.from_config` method:
```python
dataset: MazeDataset = MazeDataset.from_config(cfg)
```
This method can search for whether a dataset with matching config hash already exists on your filesystem in the expected location, and load it if so. It can also generate a dataset on the fly if needed.
## Conversions to useful formats
The elements of the dataset are [`SolvedMaze`](maze_dataset/maze/lattice_maze.py) objects:
```python
>>> m = dataset[0]
>>> type(m)
maze_dataset.maze.lattice_maze.SolvedMaze
```
Which can be converted to a variety of formats:
```python
# visual representation as ascii art
print(m.as_ascii())
# RGB image, optionally without solution or endpoints, suitable for CNNs
import matplotlib.pyplot as plt
plt.imshow(m.as_pixels())
# text format for autoreregressive transformers
from maze_dataset.tokenization import MazeTokenizerModular, TokenizationMode, PromptSequencers
m.as_tokens(maze_tokenizer=MazeTokenizerModular(
prompt_sequencer=PromptSequencers.AOTP(), # many options here
))
# advanced visualization with many features
from maze_dataset.plotting import MazePlot
MazePlot(m).plot()
```
<img src="https://understanding-search.github.io/maze-dataset/assets/output_formats.png" alt="textual and visual output formats" width="100%"/>
# Development
We use this [makefile template](https://github.com/mivanit/python-project-makefile-template) with slight modifications for our development workflow. This project uses [uv](https://docs.astral.sh/uv/) for dependency and virtual environment management.
- clone with `git clone https://github.com/understanding-search/maze-dataset`
- if you don't already have uv, [install it](https://docs.astral.sh/uv/getting-started/installation/). We only guarantee compatibility with `uv` newer than `0.8.0`
- `make dep` to install all dependencies
- `make help` will print all available commands
- `make test` will run basic tests to ensure the package is working
- run just the unit tests with `make test-unit`
- see all tests with explanations using `make help` or `make help | grep test`
- `make format` will run ruff to format and check the code
> Note: due to compatibility issues between the `rust_fst` package and Darwin/macOS systems, not all tests will pass on these systems. However, `make test-unit` and `make test-notebooks-muutils` should still pass. Please see [#57](https://github.com/understanding-search/maze-dataset/issues/57) for updates on resolving this problem.
## Contributing
We welcome contributions! We use [GitHub issues](https://github.com/understanding-search/maze-dataset/issues) to track bugs and feature requests. If you have a bug fix or a new feature to contribute, please open a [pull request](https://github.com/understanding-search/maze-dataset/pulls). We are also happy to provide usage support and answer questions about the package via issues!
While we expect that the core interface of the package is stable, we are very open to adding new features. We're particularly excited about adding [new maze generation algorithms](https://github.com/understanding-search/maze-dataset/issues?q=is%3Aissue%20state%3Aopen%20label%3Ageneration) and [new output formats](https://github.com/understanding-search/maze-dataset/issues?q=is%3Aissue%20state%3Aopen%20label%3Aexporting). Please feel free to both suggest new formats or algorithms, and to implement them and open PRs! For more info on how to add a new maze generation algorithm, see the [documentation on generators](https://understanding-search.github.io/maze-dataset/maze_dataset/generation.html).
We are also aware that like any piece of software, `maze-dataset` is not without bugs. If something isn't working as expected, please open an issue and we will do our best to fix it. It helps us keep things tidy if you first search [existing bug reports](https://github.com/understanding-search/maze-dataset/issues?q=label%3Abug) to see if your issue has already been reported.
Raw data
{
"_id": null,
"home_page": null,
"name": "maze-dataset",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "dataset, labyrinth, maze, mazes, pathfinding, procedural, tokenization",
"author": null,
"author_email": "Michael Ivanitskiy <mivanits@umich.edu>, Aaron Sandoval <aaron.sandoval10@gmail.com>, Rusheb Shah <rusheb.shah@gmail.com>, Dan Valentine <danvalentine256@gmail.com>, Lucia Quirke <luciaq@canva.com>, Can Rager <can.rager@posteo.de>, Alex Spies <alexfspies@gmail.com>, Chris Mathwin <cwmathwin@gmail.com>, Tilman Rauker <traeuker@googlemail.com>, Guillaume Corlouer <guillaume.corlouer@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/0f/a1/ea8c35fc2dd134dee0b4e37159f70125a26f9f81f102c2294d6c3b46cada/maze_dataset-1.4.0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n\n <!-- Header with logo to the left of the title -->\n <table style=\"margin: 0 auto;\">\n <tr>\n <td style=\"vertical-align: middle;\">\n <img src=\"https://understanding-search.github.io/maze-dataset/resources/logo.svg\" alt=\"Maze Dataset Logo\" width=\"50\">\n </td>\n <td style=\"vertical-align: middle; padding-left: 10px;\">\n <h1 style=\"margin: 0;\">maze-dataset</h1>\n </td>\n </tr>\n </table>\n\n<!-- Top prominent badges -->\n <p>\n <a href=\"https://pypi.org/project/maze-dataset/\">\n <img src=\"https://img.shields.io/pypi/v/maze-dataset\" alt=\"PyPI\">\n </a>\n \n <a href=\"https://understanding-search.github.io/maze-dataset/\">\n <img src=\"https://img.shields.io/badge/docs-available-brightgreen\" alt=\"Docs\">\n </a>\n \n <a href=\"https://understanding-search.github.io/maze-dataset/examples/maze_examples.html\">\n <img src=\"https://img.shields.io/badge/examples-view-blue\" alt=\"Examples\">\n </a>\n \n <a href=\"http://arxiv.org/abs/2309.10498\">\n <img src=\"https://img.shields.io/badge/arXiv-2309.10498-b31b1b.svg\" alt=\"arXiv\">\n </a>\n </p>\n \n\n <!-- Diagram -->\n <p>\n <img src=\"https://understanding-search.github.io/maze-dataset/resources/diagram.svg\" alt=\"Diagram\" width=\"95%\">\n </p>\n\n <!-- Additional badges -->\n <p>\n <a href=\"https://pypi.org/project/maze-dataset/\">\n <img src=\"https://img.shields.io/pypi/v/maze-dataset\" alt=\"PyPI\">\n </a>\n \n <img src=\"https://img.shields.io/badge/python-3.10%2B-blue?logo=python&logoColor=white\" alt=\"Python Version\">\n \n <a href=\"https://github.com/understanding-search/maze-dataset/actions/workflows/checks.yml\">\n <img src=\"https://github.com/understanding-search/maze-dataset/actions/workflows/checks.yml/badge.svg\" alt=\"Checks\">\n </a>\n \n <a href=\"https://understanding-search.github.io/maze-dataset/coverage/coverage.txt\">\n <img src=\"https://understanding-search.github.io/maze-dataset/coverage/coverage.svg\" alt=\"Coverage\">\n </a>\n \n <img src=\"https://img.shields.io/github/languages/code-size/understanding-search/maze-dataset\" alt=\"code size, bytes\">\n \n <img src=\"https://img.shields.io/github/commit-activity/t/understanding-search/maze-dataset\" alt=\"GitHub commit activity\">\n \n <img src=\"https://img.shields.io/github/issues-closed/understanding-search/maze-dataset\" alt=\"GitHub closed issues\">\n \n <img src=\"https://img.shields.io/github/issues-pr-closed/understanding-search/maze-dataset\" alt=\"GitHub closed pull requests\">\n \n <img src=\"https://img.shields.io/pypi/dm/maze-dataset\" alt=\"PyPI - Downloads\">\n \n <a href=\"https://juleskreuer.eu/projects/citation-badge\">\n <img alt=\"Citation Badge\" src=\"https://api.juleskreuer.eu/citation-badge.php?doi=10.48550/arxiv.2309.10498\">\n </a>\n </p>\n\n</div>\n\n\n\n<!-- [](https://pypi.org/project/maze-dataset/)\n\n[](https://github.com/understanding-search/maze-dataset/actions/workflows/checks.yml)\n[](docs/coverage/coverage.txt)\n\n\n -->\n\n\n# `maze-dataset`\n\nThis package provides utilities for generation, filtering, solving, visualizing, and processing of mazes for training or evaluating ML systems. Primarily built for the [maze-transformer interpretability](https://github.com/understanding-search/maze-transformer) project. You can find our paper on it here: http://arxiv.org/abs/2309.10498\n\nThis package includes a variety of maze generation algorithms, including randomized depth first search, Wilson's algorithm for uniform spanning trees, and percolation. Datasets can be filtered to select mazes of a certain length or complexity, remove duplicates, and satisfy custom properties. A variety of output formats for visualization and training ML models are provided.\n\n| | | | |\n|---|---|---|---|\n| <img src=\"https://understanding-search.github.io/maze-dataset/assets/maze_perc.png\" alt=\"Maze generated via percolation\" width=\"95%\"/> | <img src=\"https://understanding-search.github.io/maze-dataset/assets/maze_dfs_constrained.png\" alt=\"Maze generated via constrained randomized depth first search\" width=\"95%\"/> | <img src=\"https://understanding-search.github.io/maze-dataset/assets/mazeplot_heatmap.png\" alt=\"Maze with random heatmap\" width=\"95%\"/> | <img src=\"https://understanding-search.github.io/maze-dataset/assets/mazeplot_path.png\" alt=\"MazePlot with solution\" width=\"95%\"/> |\n\n\nYou can view and search through a wide variety of example mazes here: [`understanding-search.github.io/maze-dataset/examples/maze_examples`](https://understanding-search.github.io/maze-dataset/examples/maze_examples.html)\n\n# Citing\n\nIf you use this code in your research, please cite [our paper](http://arxiv.org/abs/2309.10498):\n\n```\n@misc{maze-dataset,\n title={A Configurable Library for Generating and Manipulating Maze Datasets}, \n author={Michael Igorevich Ivanitskiy and Rusheb Shah and Alex F. Spies and Tilman R\u00e4uker and Dan Valentine and Can Rager and Lucia Quirke and Chris Mathwin and Guillaume Corlouer and Cecilia Diniz Behn and Samy Wu Fung},\n year={2023},\n eprint={2309.10498},\n archivePrefix={arXiv},\n primaryClass={cs.LG},\n url={http://arxiv.org/abs/2309.10498}\n}\n```\n\n\n# Installation\nThis package is [available on PyPI](https://pypi.org/project/maze-dataset/), and can be installed via\n```\npip install maze-dataset\n```\n\n> Please note that due to an issue with the `rust-fst` package, some tokenization features are not available on macOS. Please see [#57](https://github.com/understanding-search/maze-dataset/issues/57)\n\n# Docs\n\nThe full hosted documentation is available at [https://understanding-search.github.io/maze-dataset/](https://understanding-search.github.io/maze-dataset/).\n\nAdditionally, our [notebooks](https://understanding-search.github.io/maze-dataset/notebooks) serve as a good starting point for understanding the package.\n\n# Usage\n\n## Creating a dataset\n\nTo create a `MazeDataset`, you first create a `MazeDatasetConfig`:\n\n```python\nfrom maze_dataset import MazeDataset, MazeDatasetConfig\nfrom maze_dataset.generation import LatticeMazeGenerators\ncfg: MazeDatasetConfig = MazeDatasetConfig(\n\tname=\"test\", # name is only for you to keep track of things\n\tgrid_n=5, # number of rows/columns in the lattice\n\tn_mazes=4, # number of mazes to generate\n\tmaze_ctor=LatticeMazeGenerators.gen_dfs, # algorithm to generate the maze\n maze_ctor_kwargs=dict(do_forks=False), # additional parameters to pass to the maze generation algorithm\n)\n```\n\nand then pass this config to the `MazeDataset.from_config` method:\n\n```python\ndataset: MazeDataset = MazeDataset.from_config(cfg)\n```\n\nThis method can search for whether a dataset with matching config hash already exists on your filesystem in the expected location, and load it if so. It can also generate a dataset on the fly if needed.\n\n## Conversions to useful formats\n\nThe elements of the dataset are [`SolvedMaze`](maze_dataset/maze/lattice_maze.py) objects:\n```python\n>>> m = dataset[0]\n>>> type(m)\nmaze_dataset.maze.lattice_maze.SolvedMaze\n```\n\nWhich can be converted to a variety of formats:\n```python\n# visual representation as ascii art\nprint(m.as_ascii()) \n# RGB image, optionally without solution or endpoints, suitable for CNNs\nimport matplotlib.pyplot as plt\nplt.imshow(m.as_pixels())\n# text format for autoreregressive transformers\nfrom maze_dataset.tokenization import MazeTokenizerModular, TokenizationMode, PromptSequencers\nm.as_tokens(maze_tokenizer=MazeTokenizerModular(\n\tprompt_sequencer=PromptSequencers.AOTP(), # many options here\n))\n# advanced visualization with many features\nfrom maze_dataset.plotting import MazePlot\nMazePlot(m).plot()\n```\n\n<img src=\"https://understanding-search.github.io/maze-dataset/assets/output_formats.png\" alt=\"textual and visual output formats\" width=\"100%\"/>\n\n\n# Development\n\nWe use this [makefile template](https://github.com/mivanit/python-project-makefile-template) with slight modifications for our development workflow. This project uses [uv](https://docs.astral.sh/uv/) for dependency and virtual environment management.\n\n- clone with `git clone https://github.com/understanding-search/maze-dataset`\n- if you don't already have uv, [install it](https://docs.astral.sh/uv/getting-started/installation/). We only guarantee compatibility with `uv` newer than `0.8.0`\n- `make dep` to install all dependencies\n- `make help` will print all available commands\n- `make test` will run basic tests to ensure the package is working\n - run just the unit tests with `make test-unit`\n - see all tests with explanations using `make help` or `make help | grep test`\n- `make format` will run ruff to format and check the code\n\n> Note: due to compatibility issues between the `rust_fst` package and Darwin/macOS systems, not all tests will pass on these systems. However, `make test-unit` and `make test-notebooks-muutils` should still pass. Please see [#57](https://github.com/understanding-search/maze-dataset/issues/57) for updates on resolving this problem.\n\n## Contributing\n\nWe welcome contributions! We use [GitHub issues](https://github.com/understanding-search/maze-dataset/issues) to track bugs and feature requests. If you have a bug fix or a new feature to contribute, please open a [pull request](https://github.com/understanding-search/maze-dataset/pulls). We are also happy to provide usage support and answer questions about the package via issues!\n\nWhile we expect that the core interface of the package is stable, we are very open to adding new features. We're particularly excited about adding [new maze generation algorithms](https://github.com/understanding-search/maze-dataset/issues?q=is%3Aissue%20state%3Aopen%20label%3Ageneration) and [new output formats](https://github.com/understanding-search/maze-dataset/issues?q=is%3Aissue%20state%3Aopen%20label%3Aexporting). Please feel free to both suggest new formats or algorithms, and to implement them and open PRs! For more info on how to add a new maze generation algorithm, see the [documentation on generators](https://understanding-search.github.io/maze-dataset/maze_dataset/generation.html).\n\nWe are also aware that like any piece of software, `maze-dataset` is not without bugs. If something isn't working as expected, please open an issue and we will do our best to fix it. It helps us keep things tidy if you first search [existing bug reports](https://github.com/understanding-search/maze-dataset/issues?q=label%3Abug) to see if your issue has already been reported.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "generating and working with datasets of mazes",
"version": "1.4.0",
"project_urls": {
"Documentation": "https://understanding-search.github.io/maze-dataset/",
"Homepage": "https://github.com/understanding-search/maze-dataset",
"Issues": "https://github.com/understanding-search/maze-dataset/issues",
"Repository": "https://github.com/understanding-search/maze-dataset"
},
"split_keywords": [
"dataset",
" labyrinth",
" maze",
" mazes",
" pathfinding",
" procedural",
" tokenization"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "5bcfcde17495a8fb674d8708a4f9587e2270a76892ec8858d6fe0a766d363b16",
"md5": "bb90afa86b3f5035e2444c293476a975",
"sha256": "4717e4f6abb95ee4ec03d437c0d4e3bf0e0a5355663cfccb5302f26a3370c763"
},
"downloads": -1,
"filename": "maze_dataset-1.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bb90afa86b3f5035e2444c293476a975",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 145576,
"upload_time": "2025-08-06T23:08:54",
"upload_time_iso_8601": "2025-08-06T23:08:54.980108Z",
"url": "https://files.pythonhosted.org/packages/5b/cf/cde17495a8fb674d8708a4f9587e2270a76892ec8858d6fe0a766d363b16/maze_dataset-1.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "0fa1ea8c35fc2dd134dee0b4e37159f70125a26f9f81f102c2294d6c3b46cada",
"md5": "35f5b7b878dbc1e34ac5a4416ceb31a2",
"sha256": "7288c763b4656f23dfa3621474378febf7a43c6bf1b4ad8627e681a9bd98349b"
},
"downloads": -1,
"filename": "maze_dataset-1.4.0.tar.gz",
"has_sig": false,
"md5_digest": "35f5b7b878dbc1e34ac5a4416ceb31a2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 16724929,
"upload_time": "2025-08-06T23:08:57",
"upload_time_iso_8601": "2025-08-06T23:08:57.635330Z",
"url": "https://files.pythonhosted.org/packages/0f/a1/ea8c35fc2dd134dee0b4e37159f70125a26f9f81f102c2294d6c3b46cada/maze_dataset-1.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-06 23:08:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "understanding-search",
"github_project": "maze-dataset",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "maze-dataset"
}
JSON
