# mesa-frames 🚀
mesa-frames is an extension of the [mesa](https://github.com/projectmesa/mesa) framework, designed for complex simulations with thousands of agents. By storing agents in a DataFrame, mesa-frames significantly enhances the performance and scalability of mesa, while maintaining a similar syntax. mesa-frames allows for the use of [vectorized functions](https://stackoverflow.com/a/1422198) which significantly speeds up operations whenever simultaneous activation of agents is possible.
## Why DataFrames? 📊
DataFrames are optimized for simultaneous operations through [SIMD processing](https://en.wikipedia.org/wiki/Single_instruction,_multiple_data). At the moment, mesa-frames supports the use of two main libraries: pandas and Polars.
- [pandas](https://pandas.pydata.org/) is a popular data-manipulation Python library, developed using C and Cython. pandas is known for its ease of use, allowing for declarative programming and high performance.
- [Polars](https://pola.rs/) is a new DataFrame library with a syntax similar to pandas but with several innovations, including a backend implemented in Rust, the Apache Arrow memory format, query optimization, and support for larger-than-memory DataFrames.
The following is a performance graph showing execution time using mesa and mesa-frames for the [Boltzmann Wealth model](https://mesa.readthedocs.io/en/stable/tutorials/intro_tutorial.html).
(The script used to generate the graph can be found [here](https://github.com/adamamer20/mesa_frames/blob/main/examples/boltzmann_wealth/performance_plot.py), but if you want to additionally compare vs Mesa, you have to uncomment `mesa_implementation` and its label)
## Installation
### Cloning the Repository
To get started with mesa-frames, first clone the repository from GitHub:
```bash
git clone https://github.com/adamamer20/mesa_frames.git
cd mesa_frames
```
### Installing in a Conda Environment
If you want to install it into a new environment:
```bash
conda create -n myenv
```
If you want to install it into an existing environment:
```bash
conda activate myenv
```
Then, to install mesa-frames itself:
```bash
pip install -e .
```
### Installing in a Python Virtual Environment
If you want to install it into a new environment:
```bash
python3 -m venv myenv
source myenv/bin/activate # On Windows, use `myenv\Scripts\activate`
```
If you want to install it into an existing environment:
```bash
source myenv/bin/activate # On Windows, use `myenv\Scripts\activate`
```
Then, to install mesa-frames itself:
```bash
pip install -e .
```
## Usage
**Note:** mesa-frames is currently in its early stages of development. As such, the usage patterns and API are subject to change. Breaking changes may be introduced. Reports of feedback and issues are encouraged.
You can find the API documentation [here](https://adamamer20.github.io/mesa-frames/api).
### Creation of an Agent
The agent implementation differs from base mesa. Agents are only defined at the AgentSet level. You can import either `AgentSetPandas` or `AgentSetPolars`. As in mesa, you subclass and make sure to call `super().__init__(model)`. You can use the `add` method or the `+=` operator to add agents to the AgentSet. Most methods mirror the functionality of `mesa.AgentSet`. Additionally, `mesa-frames.AgentSet` implements many dunder methods such as `AgentSet[mask, attr]` to get and set items intuitively. All operations are by default inplace, but if you'd like to use functional programming, mesa-frames implements a fast copy method which aims to reduce memory usage, relying on reference-only and native copy methods.
```python
from mesa-frames import AgentSetPolars
class MoneyAgentPolars(AgentSetPolars):
def __init__(self, n: int, model: ModelDF):
super().__init__(model)
# Adding the agents to the agent set
self += pl.DataFrame(
{"unique_id": pl.arange(n, eager=True), "wealth": pl.ones(n, eager=True)}
)
def step(self) -> None:
# The give_money method is called
self.do("give_money")
def give_money(self):
# Active agents are changed to wealthy agents
self.select(self.wealth > 0)
# Receiving agents are sampled (only native expressions currently supported)
other_agents = self.agents.sample(
n=len(self.active_agents), with_replacement=True
)
# Wealth of wealthy is decreased by 1
self["active", "wealth"] -= 1
# Compute the income of the other agents (only native expressions currently supported)
new_wealth = other_agents.group_by("unique_id").len()
# Add the income to the other agents
self[new_wealth, "wealth"] += new_wealth["len"]
```
### Creation of the Model
Creation of the model is fairly similar to the process in mesa. You subclass `ModelDF` and call `super().__init__()`. The `model.agents` attribute has the same interface as `mesa-frames.AgentSet`. You can use `+=` or `self.agents.add` with a `mesa-frames.AgentSet` (or a list of `AgentSet`) to add agents to the model.
```python
from mesa-frames import ModelDF
class MoneyModelDF(ModelDF):
def __init__(self, N: int, agents_cls):
super().__init__()
self.n_agents = N
self.agents += MoneyAgentPolars(N, self)
def step(self):
# Executes the step method for every agentset in self.agents
self.agents.do("step")
def run_model(self, n):
for _ in range(n):
self.step()
```
## What's Next? 🔮
- Refine the API to make it more understandable for someone who is already familiar with the mesa package. The goal is to provide a seamless experience for users transitioning to or incorporating mesa-frames.
- Adding support for default mesa functions to ensure that the standard mesa functionality is preserved.
- Adding GPU functionality (cuDF and Dask-cuDF).
- Creating a decorator that will automatically vectorize an existing mesa model. This feature will allow users to easily tap into the performance enhancements that mesa-frames offers without significant code alterations.
- Creating a unique class for AgentSet, independent of the backend implementation.
## License
mesa-frames is made available under the MIT License. This license allows you to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, subject to the following conditions:
- The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
- The software is provided "as is", without warranty of any kind.
For the full license text, see the [LICENSE](https://github.com/adamamer20/mesa_frames/blob/main/LICENSE) file in the GitHub repository.
Raw data
{
"_id": null,
"home_page": null,
"name": "mesa-frames",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "agent-based-modeling, agent-based-modelling, complex-systems, complexity-analysis, gis, mesa, modeling-agents, pandas, simulation, simulation-environment, simulation-framework, spatial-models",
"author": "Adam Amer",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/47/f4/39aa8de7b01a188e8a8c30db20cd8111202621ff8100dc61b4544db5438c/mesa_frames-0.1.0a0.tar.gz",
"platform": null,
"description": "# mesa-frames \ud83d\ude80\n\nmesa-frames is an extension of the [mesa](https://github.com/projectmesa/mesa) framework, designed for complex simulations with thousands of agents. By storing agents in a DataFrame, mesa-frames significantly enhances the performance and scalability of mesa, while maintaining a similar syntax. mesa-frames allows for the use of [vectorized functions](https://stackoverflow.com/a/1422198) which significantly speeds up operations whenever simultaneous activation of agents is possible.\n\n## Why DataFrames? \ud83d\udcca\n\nDataFrames are optimized for simultaneous operations through [SIMD processing](https://en.wikipedia.org/wiki/Single_instruction,_multiple_data). At the moment, mesa-frames supports the use of two main libraries: pandas and Polars.\n\n- [pandas](https://pandas.pydata.org/) is a popular data-manipulation Python library, developed using C and Cython. pandas is known for its ease of use, allowing for declarative programming and high performance.\n- [Polars](https://pola.rs/) is a new DataFrame library with a syntax similar to pandas but with several innovations, including a backend implemented in Rust, the Apache Arrow memory format, query optimization, and support for larger-than-memory DataFrames.\n\nThe following is a performance graph showing execution time using mesa and mesa-frames for the [Boltzmann Wealth model](https://mesa.readthedocs.io/en/stable/tutorials/intro_tutorial.html).\n\n\n\n\n\n(The script used to generate the graph can be found [here](https://github.com/adamamer20/mesa_frames/blob/main/examples/boltzmann_wealth/performance_plot.py), but if you want to additionally compare vs Mesa, you have to uncomment `mesa_implementation` and its label)\n\n## Installation\n\n### Cloning the Repository\n\nTo get started with mesa-frames, first clone the repository from GitHub:\n\n```bash\ngit clone https://github.com/adamamer20/mesa_frames.git\ncd mesa_frames\n```\n\n### Installing in a Conda Environment\n\nIf you want to install it into a new environment:\n\n```bash\nconda create -n myenv\n```\n\nIf you want to install it into an existing environment:\n\n```bash\nconda activate myenv\n```\n\nThen, to install mesa-frames itself:\n\n```bash\npip install -e .\n```\n\n### Installing in a Python Virtual Environment\n\nIf you want to install it into a new environment:\n\n```bash\npython3 -m venv myenv\nsource myenv/bin/activate # On Windows, use `myenv\\Scripts\\activate`\n```\n\nIf you want to install it into an existing environment:\n\n```bash\nsource myenv/bin/activate # On Windows, use `myenv\\Scripts\\activate`\n```\n\nThen, to install mesa-frames itself:\n\n```bash\npip install -e .\n```\n\n## Usage\n\n**Note:** mesa-frames is currently in its early stages of development. As such, the usage patterns and API are subject to change. Breaking changes may be introduced. Reports of feedback and issues are encouraged.\n\nYou can find the API documentation [here](https://adamamer20.github.io/mesa-frames/api).\n\n### Creation of an Agent\n\nThe agent implementation differs from base mesa. Agents are only defined at the AgentSet level. You can import either `AgentSetPandas` or `AgentSetPolars`. As in mesa, you subclass and make sure to call `super().__init__(model)`. You can use the `add` method or the `+=` operator to add agents to the AgentSet. Most methods mirror the functionality of `mesa.AgentSet`. Additionally, `mesa-frames.AgentSet` implements many dunder methods such as `AgentSet[mask, attr]` to get and set items intuitively. All operations are by default inplace, but if you'd like to use functional programming, mesa-frames implements a fast copy method which aims to reduce memory usage, relying on reference-only and native copy methods.\n\n```python\nfrom mesa-frames import AgentSetPolars\n\nclass MoneyAgentPolars(AgentSetPolars):\n def __init__(self, n: int, model: ModelDF):\n super().__init__(model)\n # Adding the agents to the agent set\n self += pl.DataFrame(\n {\"unique_id\": pl.arange(n, eager=True), \"wealth\": pl.ones(n, eager=True)}\n )\n\n def step(self) -> None:\n # The give_money method is called\n self.do(\"give_money\")\n\n def give_money(self):\n # Active agents are changed to wealthy agents\n self.select(self.wealth > 0)\n\n # Receiving agents are sampled (only native expressions currently supported)\n other_agents = self.agents.sample(\n n=len(self.active_agents), with_replacement=True\n )\n\n # Wealth of wealthy is decreased by 1\n self[\"active\", \"wealth\"] -= 1\n\n # Compute the income of the other agents (only native expressions currently supported)\n new_wealth = other_agents.group_by(\"unique_id\").len()\n\n # Add the income to the other agents\n self[new_wealth, \"wealth\"] += new_wealth[\"len\"]\n```\n\n### Creation of the Model\n\nCreation of the model is fairly similar to the process in mesa. You subclass `ModelDF` and call `super().__init__()`. The `model.agents` attribute has the same interface as `mesa-frames.AgentSet`. You can use `+=` or `self.agents.add` with a `mesa-frames.AgentSet` (or a list of `AgentSet`) to add agents to the model.\n\n```python\nfrom mesa-frames import ModelDF\n\nclass MoneyModelDF(ModelDF):\n def __init__(self, N: int, agents_cls):\n super().__init__()\n self.n_agents = N\n self.agents += MoneyAgentPolars(N, self)\n\n def step(self):\n # Executes the step method for every agentset in self.agents\n self.agents.do(\"step\")\n\n def run_model(self, n):\n for _ in range(n):\n self.step()\n```\n\n## What's Next? \ud83d\udd2e\n\n- Refine the API to make it more understandable for someone who is already familiar with the mesa package. The goal is to provide a seamless experience for users transitioning to or incorporating mesa-frames.\n- Adding support for default mesa functions to ensure that the standard mesa functionality is preserved.\n- Adding GPU functionality (cuDF and Dask-cuDF).\n- Creating a decorator that will automatically vectorize an existing mesa model. This feature will allow users to easily tap into the performance enhancements that mesa-frames offers without significant code alterations.\n- Creating a unique class for AgentSet, independent of the backend implementation.\n\n## License\n\nmesa-frames is made available under the MIT License. This license allows you to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, subject to the following conditions:\n\n- The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n- The software is provided \"as is\", without warranty of any kind.\n\nFor the full license text, see the [LICENSE](https://github.com/adamamer20/mesa_frames/blob/main/LICENSE) file in the GitHub repository.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "An extension to the Mesa framework which uses pandas/Polars DataFrames for enhanced performance",
"version": "0.1.0a0",
"project_urls": {
"Documentation": "https://adamamer20.github.io/mesa-frames",
"Repository": "https://github.com/adamamer20/mesa-frames.git"
},
"split_keywords": [
"agent-based-modeling",
" agent-based-modelling",
" complex-systems",
" complexity-analysis",
" gis",
" mesa",
" modeling-agents",
" pandas",
" simulation",
" simulation-environment",
" simulation-framework",
" spatial-models"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5f621dba443df57f57f0d27108add6da1e5f6cc1abe7d650a82ac034abcb78b5",
"md5": "4409d7592cf02b1941aadafa6ad8f4e4",
"sha256": "3534e2aee919fce28f711864d8e43668a292d6b26e2e015c52558f003d30462c"
},
"downloads": -1,
"filename": "mesa_frames-0.1.0a0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4409d7592cf02b1941aadafa6ad8f4e4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 59726,
"upload_time": "2024-08-28T07:01:52",
"upload_time_iso_8601": "2024-08-28T07:01:52.241968Z",
"url": "https://files.pythonhosted.org/packages/5f/62/1dba443df57f57f0d27108add6da1e5f6cc1abe7d650a82ac034abcb78b5/mesa_frames-0.1.0a0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "47f439aa8de7b01a188e8a8c30db20cd8111202621ff8100dc61b4544db5438c",
"md5": "80d06095cb6ea1ff7df83154ad1dcd5a",
"sha256": "2b50c35cc2df9ddc382c4e9a3c12b5fc47c10e3a1140d978422375491acd27d0"
},
"downloads": -1,
"filename": "mesa_frames-0.1.0a0.tar.gz",
"has_sig": false,
"md5_digest": "80d06095cb6ea1ff7df83154ad1dcd5a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 282024,
"upload_time": "2024-08-28T07:01:54",
"upload_time_iso_8601": "2024-08-28T07:01:54.101672Z",
"url": "https://files.pythonhosted.org/packages/47/f4/39aa8de7b01a188e8a8c30db20cd8111202621ff8100dc61b4544db5438c/mesa_frames-0.1.0a0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-28 07:01:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "adamamer20",
"github_project": "mesa-frames",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mesa-frames"
}
JSON
