| Name | mage-ai JSON |
| Version |
0.4.15
JSON |
| download |
| home_page | https://github.com/mage-ai/mage-ai |
| Summary | Mage is a tool for building and deploying data pipelines. |
| upload_time | 2022-10-03 08:51:37 |
| maintainer | |
| docs_url | None |
| author | Mage |
| requires_python | >=3.6 |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<img alt="Fire mage" height="160" src="media/mage-fire-charging-up.svg" />
[](https://pypi.org/project/mage-ai/)
[](https://app.circleci.com/pipelines/github/mage-ai/mage-ai?branch=master&filter=all)
[](https://opensource.org/licenses/Apache-2.0)
[](https://join.slack.com/t/mageai/shared_invite/zt-1adn34w4m-t~TcnPTlo3~5~d_0raOp6A)
<img
referrerpolicy="no-referrer-when-downgrade"
src="https://static.scarf.sh/a.png?x-pxid=b3c96d79-b8f0-414b-a687-8bfc164b4b7a"
/>
# ๐ง Mage
Mage is an open-source tool for [building](docs/features/README.md#notebook-for-building-data-pipelines)
and [running](docs/features/orchestration/README.md)
data pipelines that <b>transform</b> your data.
<br />
<b>Here is a sample data pipeline defined across 3 files:</b>
```python
# data_loaders/load_data_from_file.py
@data_loader
def load_csv_from_file():
return pd.read_csv('default_repo/titanic.csv')
```
```python
# transformers/select_columns.py
@transformer
def select_columns_from_df(df, *args):
return df[['Age', 'Fare', 'Survived']]
```
```python
# data_exporters/export_to_file.py
@data_exporter
def export_titanic_data_to_disk(df) -> None:
df.to_csv('default_repo/titanic_transformed.csv')
```
<br />
<b>What the data pipeline looks like in the UI:</b>
<img
alt="data pipeline overview"
src="media/data-pipeline-overview.jpg"
/>
New? We recommend reading about [blocks](docs/blocks/README.md) and
learning from a [hands-on tutorial](docs/tutorials/quick_start/etl_restaurant/README.md).
<br />
[](https://www.mage.ai/chat)
## Table of contents
1. [Quick start](#%EF%B8%8F-quick-start)
1. [Demo](#-demo)
1. [Tutorials](#-tutorials)
1. [Features](#-features)
1. [Core design principles](#%EF%B8%8F-core-design-principles)
1. [Core abstractions](#-core-abstractions)
1. [Documentation](docs/README.md)
# ๐โโ๏ธ Quick start
Install Mage using Docker or `pip`:
### Using Docker
Create a new project and launch tool (change `demo_project` to any other name if you want):
```bash
docker run -it -p 6789:6789 -v $(pwd):/home/src \
mageai/mageai mage start demo_project
```
<sub>Follow the [guide](docs/spark/setup/README.md) if you want to use PySpark kernel in your notebook.</sub>
### Using `pip` or `conda`
##### 1. Install Mage
```bash
pip install mage-ai
```
or
```bash
conda install -c conda-forge mage-ai
```
<sub>For additional packages (e.g. `spark`, `postgres`, etc), please see [Installing extra packages](docs/README.md#installing-extra-packages).</sub>
<sub>If you run into errors, please see [Install errors](docs/tutorials/quick_start/setup.md#errors).</sub>
##### 2. Create new project and launch tool (change `demo_project` to any other name if you want):
```bash
mage start demo_project
```
### Open tool in browser
Open [http://localhost:6789](http://localhost:6789) in your browser and build a pipeline.
<br />
# ๐ฎ Demo
## Live demo
Try a hosted version of the tool here: [http://demo.mage.ai](http://demo.mage.ai/).
> WARNING
>
> The live demo is public, please donโt save anything sensitive.
## Demo video (2 min)
[](https://www.youtube.com/watch?v=hrsErfPDits "Mage quick start demo")
<sub><i>Click the image to play video</i></sub>
<br />
# ๐ฉโ๐ซ Tutorials
- [Train model on Titanic dataset](docs/tutorials/quick_start/train_titanic_model/README.md)
- [Load data from API, transform it, and export it to PostgreSQL](docs/tutorials/quick_start/etl_restaurant/README.md)
- [Integrate Mage into an existing Airflow project](docs/tutorials/airflow/integrate_into_existing_project/README.md)
<br />
# ๐ฎ Features
- [Data pipeline management](docs/features/orchestration/README.md)
- [Notebook for building data pipelines](docs/features/README.md#notebook-for-building-data-pipelines)
<sub>Read more [<b>here</b>](docs/features/README.md).</sub>
<br />
# ๐๏ธ Core design principles
Every user experience and technical design decision adheres to these principles.
1. [Easy developer experience](docs/core/design_principles.md#-easy-developer-experience)
1. [Engineering best practices built-in](docs/core/design_principles.md#engineering-best-practices-built-in)
1. [Data is a first-class citizen](docs/core/design_principles.md#data-is-a-first-class-citizen)
1. [Scaling made simple](docs/core/design_principles.md#scaling-is-made-simple)
<sub>Read more [<b>here</b>](docs/core/design_principles.md).</sub>
<br />
# ๐ธ Core abstractions
These are the fundamental concepts that Mage uses to operate.
- [Project](docs/core/abstractions.md#project)
- [Pipeline](docs/core/abstractions.md#pipeline)
- [Block](docs/core/abstractions.md#block)
- [Data product](docs/core/abstractions.md#data-product)
- [Trigger](docs/core/abstractions.md#trigger)
- [Run](docs/core/abstractions.md#run)
<sub>Read more [<b>here</b>](docs/core/abstractions.md).</sub>
<br />
# ๐ Documentation
Read more [<b>here</b>](docs/README.md).
<br />
# ๐โโ๏ธ Contributing
Check out the [๐ contributing guide](/docs/contributing/README.md) to get started
by setting up your development environment and
exploring the code base.
<br />
# ๐ค Frequently Asked Questions (FAQs)
Check out our [FAQ page](https://www.notion.so/mageai/Mage-FAQs-33d93ee65f934ed39568f8a4bc823b39) to find answers to some of our most asked questions.
<br />
# ๐ง Community
We love the community of Magers (`/หmฤjษr/`);
a group of mages who help each other realize their full potential!
To live chat with the Mage team and community,
please join the free Mage [<img alt="Slack" height="20" src="https://thepostsportsbar.com/wp-content/uploads/2017/02/Slack-Logo.png" style="position: relative; top: 4px;" /> Slack](https://www.mage.ai/chat)
channel.
[](https://www.mage.ai/chat)
For real-time news and fun memes, check out the Mage
[<img alt="Twitter" height="20" src="https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/Twitter-logo.svg/2491px-Twitter-logo.svg.png" style="position: relative; top: 4px;" /> Twitter](https://twitter.com/mage_ai).
To report bugs or add your awesome code for others to enjoy,
visit [GitHub](https://github.com/mage-ai/mage-ai).
<br />
# ๐ชช License
See the [LICENSE](LICENSE) file for licensing information.
[<img alt="Water mage casting spell" height="300" src="media/mage-water-charging-up.svg" />](https://www.mage.ai/)
Raw data
{
"_id": null,
"home_page": "https://github.com/mage-ai/mage-ai",
"name": "mage-ai",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Mage",
"author_email": "eng@mage.ai",
"download_url": "https://files.pythonhosted.org/packages/4d/34/e22927a0429a9ef6155afeeafa801a3c10f12ccd0ad8184f26685e739454/mage-ai-0.4.15.tar.gz",
"platform": null,
"description": "<img alt=\"Fire mage\" height=\"160\" src=\"media/mage-fire-charging-up.svg\" />\n\n[](https://pypi.org/project/mage-ai/)\n[](https://app.circleci.com/pipelines/github/mage-ai/mage-ai?branch=master&filter=all)\n[](https://opensource.org/licenses/Apache-2.0)\n[](https://join.slack.com/t/mageai/shared_invite/zt-1adn34w4m-t~TcnPTlo3~5~d_0raOp6A)\n<img\n referrerpolicy=\"no-referrer-when-downgrade\"\n src=\"https://static.scarf.sh/a.png?x-pxid=b3c96d79-b8f0-414b-a687-8bfc164b4b7a\"\n/>\n\n# \ud83e\uddd9 Mage\n\nMage is an open-source tool for [building](docs/features/README.md#notebook-for-building-data-pipelines)\nand [running](docs/features/orchestration/README.md)\ndata pipelines that <b>transform</b> your data.\n\n<br />\n\n<b>Here is a sample data pipeline defined across 3 files:</b>\n\n```python\n# data_loaders/load_data_from_file.py\n@data_loader\ndef load_csv_from_file():\n return pd.read_csv('default_repo/titanic.csv')\n```\n\n```python\n# transformers/select_columns.py\n@transformer\ndef select_columns_from_df(df, *args):\n return df[['Age', 'Fare', 'Survived']]\n```\n\n```python\n# data_exporters/export_to_file.py\n@data_exporter\ndef export_titanic_data_to_disk(df) -> None:\n df.to_csv('default_repo/titanic_transformed.csv')\n```\n\n<br />\n\n<b>What the data pipeline looks like in the UI:</b>\n\n<img\n alt=\"data pipeline overview\"\n src=\"media/data-pipeline-overview.jpg\"\n/>\n\nNew? We recommend reading about [blocks](docs/blocks/README.md) and\nlearning from a [hands-on tutorial](docs/tutorials/quick_start/etl_restaurant/README.md).\n\n<br />\n\n[](https://www.mage.ai/chat)\n\n## Table of contents\n\n1. [Quick start](#%EF%B8%8F-quick-start)\n1. [Demo](#-demo)\n1. [Tutorials](#-tutorials)\n1. [Features](#-features)\n1. [Core design principles](#%EF%B8%8F-core-design-principles)\n1. [Core abstractions](#-core-abstractions)\n1. [Documentation](docs/README.md)\n\n# \ud83c\udfc3\u200d\u2640\ufe0f Quick start\n\nInstall Mage using Docker or `pip`:\n\n### Using Docker\n\nCreate a new project and launch tool (change `demo_project` to any other name if you want):\n\n```bash\ndocker run -it -p 6789:6789 -v $(pwd):/home/src \\\n mageai/mageai mage start demo_project\n```\n\n<sub>Follow the [guide](docs/spark/setup/README.md) if you want to use PySpark kernel in your notebook.</sub>\n\n### Using `pip` or `conda`\n\n##### 1. Install Mage\n```bash\npip install mage-ai\n```\n\nor\n\n```bash\nconda install -c conda-forge mage-ai\n```\n\n<sub>For additional packages (e.g. `spark`, `postgres`, etc), please see [Installing extra packages](docs/README.md#installing-extra-packages).</sub>\n\n<sub>If you run into errors, please see [Install errors](docs/tutorials/quick_start/setup.md#errors).</sub>\n\n##### 2. Create new project and launch tool (change `demo_project` to any other name if you want):\n\n```bash\nmage start demo_project\n```\n\n### Open tool in browser\n\nOpen [http://localhost:6789](http://localhost:6789) in your browser and build a pipeline.\n\n<br />\n\n# \ud83c\udfae Demo\n\n## Live demo\n\nTry a hosted version of the tool here: [http://demo.mage.ai](http://demo.mage.ai/).\n\n> WARNING\n>\n> The live demo is public, please don\u2019t save anything sensitive.\n\n## Demo video (2 min)\n\n[](https://www.youtube.com/watch?v=hrsErfPDits \"Mage quick start demo\")\n\n<sub><i>Click the image to play video</i></sub>\n\n<br />\n\n# \ud83d\udc69\u200d\ud83c\udfeb Tutorials\n\n- [Train model on Titanic dataset](docs/tutorials/quick_start/train_titanic_model/README.md)\n- [Load data from API, transform it, and export it to PostgreSQL](docs/tutorials/quick_start/etl_restaurant/README.md)\n- [Integrate Mage into an existing Airflow project](docs/tutorials/airflow/integrate_into_existing_project/README.md)\n\n<br />\n\n# \ud83d\udd2e Features\n\n- [Data pipeline management](docs/features/orchestration/README.md)\n- [Notebook for building data pipelines](docs/features/README.md#notebook-for-building-data-pipelines)\n\n<sub>Read more [<b>here</b>](docs/features/README.md).</sub>\n\n<br />\n\n# \ud83c\udfd4\ufe0f Core design principles\n\nEvery user experience and technical design decision adheres to these principles.\n\n1. [Easy developer experience](docs/core/design_principles.md#-easy-developer-experience)\n1. [Engineering best practices built-in](docs/core/design_principles.md#engineering-best-practices-built-in)\n1. [Data is a first-class citizen](docs/core/design_principles.md#data-is-a-first-class-citizen)\n1. [Scaling made simple](docs/core/design_principles.md#scaling-is-made-simple)\n\n<sub>Read more [<b>here</b>](docs/core/design_principles.md).</sub>\n\n<br />\n\n# \ud83d\udef8 Core abstractions\n\nThese are the fundamental concepts that Mage uses to operate.\n\n- [Project](docs/core/abstractions.md#project)\n- [Pipeline](docs/core/abstractions.md#pipeline)\n- [Block](docs/core/abstractions.md#block)\n- [Data product](docs/core/abstractions.md#data-product)\n- [Trigger](docs/core/abstractions.md#trigger)\n- [Run](docs/core/abstractions.md#run)\n\n<sub>Read more [<b>here</b>](docs/core/abstractions.md).</sub>\n\n<br />\n\n# \ud83d\udcda Documentation\n\nRead more [<b>here</b>](docs/README.md).\n\n<br />\n\n# \ud83d\ude4b\u200d\u2640\ufe0f Contributing\n\nCheck out the [\ud83c\udf81 contributing guide](/docs/contributing/README.md) to get started\nby setting up your development environment and\nexploring the code base.\n\n<br />\n\n# \ud83e\udd14 Frequently Asked Questions (FAQs)\n\nCheck out our [FAQ page](https://www.notion.so/mageai/Mage-FAQs-33d93ee65f934ed39568f8a4bc823b39) to find answers to some of our most asked questions.\n\n<br />\n\n# \ud83e\uddd9 Community\nWe love the community of Magers (`/\u02c8m\u0101j\u0259r/`);\na group of mages who help each other realize their full potential!\n\nTo live chat with the Mage team and community,\nplease join the free Mage [<img alt=\"Slack\" height=\"20\" src=\"https://thepostsportsbar.com/wp-content/uploads/2017/02/Slack-Logo.png\" style=\"position: relative; top: 4px;\" /> Slack](https://www.mage.ai/chat)\nchannel.\n\n[](https://www.mage.ai/chat)\n\nFor real-time news and fun memes, check out the Mage\n[<img alt=\"Twitter\" height=\"20\" src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/Twitter-logo.svg/2491px-Twitter-logo.svg.png\" style=\"position: relative; top: 4px;\" /> Twitter](https://twitter.com/mage_ai).\n\nTo report bugs or add your awesome code for others to enjoy,\nvisit [GitHub](https://github.com/mage-ai/mage-ai).\n\n<br />\n\n# \ud83e\udeaa License\nSee the [LICENSE](LICENSE) file for licensing information.\n\n[<img alt=\"Water mage casting spell\" height=\"300\" src=\"media/mage-water-charging-up.svg\" />](https://www.mage.ai/)\n",
"bugtrack_url": null,
"license": "",
"summary": "Mage is a tool for building and deploying data pipelines.",
"version": "0.4.15",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "c88c5aa39f5334ca6f8a7d42f13f2425",
"sha256": "7e967e10e208dd88c32b149b1b635d543c78ff342368c9b93c55cdf4686a28f2"
},
"downloads": -1,
"filename": "mage_ai-0.4.15-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c88c5aa39f5334ca6f8a7d42f13f2425",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 6467423,
"upload_time": "2022-10-03T08:51:35",
"upload_time_iso_8601": "2022-10-03T08:51:35.735585Z",
"url": "https://files.pythonhosted.org/packages/4d/d9/5f8f2de87e3777196f530a32a071c924c44af85dac28c960d23e20f856ad/mage_ai-0.4.15-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "0abddf437ec995d0f1b978353126dc05",
"sha256": "36a947bf98a497689a3b45bc499a310c941be93fc65d2161fd68cbe836598b37"
},
"downloads": -1,
"filename": "mage-ai-0.4.15.tar.gz",
"has_sig": false,
"md5_digest": "0abddf437ec995d0f1b978353126dc05",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 6225798,
"upload_time": "2022-10-03T08:51:37",
"upload_time_iso_8601": "2022-10-03T08:51:37.883459Z",
"url": "https://files.pythonhosted.org/packages/4d/34/e22927a0429a9ef6155afeeafa801a3c10f12ccd0ad8184f26685e739454/mage-ai-0.4.15.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-10-03 08:51:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "mage-ai",
"github_project": "mage-ai",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "mage-ai"
}
