# Airflow Code Editor Plugin
This plugin for [Apache Airflow](https://github.com/apache/airflow) allows you to edit DAGs directly within your browser,
providing a seamless and efficient workflow for managing your pipelines.
Offering a user-friendly file management interface within designated directories, it facilitates effortless editing,
uploading, and downloading of files.
With Git support enabled, DAGs are stored in a Git repository, enabling users to explore Git history, review local modifications, and commit changes.
[](https://github.com/andreax79/airflow-code-editor/actions)
[](https://badge.fury.io/py/airflow-code-editor)
[](https://pypi.org/project/airflow-code-editor)
[](https://pepy.tech/project/airflow-code-editor)
[](https://github.com/psf/black)
### System Requirements
* Airflow Versions
* 1.10.3 or newer
* git Versions (git is not required if git support is disabled)
* 2.0 or newer
### Screenshots
#### File Manager
#### Editor
#### Search
#### Git History
#### Git Workspace
### Install Instructions
#### Docker Images
For the ease of deployment, use the production-ready reference container image.
The image is based on the reference images for Apache Airflow.
You can find the following images there:
* andreax79/airflow-code-editor:**latest** - the latest released Airflow Code Editor image with the latest Apache Airflow version
* andreax79/airflow-code-editor:**2.10.0** - the latest released Airflow Code Editor with specific Airflow version
* andreax79/airflow-code-editor:**2.10.0-7.7.0** - specific version of Airflow and Airflow Code Editor
#### Installing from PyPI
1. Install the plugin
```bash
pip install airflow-code-editor
```
2. Install optional dependencies
* black - Black Python code formatter
* isort - A Python utility/library to sort imports
* fs-s3fs - S3FS Amazon S3 Filesystem
* fs-gcsfs - Google Cloud Storage Filesystem
* ... other filesystems supported by PyFilesystem - see https://www.pyfilesystem.org/page/index-of-filesystems/
```bash
pip install black isort fs-s3fs fs-gcsfs
```
3. Restart the Airflow Web Server
4. Open Admin - DAGs Code Editor
### Config Options
You can set options editing the Airflow's configuration file or setting environment variables.
You can edit your *airflow.cfg* adding any of the following settings in the \[code_editor\] section.
All the settings are optional.
* **enabled** enable this plugin (default: True).
* **git_enabled** enable git support (default: True). If git is not installed, disable this option.
* **git_cmd** git command (path)
* **git_default_args** git arguments added to each call (default: -c color.ui=true)
* **git_author_name** human-readable name in the author/committer (default logged user first and last names)
* **git_author_email** email for the author/committer (default: logged user email)
* **git_init_repo** initialize a git repo in DAGs folder (default: True)
* **root_directory** root folder (default: Airflow DAGs folder)
* **line_length** Python code formatter - max line length (default: 88)
* **string_normalization** Python code formatter - if true normalize string quotes and prefixes (default: False)
* **mount**, **mount1**, ... configure additional folder (mount point) - format: name=xxx,path=yyy
* **ignored_entries** comma-separated list of entries to be excluded from file/directory list (default: .\*,\_\_pycache\_\_)
```
[code_editor]
enabled = True
git_enabled = True
git_cmd = /usr/bin/git
git_default_args = -c color.ui=true
git_init_repo = False
root_directory = /home/airflow/dags
line_length = 88
string_normalization = False
mount = name=data,path=/home/airflow/data
mount1 = name=logs,path=/home/airflow/logs
mount2 = name=data,path=s3://example
```
Mount Options:
* **name**: mount name (destination)
* **path**: local path or PyFilesystem FS URLs - see https://docs.pyfilesystem.org/en/latest/openers.html
Example:
* name=ftp_server,path=ftp://user:pass@ftp.example.org/private
* name=data,path=s3://example
* name=tmp,path=/tmp
You can also set options with the following environment variables:
* AIRFLOW__CODE_EDITOR__ENABLED
* AIRFLOW__CODE_EDITOR__GIT_ENABLED
* AIRFLOW__CODE_EDITOR__GIT_CMD
* AIRFLOW__CODE_EDITOR__GIT_DEFAULT_ARGS
* AIRFLOW__CODE_EDITOR__GIT_AUTHOR_NAME
* AIRFLOW__CODE_EDITOR__GIT_AUTHOR_EMAIL
* AIRFLOW__CODE_EDITOR__GIT_INIT_REPO
* AIRFLOW__CODE_EDITOR__ROOT_DIRECTORY
* AIRFLOW__CODE_EDITOR__LINE_LENGTH
* AIRFLOW__CODE_EDITOR__STRING_NORMALIZATION
* AIRFLOW__CODE_EDITOR__MOUNT, AIRFLOW__CODE_EDITOR__MOUNT1, AIRFLOW__CODE_EDITOR__MOUNT2, ...
* AIRFLOW__CODE_EDITOR__IGNORED_ENTRIES
Example:
```
export AIRFLOW__CODE_EDITOR__STRING_NORMALIZATION=True
export AIRFLOW__CODE_EDITOR__MOUNT='name=data,path=/home/airflow/data'
export AIRFLOW__CODE_EDITOR__MOUNT1='name=logs,path=/home/airflow/logs'
export AIRFLOW__CODE_EDITOR__MOUNT2='name=tmp,path=/tmp'
```
### REST API
Airflow Code Editor provides a REST API. Through this API, users can interact with the application
programmatically, enabling automation, data retrieval, and integration with other software.
For detailed information on how to use each endpoint, refer to the
[API documentation](https://petstore.swagger.io/?url=https://raw.githubusercontent.com/andreax79/airflow-code-editor/main/airflow_code_editor/api/code_editor.yaml).
### REST API Authentication
The API authentication is inherited from the Apache Airflow.
If you want to check which auth backend is currently set, you can use
`airflow config get-value api auth_backends` command as in the example below.
```bash
$ airflow config get-value api auth_backends
airflow.api.auth.backend.basic_auth
```
For details on configuring the authentication, see
[API Authorization](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html).
### Development Instructions
1. Fork the repo
2. Clone it on the local machine
```bash
git clone https://github.com/andreax79/airflow-code-editor.git
cd airflow-code-editor
```
3. Create dev image
```bash
make dev-image
```
4. Switch node version
```bash
nvm use
```
5. Make changes you need. Build npm package with:
```bash
make npm-build
```
6. You can start Airflow webserver with:
```bash
make webserver
```
7. Run tests
```bash
make test
```
8. Commit and push changes
9. Create [pull request](https://github.com/andreax79/airflow-code-editor/compare) to the original repo
### Links
* [Apache Airflow](https://github.com/apache/airflow)
* [Codemirror, In-browser code editor](https://github.com/codemirror/codemirror)
* [Git WebUI, A standalone local web based user interface for git repositories](https://github.com/alberthier/git-webui)
* [Black, The Uncompromising Code Formatter](https://github.com/psf/black)
* [isort, A Python utility/library to sort imports](https://github.com/psf/black)
* [pss, power-tool for searching source files](https://github.com/eliben/pss)
* [Vue.js](https://github.com/vuejs/vue)
* [Vue-good-table, data table for VueJS](https://github.com/xaksis/vue-good-table)
* [Vue-tree, TreeView control for VueJS](https://github.com/grapoza/vue-tree)
* [Vue-universal-modal Universal modal plugin for Vue@3](https://github.com/hoiheart/vue-universal-modal)
* [Vue-simple-context-menu](https://github.com/johndatserakis/vue-simple-context-menu)
* [Splitpanes](https://github.com/antoniandre/splitpanes)
* [Axios, Promise based HTTP client for the browser and node.js](https://github.com/axios/axios)
* [PyFilesystem2, Python's Filesystem abstraction layer](https://github.com/PyFilesystem/pyfilesystem2)
* [Amazon S3 PyFilesystem](https://github.com/PyFilesystem/s3fs)
* [Google Cloud Storage PyFilesystem](https://github.com/Othoz/gcsfs)
Raw data
{
"_id": null,
"home_page": "https://github.com/andreax79/airflow-code-editor",
"name": "airflow-code-editor",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "airflow, airflow-plugin, apache-airflow, editor",
"author": "Andrea Bonomi",
"author_email": "andrea.bonomi@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/66/39/e435a4ec5c93a2d688bf852b6a05f587098e7b93df24ab7fbe94322d639d/airflow_code_editor-7.7.0.tar.gz",
"platform": "any",
"description": "# Airflow Code Editor Plugin\n\nThis plugin for [Apache Airflow](https://github.com/apache/airflow) allows you to edit DAGs directly within your browser,\nproviding a seamless and efficient workflow for managing your pipelines.\nOffering a user-friendly file management interface within designated directories, it facilitates effortless editing,\nuploading, and downloading of files.\nWith Git support enabled, DAGs are stored in a Git repository, enabling users to explore Git history, review local modifications, and commit changes.\n\n[](https://github.com/andreax79/airflow-code-editor/actions)\n[](https://badge.fury.io/py/airflow-code-editor)\n[](https://pypi.org/project/airflow-code-editor)\n[](https://pepy.tech/project/airflow-code-editor)\n[](https://github.com/psf/black)\n\n### System Requirements\n\n* Airflow Versions\n * 1.10.3 or newer\n* git Versions (git is not required if git support is disabled)\n * 2.0 or newer\n\n### Screenshots\n\n#### File Manager\n\n\n\n#### Editor\n\n\n\n#### Search\n\n\n\n#### Git History\n\n\n\n#### Git Workspace\n\n\n\n\n### Install Instructions\n\n#### Docker Images\n\nFor the ease of deployment, use the production-ready reference container image.\nThe image is based on the reference images for Apache Airflow.\n\nYou can find the following images there:\n* andreax79/airflow-code-editor:**latest** - the latest released Airflow Code Editor image with the latest Apache Airflow version\n* andreax79/airflow-code-editor:**2.10.0** - the latest released Airflow Code Editor with specific Airflow version\n* andreax79/airflow-code-editor:**2.10.0-7.7.0** - specific version of Airflow and Airflow Code Editor\n\n#### Installing from PyPI\n\n1. Install the plugin\n\n ```bash\n pip install airflow-code-editor\n ```\n\n2. Install optional dependencies\n\n* black - Black Python code formatter\n* isort - A Python utility/library to sort imports\n* fs-s3fs - S3FS Amazon S3 Filesystem\n* fs-gcsfs - Google Cloud Storage Filesystem\n* ... other filesystems supported by PyFilesystem - see https://www.pyfilesystem.org/page/index-of-filesystems/\n\n ```bash\n pip install black isort fs-s3fs fs-gcsfs\n ```\n\n3. Restart the Airflow Web Server\n\n4. Open Admin - DAGs Code Editor\n\n\n### Config Options\n\nYou can set options editing the Airflow's configuration file or setting environment variables.\nYou can edit your *airflow.cfg* adding any of the following settings in the \\[code_editor\\] section.\nAll the settings are optional.\n\n* **enabled** enable this plugin (default: True).\n* **git_enabled** enable git support (default: True). If git is not installed, disable this option.\n* **git_cmd** git command (path)\n* **git_default_args** git arguments added to each call (default: -c color.ui=true)\n* **git_author_name** human-readable name in the author/committer (default logged user first and last names)\n* **git_author_email** email for the author/committer (default: logged user email)\n* **git_init_repo** initialize a git repo in DAGs folder (default: True)\n* **root_directory** root folder (default: Airflow DAGs folder)\n* **line_length** Python code formatter - max line length (default: 88)\n* **string_normalization** Python code formatter - if true normalize string quotes and prefixes (default: False)\n* **mount**, **mount1**, ... configure additional folder (mount point) - format: name=xxx,path=yyy\n* **ignored_entries** comma-separated list of entries to be excluded from file/directory list (default: .\\*,\\_\\_pycache\\_\\_)\n\n```\n [code_editor]\n enabled = True\n git_enabled = True\n git_cmd = /usr/bin/git\n git_default_args = -c color.ui=true\n git_init_repo = False\n root_directory = /home/airflow/dags\n line_length = 88\n string_normalization = False\n mount = name=data,path=/home/airflow/data\n mount1 = name=logs,path=/home/airflow/logs\n mount2 = name=data,path=s3://example\n```\n\nMount Options:\n\n* **name**: mount name (destination)\n* **path**: local path or PyFilesystem FS URLs - see https://docs.pyfilesystem.org/en/latest/openers.html\n\nExample:\n* name=ftp_server,path=ftp://user:pass@ftp.example.org/private\n* name=data,path=s3://example\n* name=tmp,path=/tmp\n\nYou can also set options with the following environment variables:\n\n* AIRFLOW__CODE_EDITOR__ENABLED\n* AIRFLOW__CODE_EDITOR__GIT_ENABLED\n* AIRFLOW__CODE_EDITOR__GIT_CMD\n* AIRFLOW__CODE_EDITOR__GIT_DEFAULT_ARGS\n* AIRFLOW__CODE_EDITOR__GIT_AUTHOR_NAME\n* AIRFLOW__CODE_EDITOR__GIT_AUTHOR_EMAIL\n* AIRFLOW__CODE_EDITOR__GIT_INIT_REPO\n* AIRFLOW__CODE_EDITOR__ROOT_DIRECTORY\n* AIRFLOW__CODE_EDITOR__LINE_LENGTH\n* AIRFLOW__CODE_EDITOR__STRING_NORMALIZATION\n* AIRFLOW__CODE_EDITOR__MOUNT, AIRFLOW__CODE_EDITOR__MOUNT1, AIRFLOW__CODE_EDITOR__MOUNT2, ...\n* AIRFLOW__CODE_EDITOR__IGNORED_ENTRIES\n\nExample:\n```\n export AIRFLOW__CODE_EDITOR__STRING_NORMALIZATION=True\n export AIRFLOW__CODE_EDITOR__MOUNT='name=data,path=/home/airflow/data'\n export AIRFLOW__CODE_EDITOR__MOUNT1='name=logs,path=/home/airflow/logs'\n export AIRFLOW__CODE_EDITOR__MOUNT2='name=tmp,path=/tmp'\n```\n\n### REST API\n\nAirflow Code Editor provides a REST API. Through this API, users can interact with the application\nprogrammatically, enabling automation, data retrieval, and integration with other software.\n\nFor detailed information on how to use each endpoint, refer to the\n[API documentation](https://petstore.swagger.io/?url=https://raw.githubusercontent.com/andreax79/airflow-code-editor/main/airflow_code_editor/api/code_editor.yaml).\n\n### REST API Authentication\n\nThe API authentication is inherited from the Apache Airflow.\n\nIf you want to check which auth backend is currently set, you can use\n`airflow config get-value api auth_backends` command as in the example below.\n```bash\n$ airflow config get-value api auth_backends\nairflow.api.auth.backend.basic_auth\n```\n\nFor details on configuring the authentication, see\n[API Authorization](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html).\n\n### Development Instructions\n\n1. Fork the repo\n\n2. Clone it on the local machine\n\n ```bash\n git clone https://github.com/andreax79/airflow-code-editor.git\n cd airflow-code-editor\n ```\n\n3. Create dev image\n\n ```bash\n make dev-image\n ```\n\n4. Switch node version\n\n ```bash\n nvm use\n ```\n\n5. Make changes you need. Build npm package with:\n\n ```bash\n make npm-build\n ```\n\n6. You can start Airflow webserver with:\n\n ```bash\n make webserver\n ```\n\n7. Run tests\n\n ```bash\n make test\n ```\n\n8. Commit and push changes\n\n9. Create [pull request](https://github.com/andreax79/airflow-code-editor/compare) to the original repo\n\n### Links\n\n* [Apache Airflow](https://github.com/apache/airflow)\n* [Codemirror, In-browser code editor](https://github.com/codemirror/codemirror)\n* [Git WebUI, A standalone local web based user interface for git repositories](https://github.com/alberthier/git-webui)\n* [Black, The Uncompromising Code Formatter](https://github.com/psf/black)\n* [isort, A Python utility/library to sort imports](https://github.com/psf/black)\n* [pss, power-tool for searching source files](https://github.com/eliben/pss)\n* [Vue.js](https://github.com/vuejs/vue)\n* [Vue-good-table, data table for VueJS](https://github.com/xaksis/vue-good-table)\n* [Vue-tree, TreeView control for VueJS](https://github.com/grapoza/vue-tree)\n* [Vue-universal-modal Universal modal plugin for Vue@3](https://github.com/hoiheart/vue-universal-modal)\n* [Vue-simple-context-menu](https://github.com/johndatserakis/vue-simple-context-menu)\n* [Splitpanes](https://github.com/antoniandre/splitpanes)\n* [Axios, Promise based HTTP client for the browser and node.js](https://github.com/axios/axios)\n* [PyFilesystem2, Python's Filesystem abstraction layer](https://github.com/PyFilesystem/pyfilesystem2)\n* [Amazon S3 PyFilesystem](https://github.com/PyFilesystem/s3fs)\n* [Google Cloud Storage PyFilesystem](https://github.com/Othoz/gcsfs)\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Apache Airflow code editor and file manager",
"version": "7.7.0",
"project_urls": {
"Bug Tracker": "http://github.com/andreax79/airflow-code-editor/issues",
"Homepage": "https://github.com/andreax79/airflow-code-editor",
"Source Code": "http://github.com/andreax79/airflow-code-editor"
},
"split_keywords": [
"airflow",
" airflow-plugin",
" apache-airflow",
" editor"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bea6a98c5184afd5da64bf7f32012d20384a1d936482652130370585ab6bdbc5",
"md5": "546c1ce7d0a2d437d9d75064c5b640a0",
"sha256": "2f8e40b9357900f462927fd880ab28d25595cb78e950f00af8da13e96a7b7a06"
},
"downloads": -1,
"filename": "airflow_code_editor-7.7.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "546c1ce7d0a2d437d9d75064c5b640a0",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.6",
"size": 1826063,
"upload_time": "2024-08-26T12:45:53",
"upload_time_iso_8601": "2024-08-26T12:45:53.146710Z",
"url": "https://files.pythonhosted.org/packages/be/a6/a98c5184afd5da64bf7f32012d20384a1d936482652130370585ab6bdbc5/airflow_code_editor-7.7.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6639e435a4ec5c93a2d688bf852b6a05f587098e7b93df24ab7fbe94322d639d",
"md5": "05d63d2838bab123aa894972c2a0cfb8",
"sha256": "e9327e0020be6b1595fbc754f09e6463d358116656791f83f9b29dd4a0bc6025"
},
"downloads": -1,
"filename": "airflow_code_editor-7.7.0.tar.gz",
"has_sig": false,
"md5_digest": "05d63d2838bab123aa894972c2a0cfb8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 1585373,
"upload_time": "2024-08-26T12:45:55",
"upload_time_iso_8601": "2024-08-26T12:45:55.380830Z",
"url": "https://files.pythonhosted.org/packages/66/39/e435a4ec5c93a2d688bf852b6a05f587098e7b93df24ab7fbe94322d639d/airflow_code_editor-7.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-26 12:45:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "andreax79",
"github_project": "airflow-code-editor",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "airflow-code-editor"
}
JSON
