# Kedro Airflow K8S Plugin
[](https://github.com/getindata/kedro-airflow-k8s)
[](https://opensource.org/licenses/Apache-2.0)
[](https://semver.org/)
[](https://pypi.org/project/kedro-airflow-k8s/)
[](https://img.shields.io/pypi/dm/kedro-airflow-k8s)
[](https://codeclimate.com/github/getindata/kedro-airflow-k8s/maintainability)
[](https://codeclimate.com/github/getindata/kedro-airflow-k8s/test_coverage)
[](https://kedro-airflow-k8s.readthedocs.io/en/latest/?badge=latest)
## About
The main purpose of this plugin is to enable running kedro pipeline with Airflow on Kubernetes Cluster. In difference to
[kedro-airflow](https://github.com/quantumblacklabs/kedro-airflow) this plugin does not require additional libraries installed
in airflow runtime, it uses K8S infrastructure instead. It supports translation
from Kedro pipeline DSL to [airflow](https://airflow.apache.org/docs/apache-airflow/stable/python-api-ref.html) (python API)
and generation of airflow [DAGs](https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#dags).
The plugin can be used together with `kedro-docker` to simplify preparation of docker image for pipeline execution.
## Documentation
For detailed documentation refer to https://kedro-airflow-k8s.readthedocs.io/
## Usage guide
```
Usage: kedro airflow-k8s [OPTIONS] COMMAND [ARGS]...
Options:
-e, --env TEXT Environment to use.
-p, --pipeline TEXT Pipeline name to pick.
-h, --help Show this message and exit.
Commands:
compile Create an Airflow DAG for a project
init Initializes configuration for the plugin
list-pipelines List pipelines generated by this plugin
run-once Uploads pipeline to Airflow and runs once
schedule Uploads pipeline to Airflow with given schedule
ui Open Apache Airflow UI in new browser tab
upload-pipeline Uploads pipeline to Airflow DAG location
```
Raw data
{
"_id": null,
"home_page": "https://github.com/getindata/kedro-airflow-k8s/",
"name": "kedro-airflow-k8s",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3",
"maintainer_email": "",
"keywords": "kedro airflow kubernetes k8s ml mlops plugin",
"author": "Michal Zelechowski, Mariusz Strzelecki, Mateusz Pytel",
"author_email": "mateusz@getindata.com",
"download_url": "https://files.pythonhosted.org/packages/8d/8f/efc44f273f6333c63759d513ce013a7171cc5611abc5f71095af84aecb54/kedro-airflow-k8s-0.8.2.tar.gz",
"platform": null,
"description": "# Kedro Airflow K8S Plugin\n\n[](https://github.com/getindata/kedro-airflow-k8s)\n[](https://opensource.org/licenses/Apache-2.0) \n[](https://semver.org/)\n[](https://pypi.org/project/kedro-airflow-k8s/)\n[](https://img.shields.io/pypi/dm/kedro-airflow-k8s) \n\n[](https://codeclimate.com/github/getindata/kedro-airflow-k8s/maintainability)\n[](https://codeclimate.com/github/getindata/kedro-airflow-k8s/test_coverage)\n[](https://kedro-airflow-k8s.readthedocs.io/en/latest/?badge=latest)\n\n## About\n\nThe main purpose of this plugin is to enable running kedro pipeline with Airflow on Kubernetes Cluster. In difference to \n[kedro-airflow](https://github.com/quantumblacklabs/kedro-airflow) this plugin does not require additional libraries installed\nin airflow runtime, it uses K8S infrastructure instead. It supports translation\nfrom Kedro pipeline DSL to [airflow](https://airflow.apache.org/docs/apache-airflow/stable/python-api-ref.html) (python API)\nand generation of airflow [DAGs](https://airflow.apache.org/docs/apache-airflow/stable/concepts.html#dags).\n\nThe plugin can be used together with `kedro-docker` to simplify preparation of docker image for pipeline execution. \n\n## Documentation\n\nFor detailed documentation refer to https://kedro-airflow-k8s.readthedocs.io/\n\n## Usage guide\n\n```\nUsage: kedro airflow-k8s [OPTIONS] COMMAND [ARGS]...\n \nOptions:\n -e, --env TEXT Environment to use.\n -p, --pipeline TEXT Pipeline name to pick.\n -h, --help Show this message and exit.\n\nCommands:\n compile Create an Airflow DAG for a project\n init Initializes configuration for the plugin\n list-pipelines List pipelines generated by this plugin\n run-once Uploads pipeline to Airflow and runs once\n schedule Uploads pipeline to Airflow with given schedule\n ui Open Apache Airflow UI in new browser tab\n upload-pipeline Uploads pipeline to Airflow DAG location\n```",
"bugtrack_url": null,
"license": "Apache Software License (Apache 2.0)",
"summary": "Kedro plugin with Airflow on Kubernetes support",
"version": "0.8.2",
"project_urls": {
"Homepage": "https://github.com/getindata/kedro-airflow-k8s/"
},
"split_keywords": [
"kedro",
"airflow",
"kubernetes",
"k8s",
"ml",
"mlops",
"plugin"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8d8fefc44f273f6333c63759d513ce013a7171cc5611abc5f71095af84aecb54",
"md5": "1df899e21d0d855d1fda67265f4cd37b",
"sha256": "7e1b1563cc111b71721fb96002141b84cffd7aeaccd680cd5b1ddb4fa8d8f70f"
},
"downloads": -1,
"filename": "kedro-airflow-k8s-0.8.2.tar.gz",
"has_sig": false,
"md5_digest": "1df899e21d0d855d1fda67265f4cd37b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3",
"size": 34371,
"upload_time": "2023-08-29T14:25:29",
"upload_time_iso_8601": "2023-08-29T14:25:29.388352Z",
"url": "https://files.pythonhosted.org/packages/8d/8f/efc44f273f6333c63759d513ce013a7171cc5611abc5f71095af84aecb54/kedro-airflow-k8s-0.8.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-29 14:25:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "getindata",
"github_project": "kedro-airflow-k8s",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "kedro-airflow-k8s"
}
JSON
