# Airflow Dremio Provider
This is the official provider package for airflow that integrates with Dremio. It contains the following operators and sensors
* `DremioCreateReflectionOperator` - Operator to manage reflections in Dremio
* `DremioCreateSourceOperator` - Operator to create a source in Airflow
* `DremioJobSensor` - Sensor to check the status of a job submitted in dremio
## Installation
You can use `pip` to install this package
```commandline
pip install airflow-dremio-provider
```
If you want to build from source, you can do so using [hatch](https://hatch.pypa.io/). First create the .whl file usin the command `hatch build`. This will create a wheel file in the `dist` directory. To install the package run `pip install *path-to-.whl-file*`
## Creating a Dremio Connection
A dremio connection is the same as an Http connection. Therefor you can just create a connection with type being `HTTP`. Once you have created a connection enter all the fields as required. The login field must contain your username For providing the authentication there are two supported ways
#### Authentication using basic auth
This simply requires authentication to be done using the username and password. The Dremio hook automatically uses this to create an auth token from Dremio and uses this token for all REST API authentication. To use this method of authentication, the `extra` field of the connection add the following - `{"auth":"AuthToken"}`.
#### Authentication using PAT token
This method of authentication is to be done using the user generated PAT token in Dremio. The Dremio hook uses this token for all REST API authentication. To use this method of authentication, the `extra` field of the connection add the following - `{"auth":"PAT", "pat":"your-pat-token-value"}`.
# Contributing
We welcome your contributions! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute to this provider package
# License
This project is licensed under the [Apache 2.0 License](LICENSE).
Raw data
{
"_id": null,
"home_page": null,
"name": "airflow-dremio-provider",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "airflow, apache-airflow, dremio, provider",
"author": null,
"author_email": "Sanchit Sreekanth <sanchitsreekanth@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/b8/e1/a037c4b63ca9dbc65753de19c61aba31238cb90d5d46bcaa220fc5c17257/airflow_dremio_provider-0.1.1.tar.gz",
"platform": null,
"description": "# Airflow Dremio Provider\n\nThis is the official provider package for airflow that integrates with Dremio. It contains the following operators and sensors\n* `DremioCreateReflectionOperator` - Operator to manage reflections in Dremio\n* `DremioCreateSourceOperator` - Operator to create a source in Airflow\n* `DremioJobSensor` - Sensor to check the status of a job submitted in dremio\n\n## Installation\n You can use `pip` to install this package\n```commandline\npip install airflow-dremio-provider\n```\n\nIf you want to build from source, you can do so using [hatch](https://hatch.pypa.io/). First create the .whl file usin the command `hatch build`. This will create a wheel file in the `dist` directory. To install the package run `pip install *path-to-.whl-file*`\n\n## Creating a Dremio Connection\nA dremio connection is the same as an Http connection. Therefor you can just create a connection with type being `HTTP`. Once you have created a connection enter all the fields as required. The login field must contain your username For providing the authentication there are two supported ways\n\n#### Authentication using basic auth\nThis simply requires authentication to be done using the username and password. The Dremio hook automatically uses this to create an auth token from Dremio and uses this token for all REST API authentication. To use this method of authentication, the `extra` field of the connection add the following - `{\"auth\":\"AuthToken\"}`.\n\n#### Authentication using PAT token\nThis method of authentication is to be done using the user generated PAT token in Dremio. The Dremio hook uses this token for all REST API authentication. To use this method of authentication, the `extra` field of the connection add the following - `{\"auth\":\"PAT\", \"pat\":\"your-pat-token-value\"}`.\n\n\n# Contributing\nWe welcome your contributions! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute to this provider package\n\n# License\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n",
"bugtrack_url": null,
"license": null,
"summary": "A provider package for managing reflections and executing queries via Dremio.",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/sanchitsreekanth/airflow-provider-dremio"
},
"split_keywords": [
"airflow",
" apache-airflow",
" dremio",
" provider"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "36f0f244bb1dd3e6d69a2991c845e685d6ec8bae1fd4e8449285e452b71e3899",
"md5": "968cbff17765aac81b0c0e7755b0c061",
"sha256": "f0a27240920465c10c6f04c69f905344eff3fe29a7a6f34f38e2b75991faf4e7"
},
"downloads": -1,
"filename": "airflow_dremio_provider-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "968cbff17765aac81b0c0e7755b0c061",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 17172,
"upload_time": "2024-05-01T16:03:57",
"upload_time_iso_8601": "2024-05-01T16:03:57.790151Z",
"url": "https://files.pythonhosted.org/packages/36/f0/f244bb1dd3e6d69a2991c845e685d6ec8bae1fd4e8449285e452b71e3899/airflow_dremio_provider-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b8e1a037c4b63ca9dbc65753de19c61aba31238cb90d5d46bcaa220fc5c17257",
"md5": "fcabca4cf54538dad0937f5199fac061",
"sha256": "9945aa0a4c6cc8dc8d7b1beb093d05c8dc6ee68ede3cb660eb5c229a50050c83"
},
"downloads": -1,
"filename": "airflow_dremio_provider-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "fcabca4cf54538dad0937f5199fac061",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 7167,
"upload_time": "2024-05-01T16:03:55",
"upload_time_iso_8601": "2024-05-01T16:03:55.291164Z",
"url": "https://files.pythonhosted.org/packages/b8/e1/a037c4b63ca9dbc65753de19c61aba31238cb90d5d46bcaa220fc5c17257/airflow_dremio_provider-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-01 16:03:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sanchitsreekanth",
"github_project": "airflow-provider-dremio",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "airflow-dremio-provider"
}
JSON
