<p align="center">
<a href="https://www.airflow.apache.org">
<img alt="Airflow" src="https://cwiki.apache.org/confluence/download/attachments/145723561/airflow_transparent.png?api=v2" width="60" />
</a>
<a href="https://cloud.google.com/bigquery/docs">
<img alt="BigQuery Pricing" src="https://storage.googleapis.com/data-fullstack-utils-public/logo_bq_pricing_bg.png" width="80" />
</a>
</p>
<h1 align="center">
Airflow BigQuery Reservation Provider
</h1>
<br/>
[](https://badge.fury.io/py/airflow-provider-bigquery-reservation)
[](https://codecov.io/gh/PierreC1024/airflow-provider-bigquery-reservation)
> **Warning**
> This package is a pre-released of the official apache-airflow-providers-google package. All of these operators will be integrated to the official package, soon.
This repository provides an Apache Airflow provider based on [BigQuery Reservation API](https://cloud.google.com/python/docs/reference/bigqueryreservation/latest).
## Airflow Operators
* `BigQueryReservationCreateOperator`: Buy BigQuery slots (commitments) and assign them to a GCP project (reserve and assign).
* `BigQueryReservationDeleteOperator`: Delete BigQuery commitments and remove associated ressources (rservation and assignment).
* `BigQueryBiEngineReservationCreateOperator`: Create or Update a BI engine reservation.
* `BigQueryBiEngineReservationDeleteOperator`: Delete or Update a BI engine reservation.
You could find DAG samples [here](https://github.com/PierreC1024/airflow-provider-bigquery-reservation/tree/main/airflow_provider_bigquery_reservation/example_dags).
### Requirements
* A [Google Cloud connection](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/connections/gcp.html) has to be defined.
By default, all hooks and operators use `google_cloud_default`.
* This connection requires the following roles on the Google Cloud project(s) used in these operators:
* [BigQuery Resource Admin](https://cloud.google.com/iam/docs/understanding-roles#bigquery.resourceAdmin)
* [BigQuery Job User](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser) - *Required for `BigQueryReservationCreateOperator` because of the reservation attachment check.*
*Defining a new dedicated connection and custom GCP role could be good practices to respect the principle of least privilege.*
## How to install
```bash
pip install --user airflow-provider-bigquery-reservation
```
Raw data
{
"_id": null,
"home_page": "https://github.com/PierreC1024/airflow-provider-bigquery-reservation",
"name": "airflow-provider-bigquery-reservation",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "airflow,bigquery",
"author": "Pierre Cardona",
"author_email": "pierre@data-fullstack.com",
"download_url": "https://github.com/PierreC1024/airflow-provider-bigquery-reservation/archive/refs/tags/0.2.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://www.airflow.apache.org\">\n <img alt=\"Airflow\" src=\"https://cwiki.apache.org/confluence/download/attachments/145723561/airflow_transparent.png?api=v2\" width=\"60\" />\n </a>\n <a href=\"https://cloud.google.com/bigquery/docs\">\n <img alt=\"BigQuery Pricing\" src=\"https://storage.googleapis.com/data-fullstack-utils-public/logo_bq_pricing_bg.png\" width=\"80\" />\n </a>\n</p>\n<h1 align=\"center\">\n Airflow BigQuery Reservation Provider\n</h1>\n<br/>\n\n[](https://badge.fury.io/py/airflow-provider-bigquery-reservation)\n[](https://codecov.io/gh/PierreC1024/airflow-provider-bigquery-reservation)\n\n\n> **Warning**\n> This package is a pre-released of the official apache-airflow-providers-google package. All of these operators will be integrated to the official package, soon.\n\nThis repository provides an Apache Airflow provider based on [BigQuery Reservation API](https://cloud.google.com/python/docs/reference/bigqueryreservation/latest).\n\n## Airflow Operators\n* `BigQueryReservationCreateOperator`: Buy BigQuery slots (commitments) and assign them to a GCP project (reserve and assign).\n* `BigQueryReservationDeleteOperator`: Delete BigQuery commitments and remove associated ressources (rservation and assignment).\n* `BigQueryBiEngineReservationCreateOperator`: Create or Update a BI engine reservation.\n* `BigQueryBiEngineReservationDeleteOperator`: Delete or Update a BI engine reservation.\n\nYou could find DAG samples [here](https://github.com/PierreC1024/airflow-provider-bigquery-reservation/tree/main/airflow_provider_bigquery_reservation/example_dags).\n\n### Requirements\n\n* A [Google Cloud connection](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/connections/gcp.html) has to be defined.\nBy default, all hooks and operators use `google_cloud_default`.\n* This connection requires the following roles on the Google Cloud project(s) used in these operators:\n * [BigQuery Resource Admin](https://cloud.google.com/iam/docs/understanding-roles#bigquery.resourceAdmin)\n * [BigQuery Job User](https://cloud.google.com/iam/docs/understanding-roles#bigquery.jobUser) - *Required for `BigQueryReservationCreateOperator` because of the reservation attachment check.*\n\n*Defining a new dedicated connection and custom GCP role could be good practices to respect the principle of least privilege.*\n\n## How to install\n\n```bash\npip install --user airflow-provider-bigquery-reservation\n```\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "An Apache Airflow provider concerning BigQuery reservation.",
"version": "1.1.1",
"split_keywords": [
"airflow",
"bigquery"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6ce41e0fb487177f6e84b2299a5de1bf00d2b266325de8b82cf3f71140dac9db",
"md5": "4a4b25aeabf1dfa8fb8ffb69b0f53086",
"sha256": "9b5cacba58eeb927947d9bcb66900002d2358f95965a26116a50164e0be5e053"
},
"downloads": -1,
"filename": "airflow_provider_bigquery_reservation-1.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4a4b25aeabf1dfa8fb8ffb69b0f53086",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 16065,
"upload_time": "2023-03-24T08:25:21",
"upload_time_iso_8601": "2023-03-24T08:25:21.401796Z",
"url": "https://files.pythonhosted.org/packages/6c/e4/1e0fb487177f6e84b2299a5de1bf00d2b266325de8b82cf3f71140dac9db/airflow_provider_bigquery_reservation-1.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-24 08:25:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "PierreC1024",
"github_project": "airflow-provider-bigquery-reservation",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "airflow-provider-bigquery-reservation"
}
JSON
