| Name | bigquery-advanced-utils JSON |
| Version |
0.0.1.dev5
JSON |
| download |
| home_page | None |
| Summary | A utility library that enhances the official BigQuery Python client by providing additional tools for efficient query management, data processing, and automation. Designed for developers and data scientists, `bigquery-advanced-utils` simplifies working with Google BigQuery. |
| upload_time | 2024-12-27 11:33:12 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | GPL-3.0 |
| keywords |
python
bigquery
google-cloud
datatransfer
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# bigquery-advanced-utils
[](https://codecov.io/gh/Alessio-Siciliano/bigquery-advanced-utils)
**BigQuery-advanced-utils** is a lightweight utility library that extends the official Google BigQuery Python client.
It simplifies tasks like **query management**, *data processing*, and *automation*.
Aimed at **developers** and **data scientists**, the project is open to *contributions* to improve and enhance its functionality.
For full documentation, please visit: [BigQuery-Advanced-Utils Documentation](https://alessio-siciliano.github.io/bigquery-utils/)
## Why This Library?
I created **bigquery-advanced-utils** because I often found myself facing complex or uncommon tasks when working with BigQuery, and there was little or no support available online. Rather than spending time reinventing the wheel, I decided to create this library to help others avoid the same challenges. I hope that everyone can contribute in the same spirit, so feel free to get involved and make this library even better!
## Requirements
- Python 3.10+
## Installation 📦
### Install via pip (recommended)
Run the following command in your terminal:
```bash
pip install bigquery-advanced-utils
```
### Install in a Virtual Environment
1. Create a virtual environment:
```bash
python -m venv venv
```
2. Activate the environment and install:
```bash
source venv/bin/activate # on macOS/Linux
venv\Scripts\activate # on Windows
pip install bigquery-advanced-utils
```
## Usage Examples 🚀
### Quick Start
```python
from bigquery_advanced_utils.bigquery import BigQueryClient
# Initialize helper with your project ID
helper = BigQueryClient(project_id="your_project_id")
# Load data from CSV
helper.load_data_from_csv(file_path="your_data.csv")
```
### Data Validation
```python
test_functions=[
partial(
# Check if any null values exist in the "age" column
b.check_no_nulls, columns_to_test=["age"],
),
partial(
# Ensure values in the "email" column are unique
b.check_unique_column, columns_to_test=["email"]
)
]
```
### Search by table or owner in Datatransfer
```python
from bigquery_advanced_utils.datatransfer import DataTransferClient
helper = DataTransferClient()
# Call the function with two parameters: owner email and project id
list_transfer_config_by_owner_email(owner_email="my-email@email.com", project_id="my-project")
# Get the scheduled queries by the name of a table (it's case sensitive)
list_transfer_configs_by_table(table_id="my-table", project_id="my-project")
```
## Planned features 🚧
- A new query builder.
- Custom data transformation and processing functions.
- Exclusive features with datatransfer.
- Utility functions to manipulate strings and query.
## Contributing
We are always open to contributions! Whether you have a bug fix, a feature request, or a general improvement to make, your help is appreciated. Here are some ways you can contribute:
- **Bug reports**: Help us catch issues before they affect users.
- **New features**: Suggest new functionalities that could improve the usability of the package.
- **Code improvements**: Review the code and suggest optimizations or fixes.
Please follow the [contributing guide](CONTRIBUTING.md) for more details on how to get started.
## License
This project is licensed under the **GNU General Public License**. See the [LICENSE](LICENSE) file for details.
## Contact
For questions or feedback, feel free to open an issue or reach out to me.
Raw data
{
"_id": null,
"home_page": null,
"name": "bigquery-advanced-utils",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "python, bigquery, google-cloud, datatransfer",
"author": null,
"author_email": "Alessio Siciliano <alessiosiciliano@outlook.it>",
"download_url": "https://files.pythonhosted.org/packages/9a/c5/973cdff4fab00e7a8886e0d287ba3e05ef01b568eb3ff2ab6becf817758f/bigquery_advanced_utils-0.0.1.dev5.tar.gz",
"platform": null,
"description": "# bigquery-advanced-utils\n\n\n\n\n[](https://codecov.io/gh/Alessio-Siciliano/bigquery-advanced-utils)\n\n\n\n\n\n\n**BigQuery-advanced-utils** is a lightweight utility library that extends the official Google BigQuery Python client. \nIt simplifies tasks like **query management**, *data processing*, and *automation*. \n\nAimed at **developers** and **data scientists**, the project is open to *contributions* to improve and enhance its functionality.\n\nFor full documentation, please visit: [BigQuery-Advanced-Utils Documentation](https://alessio-siciliano.github.io/bigquery-utils/)\n\n## Why This Library?\n\nI created **bigquery-advanced-utils** because I often found myself facing complex or uncommon tasks when working with BigQuery, and there was little or no support available online. Rather than spending time reinventing the wheel, I decided to create this library to help others avoid the same challenges. I hope that everyone can contribute in the same spirit, so feel free to get involved and make this library even better!\n\n## Requirements\n- Python 3.10+\n\n## Installation \ud83d\udce6\n\n### Install via pip (recommended)\n\nRun the following command in your terminal:\n```bash\npip install bigquery-advanced-utils\n```\n### Install in a Virtual Environment\n\n1. Create a virtual environment:\n```bash\npython -m venv venv\n```\n2. Activate the environment and install:\n```bash\nsource venv/bin/activate # on macOS/Linux \nvenv\\Scripts\\activate # on Windows \npip install bigquery-advanced-utils\n```\n\n## Usage Examples \ud83d\ude80\n### Quick Start\n\n```python\nfrom bigquery_advanced_utils.bigquery import BigQueryClient\n\n# Initialize helper with your project ID\nhelper = BigQueryClient(project_id=\"your_project_id\")\n\n# Load data from CSV\nhelper.load_data_from_csv(file_path=\"your_data.csv\")\n```\n### Data Validation\n```python\ntest_functions=[\n partial(\n # Check if any null values exist in the \"age\" column\n b.check_no_nulls, columns_to_test=[\"age\"],\n ),\n partial(\n # Ensure values in the \"email\" column are unique\n b.check_unique_column, columns_to_test=[\"email\"]\n )\n]\n```\n### Search by table or owner in Datatransfer\n```python\nfrom bigquery_advanced_utils.datatransfer import DataTransferClient\n\nhelper = DataTransferClient()\n# Call the function with two parameters: owner email and project id\nlist_transfer_config_by_owner_email(owner_email=\"my-email@email.com\", project_id=\"my-project\")\n\n# Get the scheduled queries by the name of a table (it's case sensitive) \nlist_transfer_configs_by_table(table_id=\"my-table\", project_id=\"my-project\")\n```\n\n## Planned features \ud83d\udea7\n- A new query builder.\n- Custom data transformation and processing functions.\n- Exclusive features with datatransfer.\n- Utility functions to manipulate strings and query.\n\n## Contributing\n\nWe are always open to contributions! Whether you have a bug fix, a feature request, or a general improvement to make, your help is appreciated. Here are some ways you can contribute:\n\n- **Bug reports**: Help us catch issues before they affect users.\n- **New features**: Suggest new functionalities that could improve the usability of the package.\n- **Code improvements**: Review the code and suggest optimizations or fixes.\n\nPlease follow the [contributing guide](CONTRIBUTING.md) for more details on how to get started.\n\n## License\nThis project is licensed under the **GNU General Public License**. See the [LICENSE](LICENSE) file for details.\n\n\n## Contact\nFor questions or feedback, feel free to open an issue or reach out to me.\n",
"bugtrack_url": null,
"license": "GPL-3.0",
"summary": "A utility library that enhances the official BigQuery Python client by providing additional tools for efficient query management, data processing, and automation. Designed for developers and data scientists, `bigquery-advanced-utils` simplifies working with Google BigQuery.",
"version": "0.0.1.dev5",
"project_urls": null,
"split_keywords": [
"python",
" bigquery",
" google-cloud",
" datatransfer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "91e66379efadb593ea3e4055b0f577e69f54f897718ce64465bfec85587f381c",
"md5": "b6ffe10913d421f09c59e22ba84e3955",
"sha256": "e549e9eb3172333325eb6de3430e3bf1670122e4371d031637e1e011854cc737"
},
"downloads": -1,
"filename": "bigquery_advanced_utils-0.0.1.dev5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b6ffe10913d421f09c59e22ba84e3955",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 39324,
"upload_time": "2024-12-27T11:33:08",
"upload_time_iso_8601": "2024-12-27T11:33:08.715073Z",
"url": "https://files.pythonhosted.org/packages/91/e6/6379efadb593ea3e4055b0f577e69f54f897718ce64465bfec85587f381c/bigquery_advanced_utils-0.0.1.dev5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9ac5973cdff4fab00e7a8886e0d287ba3e05ef01b568eb3ff2ab6becf817758f",
"md5": "f8a32048486707c11a2397fea5ded08c",
"sha256": "28d1ff8c750c71038e268d643ca4779d0e5dde8660f623764a65e6147e17d3d7"
},
"downloads": -1,
"filename": "bigquery_advanced_utils-0.0.1.dev5.tar.gz",
"has_sig": false,
"md5_digest": "f8a32048486707c11a2397fea5ded08c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 55795,
"upload_time": "2024-12-27T11:33:12",
"upload_time_iso_8601": "2024-12-27T11:33:12.578716Z",
"url": "https://files.pythonhosted.org/packages/9a/c5/973cdff4fab00e7a8886e0d287ba3e05ef01b568eb3ff2ab6becf817758f/bigquery_advanced_utils-0.0.1.dev5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-27 11:33:12",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "bigquery-advanced-utils"
}
