sqlalchemy-bigquery


Namesqlalchemy-bigquery JSON
Version 1.12.0 PyPI version JSON
download
home_pagehttps://github.com/googleapis/python-bigquery-sqlalchemy
SummarySQLAlchemy dialect for BigQuery
upload_time2024-10-02 21:32:50
maintainerNone
docs_urlNone
authorThe Sqlalchemy-Bigquery Authors
requires_python<3.13,>=3.8
licenseNone
keywords bigquery sqlalchemy
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage
            SQLAlchemy Dialect for BigQuery
===============================

|GA| |pypi| |versions|

`SQLALchemy Dialects`_

- `Dialect Documentation`_
- `Product Documentation`_

.. |GA| image:: https://img.shields.io/badge/support-GA-gold.svg
   :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#general-availability
.. |pypi| image:: https://img.shields.io/pypi/v/sqlalchemy-bigquery.svg
   :target: https://pypi.org/project/sqlalchemy-bigquery/
.. |versions| image:: https://img.shields.io/pypi/pyversions/sqlalchemy-bigquery.svg
   :target: https://pypi.org/project/sqlalchemy-bigquery/
.. _SQLAlchemy Dialects: https://docs.sqlalchemy.org/en/14/dialects/
.. _Dialect Documentation: https://googleapis.dev/python/sqlalchemy-bigquery/latest
.. _Product Documentation: https://cloud.google.com/bigquery/docs/


Quick Start
-----------

In order to use this library, you first need to go through the following steps:

1. `Select or create a Cloud Platform project.`_
2. [Optional] `Enable billing for your project.`_
3. `Enable the BigQuery Storage API.`_
4. `Setup Authentication.`_

.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project
.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project
.. _Enable the BigQuery Storage API.: https://console.cloud.google.com/apis/library/bigquery.googleapis.com
.. _Setup Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html


Installation
------------

Install this library in a `virtualenv`_ using pip. `virtualenv`_ is a tool to
create isolated Python environments. The basic problem it addresses is one of
dependencies and versions, and indirectly permissions.

With `virtualenv`_, it's possible to install this library without needing system
install permissions, and without clashing with the installed system
dependencies.

.. _`virtualenv`: https://virtualenv.pypa.io/en/latest/


Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.8

Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python <= 3.7.


Mac/Linux
^^^^^^^^^

.. code-block:: console

    pip install virtualenv
    virtualenv <your-env>
    source <your-env>/bin/activate
    <your-env>/bin/pip install sqlalchemy-bigquery


Windows
^^^^^^^

.. code-block:: console

    pip install virtualenv
    virtualenv <your-env>
    <your-env>\Scripts\activate
    <your-env>\Scripts\pip.exe install sqlalchemy-bigquery


Installations when processing large datasets
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When handling large datasets, you may see speed increases by also installing the
`bqstorage` dependencies. See the instructions above about creating a virtual 
environment and then install `sqlalchemy-bigquery` using the `bqstorage` extras:

.. code-block:: console

    source <your-env>/bin/activate
    <your-env>/bin/pip install sqlalchemy-bigquery[bqstorage]


Usage
-----

SQLAlchemy
^^^^^^^^^^

.. code-block:: python

    from sqlalchemy import *
    from sqlalchemy.engine import create_engine
    from sqlalchemy.schema import *
    engine = create_engine('bigquery://project')
    table = Table('dataset.table', MetaData(bind=engine), autoload=True)
    print(select([func.count('*')], from_obj=table().scalar()))


Project
^^^^^^^

``project`` in ``bigquery://project`` is used to instantiate BigQuery client with the specific project ID. To infer project from the environment, use ``bigquery://`` – without ``project``

Authentication
^^^^^^^^^^^^^^

Follow the `Google Cloud library guide <https://google-cloud-python.readthedocs.io/en/latest/core/auth.html>`_ for authentication. 

Alternatively, you can choose either of the following approaches:

* provide the path to a service account JSON file in ``create_engine()`` using the ``credentials_path`` parameter:

.. code-block:: python

    # provide the path to a service account JSON file
    engine = create_engine('bigquery://', credentials_path='/path/to/keyfile.json')

* pass the credentials in ``create_engine()`` as a Python dictionary using the ``credentials_info`` parameter:

.. code-block:: python
    
    # provide credentials as a Python dictionary
    credentials_info = {
        "type": "service_account", 
        "project_id": "your-service-account-project-id"
    },
    engine = create_engine('bigquery://', credentials_info=credentials_info)

Location
^^^^^^^^

To specify location of your datasets pass ``location`` to ``create_engine()``:

.. code-block:: python

    engine = create_engine('bigquery://project', location="asia-northeast1")


Table names
^^^^^^^^^^^

To query tables from non-default projects or datasets, use the following format for the SQLAlchemy schema name: ``[project.]dataset``, e.g.:

.. code-block:: python

    # If neither dataset nor project are the default
    sample_table_1 = Table('natality', schema='bigquery-public-data.samples')
    # If just dataset is not the default
    sample_table_2 = Table('natality', schema='bigquery-public-data')

Batch size
^^^^^^^^^^

By default, ``arraysize`` is set to ``5000``. ``arraysize`` is used to set the batch size for fetching results. To change it, pass ``arraysize`` to ``create_engine()``:

.. code-block:: python

    engine = create_engine('bigquery://project', arraysize=1000)

Page size for dataset.list_tables
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

By default, ``list_tables_page_size`` is set to ``1000``. ``list_tables_page_size`` is used to set the max_results for `dataset.list_tables`_ operation. To change it, pass ``list_tables_page_size`` to ``create_engine()``:

.. _`dataset.list_tables`: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list
.. code-block:: python

    engine = create_engine('bigquery://project', list_tables_page_size=100)

Adding a Default Dataset
^^^^^^^^^^^^^^^^^^^^^^^^

If you want to have the ``Client`` use a default dataset, specify it as the "database" portion of the connection string.

.. code-block:: python

    engine = create_engine('bigquery://project/dataset')

When using a default dataset, don't include the dataset name in the table name, e.g.:

.. code-block:: python

    table = Table('table_name')

Note that specifying a default dataset doesn't restrict execution of queries to that particular dataset when using raw queries, e.g.:

.. code-block:: python

    # Set default dataset to dataset_a
    engine = create_engine('bigquery://project/dataset_a')

    # This will still execute and return rows from dataset_b
    engine.execute('SELECT * FROM dataset_b.table').fetchall()


Connection String Parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There are many situations where you can't call ``create_engine`` directly, such as when using tools like `Flask SQLAlchemy <http://flask-sqlalchemy.pocoo.org/2.3/>`_. For situations like these, or for situations where you want the ``Client`` to have a `default_query_job_config <https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/generated/google.cloud.bigquery.client.Client.html#google.cloud.bigquery.client.Client>`_, you can pass many arguments in the query of the connection string.

The ``credentials_path``, ``credentials_info``, ``credentials_base64``, ``location``, ``arraysize`` and ``list_tables_page_size`` parameters are used by this library, and the rest are used to create a `QueryJobConfig <https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/generated/google.cloud.bigquery.job.QueryJobConfig.html#google.cloud.bigquery.job.QueryJobConfig>`_

Note that if you want to use query strings, it will be more reliable if you use three slashes, so ``'bigquery:///?a=b'`` will work reliably, but ``'bigquery://?a=b'`` might be interpreted as having a "database" of ``?a=b``, depending on the system being used to parse the connection string.

Here are examples of all the supported arguments. Any not present are either for legacy sql (which isn't supported by this library), or are too complex and are not implemented.

.. code-block:: python

    engine = create_engine(
        'bigquery://some-project/some-dataset' '?'
        'credentials_path=/some/path/to.json' '&'
        'location=some-location' '&'
        'arraysize=1000' '&'
        'list_tables_page_size=100' '&'
        'clustering_fields=a,b,c' '&'
        'create_disposition=CREATE_IF_NEEDED' '&'
        'destination=different-project.different-dataset.table' '&'
        'destination_encryption_configuration=some-configuration' '&'
        'dry_run=true' '&'
        'labels=a:b,c:d' '&'
        'maximum_bytes_billed=1000' '&'
        'priority=INTERACTIVE' '&'
        'schema_update_options=ALLOW_FIELD_ADDITION,ALLOW_FIELD_RELAXATION' '&'
        'use_query_cache=true' '&'
        'write_disposition=WRITE_APPEND'
    )

In cases where you wish to include the full credentials in the connection URI you can base64 the credentials JSON file and supply the encoded string to the ``credentials_base64`` parameter.

.. code-block:: python

    engine = create_engine(
        'bigquery://some-project/some-dataset' '?'
        'credentials_base64=eyJrZXkiOiJ2YWx1ZSJ9Cg==' '&'
        'location=some-location' '&'
        'arraysize=1000' '&'
        'list_tables_page_size=100' '&'
        'clustering_fields=a,b,c' '&'
        'create_disposition=CREATE_IF_NEEDED' '&'
        'destination=different-project.different-dataset.table' '&'
        'destination_encryption_configuration=some-configuration' '&'
        'dry_run=true' '&'
        'labels=a:b,c:d' '&'
        'maximum_bytes_billed=1000' '&'
        'priority=INTERACTIVE' '&'
        'schema_update_options=ALLOW_FIELD_ADDITION,ALLOW_FIELD_RELAXATION' '&'
        'use_query_cache=true' '&'
        'write_disposition=WRITE_APPEND'
    )

To create the base64 encoded string you can use the command line tool ``base64``, or ``openssl base64``, or ``python -m base64``.

Alternatively, you can use an online generator like `www.base64encode.org <https://www.base64encode.org>_` to paste your credentials JSON file to be encoded.


Supplying Your Own BigQuery Client
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The above connection string parameters allow you to influence how the BigQuery client used to execute your queries will be instantiated.
If you need additional control, you can supply a BigQuery client of your own:

.. code-block:: python

    from google.cloud import bigquery

    custom_bq_client = bigquery.Client(...)

    engine = create_engine(
        'bigquery://some-project/some-dataset?user_supplied_client=True',
	    connect_args={'client': custom_bq_client},
    )


Creating tables
^^^^^^^^^^^^^^^

To add metadata to a table:

.. code-block:: python

    table = Table('mytable', ...,
        bigquery_description='my table description',
        bigquery_friendly_name='my table friendly name',
        bigquery_default_rounding_mode="ROUND_HALF_EVEN",
        bigquery_expiration_timestamp=datetime.datetime.fromisoformat("2038-01-01T00:00:00+00:00"),
    )

To add metadata to a column:

.. code-block:: python

    Column('mycolumn', doc='my column description')

To create a clustered table:

.. code-block:: python

    table = Table('mytable', ..., bigquery_clustering_fields=["a", "b", "c"])

To create a time-unit column-partitioned table:

.. code-block:: python

    from google.cloud import bigquery

    table = Table('mytable', ...,
        bigquery_time_partitioning=bigquery.TimePartitioning(
            field="mytimestamp",
            type_="MONTH",
            expiration_ms=1000 * 60 * 60 * 24 * 30 * 6, # 6 months
        ),
        bigquery_require_partition_filter=True,
    )

To create an ingestion-time partitioned table:

.. code-block:: python

    from google.cloud import bigquery

    table = Table('mytable', ...,
        bigquery_time_partitioning=bigquery.TimePartitioning(),
        bigquery_require_partition_filter=True,
    )

To create an integer-range partitioned table

.. code-block:: python

    from google.cloud import bigquery

    table = Table('mytable', ...,
        bigquery_range_partitioning=bigquery.RangePartitioning(
            field="zipcode",
            range_=bigquery.PartitionRange(start=0, end=100000, interval=10),
        ),
        bigquery_require_partition_filter=True,
    )


Threading and Multiprocessing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Because this client uses the `grpc` library, it's safe to
share instances across threads.

In multiprocessing scenarios, the best
practice is to create client instances *after* the invocation of
`os.fork` by `multiprocessing.pool.Pool` or
`multiprocessing.Process`.

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/googleapis/python-bigquery-sqlalchemy",
    "name": "sqlalchemy-bigquery",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<3.13,>=3.8",
    "maintainer_email": null,
    "keywords": "bigquery, sqlalchemy",
    "author": "The Sqlalchemy-Bigquery Authors",
    "author_email": "googleapis-packages@google.com",
    "download_url": "https://files.pythonhosted.org/packages/75/39/6d2fb718e61d18c07cfc3de84362c198aa429e3dcf3c1d0a1e476e474196/sqlalchemy_bigquery-1.12.0.tar.gz",
    "platform": "Posix; MacOS X; Windows",
    "description": "SQLAlchemy Dialect for BigQuery\n===============================\n\n|GA| |pypi| |versions|\n\n`SQLALchemy Dialects`_\n\n- `Dialect Documentation`_\n- `Product Documentation`_\n\n.. |GA| image:: https://img.shields.io/badge/support-GA-gold.svg\n   :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#general-availability\n.. |pypi| image:: https://img.shields.io/pypi/v/sqlalchemy-bigquery.svg\n   :target: https://pypi.org/project/sqlalchemy-bigquery/\n.. |versions| image:: https://img.shields.io/pypi/pyversions/sqlalchemy-bigquery.svg\n   :target: https://pypi.org/project/sqlalchemy-bigquery/\n.. _SQLAlchemy Dialects: https://docs.sqlalchemy.org/en/14/dialects/\n.. _Dialect Documentation: https://googleapis.dev/python/sqlalchemy-bigquery/latest\n.. _Product Documentation: https://cloud.google.com/bigquery/docs/\n\n\nQuick Start\n-----------\n\nIn order to use this library, you first need to go through the following steps:\n\n1. `Select or create a Cloud Platform project.`_\n2. [Optional] `Enable billing for your project.`_\n3. `Enable the BigQuery Storage API.`_\n4. `Setup Authentication.`_\n\n.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project\n.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project\n.. _Enable the BigQuery Storage API.: https://console.cloud.google.com/apis/library/bigquery.googleapis.com\n.. _Setup Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html\n\n\nInstallation\n------------\n\nInstall this library in a `virtualenv`_ using pip. `virtualenv`_ is a tool to\ncreate isolated Python environments. The basic problem it addresses is one of\ndependencies and versions, and indirectly permissions.\n\nWith `virtualenv`_, it's possible to install this library without needing system\ninstall permissions, and without clashing with the installed system\ndependencies.\n\n.. _`virtualenv`: https://virtualenv.pypa.io/en/latest/\n\n\nSupported Python Versions\n^^^^^^^^^^^^^^^^^^^^^^^^^\nPython >= 3.8\n\nUnsupported Python Versions\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\nPython <= 3.7.\n\n\nMac/Linux\n^^^^^^^^^\n\n.. code-block:: console\n\n    pip install virtualenv\n    virtualenv <your-env>\n    source <your-env>/bin/activate\n    <your-env>/bin/pip install sqlalchemy-bigquery\n\n\nWindows\n^^^^^^^\n\n.. code-block:: console\n\n    pip install virtualenv\n    virtualenv <your-env>\n    <your-env>\\Scripts\\activate\n    <your-env>\\Scripts\\pip.exe install sqlalchemy-bigquery\n\n\nInstallations when processing large datasets\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWhen handling large datasets, you may see speed increases by also installing the\n`bqstorage` dependencies. See the instructions above about creating a virtual \nenvironment and then install `sqlalchemy-bigquery` using the `bqstorage` extras:\n\n.. code-block:: console\n\n    source <your-env>/bin/activate\n    <your-env>/bin/pip install sqlalchemy-bigquery[bqstorage]\n\n\nUsage\n-----\n\nSQLAlchemy\n^^^^^^^^^^\n\n.. code-block:: python\n\n    from sqlalchemy import *\n    from sqlalchemy.engine import create_engine\n    from sqlalchemy.schema import *\n    engine = create_engine('bigquery://project')\n    table = Table('dataset.table', MetaData(bind=engine), autoload=True)\n    print(select([func.count('*')], from_obj=table().scalar()))\n\n\nProject\n^^^^^^^\n\n``project`` in ``bigquery://project`` is used to instantiate BigQuery client with the specific project ID. To infer project from the environment, use ``bigquery://`` \u2013\u00a0without ``project``\n\nAuthentication\n^^^^^^^^^^^^^^\n\nFollow the `Google Cloud library guide <https://google-cloud-python.readthedocs.io/en/latest/core/auth.html>`_ for authentication. \n\nAlternatively, you can choose either of the following approaches:\n\n* provide the path to a service account JSON file in ``create_engine()`` using the ``credentials_path`` parameter:\n\n.. code-block:: python\n\n    # provide the path to a service account JSON file\n    engine = create_engine('bigquery://', credentials_path='/path/to/keyfile.json')\n\n* pass the credentials in ``create_engine()`` as a Python dictionary using the ``credentials_info`` parameter:\n\n.. code-block:: python\n    \n    # provide credentials as a Python dictionary\n    credentials_info = {\n        \"type\": \"service_account\", \n        \"project_id\": \"your-service-account-project-id\"\n    },\n    engine = create_engine('bigquery://', credentials_info=credentials_info)\n\nLocation\n^^^^^^^^\n\nTo specify location of your datasets pass ``location`` to ``create_engine()``:\n\n.. code-block:: python\n\n    engine = create_engine('bigquery://project', location=\"asia-northeast1\")\n\n\nTable names\n^^^^^^^^^^^\n\nTo query tables from non-default projects or datasets, use the following format for the SQLAlchemy schema name: ``[project.]dataset``, e.g.:\n\n.. code-block:: python\n\n    # If neither dataset nor project are the default\n    sample_table_1 = Table('natality', schema='bigquery-public-data.samples')\n    # If just dataset is not the default\n    sample_table_2 = Table('natality', schema='bigquery-public-data')\n\nBatch size\n^^^^^^^^^^\n\nBy default, ``arraysize`` is set to ``5000``. ``arraysize`` is used to set the batch size for fetching results. To change it, pass ``arraysize`` to ``create_engine()``:\n\n.. code-block:: python\n\n    engine = create_engine('bigquery://project', arraysize=1000)\n\nPage size for dataset.list_tables\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBy default, ``list_tables_page_size`` is set to ``1000``. ``list_tables_page_size`` is used to set the max_results for `dataset.list_tables`_ operation. To change it, pass ``list_tables_page_size`` to ``create_engine()``:\n\n.. _`dataset.list_tables`: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list\n.. code-block:: python\n\n    engine = create_engine('bigquery://project', list_tables_page_size=100)\n\nAdding a Default Dataset\n^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf you want to have the ``Client`` use a default dataset, specify it as the \"database\" portion of the connection string.\n\n.. code-block:: python\n\n    engine = create_engine('bigquery://project/dataset')\n\nWhen using a default dataset, don't include the dataset name in the table name, e.g.:\n\n.. code-block:: python\n\n    table = Table('table_name')\n\nNote that specifying a default dataset doesn't restrict execution of queries to that particular dataset when using raw queries, e.g.:\n\n.. code-block:: python\n\n    # Set default dataset to dataset_a\n    engine = create_engine('bigquery://project/dataset_a')\n\n    # This will still execute and return rows from dataset_b\n    engine.execute('SELECT * FROM dataset_b.table').fetchall()\n\n\nConnection String Parameters\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThere are many situations where you can't call ``create_engine`` directly, such as when using tools like `Flask SQLAlchemy <http://flask-sqlalchemy.pocoo.org/2.3/>`_. For situations like these, or for situations where you want the ``Client`` to have a `default_query_job_config <https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/generated/google.cloud.bigquery.client.Client.html#google.cloud.bigquery.client.Client>`_, you can pass many arguments in the query of the connection string.\n\nThe ``credentials_path``, ``credentials_info``, ``credentials_base64``, ``location``, ``arraysize`` and ``list_tables_page_size`` parameters are used by this library, and the rest are used to create a `QueryJobConfig <https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/generated/google.cloud.bigquery.job.QueryJobConfig.html#google.cloud.bigquery.job.QueryJobConfig>`_\n\nNote that if you want to use query strings, it will be more reliable if you use three slashes, so ``'bigquery:///?a=b'`` will work reliably, but ``'bigquery://?a=b'`` might be interpreted as having a \"database\" of ``?a=b``, depending on the system being used to parse the connection string.\n\nHere are examples of all the supported arguments. Any not present are either for legacy sql (which isn't supported by this library), or are too complex and are not implemented.\n\n.. code-block:: python\n\n    engine = create_engine(\n        'bigquery://some-project/some-dataset' '?'\n        'credentials_path=/some/path/to.json' '&'\n        'location=some-location' '&'\n        'arraysize=1000' '&'\n        'list_tables_page_size=100' '&'\n        'clustering_fields=a,b,c' '&'\n        'create_disposition=CREATE_IF_NEEDED' '&'\n        'destination=different-project.different-dataset.table' '&'\n        'destination_encryption_configuration=some-configuration' '&'\n        'dry_run=true' '&'\n        'labels=a:b,c:d' '&'\n        'maximum_bytes_billed=1000' '&'\n        'priority=INTERACTIVE' '&'\n        'schema_update_options=ALLOW_FIELD_ADDITION,ALLOW_FIELD_RELAXATION' '&'\n        'use_query_cache=true' '&'\n        'write_disposition=WRITE_APPEND'\n    )\n\nIn cases where you wish to include the full credentials in the connection URI you can base64 the credentials JSON file and supply the encoded string to the ``credentials_base64`` parameter.\n\n.. code-block:: python\n\n    engine = create_engine(\n        'bigquery://some-project/some-dataset' '?'\n        'credentials_base64=eyJrZXkiOiJ2YWx1ZSJ9Cg==' '&'\n        'location=some-location' '&'\n        'arraysize=1000' '&'\n        'list_tables_page_size=100' '&'\n        'clustering_fields=a,b,c' '&'\n        'create_disposition=CREATE_IF_NEEDED' '&'\n        'destination=different-project.different-dataset.table' '&'\n        'destination_encryption_configuration=some-configuration' '&'\n        'dry_run=true' '&'\n        'labels=a:b,c:d' '&'\n        'maximum_bytes_billed=1000' '&'\n        'priority=INTERACTIVE' '&'\n        'schema_update_options=ALLOW_FIELD_ADDITION,ALLOW_FIELD_RELAXATION' '&'\n        'use_query_cache=true' '&'\n        'write_disposition=WRITE_APPEND'\n    )\n\nTo create the base64 encoded string you can use the command line tool ``base64``, or ``openssl base64``, or ``python -m base64``.\n\nAlternatively, you can use an online generator like `www.base64encode.org <https://www.base64encode.org>_` to paste your credentials JSON file to be encoded.\n\n\nSupplying Your Own BigQuery Client\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThe above connection string parameters allow you to influence how the BigQuery client used to execute your queries will be instantiated.\nIf you need additional control, you can supply a BigQuery client of your own:\n\n.. code-block:: python\n\n    from google.cloud import bigquery\n\n    custom_bq_client = bigquery.Client(...)\n\n    engine = create_engine(\n        'bigquery://some-project/some-dataset?user_supplied_client=True',\n\t    connect_args={'client': custom_bq_client},\n    )\n\n\nCreating tables\n^^^^^^^^^^^^^^^\n\nTo add metadata to a table:\n\n.. code-block:: python\n\n    table = Table('mytable', ...,\n        bigquery_description='my table description',\n        bigquery_friendly_name='my table friendly name',\n        bigquery_default_rounding_mode=\"ROUND_HALF_EVEN\",\n        bigquery_expiration_timestamp=datetime.datetime.fromisoformat(\"2038-01-01T00:00:00+00:00\"),\n    )\n\nTo add metadata to a column:\n\n.. code-block:: python\n\n    Column('mycolumn', doc='my column description')\n\nTo create a clustered table:\n\n.. code-block:: python\n\n    table = Table('mytable', ..., bigquery_clustering_fields=[\"a\", \"b\", \"c\"])\n\nTo create a time-unit column-partitioned table:\n\n.. code-block:: python\n\n    from google.cloud import bigquery\n\n    table = Table('mytable', ...,\n        bigquery_time_partitioning=bigquery.TimePartitioning(\n            field=\"mytimestamp\",\n            type_=\"MONTH\",\n            expiration_ms=1000 * 60 * 60 * 24 * 30 * 6, # 6 months\n        ),\n        bigquery_require_partition_filter=True,\n    )\n\nTo create an ingestion-time partitioned table:\n\n.. code-block:: python\n\n    from google.cloud import bigquery\n\n    table = Table('mytable', ...,\n        bigquery_time_partitioning=bigquery.TimePartitioning(),\n        bigquery_require_partition_filter=True,\n    )\n\nTo create an integer-range partitioned table\n\n.. code-block:: python\n\n    from google.cloud import bigquery\n\n    table = Table('mytable', ...,\n        bigquery_range_partitioning=bigquery.RangePartitioning(\n            field=\"zipcode\",\n            range_=bigquery.PartitionRange(start=0, end=100000, interval=10),\n        ),\n        bigquery_require_partition_filter=True,\n    )\n\n\nThreading and Multiprocessing\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBecause this client uses the `grpc` library, it's safe to\nshare instances across threads.\n\nIn multiprocessing scenarios, the best\npractice is to create client instances *after* the invocation of\n`os.fork` by `multiprocessing.pool.Pool` or\n`multiprocessing.Process`.\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "SQLAlchemy dialect for BigQuery",
    "version": "1.12.0",
    "project_urls": {
        "Homepage": "https://github.com/googleapis/python-bigquery-sqlalchemy"
    },
    "split_keywords": [
        "bigquery",
        " sqlalchemy"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "7dba684540c3970f70ba68405283671dd23fd42fd7de559bf0aea5bf4117c9e7",
                "md5": "c17d7938cad56c0cd20fa4307670d5bf",
                "sha256": "5b2b77bdaefe9c0663db213d9475a5abbae88fa46108c352d19fa6fc51a47a1a"
            },
            "downloads": -1,
            "filename": "sqlalchemy_bigquery-1.12.0-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c17d7938cad56c0cd20fa4307670d5bf",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": "<3.13,>=3.8",
            "size": 38258,
            "upload_time": "2024-10-02T21:32:48",
            "upload_time_iso_8601": "2024-10-02T21:32:48.593080Z",
            "url": "https://files.pythonhosted.org/packages/7d/ba/684540c3970f70ba68405283671dd23fd42fd7de559bf0aea5bf4117c9e7/sqlalchemy_bigquery-1.12.0-py2.py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "75396d2fb718e61d18c07cfc3de84362c198aa429e3dcf3c1d0a1e476e474196",
                "md5": "08f739c02a301f6e1778a3bbd99c08f9",
                "sha256": "12783ad83ffad34e8e6e14046cb14bb2f1a3e7fb52676f5a24e940ff5cdeb864"
            },
            "downloads": -1,
            "filename": "sqlalchemy_bigquery-1.12.0.tar.gz",
            "has_sig": false,
            "md5_digest": "08f739c02a301f6e1778a3bbd99c08f9",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.13,>=3.8",
            "size": 113993,
            "upload_time": "2024-10-02T21:32:50",
            "upload_time_iso_8601": "2024-10-02T21:32:50.685000Z",
            "url": "https://files.pythonhosted.org/packages/75/39/6d2fb718e61d18c07cfc3de84362c198aa429e3dcf3c1d0a1e476e474196/sqlalchemy_bigquery-1.12.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-10-02 21:32:50",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "googleapis",
    "github_project": "python-bigquery-sqlalchemy",
    "travis_ci": false,
    "coveralls": true,
    "github_actions": false,
    "lcname": "sqlalchemy-bigquery"
}
        
Elapsed time: 0.33344s