aurora-data-api


Nameaurora-data-api JSON
Version 0.5.0 PyPI version JSON
download
home_pagehttps://github.com/chanzuckerberg/aurora-data-api
SummaryA Python DB-API 2.0 client for the AWS Aurora Serverless Data API
upload_time2023-12-29 18:37:05
maintainer
docs_urlNone
authorAndrey Kislyuk
requires_python
licenseApache Software License
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            aurora-data-api - A Python DB-API 2.0 client for the AWS Aurora Serverless Data API
===================================================================================

Installation
------------
::

    pip install aurora-data-api

Prerequisites
-------------
* Set up an
  `AWS Aurora Serverless cluster <https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html>`_
  and enable Data API access for it. If you have previously set up an Aurora Serverless cluster, you can enable Data API
  with the following `AWS CLI <https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html>`_ command::

      aws rds modify-db-cluster --db-cluster-identifier DB_CLUSTER_NAME --enable-http-endpoint --apply-immediately

* Save the database credentials in
  `AWS Secrets Manager <https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html>`_ using a format
  expected by the Data API (a JSON object with the keys ``username`` and ``password``)::

      aws secretsmanager create-secret --name rds-db-credentials/MY_DB
      aws secretsmanager put-secret-value --secret-id rds-db-credentials/MY_DB --secret-string "$(jq -n '.username=env.PGUSER | .password=env.PGPASSWORD')"

* Configure your AWS command line credentials using
  `standard AWS conventions <https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html>`_.
  You can verify that everything works correctly by running a test query via the AWS CLI::

      aws rds-data execute-statement --resource-arn RESOURCE_ARN --secret-arn SECRET_ARN --sql "select * from pg_catalog.pg_tables"

  * Here, RESOURCE_ARN refers to the Aurora RDS database ARN, which can be found in the
    `AWS RDS Console <https://console.aws.amazon.com/rds/home#databases:>`_ (click on your database, then "Configuration")
    or in the CLI by running ``aws rds describe-db-clusters``. SECRET_ARN refers to the AWS Secrets Manager secret
    created above.

  * When running deployed code (on an EC2 instance, ECS/EKS container, or Lambda), you can use the managed IAM policy
    **AmazonRDSDataFullAccess** to grant your IAM role permissions to access the RDS Data API (while this policy is
    convenient for testing, we recommend that you create your own scoped down least-privilege policy for production
    applications).

Usage
-----
Use this module as you would use any DB-API compatible driver module. The ``aurora_data_api.connect()`` method is
the standard main entry point, and accepts two implementation-specific keyword arguments:

* ``aurora_cluster_arn`` (also referred to as ``resourceArn`` in the
  `Data API documentation <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rds-data.html>`_)

  * If not given as a keyword argument, this can also be specified using the ``AURORA_CLUSTER_ARN`` environment variable

* ``secret_arn`` (the database credentials secret)

  * If not given as a keyword argument, this can also be specified using the ``AURORA_SECRET_ARN`` environment variable

.. code-block:: python

    import aurora_data_api

    cluster_arn = "arn:aws:rds:us-east-1:123456789012:cluster:my-aurora-serverless-cluster"
    secret_arn = "arn:aws:secretsmanager:us-east-1:123456789012:secret:rds-db-credentials/MY_DB"
    with aurora_data_api.connect(aurora_cluster_arn=cluster_arn, secret_arn=secret_arn, database="my_db") as conn:
        with conn.cursor() as cursor:
            cursor.execute("select * from pg_catalog.pg_tables")
            print(cursor.fetchall())

The cursor supports iteration (and automatically wraps the query in a server-side cursor and paginates it if required):

.. code-block:: python

    with conn.cursor() as cursor:
        for row in cursor.execute("select * from pg_catalog.pg_tables"):
            print(row)

Motivation
----------
The `RDS Data API <https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html>`_ is the link between the
AWS Lambda serverless environment and the sophisticated features provided by PostgreSQL and MySQL. The Data API tunnels
SQL over HTTP, which has advantages in the context of AWS Lambda:

* It eliminates the need to open database ports to the AWS Lambda public IP address pool
* It uses stateless HTTP connections instead of stateful internal TCP connection pools used by most database drivers
  (the stateful pools become invalid after going through
  `AWS Lambda freeze-thaw cycles <https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html>`_, causing
  connection errors and burdening the database server with abandoned invalid connections)
* It uses AWS role-based authentication, eliminating the need for the Lambda to handle database credentials directly

Links
-----
* `Project home page (GitHub) <https://github.com/chanzuckerberg/aurora-data-api>`_
* `Package distribution (PyPI) <https://pypi.python.org/pypi/aurora-data-api>`_
* `Change log <https://github.com/chanzuckerberg/aurora-data-api/blob/master/Changes.rst>`_
* `sqlalchemy-aurora-data-api <https://github.com/chanzuckerberg/sqlalchemy-aurora-data-api>`_, a SQLAlchemy dialect
  that uses aurora-data-api

Bugs
~~~~
Please report bugs, issues, feature requests, etc. on `GitHub <https://github.com/chanzuckerberg/aurora-data-api/issues>`_.

License
-------
Licensed under the terms of the `Apache License, Version 2.0 <http://www.apache.org/licenses/LICENSE-2.0>`_.

.. image:: https://travis-ci.org/chanzuckerberg/aurora-data-api.png
        :target: https://travis-ci.org/chanzuckerberg/aurora-data-api
.. image:: https://codecov.io/github/chanzuckerberg/aurora-data-api/coverage.svg?branch=master
        :target: https://codecov.io/github/chanzuckerberg/aurora-data-api?branch=master
.. image:: https://img.shields.io/pypi/v/aurora-data-api.svg
        :target: https://pypi.python.org/pypi/aurora-data-api
.. image:: https://img.shields.io/pypi/l/aurora-data-api.svg
        :target: https://pypi.python.org/pypi/aurora-data-api

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/chanzuckerberg/aurora-data-api",
    "name": "aurora-data-api",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "",
    "author": "Andrey Kislyuk",
    "author_email": "kislyuk@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/46/02/14d35e980880ce22b00801e6008ad2c58c5796cdb8deaef0d4bc985d94cc/aurora-data-api-0.5.0.tar.gz",
    "platform": "MacOS X",
    "description": "aurora-data-api - A Python DB-API 2.0 client for the AWS Aurora Serverless Data API\n===================================================================================\n\nInstallation\n------------\n::\n\n    pip install aurora-data-api\n\nPrerequisites\n-------------\n* Set up an\n  `AWS Aurora Serverless cluster <https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html>`_\n  and enable Data API access for it. If you have previously set up an Aurora Serverless cluster, you can enable Data API\n  with the following `AWS CLI <https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html>`_ command::\n\n      aws rds modify-db-cluster --db-cluster-identifier DB_CLUSTER_NAME --enable-http-endpoint --apply-immediately\n\n* Save the database credentials in\n  `AWS Secrets Manager <https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html>`_ using a format\n  expected by the Data API (a JSON object with the keys ``username`` and ``password``)::\n\n      aws secretsmanager create-secret --name rds-db-credentials/MY_DB\n      aws secretsmanager put-secret-value --secret-id rds-db-credentials/MY_DB --secret-string \"$(jq -n '.username=env.PGUSER | .password=env.PGPASSWORD')\"\n\n* Configure your AWS command line credentials using\n  `standard AWS conventions <https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html>`_.\n  You can verify that everything works correctly by running a test query via the AWS CLI::\n\n      aws rds-data execute-statement --resource-arn RESOURCE_ARN --secret-arn SECRET_ARN --sql \"select * from pg_catalog.pg_tables\"\n\n  * Here, RESOURCE_ARN refers to the Aurora RDS database ARN, which can be found in the\n    `AWS RDS Console <https://console.aws.amazon.com/rds/home#databases:>`_ (click on your database, then \"Configuration\")\n    or in the CLI by running ``aws rds describe-db-clusters``. SECRET_ARN refers to the AWS Secrets Manager secret\n    created above.\n\n  * When running deployed code (on an EC2 instance, ECS/EKS container, or Lambda), you can use the managed IAM policy\n    **AmazonRDSDataFullAccess** to grant your IAM role permissions to access the RDS Data API (while this policy is\n    convenient for testing, we recommend that you create your own scoped down least-privilege policy for production\n    applications).\n\nUsage\n-----\nUse this module as you would use any DB-API compatible driver module. The ``aurora_data_api.connect()`` method is\nthe standard main entry point, and accepts two implementation-specific keyword arguments:\n\n* ``aurora_cluster_arn`` (also referred to as ``resourceArn`` in the\n  `Data API documentation <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rds-data.html>`_)\n\n  * If not given as a keyword argument, this can also be specified using the ``AURORA_CLUSTER_ARN`` environment variable\n\n* ``secret_arn`` (the database credentials secret)\n\n  * If not given as a keyword argument, this can also be specified using the ``AURORA_SECRET_ARN`` environment variable\n\n.. code-block:: python\n\n    import aurora_data_api\n\n    cluster_arn = \"arn:aws:rds:us-east-1:123456789012:cluster:my-aurora-serverless-cluster\"\n    secret_arn = \"arn:aws:secretsmanager:us-east-1:123456789012:secret:rds-db-credentials/MY_DB\"\n    with aurora_data_api.connect(aurora_cluster_arn=cluster_arn, secret_arn=secret_arn, database=\"my_db\") as conn:\n        with conn.cursor() as cursor:\n            cursor.execute(\"select * from pg_catalog.pg_tables\")\n            print(cursor.fetchall())\n\nThe cursor supports iteration (and automatically wraps the query in a server-side cursor and paginates it if required):\n\n.. code-block:: python\n\n    with conn.cursor() as cursor:\n        for row in cursor.execute(\"select * from pg_catalog.pg_tables\"):\n            print(row)\n\nMotivation\n----------\nThe `RDS Data API <https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html>`_ is the link between the\nAWS Lambda serverless environment and the sophisticated features provided by PostgreSQL and MySQL. The Data API tunnels\nSQL over HTTP, which has advantages in the context of AWS Lambda:\n\n* It eliminates the need to open database ports to the AWS Lambda public IP address pool\n* It uses stateless HTTP connections instead of stateful internal TCP connection pools used by most database drivers\n  (the stateful pools become invalid after going through\n  `AWS Lambda freeze-thaw cycles <https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html>`_, causing\n  connection errors and burdening the database server with abandoned invalid connections)\n* It uses AWS role-based authentication, eliminating the need for the Lambda to handle database credentials directly\n\nLinks\n-----\n* `Project home page (GitHub) <https://github.com/chanzuckerberg/aurora-data-api>`_\n* `Package distribution (PyPI) <https://pypi.python.org/pypi/aurora-data-api>`_\n* `Change log <https://github.com/chanzuckerberg/aurora-data-api/blob/master/Changes.rst>`_\n* `sqlalchemy-aurora-data-api <https://github.com/chanzuckerberg/sqlalchemy-aurora-data-api>`_, a SQLAlchemy dialect\n  that uses aurora-data-api\n\nBugs\n~~~~\nPlease report bugs, issues, feature requests, etc. on `GitHub <https://github.com/chanzuckerberg/aurora-data-api/issues>`_.\n\nLicense\n-------\nLicensed under the terms of the `Apache License, Version 2.0 <http://www.apache.org/licenses/LICENSE-2.0>`_.\n\n.. image:: https://travis-ci.org/chanzuckerberg/aurora-data-api.png\n        :target: https://travis-ci.org/chanzuckerberg/aurora-data-api\n.. image:: https://codecov.io/github/chanzuckerberg/aurora-data-api/coverage.svg?branch=master\n        :target: https://codecov.io/github/chanzuckerberg/aurora-data-api?branch=master\n.. image:: https://img.shields.io/pypi/v/aurora-data-api.svg\n        :target: https://pypi.python.org/pypi/aurora-data-api\n.. image:: https://img.shields.io/pypi/l/aurora-data-api.svg\n        :target: https://pypi.python.org/pypi/aurora-data-api\n",
    "bugtrack_url": null,
    "license": "Apache Software License",
    "summary": "A Python DB-API 2.0 client for the AWS Aurora Serverless Data API",
    "version": "0.5.0",
    "project_urls": {
        "Homepage": "https://github.com/chanzuckerberg/aurora-data-api"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "5da42a70f608a769f1cdf6ea58cf23aacd3b8a628ce8d8865a032672bb50bf4f",
                "md5": "23fcfa5bdbed6fa6dc26ba2923603e46",
                "sha256": "defe1e7b2a1d4e943538301240e1d161068129db1a534b374dad29aa76445db5"
            },
            "downloads": -1,
            "filename": "aurora_data_api-0.5.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "23fcfa5bdbed6fa6dc26ba2923603e46",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 24396,
            "upload_time": "2023-12-29T18:37:03",
            "upload_time_iso_8601": "2023-12-29T18:37:03.572542Z",
            "url": "https://files.pythonhosted.org/packages/5d/a4/2a70f608a769f1cdf6ea58cf23aacd3b8a628ce8d8865a032672bb50bf4f/aurora_data_api-0.5.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "460214d35e980880ce22b00801e6008ad2c58c5796cdb8deaef0d4bc985d94cc",
                "md5": "639cab10933b776e0321db7f68022005",
                "sha256": "27db80b7ba2f4e8c0e9b8b5e4fd4d1a14bc0442f8db367812954e6f4eae09eed"
            },
            "downloads": -1,
            "filename": "aurora-data-api-0.5.0.tar.gz",
            "has_sig": false,
            "md5_digest": "639cab10933b776e0321db7f68022005",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 28756,
            "upload_time": "2023-12-29T18:37:05",
            "upload_time_iso_8601": "2023-12-29T18:37:05.466823Z",
            "url": "https://files.pythonhosted.org/packages/46/02/14d35e980880ce22b00801e6008ad2c58c5796cdb8deaef0d4bc985d94cc/aurora-data-api-0.5.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-12-29 18:37:05",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "chanzuckerberg",
    "github_project": "aurora-data-api",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "aurora-data-api"
}
        
Elapsed time: 0.19976s