==============
py-hive-iomete
==============
py-hive-iomete is a collection of Python `DB-API <http://www.python.org/dev/peps/pep-0249/>`_ and
`SQLAlchemy <http://www.sqlalchemy.org/>`_ interfaces for
`iomete hive <http://hive.apache.org/>`_.
Usage
=====
DB-API
------
.. code-block:: python
from pyhive import hive
connection = hive.connect(
host="<data_plane_host>",
port=<data_plane_port>,
scheme="http", # or "https"
lakehouse="<lakehouse_cluster_name>",
database="default",
username="<username>",
password="<password>"
)
cursor = connection.cursor()
cursor.execute("SELECT * FROM my_awesome_data LIMIT 10")
print(cursor.fetchone())
print(cursor.fetchall())
DB-API (asynchronous)
---------------------
.. code-block:: python
from pyhive import hive
from TCLIService.ttypes import TOperationState
connection = hive.connect(
host="<data_plane_host>",
port=<data_plane_port>,
scheme="http", # or "https"
lakehouse="<lakehouse_cluster_name>",
database="default",
username="<username>",
password="<password>"
)
cursor = connection.cursor()
cursor.execute("SELECT * FROM my_awesome_data LIMIT 10", async_=True)
status = cursor.poll().operationState
while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):
logs = cursor.fetch_logs()
for message in logs:
print(message)
# If needed, an asynchronous query can be cancelled at any time with:
# cursor.cancel()
status = cursor.poll().operationState
print(cursor.fetchall())
SQLAlchemy
----------
First install this package to register it with SQLAlchemy (see ``setup.py``).
.. code-block:: python
from sqlalchemy.engine import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.schema import *
# Possible dialects (hive and iomete are both operate identically):
# hive+http
# hive+https
# iomete+http
# iomete+https
engine = create_engine(
'iomete+https://<username>:<password>@<data_plane_host>:<data_plane_port>/<database>?lakehouse=<lakehouse_cluster_name>')
# Alternatively, "hive" driver could be used as well
# engine = create_engine(
# 'hive+https://<username>:<password>@<data_plane_host>:<data_plane_port>/<database>?lakehouse=<lakehouse_cluster_name>')
session = sessionmaker(bind=engine)()
records = session.query(Table('my_awesome_data', MetaData(bind=engine), autoload=True)) \
.limit(10) \
.all()
print(records)
Note: query generation functionality is not exhaustive or fully tested, but there should be no
problem with raw SQL.
Requirements
============
Install using
- ``pip install 'py-hive-iomete'`` for the DB-API interface
- ``pip install 'py-hive-iomete[sqlalchemy]'`` for the SQLAlchemy interface
py-hive-iomete works with
- Python 2.7 / Python 3
Changelog
=========
See https://github.com/iomete/py-hive-iomete/releases.
Contributing
============
- Changes must come with tests, with the exception of trivial things like fixing comments. See .travis.yml for the test environment setup.
- Notes on project scope:
- This project is intended to be a minimal iomete (hive) client that does that one thing and nothing else.
Features that can be implemented on top of py-hive-iomete, such integration with your favorite data analysis library, are likely out of scope.
- We prefer having a small number of generic features over a large number of specialized, inflexible features.
Updating TCLIService
====================
The TCLIService module is autogenerated using a ``TCLIService.thrift`` file. To update it, the
``generate.py`` file can be used: ``python generate.py <TCLIServiceURL>``. When left blank, the
version for Hive 2.3 will be downloaded.
Raw data
{
"_id": null,
"home_page": "https://github.com/iomete/py-hive-iomete",
"name": "py-hive-iomete",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Vusal Dadalov",
"author_email": "vusal@iomete.com",
"download_url": "https://files.pythonhosted.org/packages/17/00/c170588be1dd0f2bce790d5277a2f4f5230ef85ede8874b040954d7f90f8/py-hive-iomete-1.3.0.tar.gz",
"platform": null,
"description": "==============\npy-hive-iomete\n==============\n\npy-hive-iomete is a collection of Python `DB-API <http://www.python.org/dev/peps/pep-0249/>`_ and\n`SQLAlchemy <http://www.sqlalchemy.org/>`_ interfaces for\n`iomete hive <http://hive.apache.org/>`_.\n\nUsage\n=====\n\nDB-API\n------\n.. code-block:: python\n\n from pyhive import hive\n\n connection = hive.connect(\n host=\"<data_plane_host>\",\n port=<data_plane_port>,\n scheme=\"http\", # or \"https\"\n lakehouse=\"<lakehouse_cluster_name>\",\n database=\"default\",\n username=\"<username>\",\n password=\"<password>\"\n )\n\n cursor = connection.cursor()\n cursor.execute(\"SELECT * FROM my_awesome_data LIMIT 10\")\n\n print(cursor.fetchone())\n print(cursor.fetchall())\n\nDB-API (asynchronous)\n---------------------\n.. code-block:: python\n\n from pyhive import hive\n from TCLIService.ttypes import TOperationState\n\n connection = hive.connect(\n host=\"<data_plane_host>\",\n port=<data_plane_port>,\n scheme=\"http\", # or \"https\"\n lakehouse=\"<lakehouse_cluster_name>\",\n database=\"default\",\n username=\"<username>\",\n password=\"<password>\"\n )\n\n cursor = connection.cursor()\n\n cursor.execute(\"SELECT * FROM my_awesome_data LIMIT 10\", async_=True)\n\n status = cursor.poll().operationState\n\n while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):\n logs = cursor.fetch_logs()\n for message in logs:\n print(message)\n\n # If needed, an asynchronous query can be cancelled at any time with:\n # cursor.cancel()\n\n status = cursor.poll().operationState\n\n print(cursor.fetchall())\n\n\nSQLAlchemy\n----------\nFirst install this package to register it with SQLAlchemy (see ``setup.py``).\n\n.. code-block:: python\n\n from sqlalchemy.engine import create_engine\n from sqlalchemy.orm import sessionmaker\n from sqlalchemy.schema import *\n\n # Possible dialects (hive and iomete are both operate identically):\n # hive+http\n # hive+https\n # iomete+http\n # iomete+https\n engine = create_engine(\n 'iomete+https://<username>:<password>@<data_plane_host>:<data_plane_port>/<database>?lakehouse=<lakehouse_cluster_name>')\n\n # Alternatively, \"hive\" driver could be used as well\n # engine = create_engine(\n # 'hive+https://<username>:<password>@<data_plane_host>:<data_plane_port>/<database>?lakehouse=<lakehouse_cluster_name>')\n\n session = sessionmaker(bind=engine)()\n records = session.query(Table('my_awesome_data', MetaData(bind=engine), autoload=True)) \\\n .limit(10) \\\n .all()\n print(records)\n\nNote: query generation functionality is not exhaustive or fully tested, but there should be no\nproblem with raw SQL.\n\n\nRequirements\n============\n\nInstall using\n\n- ``pip install 'py-hive-iomete'`` for the DB-API interface\n- ``pip install 'py-hive-iomete[sqlalchemy]'`` for the SQLAlchemy interface\n\npy-hive-iomete works with\n\n- Python 2.7 / Python 3\n\nChangelog\n=========\nSee https://github.com/iomete/py-hive-iomete/releases.\n\nContributing\n============\n- Changes must come with tests, with the exception of trivial things like fixing comments. See .travis.yml for the test environment setup.\n- Notes on project scope:\n\n - This project is intended to be a minimal iomete (hive) client that does that one thing and nothing else.\n Features that can be implemented on top of py-hive-iomete, such integration with your favorite data analysis library, are likely out of scope.\n - We prefer having a small number of generic features over a large number of specialized, inflexible features.\n\nUpdating TCLIService\n====================\n\nThe TCLIService module is autogenerated using a ``TCLIService.thrift`` file. To update it, the\n``generate.py`` file can be used: ``python generate.py <TCLIServiceURL>``. When left blank, the\nversion for Hive 2.3 will be downloaded.\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "Python interface to iomete (Hive)",
"version": "1.3.0",
"project_urls": {
"Homepage": "https://github.com/iomete/py-hive-iomete"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7208e2edf7a902bfd7af5afc520280916989f5f1c3925b2d7b39ee7ce74afe7d",
"md5": "e162bc5ecc6a4d61ef2e7653d2975adb",
"sha256": "60c7a7530466cbfefe02aa5f103614ac051cb1d760528599ad91ec533cce9f42"
},
"downloads": -1,
"filename": "py_hive_iomete-1.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e162bc5ecc6a4d61ef2e7653d2975adb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 50640,
"upload_time": "2023-10-19T23:58:56",
"upload_time_iso_8601": "2023-10-19T23:58:56.972181Z",
"url": "https://files.pythonhosted.org/packages/72/08/e2edf7a902bfd7af5afc520280916989f5f1c3925b2d7b39ee7ce74afe7d/py_hive_iomete-1.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1700c170588be1dd0f2bce790d5277a2f4f5230ef85ede8874b040954d7f90f8",
"md5": "fc8e0c93daebda4f03f228131ce8daa5",
"sha256": "44ee33080f4c81bfc339c38fcdbb1d536adb3cd597e304d4c7671f64a0058d37"
},
"downloads": -1,
"filename": "py-hive-iomete-1.3.0.tar.gz",
"has_sig": false,
"md5_digest": "fc8e0c93daebda4f03f228131ce8daa5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 44830,
"upload_time": "2023-10-19T23:58:58",
"upload_time_iso_8601": "2023-10-19T23:58:58.816590Z",
"url": "https://files.pythonhosted.org/packages/17/00/c170588be1dd0f2bce790d5277a2f4f5230ef85ede8874b040954d7f90f8/py-hive-iomete-1.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-19 23:58:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "iomete",
"github_project": "py-hive-iomete",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"lcname": "py-hive-iomete"
}
JSON
