# enverus-developer-api
[](https://badge.fury.io/py/enverus-developer-api)
A thin wrapper around Enverus' Developer API. Handles authentication and token management, pagination and
network-related error handling/retries.
This module is built and tested on Python 3.9 but should work on Python 2.7 and up.
## Install
```commandline
pip install enverus-developer-api
```
## Clients
### Developer API - Version 3
DirectAccess has been rebranded as DeveloperAPI. For version 3 of the API, create an instance of the DeveloperAPIv3 class, providing it your secret_key (not the same as the v2 api_key).
The returned access token will be available as an attribute on the instance (v3.access_token) and the Authorization
header is set automatically.
```python
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(secret_key='<your-secret-key>')
```
Your secret_key can be generated, retrieved and revoked at https://app.enverus.com/provisioning/directaccess
The Developer API Version 3 endpoint documentation can be found at https://app.enverus.com/direct/#/api/explorer/v3/gettingStarted
### Direct Access - Version 2
For version 2 of the API, create an instance of the DirectAccessV2 class, providing it your API key, client id and client secret.
The returned access token will be available as an attribute on the instance (d2.access_token) and the Authorization
header is set automatically
```python
from enverus_developer_api import DirectAccessV2
d2 = DirectAccessV2(
api_key='<your-api-key>',
client_id='<your-client-id>',
client_secret='<your-client-secret>',
)
```
The Direct Access Version 2 endpoint documentation can be found at https://app.enverus.com/direct/#/api/explorer/v2/gettingStarted
## Usage
The functionality outlined below exists for **both** DeveloperAPIv3 and DirectAccessV2 clients.
Only 1 instance of the client needs to be created to perform all your queries. It can execute multiple simultaneous requests if needed,
and will automatically refresh the access_token for the Authorization header if expired.
An access_token is valid for 8 hours, and there is rate limit on the number of access_tokens that can be requested per minute
which is why we recommend creating and reusing a single DeveloperAPIv3 client instance for all of your querying.
Provide the query method the dataset and query params. All query parameters must match the valid
Request Parameters found in the Developer API documentation for a given dataset and be passed as keyword arguments.
```python
for row in v3.query('wells', county='REEVES', deleteddate='null'):
print(row)
```
### Filter functions
Developer API supports filter functions. These can be passed as strings on the keyword arguments.
Some common filters are greater than (`gt()`), less than (`lt()`), null, not null (`not(null)`) and between (`btw()`).
See the Developer API documentation for a list of all available filters.
```python
# Get well records updated after 2018-08-01 and without deleted dates
for row in v3.query('wells', updateddate='gt(2018-08-01)', deleteddate='null'):
print(row)
# Get permit records with approved dates between 2018-03-01 and 2018-06-01
for row in v3.query('rigs', spuddate='btw(2018-03-01,2018-06-01)'):
print(row)
```
You can use the `fields` keyword to limit the returned fields in your request.
```python
for row in v3.query('rigs', fields='PermitApprovedDate,LeaseName,RigName_Number,MD_FT'):
print(row)
```
### Escaping
When making requests containing certain characters like commas, use a backslash to escape them.
```python
# Escaping the comma before LLC
for row in v3.query('rigs', envoperator='PERCUSSION PETROLEUM OPERATING\, LLC'):
print(row)
```
### Network request handling
This module exposes functionality in python-requests for modifying network requests handling, namely:
* retries and backoff
* network proxies
* ssl verification
#### Retries and backoff
Specify the number of retry attempts in `retries` and the backoff factor in `backoff_factor`. See the urllib3
[Retry](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) utility API for more info
```python
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(
secret_key='<your-secret-key>',
retries=5,
backoff_factor=1
)
```
You can specify a network proxy by passing a dictionary with the host and port of your proxy to `proxies`. See the
[proxies](https://requests.readthedocs.io/en/master/user/advanced/#proxies) section of the python-requests documentation
for more info.
```python
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(
secret_key='<your-secret-key>',
proxies={'https': 'http://10.10.1.10:1080'}
)
```
Finally, if you're in an environment that provides its own SSL certificates that might not be in your trusted store,
you can choose to ignore SSL verification altogether. This is typically not a good idea and you should seek to resolve
certificate errors instead of ignore them.
```python
from enverus_developer_api import DeveloperAPIv3
v3 = DeveloperAPIv3(
secret_key='<your-secret-key>',
verify=False
)
```
## Functions
### docs
Returns a sample response for a given dataset
```python
docs = v3.docs("casings")
```
### ddl
Returns a CREATE TABLE DDL statement for a given dataset. Must specify either
"mssql" for MS SQL Server or "pg" for PostgreSQL as the database argument
```python
from tempfile import TemporaryFile
ddl = v3.ddl("casings", database="pg")
with TemporaryFile(mode="w+") as f:
f.write(ddl)
f.seek(0)
for line in f:
print(line, end='')
```
### count
Returns the count of records for a given dataset and query options in the
X-QUERY-RECORD-COUNT response header value
```python
count = v3.count("rigs", deleteddate="null")
```
### query
Accepts a dataset name, request headers and a variable number of keyword arguments that correspond to the fields specified
in the ‘Request Parameters’ section for each dataset in the Developer API documentation.
This method only supports the JSON output provided by the API and yields dicts for each record
```python
for row in v3.query("rigs", pagesize=1000, deleteddate="null"):
print(row)
```
##### X-Omit-Header-Next-Links header
Omit the Next Link in the Response Header section, add the Next Link to the JSON Response Body.
```python
for row in v3.query("rigs", pagesize=1000, deleteddate="null", _headers={'X-Omit-Header-Next-Links': 'true'}):
print(row)
```
### to_csv
Write query results to CSV. Optional keyword arguments are provided to the csv writer object,
allowing control over delimiters, quoting, etc. The default is comma-separated with csv.QUOTE_MINIMAL
```python
import csv, os
from tempfile import mkdtemp
tempdir = mkdtemp()
path = os.path.join(tempdir, "rigs.csv")
dataset = "rigs"
options = dict(pagesize=10000, deleteddate="null")
v3.query(dataset, **options)
v3.to_csv(query, path, log_progress=True, delimiter=",", quoting=csv.QUOTE_MINIMAL)
with open(path, mode="r") as f:
reader = csv.reader(f)
```
### to_dataframe
Write query results to a pandas Dataframe with properly set dtypes and index columns.
This works by requesting the DDL for a given dataset and manipulating the text to build a list of dtypes, date columns and the index column(s).
It then makes a query request for the dataset to ensure we know the exact fields to expect,
(ie, if fields was a provided query parameter and the result will have fewer fields than the DDL).
For endpoints with composite primary keys, a pandas MultiIndex is created.
Query results are written to a temporary CSV file and then read into the dataframe. The CSV is removed afterwards.
Pandas version 0.24.0 or higher is required for use of the Int64 dtype allowing integers with NaN values.
It is not possible to coerce missing values for columns of dtype bool and so these are set to dtype object.
You will need to have pandas installed to use the to_dataframe function
```python
pip install pandas
```
Create a pandas dataframe from a dataset query
```python
df = v3.to_dataframe("rigs", pagesize=10000, deleteddate="null")
```
Create a Texas rigs dataframe, replacing the state abbreviation with the complete name
and removing commas from Operator names
```python
df = v3.to_dataframe(
dataset="rigs",
deleteddate="null",
pagesize=100000,
stateprovince="TX",
converters={
"StateProvince": lambda x: "TEXAS",
"ENVOperator": lambda x: x.replace(",", "")
}
)
df.head(10)
```
Reset the index of the DataFrame, and use the default one instead. [reset_index()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reset_index.html)
```python
df = v3.to_dataframe(dataset, pagesize=10000, ENVBasin="SACRAMENTO")
df.reset_index(inplace=True)
df.head(10)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/enverus-ea/enverus-developer-api",
"name": "enverus-developer-api",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "enverus,drillinginfo,directaccess,oil,gas",
"author": "Direct Access",
"author_email": "directaccess@enverus.com",
"download_url": "https://files.pythonhosted.org/packages/ff/4c/715d78c970960b415ef48b8598281eec272ac39e216a44e63776854bbe2c/enverus-developer-api-3.2.2.tar.gz",
"platform": null,
"description": "# enverus-developer-api\r\n\r\n[](https://badge.fury.io/py/enverus-developer-api)\r\n\r\nA thin wrapper around Enverus' Developer API. Handles authentication and token management, pagination and\r\nnetwork-related error handling/retries. \r\n\r\nThis module is built and tested on Python 3.9 but should work on Python 2.7 and up.\r\n\r\n\r\n## Install\r\n```commandline\r\npip install enverus-developer-api\r\n```\r\n\r\n## Clients\r\n\r\n### Developer API - Version 3\r\nDirectAccess has been rebranded as DeveloperAPI. For version 3 of the API, create an instance of the DeveloperAPIv3 class, providing it your secret_key (not the same as the v2 api_key).\r\nThe returned access token will be available as an attribute on the instance (v3.access_token) and the Authorization\r\nheader is set automatically.\r\n```python\r\nfrom enverus_developer_api import DeveloperAPIv3\r\n\r\nv3 = DeveloperAPIv3(secret_key='<your-secret-key>')\r\n```\r\nYour secret_key can be generated, retrieved and revoked at https://app.enverus.com/provisioning/directaccess\r\n\r\nThe Developer API Version 3 endpoint documentation can be found at https://app.enverus.com/direct/#/api/explorer/v3/gettingStarted\r\n\r\n### Direct Access - Version 2\r\nFor version 2 of the API, create an instance of the DirectAccessV2 class, providing it your API key, client id and client secret.\r\nThe returned access token will be available as an attribute on the instance (d2.access_token) and the Authorization\r\nheader is set automatically\r\n```python\r\nfrom enverus_developer_api import DirectAccessV2\r\n\r\nd2 = DirectAccessV2(\r\n api_key='<your-api-key>',\r\n client_id='<your-client-id>',\r\n client_secret='<your-client-secret>',\r\n)\r\n```\r\nThe Direct Access Version 2 endpoint documentation can be found at https://app.enverus.com/direct/#/api/explorer/v2/gettingStarted\r\n\r\n## Usage\r\n\r\nThe functionality outlined below exists for **both** DeveloperAPIv3 and DirectAccessV2 clients.\r\n\r\nOnly 1 instance of the client needs to be created to perform all your queries. It can execute multiple simultaneous requests if needed,\r\nand will automatically refresh the access_token for the Authorization header if expired.\r\nAn access_token is valid for 8 hours, and there is rate limit on the number of access_tokens that can be requested per minute\r\nwhich is why we recommend creating and reusing a single DeveloperAPIv3 client instance for all of your querying.\r\n\r\nProvide the query method the dataset and query params. All query parameters must match the valid\r\nRequest Parameters found in the Developer API documentation for a given dataset and be passed as keyword arguments.\r\n\r\n```python\r\nfor row in v3.query('wells', county='REEVES', deleteddate='null'):\r\n print(row)\r\n```\r\n\r\n### Filter functions\r\nDeveloper API supports filter functions. These can be passed as strings on the keyword arguments.\r\n\r\nSome common filters are greater than (`gt()`), less than (`lt()`), null, not null (`not(null)`) and between (`btw()`). \r\nSee the Developer API documentation for a list of all available filters.\r\n\r\n```python\r\n# Get well records updated after 2018-08-01 and without deleted dates\r\nfor row in v3.query('wells', updateddate='gt(2018-08-01)', deleteddate='null'):\r\n print(row)\r\n\r\n# Get permit records with approved dates between 2018-03-01 and 2018-06-01\r\nfor row in v3.query('rigs', spuddate='btw(2018-03-01,2018-06-01)'):\r\n print(row) \r\n```\r\n\r\nYou can use the `fields` keyword to limit the returned fields in your request.\r\n\r\n```python\r\nfor row in v3.query('rigs', fields='PermitApprovedDate,LeaseName,RigName_Number,MD_FT'):\r\n print(row)\r\n```\r\n\r\n### Escaping\r\nWhen making requests containing certain characters like commas, use a backslash to escape them.\r\n\r\n```python\r\n# Escaping the comma before LLC\r\nfor row in v3.query('rigs', envoperator='PERCUSSION PETROLEUM OPERATING\\, LLC'):\r\n print(row)\r\n```\r\n\r\n### Network request handling\r\nThis module exposes functionality in python-requests for modifying network requests handling, namely:\r\n* retries and backoff\r\n* network proxies\r\n* ssl verification\r\n\r\n#### Retries and backoff\r\nSpecify the number of retry attempts in `retries` and the backoff factor in `backoff_factor`. See the urllib3\r\n[Retry](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) utility API for more info\r\n```python\r\nfrom enverus_developer_api import DeveloperAPIv3\r\n\r\nv3 = DeveloperAPIv3(\r\n secret_key='<your-secret-key>',\r\n retries=5,\r\n backoff_factor=1\r\n)\r\n```\r\n\r\nYou can specify a network proxy by passing a dictionary with the host and port of your proxy to `proxies`. See the\r\n[proxies](https://requests.readthedocs.io/en/master/user/advanced/#proxies) section of the python-requests documentation\r\nfor more info.\r\n```python\r\nfrom enverus_developer_api import DeveloperAPIv3\r\n\r\nv3 = DeveloperAPIv3(\r\n secret_key='<your-secret-key>',\r\n proxies={'https': 'http://10.10.1.10:1080'}\r\n)\r\n```\r\n\r\nFinally, if you're in an environment that provides its own SSL certificates that might not be in your trusted store,\r\nyou can choose to ignore SSL verification altogether. This is typically not a good idea and you should seek to resolve\r\ncertificate errors instead of ignore them.\r\n```python\r\nfrom enverus_developer_api import DeveloperAPIv3\r\n\r\nv3 = DeveloperAPIv3(\r\n secret_key='<your-secret-key>',\r\n verify=False\r\n)\r\n```\r\n\r\n## Functions\r\n\r\n### docs\r\nReturns a sample response for a given dataset\r\n```python\r\ndocs = v3.docs(\"casings\")\r\n```\r\n\r\n### ddl\r\nReturns a CREATE TABLE DDL statement for a given dataset. Must specify either \r\n\"mssql\" for MS SQL Server or \"pg\" for PostgreSQL as the database argument\r\n```python\r\nfrom tempfile import TemporaryFile\r\n\r\nddl = v3.ddl(\"casings\", database=\"pg\")\r\nwith TemporaryFile(mode=\"w+\") as f:\r\n f.write(ddl)\r\n f.seek(0)\r\n for line in f:\r\n print(line, end='')\r\n```\r\n\r\n### count\r\nReturns the count of records for a given dataset and query options in the \r\nX-QUERY-RECORD-COUNT response header value\r\n```python\r\ncount = v3.count(\"rigs\", deleteddate=\"null\")\r\n```\r\n\r\n### query\r\nAccepts a dataset name, request headers and a variable number of keyword arguments that correspond to the fields specified \r\nin the \u00e2\u20ac\u02dcRequest Parameters\u00e2\u20ac\u2122 section for each dataset in the Developer API documentation.\r\n\r\nThis method only supports the JSON output provided by the API and yields dicts for each record\r\n```python\r\nfor row in v3.query(\"rigs\", pagesize=1000, deleteddate=\"null\"):\r\n print(row)\r\n```\r\n##### X-Omit-Header-Next-Links header\r\nOmit the Next Link in the Response Header section, add the Next Link to the JSON Response Body.\r\n```python\r\nfor row in v3.query(\"rigs\", pagesize=1000, deleteddate=\"null\", _headers={'X-Omit-Header-Next-Links': 'true'}):\r\n print(row)\r\n```\r\n\r\n### to_csv\r\nWrite query results to CSV. Optional keyword arguments are provided to the csv writer object, \r\nallowing control over delimiters, quoting, etc. The default is comma-separated with csv.QUOTE_MINIMAL\r\n```python\r\nimport csv, os\r\nfrom tempfile import mkdtemp\r\n\r\ntempdir = mkdtemp()\r\npath = os.path.join(tempdir, \"rigs.csv\")\r\n\r\ndataset = \"rigs\"\r\noptions = dict(pagesize=10000, deleteddate=\"null\")\r\n\r\nv3.query(dataset, **options)\r\nv3.to_csv(query, path, log_progress=True, delimiter=\",\", quoting=csv.QUOTE_MINIMAL)\r\n\r\nwith open(path, mode=\"r\") as f:\r\n reader = csv.reader(f)\r\n```\r\n\r\n### to_dataframe\r\nWrite query results to a pandas Dataframe with properly set dtypes and index columns.\r\n\r\nThis works by requesting the DDL for a given dataset and manipulating the text to build a list of dtypes, date columns and the index column(s). \r\nIt then makes a query request for the dataset to ensure we know the exact fields to expect, \r\n(ie, if fields was a provided query parameter and the result will have fewer fields than the DDL).\r\n\r\nFor endpoints with composite primary keys, a pandas MultiIndex is created.\r\n\r\nQuery results are written to a temporary CSV file and then read into the dataframe. The CSV is removed afterwards.\r\n\r\nPandas version 0.24.0 or higher is required for use of the Int64 dtype allowing integers with NaN values. \r\nIt is not possible to coerce missing values for columns of dtype bool and so these are set to dtype object.\r\n\r\nYou will need to have pandas installed to use the to_dataframe function\r\n```python\r\npip install pandas\r\n```\r\n\r\nCreate a pandas dataframe from a dataset query\r\n```python\r\ndf = v3.to_dataframe(\"rigs\", pagesize=10000, deleteddate=\"null\")\r\n```\r\n\r\nCreate a Texas rigs dataframe, replacing the state abbreviation with the complete name\r\nand removing commas from Operator names\r\n```python\r\ndf = v3.to_dataframe(\r\n dataset=\"rigs\",\r\n deleteddate=\"null\",\r\n pagesize=100000,\r\n stateprovince=\"TX\",\r\n converters={\r\n \"StateProvince\": lambda x: \"TEXAS\",\r\n \"ENVOperator\": lambda x: x.replace(\",\", \"\")\r\n }\r\n)\r\ndf.head(10)\r\n```\r\nReset the index of the DataFrame, and use the default one instead. [reset_index()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reset_index.html)\r\n```python\r\n df = v3.to_dataframe(dataset, pagesize=10000, ENVBasin=\"SACRAMENTO\")\r\n df.reset_index(inplace=True)\r\n df.head(10)\r\n```\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Enverus Developer API Python Client",
"version": "3.2.2",
"project_urls": {
"Homepage": "https://github.com/enverus-ea/enverus-developer-api"
},
"split_keywords": [
"enverus",
"drillinginfo",
"directaccess",
"oil",
"gas"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e0174499bbb5cfc9d2392e0bb16a2a2df7aa5fab9a3f9d17f6239ad3d19da037",
"md5": "a5444895c20d34766b3265b7fdc4756b",
"sha256": "d24dfe5d4e33d2136b3d2639f33c0649b7f1b4e92c2215df583a607005acf7a2"
},
"downloads": -1,
"filename": "enverus_developer_api-3.2.2-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "a5444895c20d34766b3265b7fdc4756b",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 12745,
"upload_time": "2023-06-20T17:34:10",
"upload_time_iso_8601": "2023-06-20T17:34:10.137421Z",
"url": "https://files.pythonhosted.org/packages/e0/17/4499bbb5cfc9d2392e0bb16a2a2df7aa5fab9a3f9d17f6239ad3d19da037/enverus_developer_api-3.2.2-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ff4c715d78c970960b415ef48b8598281eec272ac39e216a44e63776854bbe2c",
"md5": "2665d5da144d3ed67688f4bac544ea0d",
"sha256": "3ebcb84604dd76d023a2048620bebb6d0ac5b7b12ee80e65a71fb1450371b74e"
},
"downloads": -1,
"filename": "enverus-developer-api-3.2.2.tar.gz",
"has_sig": false,
"md5_digest": "2665d5da144d3ed67688f4bac544ea0d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 17182,
"upload_time": "2023-06-20T17:34:12",
"upload_time_iso_8601": "2023-06-20T17:34:12.135615Z",
"url": "https://files.pythonhosted.org/packages/ff/4c/715d78c970960b415ef48b8598281eec272ac39e216a44e63776854bbe2c/enverus-developer-api-3.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-20 17:34:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "enverus-ea",
"github_project": "enverus-developer-api",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"circle": true,
"requirements": [],
"lcname": "enverus-developer-api"
}
JSON
