| Name | downerhelper JSON |
| Version |
0.1.18
JSON |
| download |
| home_page | None |
| Summary | Collection of functions to wrap the Azure SDK |
| upload_time | 2025-02-16 22:17:26 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Downer Azure Helper
Collection of functions to wrap the Azure SDK and ArcGIS.
Assume argument are of type `str` unless stated otherwise.
## Logging
### Postgres Log Queue
Simple handler to enter logs directly to postgres databases; uses psycopg2 for connection. Creates a new `table` if does not already exist, and groups logs by `job_id`. Logs are stored in a queue and dumped on `save` method invocation. Set `print_logs=True` to print immediately to console as well. Valid log levels are `INFO`, `WARN`, `ERROR`.
#### Quick Setup
```python
from downerhelper.logs import setup_queue
queue = setup_queue(logger_name, job_id, table, db_config_name, keyvault_url, print_logs=False)
queue.add('INFO', "Test message!")
queue.save(throw_error=False: boolean)
```
Store database config in Azure Key Vault with format `<dbname>,<user>,<password>,<host>`.
#### Manual Setup
```python
from downerhelper.logs import PostgresLogQueue
db_config = {
'dbname': <dbname>,
'user': <user>,
'password': <password>,
'host': <host>,
}
queue = PostgresLogQueue(logger_name, job_id, table, db_config: dict(str, str), print_logs=False)
```
Provide database config dictionary.
### Log Check
```python
from downerhelper.logs import setup_queue
queue = setup_queue(logger_name, job_id, table, db_config_name, keyvault_url, print_logs=False)
queue.check_logs(key_url: tuple(str, str), recipients, interval_hours=24: int)
```
Check logs from the past `interval_hours`, sends an email to `recipients` for any log levels that are not `INFO`. Recipients is a string of emails with `;` delimeter. `key_url` is for the logic app to send the email. Throws an exception if error encountered.
## Key Vault Secrets
### Get Secret Value
```python
from downerhelper.secrets import get_secret_value
value = get_secret_value(secret_name, keyvault_url, queue=None: PostgresLogQueue, credential=DefaultAzureCredential()) -> str | None
```
Retrieve the value of a keyvault secret. Returns `None` on error.
### Get Config Dictionary
```python
from downerhelper.secrets import get_config_dict
config = get_config_dict(secret_name, keyvault_url, dbname=None, queue=None: PostgresLogQueue, credential=DefaultAzureCredential()) -> dict
```
Retrieve a database config secret stored like `<dbname>,<user>,<password>,<host>`. Raises an error if config cannot be read.
### Get Key Url Combination
```python
from downerhelper.secrets import get_key_url
key, url = get_key_url(secret_name, keyvault_url, queue=None: PostgresLogQueue, credential=DefaultAzureCredential()) -> tuple(str, str)
```
Retrieve the key/url combination for a logic app. Store secret as `<key>/<url>` to require only one request to get the access info for a logic app (instead of two). Raises an error if secret has incorrect format.
### Form Connection String
```python
from downerhelper.secrets import form_connection_string
string = form_connection_string(storage_account, storage_account_key) -> str
```
Returns the connection string for the target storage account.
## GIS
### ArcGIS Functions
#### Get Access Token
```python
from downerhelper.gis import get_access_token
token = get_access_token(root, username, pwd, queue: PostgresLogQueue) -> str
```
Token has a 60 minute expiry. Raises an error on failure.
#### Query Feature Service/Layer
```python
from downerhelper.gis import query_feature_service
data = query_feature_service(token, feature_service, where, queue: PostgresLogQueue, add_where=None) -> dict
```
`add_where` is an additional `where` clause, to be used if some calls have similar `where` structures. Returns arcgis return structure. Raises an error on failure.
#### Query Attachments
```python
from downerhelper.gis import query_attachments
data = query_attachments(token, feature_service, object_id, queue: PostgresLogQueue) -> dict
```
Find attachments for `object_id`. Returns arcgis return structure. Raises an error on failure.
#### Get Attachment
```python
from downerhelper.gis import get_attachment
attachment = get_attachment(token, feature_service, object_id, attachment_id, queue: PostgresLogQueue)
```
Get attachment data for `attachment_id` on `object_id`. Returns attachment content directly. Raises an error on failure.
#### Update Feature Service
```python
from downerhelper.gis import update_feature_service
update_feature_service(token, feature_service, features: list, queue: PostgresLogQueue)
```
Argument `features` is defined by arcgis docs. Raises an error on failure.
#### Timestamp To Datetime
```python
from downerhelper.gis import timestamp_to_datetime
date_str = timestamp_to_datetime(timestamp: int | str, queue: PostgresLogQueue, tz_str='Pacific/Auckland': str)
```
Convert arcgis timestamp to datetime string. `tz_str` is the target timezone in `pytz.timezone` format. Raises an error on failure.
## Sharepoint
Note: sharepoint list display names (column names) are often not the same as the data label. To find data label you can,
1. sort by the target column, in the url find the `sortField` param.
2. go to column settings and select the target column, then in the url find the `Field` param.
### Get List
```python
from downerhelper.sharepoint import get_list
sp_list = get_list(site_address, list_name, keyvault_url, queue: PostgresLogQueue, pagination_limit=100: int) -> list(dict)
```
Returns sharepoint list items as a list of dictionary data. Keys are sharepoint data labels not display names. Raises an error on failure.
### Form List Item
```python
from downerhelper.sharepoint import form_list_item
form_list_item(attributes: dict(str, str), title, pairs: dict(str, str), queue: PostgresLogQueue, delims=[', ']: list(str)) -> dict
```
Uses arcgis record `attributes` from `query_feature_service` functions. Argument `pairs` represents the mapping of sharepoint data labels (keys of return dict) to arcgis attribute keys (values of return dict). Raises an error on failure.
### Create List Item
```python
from downerhelper.sharepoint import create_list_item
item = create_list_item(item: dict(str, str), site_address, list_name, key_url: tuple(str, str), queue: PostgresLogQueue)
```
Uploads an item to target sharepoint list, where `key_url` represents the key/url access combination of the logic app. Raises an error on failure.
### Upload Attachments
```python
from downerhelper.sharepoint import upload_attachments
upload_attachments(token, feature_service, object_id, site_address, list_name, item_id, key_url: tuple(str, str), queue: PostgresLogQueue)
```
Uploads all attachments for arcgis target `object_id` to sharepoint. Raises an error on failure (one or more attachments failing to upload triggers failure).
### Get List Item Attachments
```python
from downerhelper.sharepoint import get_list_attachments
attachmnets = get_list_attachments(item_id, site_address, list_name, key_url: tuple(str, str), queue: PostgresLogQueue) -> list
```
Returns empty list on failure.
## Logic App
### Send Email (Outlook)
```python
from downerhelper.logicapp import send_email
send_email(key_url: tuple(str, str), recipients: str | list(str), subject, body, attachments=[], cc=[]: str | list(str), bcc=[]: str | list(str), importance="Normal": ["Normal", "Low", "High"]) -> bool
```
`key_url` is for the logic app access. All `recipients`, `cc` and `bcc` are emails, supplied as a string delimited by `;` or a list of strings. `attachments` is a list of dictionaries like
```python
{
"Name": filename: str,
"ContentBytes": content: str | bytes
}
```
where `ContentBytes` is either a base64 encoded string or bytes that will be encoded.
Raises an exception on failure.
## Warning!
The following snippet shows incorrect usage. Modules must be declared and imported seperately.
```python
import downerhelper
value = downerhelper.secrets.get_secret_value(secret_name, keyvault_url)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "downerhelper",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Marcus Oates <marcus.oates@downergroup.com>",
"download_url": "https://files.pythonhosted.org/packages/3f/66/e5af5421d1c7a2b5c78a2a003bc8cd1dacebb67c2f04ffd5ea625b9732a7/downerhelper-0.1.18.tar.gz",
"platform": null,
"description": "# Downer Azure Helper\n\nCollection of functions to wrap the Azure SDK and ArcGIS.\n\nAssume argument are of type `str` unless stated otherwise.\n\n## Logging\n\n### Postgres Log Queue\n\nSimple handler to enter logs directly to postgres databases; uses psycopg2 for connection. Creates a new `table` if does not already exist, and groups logs by `job_id`. Logs are stored in a queue and dumped on `save` method invocation. Set `print_logs=True` to print immediately to console as well. Valid log levels are `INFO`, `WARN`, `ERROR`.\n\n#### Quick Setup\n\n```python\nfrom downerhelper.logs import setup_queue\n\nqueue = setup_queue(logger_name, job_id, table, db_config_name, keyvault_url, print_logs=False)\nqueue.add('INFO', \"Test message!\")\nqueue.save(throw_error=False: boolean)\n```\n\nStore database config in Azure Key Vault with format `<dbname>,<user>,<password>,<host>`.\n\n#### Manual Setup \n\n```python\nfrom downerhelper.logs import PostgresLogQueue\n\ndb_config = {\n 'dbname': <dbname>,\n 'user': <user>,\n 'password': <password>,\n 'host': <host>,\n}\n\nqueue = PostgresLogQueue(logger_name, job_id, table, db_config: dict(str, str), print_logs=False)\n```\n\nProvide database config dictionary.\n\n### Log Check\n\n```python\nfrom downerhelper.logs import setup_queue\n\nqueue = setup_queue(logger_name, job_id, table, db_config_name, keyvault_url, print_logs=False)\nqueue.check_logs(key_url: tuple(str, str), recipients, interval_hours=24: int)\n```\n\nCheck logs from the past `interval_hours`, sends an email to `recipients` for any log levels that are not `INFO`. Recipients is a string of emails with `;` delimeter. `key_url` is for the logic app to send the email. Throws an exception if error encountered.\n\n## Key Vault Secrets\n\n### Get Secret Value\n\n```python\nfrom downerhelper.secrets import get_secret_value\n\nvalue = get_secret_value(secret_name, keyvault_url, queue=None: PostgresLogQueue, credential=DefaultAzureCredential()) -> str | None\n```\n\nRetrieve the value of a keyvault secret. Returns `None` on error.\n\n### Get Config Dictionary\n\n```python\nfrom downerhelper.secrets import get_config_dict\n\nconfig = get_config_dict(secret_name, keyvault_url, dbname=None, queue=None: PostgresLogQueue, credential=DefaultAzureCredential()) -> dict\n```\n\nRetrieve a database config secret stored like `<dbname>,<user>,<password>,<host>`. Raises an error if config cannot be read.\n\n### Get Key Url Combination\n\n```python\nfrom downerhelper.secrets import get_key_url\n\nkey, url = get_key_url(secret_name, keyvault_url, queue=None: PostgresLogQueue, credential=DefaultAzureCredential()) -> tuple(str, str)\n```\n\nRetrieve the key/url combination for a logic app. Store secret as `<key>/<url>` to require only one request to get the access info for a logic app (instead of two). Raises an error if secret has incorrect format.\n\n### Form Connection String\n\n```python\nfrom downerhelper.secrets import form_connection_string\n\nstring = form_connection_string(storage_account, storage_account_key) -> str\n```\n\nReturns the connection string for the target storage account.\n\n## GIS\n\n### ArcGIS Functions\n\n#### Get Access Token\n\n```python\nfrom downerhelper.gis import get_access_token\n\ntoken = get_access_token(root, username, pwd, queue: PostgresLogQueue) -> str\n```\n\nToken has a 60 minute expiry. Raises an error on failure. \n\n#### Query Feature Service/Layer\n\n```python\nfrom downerhelper.gis import query_feature_service\n\ndata = query_feature_service(token, feature_service, where, queue: PostgresLogQueue, add_where=None) -> dict\n```\n\n`add_where` is an additional `where` clause, to be used if some calls have similar `where` structures. Returns arcgis return structure. Raises an error on failure.\n\n#### Query Attachments\n\n```python\nfrom downerhelper.gis import query_attachments\n\ndata = query_attachments(token, feature_service, object_id, queue: PostgresLogQueue) -> dict\n```\n\nFind attachments for `object_id`. Returns arcgis return structure. Raises an error on failure.\n\n#### Get Attachment\n\n```python\nfrom downerhelper.gis import get_attachment\n\nattachment = get_attachment(token, feature_service, object_id, attachment_id, queue: PostgresLogQueue)\n```\n\nGet attachment data for `attachment_id` on `object_id`. Returns attachment content directly. Raises an error on failure.\n\n#### Update Feature Service\n\n```python\nfrom downerhelper.gis import update_feature_service\n\nupdate_feature_service(token, feature_service, features: list, queue: PostgresLogQueue)\n```\n\nArgument `features` is defined by arcgis docs. Raises an error on failure.\n\n#### Timestamp To Datetime\n\n```python\nfrom downerhelper.gis import timestamp_to_datetime\n\ndate_str = timestamp_to_datetime(timestamp: int | str, queue: PostgresLogQueue, tz_str='Pacific/Auckland': str)\n```\n\nConvert arcgis timestamp to datetime string. `tz_str` is the target timezone in `pytz.timezone` format. Raises an error on failure.\n\n## Sharepoint\n\nNote: sharepoint list display names (column names) are often not the same as the data label. To find data label you can,\n \n1. sort by the target column, in the url find the `sortField` param.\n2. go to column settings and select the target column, then in the url find the `Field` param.\n\n### Get List\n\n```python\nfrom downerhelper.sharepoint import get_list\n\nsp_list = get_list(site_address, list_name, keyvault_url, queue: PostgresLogQueue, pagination_limit=100: int) -> list(dict)\n```\n\nReturns sharepoint list items as a list of dictionary data. Keys are sharepoint data labels not display names. Raises an error on failure.\n\n### Form List Item\n\n```python\nfrom downerhelper.sharepoint import form_list_item\n\nform_list_item(attributes: dict(str, str), title, pairs: dict(str, str), queue: PostgresLogQueue, delims=[', ']: list(str)) -> dict\n```\n\nUses arcgis record `attributes` from `query_feature_service` functions. Argument `pairs` represents the mapping of sharepoint data labels (keys of return dict) to arcgis attribute keys (values of return dict). Raises an error on failure.\n\n### Create List Item\n\n```python\nfrom downerhelper.sharepoint import create_list_item\n\nitem = create_list_item(item: dict(str, str), site_address, list_name, key_url: tuple(str, str), queue: PostgresLogQueue)\n```\n\nUploads an item to target sharepoint list, where `key_url` represents the key/url access combination of the logic app. Raises an error on failure.\n\n### Upload Attachments\n\n```python\nfrom downerhelper.sharepoint import upload_attachments\n\nupload_attachments(token, feature_service, object_id, site_address, list_name, item_id, key_url: tuple(str, str), queue: PostgresLogQueue)\n```\n\nUploads all attachments for arcgis target `object_id` to sharepoint. Raises an error on failure (one or more attachments failing to upload triggers failure).\n\n### Get List Item Attachments\n\n```python\nfrom downerhelper.sharepoint import get_list_attachments\n\nattachmnets = get_list_attachments(item_id, site_address, list_name, key_url: tuple(str, str), queue: PostgresLogQueue) -> list\n```\n\nReturns empty list on failure.\n\n## Logic App\n\n### Send Email (Outlook)\n\n```python\nfrom downerhelper.logicapp import send_email\n\nsend_email(key_url: tuple(str, str), recipients: str | list(str), subject, body, attachments=[], cc=[]: str | list(str), bcc=[]: str | list(str), importance=\"Normal\": [\"Normal\", \"Low\", \"High\"]) -> bool\n```\n\n`key_url` is for the logic app access. All `recipients`, `cc` and `bcc` are emails, supplied as a string delimited by `;` or a list of strings. `attachments` is a list of dictionaries like \n```python\n{\n \"Name\": filename: str,\n \"ContentBytes\": content: str | bytes\n}\n```\nwhere `ContentBytes` is either a base64 encoded string or bytes that will be encoded.\n\nRaises an exception on failure.\n\n## Warning!\n\nThe following snippet shows incorrect usage. Modules must be declared and imported seperately.\n\n```python\nimport downerhelper\n\nvalue = downerhelper.secrets.get_secret_value(secret_name, keyvault_url)\n```",
"bugtrack_url": null,
"license": "MIT",
"summary": "Collection of functions to wrap the Azure SDK",
"version": "0.1.18",
"project_urls": {
"Homepage": "https://github.com/DownerEU/geospatialDesign-azure_helper"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "634068506e45aa5f04434faecc9044f208667c4b25ea5554a650471aa765a275",
"md5": "4cb62df6231f036bcb787a37632c19fa",
"sha256": "d862c3685519d720b7476d1bc80d6a2bc525304e7e53ccbcb69a435dae569bba"
},
"downloads": -1,
"filename": "downerhelper-0.1.18-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4cb62df6231f036bcb787a37632c19fa",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 12395,
"upload_time": "2025-02-16T22:17:24",
"upload_time_iso_8601": "2025-02-16T22:17:24.987537Z",
"url": "https://files.pythonhosted.org/packages/63/40/68506e45aa5f04434faecc9044f208667c4b25ea5554a650471aa765a275/downerhelper-0.1.18-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "3f66e5af5421d1c7a2b5c78a2a003bc8cd1dacebb67c2f04ffd5ea625b9732a7",
"md5": "5683aeede320edee86a248f6ba50b797",
"sha256": "e89ef0ad602a435bd50940b86c4fdfbb55b0c6d4aee9f61fbfdda2f4801e013c"
},
"downloads": -1,
"filename": "downerhelper-0.1.18.tar.gz",
"has_sig": false,
"md5_digest": "5683aeede320edee86a248f6ba50b797",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 13321,
"upload_time": "2025-02-16T22:17:26",
"upload_time_iso_8601": "2025-02-16T22:17:26.753580Z",
"url": "https://files.pythonhosted.org/packages/3f/66/e5af5421d1c7a2b5c78a2a003bc8cd1dacebb67c2f04ffd5ea625b9732a7/downerhelper-0.1.18.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-16 22:17:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DownerEU",
"github_project": "geospatialDesign-azure_helper",
"github_not_found": true,
"lcname": "downerhelper"
}
