# Pycarlo
Monte Carlo's Alpha Python SDK!
## Installation
Requires Python 3.7 or greater. Normally you can install and update using pip. For instance:
```shell
virtualenv venv
. venv/bin/activate
pip install -U pycarlo
```
Developers of the SDK can use:
```shell
make install-with-tests
. venv/bin/activate
pre-commit install
```
## Overview
Pycarlo comprises two components: `core` and `features`.
All Monte Carlo API queries and mutations that you could execute via the API are supported via the `core` library. Operations can be executed as first class objects, using [sgqlc](https://github.com/profusion/sgqlc), or as raw GQL with variables. In both cases, a consistent object where fields can be referenced by dot notation and the more pythonic snake_case is returned for ease of use.
The `features` library provides additional convenience for performing common operations like with dbt, circuit breaking, and pii filtering.
Note that an API Key is required to use the SDK. See [here](https://docs.getmontecarlo.com/docs/developer-resources#creating-an-api-key) for details on how to generate one.
## Basic usage
### Core
```python
from pycarlo.core import Client, Query, Mutation
# First create a client. This creates a session using the 'default' profile from
# '~/.mcd/profiles.ini'. This profile is created automatically via `montecarlo configure` on the
# CLI. See the session subsection for customizations, options and alternatives (e.g. using the
# environment, params, named profiles, etc.)
client = Client()
# Now you can can execute a query. For instance, getUser (selecting the email field).
# This would be like executing -
# curl --location --request POST 'https://api.getmontecarlo.com/graphql' \
# --header 'x-mcd-id: <ID>' \
# --header 'x-mcd-token: <TOKEN>' \
# --header 'Content-Type: application/json' \
# --data-raw '{"query": "query {getUser {email}}"}'
# Notice how the CamelCase from the Graphql query is converted to snake_case in both the request
# and response.
query = Query()
query.get_user.__fields__('email')
print(client(query).get_user.email)
# You can also execute a query that requires variables. For instance,
# testTelnetConnection (selecting all fields).
query = Query()
query.test_telnet_connection(host='montecarlodata.com', port=443)
print(client(query))
# If necessary, you can always generate (e.g. print) the raw query that would be executed.
print(query)
# query {
# testTelnetConnection(host: "montecarlodata.com", port: 443) {
# success
# validations {
# type
# message
# }
# warnings {
# type
# message
# }
# }
# }
# If you are not a fan of sgqlc operations (Query and Mutation) you can also execute any
# raw query using the client. For instance, if we want the first 10 tables from getTables.
get_table_query = """
query getTables{
getTables(first: 10) {
edges {
node {
fullTableId
}
}
}
}
"""
response = client(get_table_query)
# This returns a Box object where fields can be accessed using dot notation.
# Notice how unlike with the API the response uses the more Pythonic snake_case.
for edge in response.get_tables.edges:
print(edge.node.full_table_id)
# The response can still be processed as a standard dictionary.
print(response['get_tables']['edges'][0]['node']['full_table_id'])
# You can also execute any mutations too. For instance, generateCollectorTemplate
# (selecting the templateLaunchUrl).
mutation = Mutation()
mutation.generate_collector_template().dc.template_launch_url()
print(client(mutation))
# Any errors will raise a GqlError with details. For instance, executing above with an
# invalid region.
mutation = Mutation()
mutation.generate_collector_template(region='artemis')
print(client(mutation))
# pycarlo.common.errors.GqlError: [{'message': 'Region "\'artemis\'" not currently active.'...]
```
Note that you can find Monte Carlo's API reference [here](https://apidocs.getmontecarlo.com/).
For details and additional examples on how to map (convert) GraphQL queries to `sgqlc` operations please refer to the docs [here](https://sgqlc.readthedocs.io/en/latest/sgqlc.operation.html).
### Features
You can use [pydoc](https://docs.python.org/3.8/library/pydoc.html) to retrieve documentation on any feature packages (`pydoc pycarlo.features`).
For instance for [circuit breakers](https://docs.getmontecarlo.com/docs/circuit-breakers):
```shell
pydoc pycarlo.features.circuit_breakers.service
```
## Session configuration
By default, when creating a client the `default` profile from `~/.mcd/profiles.ini` is used. This file created via [montecarlo configure](https://docs.getmontecarlo.com/docs/using-the-cli#setting-up-the-cli) on the CLI. Note that you can find Monte Carlo's CLI reference [here](https://clidocs.getmontecarlo.com/).
You can override this usage by creating a custom `Session`. For instance, if you want to pass the ID and Token:
```python
from pycarlo.core import Client, Session
client = Client(session=Session(mcd_id='foo', mcd_token='bar'))
```
Sessions support the following params:
- `mcd_id`: API Key ID.
- `mcd_token`: API secret.
- `mcd_profile`: Named profile containing credentials. This is created via the CLI (e.g. `montecarlo configure --profile-name zeus`).
- `mcd_config_path`: Path to file containing credentials. Defaults to `~/.mcd/`.
You can also specify the API Key, secret or profile name using the following environment variables:
- `MCD_DEFAULT_API_ID`
- `MCD_DEFAULT_API_TOKEN`
- `MCD_DEFAULT_PROFILE`
When creating a session any explicitly passed `mcd_id` and `mcd_token` params take precedence, followed by environmental variables and then any config-file options.
Environment variables can be mixed with passed credentials, but not the config-file profile.
**We do not recommend passing `mcd_token` as it is a secret and can be accidentally committed.**
## Integration Gateway API
There are features that require the Integration Gateway API instead of the regular GraphQL Application API, for example Airflow Callbacks invoked by the `airflow-mcd` library.
To use the Gateway you need to initialize the `Session` object passing a `scope` parameter and then use `make_request` to invoke Gateway endpoints:
```python
from pycarlo.core import Client, Session
client = Client(session=Session(mcd_id='foo', mcd_token='bar', scope='AirflowCallbacks'))
response = client.make_request(path='/airflow/callbacks', method='POST', body={}, timeout_in_seconds=20)
```
## Advanced configuration
The following values also be set by the environment:
- `MCD_VERBOSE_ERRORS`: Enable logging. This includes a trace ID for each session and request.
- `MCD_API_ENDPOINT`: Customize the endpoint where queries and mutations are executed.
## Tests and releases
To update queries and mutations via introspection, use `make generate`.
`make test` can be used to run all tests locally. CircleCI manages all testing for deployment. When ready for a review, create a PR against `main`.
When ready to release, create a new [Github release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository) with a tag using semantic versioning (e.g. `v0.42.0`) and CircleCI will test and publish to PyPI. Note that an existing version will not be deployed.
## References
- Dashboard: <https://getmontecarlo.com>
- Product docs: <https://docs.getmontecarlo.com>
- Status page: <https://status.getmontecarlo.com>
- API (and SDK): <https://apidocs.getmontecarlo.com>
- CLI: <https://clidocs.getmontecarlo.com>
## License
Apache 2.0 - See the [LICENSE](http://www.apache.org/licenses/LICENSE-2.0) for more information.
Raw data
{
"_id": null,
"home_page": "https://www.montecarlodata.com/",
"name": "pycarlo",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Monte Carlo Data, Inc",
"author_email": "info@montecarlodata.com",
"download_url": "https://files.pythonhosted.org/packages/61/f6/cddc0cb38513398d0e65152c7ec435649f23a85d7c52d31af0167498d839/pycarlo-0.10.0.tar.gz",
"platform": null,
"description": "# Pycarlo\n\nMonte Carlo's Alpha Python SDK!\n\n## Installation\n\nRequires Python 3.7 or greater. Normally you can install and update using pip. For instance:\n\n```shell\nvirtualenv venv\n. venv/bin/activate\n\npip install -U pycarlo\n```\n\nDevelopers of the SDK can use:\n\n```shell\nmake install-with-tests\n. venv/bin/activate\npre-commit install\n```\n\n## Overview\n\nPycarlo comprises two components: `core` and `features`.\n\nAll Monte Carlo API queries and mutations that you could execute via the API are supported via the `core` library. Operations can be executed as first class objects, using [sgqlc](https://github.com/profusion/sgqlc), or as raw GQL with variables. In both cases, a consistent object where fields can be referenced by dot notation and the more pythonic snake_case is returned for ease of use.\n\nThe `features` library provides additional convenience for performing common operations like with dbt, circuit breaking, and pii filtering.\n\nNote that an API Key is required to use the SDK. See [here](https://docs.getmontecarlo.com/docs/developer-resources#creating-an-api-key) for details on how to generate one.\n\n## Basic usage\n\n### Core\n\n```python\nfrom pycarlo.core import Client, Query, Mutation\n\n# First create a client. This creates a session using the 'default' profile from\n# '~/.mcd/profiles.ini'. This profile is created automatically via `montecarlo configure` on the\n# CLI. See the session subsection for customizations, options and alternatives (e.g. using the\n# environment, params, named profiles, etc.)\nclient = Client()\n\n# Now you can can execute a query. For instance, getUser (selecting the email field).\n# This would be like executing -\n# curl --location --request POST 'https://api.getmontecarlo.com/graphql' \\\n# --header 'x-mcd-id: <ID>' \\\n# --header 'x-mcd-token: <TOKEN>' \\\n# --header 'Content-Type: application/json' \\\n# --data-raw '{\"query\": \"query {getUser {email}}\"}'\n# Notice how the CamelCase from the Graphql query is converted to snake_case in both the request\n# and response.\nquery = Query()\nquery.get_user.__fields__('email')\nprint(client(query).get_user.email)\n\n# You can also execute a query that requires variables. For instance,\n# testTelnetConnection (selecting all fields).\nquery = Query()\nquery.test_telnet_connection(host='montecarlodata.com', port=443)\nprint(client(query))\n\n# If necessary, you can always generate (e.g. print) the raw query that would be executed.\nprint(query)\n# query {\n# testTelnetConnection(host: \"montecarlodata.com\", port: 443) {\n# success\n# validations {\n# type\n# message\n# }\n# warnings {\n# type\n# message\n# }\n# }\n# }\n\n# If you are not a fan of sgqlc operations (Query and Mutation) you can also execute any\n# raw query using the client. For instance, if we want the first 10 tables from getTables.\nget_table_query = \"\"\"\nquery getTables{\n getTables(first: 10) {\n edges {\n node {\n fullTableId\n }\n }\n }\n}\n\"\"\"\nresponse = client(get_table_query)\n# This returns a Box object where fields can be accessed using dot notation.\n# Notice how unlike with the API the response uses the more Pythonic snake_case.\nfor edge in response.get_tables.edges:\n print(edge.node.full_table_id)\n# The response can still be processed as a standard dictionary.\nprint(response['get_tables']['edges'][0]['node']['full_table_id'])\n\n# You can also execute any mutations too. For instance, generateCollectorTemplate\n# (selecting the templateLaunchUrl).\nmutation = Mutation()\nmutation.generate_collector_template().dc.template_launch_url()\nprint(client(mutation))\n\n# Any errors will raise a GqlError with details. For instance, executing above with an\n# invalid region.\nmutation = Mutation()\nmutation.generate_collector_template(region='artemis')\nprint(client(mutation))\n# pycarlo.common.errors.GqlError: [{'message': 'Region \"\\'artemis\\'\" not currently active.'...]\n```\n\nNote that you can find Monte Carlo's API reference [here](https://apidocs.getmontecarlo.com/).\n\nFor details and additional examples on how to map (convert) GraphQL queries to `sgqlc` operations please refer to the docs [here](https://sgqlc.readthedocs.io/en/latest/sgqlc.operation.html).\n\n### Features\n\nYou can use [pydoc](https://docs.python.org/3.8/library/pydoc.html) to retrieve documentation on any feature packages (`pydoc pycarlo.features`).\n\nFor instance for [circuit breakers](https://docs.getmontecarlo.com/docs/circuit-breakers):\n\n```shell\npydoc pycarlo.features.circuit_breakers.service\n```\n\n## Session configuration\n\nBy default, when creating a client the `default` profile from `~/.mcd/profiles.ini` is used. This file created via [montecarlo configure](https://docs.getmontecarlo.com/docs/using-the-cli#setting-up-the-cli) on the CLI. Note that you can find Monte Carlo's CLI reference [here](https://clidocs.getmontecarlo.com/).\n\nYou can override this usage by creating a custom `Session`. For instance, if you want to pass the ID and Token:\n\n```python\nfrom pycarlo.core import Client, Session\n\nclient = Client(session=Session(mcd_id='foo', mcd_token='bar'))\n```\n\nSessions support the following params:\n\n- `mcd_id`: API Key ID.\n- `mcd_token`: API secret.\n- `mcd_profile`: Named profile containing credentials. This is created via the CLI (e.g. `montecarlo configure --profile-name zeus`).\n- `mcd_config_path`: Path to file containing credentials. Defaults to `~/.mcd/`.\n\nYou can also specify the API Key, secret or profile name using the following environment variables:\n\n- `MCD_DEFAULT_API_ID`\n- `MCD_DEFAULT_API_TOKEN`\n- `MCD_DEFAULT_PROFILE`\n\nWhen creating a session any explicitly passed `mcd_id` and `mcd_token` params take precedence, followed by environmental variables and then any config-file options.\n\nEnvironment variables can be mixed with passed credentials, but not the config-file profile.\n\n**We do not recommend passing `mcd_token` as it is a secret and can be accidentally committed.**\n\n## Integration Gateway API\n\nThere are features that require the Integration Gateway API instead of the regular GraphQL Application API, for example Airflow Callbacks invoked by the `airflow-mcd` library.\n\nTo use the Gateway you need to initialize the `Session` object passing a `scope` parameter and then use `make_request` to invoke Gateway endpoints:\n\n```python\nfrom pycarlo.core import Client, Session\n\nclient = Client(session=Session(mcd_id='foo', mcd_token='bar', scope='AirflowCallbacks'))\nresponse = client.make_request(path='/airflow/callbacks', method='POST', body={}, timeout_in_seconds=20)\n```\n\n## Advanced configuration\n\nThe following values also be set by the environment:\n\n- `MCD_VERBOSE_ERRORS`: Enable logging. This includes a trace ID for each session and request.\n- `MCD_API_ENDPOINT`: Customize the endpoint where queries and mutations are executed.\n\n## Tests and releases\n\nTo update queries and mutations via introspection, use `make generate`.\n\n`make test` can be used to run all tests locally. CircleCI manages all testing for deployment. When ready for a review, create a PR against `main`.\n\nWhen ready to release, create a new [Github release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository) with a tag using semantic versioning (e.g. `v0.42.0`) and CircleCI will test and publish to PyPI. Note that an existing version will not be deployed.\n\n## References\n\n- Dashboard: <https://getmontecarlo.com>\n- Product docs: <https://docs.getmontecarlo.com>\n- Status page: <https://status.getmontecarlo.com>\n- API (and SDK): <https://apidocs.getmontecarlo.com>\n- CLI: <https://clidocs.getmontecarlo.com>\n\n## License\n\nApache 2.0 - See the [LICENSE](http://www.apache.org/licenses/LICENSE-2.0) for more information.\n",
"bugtrack_url": null,
"license": "Apache Software License (Apache 2.0)",
"summary": "Monte Carlo's Python SDK",
"version": "0.10.0",
"project_urls": {
"Homepage": "https://www.montecarlodata.com/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dde1809866595f3448427b2a0fae780f745e4506af3c31e1d51127e02278cb37",
"md5": "17dd1a98492985988eaa74404a64e9d7",
"sha256": "6b5f34dc4dbf1ccf5fad18cb233dc51a48007cfda5d14c2a6e7b85ab2d1a0cf5"
},
"downloads": -1,
"filename": "pycarlo-0.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "17dd1a98492985988eaa74404a64e9d7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 561662,
"upload_time": "2024-12-17T19:05:25",
"upload_time_iso_8601": "2024-12-17T19:05:25.078588Z",
"url": "https://files.pythonhosted.org/packages/dd/e1/809866595f3448427b2a0fae780f745e4506af3c31e1d51127e02278cb37/pycarlo-0.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "61f6cddc0cb38513398d0e65152c7ec435649f23a85d7c52d31af0167498d839",
"md5": "c826cf4874af4468a9baf390c98ff995",
"sha256": "e33ea5fdb6afc1fcd5c0cc84d8aa2720c90e02c773abdc6d3af8adccdd6d20b0"
},
"downloads": -1,
"filename": "pycarlo-0.10.0.tar.gz",
"has_sig": false,
"md5_digest": "c826cf4874af4468a9baf390c98ff995",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 605788,
"upload_time": "2024-12-17T19:05:27",
"upload_time_iso_8601": "2024-12-17T19:05:27.777098Z",
"url": "https://files.pythonhosted.org/packages/61/f6/cddc0cb38513398d0e65152c7ec435649f23a85d7c52d31af0167498d839/pycarlo-0.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-17 19:05:27",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pycarlo"
}
JSON