# orchestra-logger
This is a lightweight sdk that allows Orchestra (getorchestra.io) to interact with self-hosted applications.
The basic premise is for your self-hosted application to be triggerable via Orchestra, so accepts a
webhook or HTTP request. Within the application, the orchestra-logger is added to send checkpoint information
to orchestra, such as whether the request was successful, whether it failed, why etc. That's it!
## Quickstart - local testing
Firstly, run
```
pip install orchestra-logger==0.x.x
```
With your chosen version. The current version is 0.0.8.
Next, head to orchestra_logger/example/example.py:
```
from orchestra_logger.orcProcess.orcProcess import orcProcess
def run_example_process(correlation_id, creds):
orcProcessInstance = orcProcess(correlation_id, creds)
print('Starting complicated process')
try:
print('Trying something complicated')
raise Exception
except Exception as e:
print('Failed to do something complicated')
orcProcessInstance.sendFailure(message = str(e), data={'some':'arbitrary stuff'})
finally:
print('Completed')
orcProcessInstance.sendCompletion(message = 'Completed')
creds = {'apikey':'my_api_key'}
run_example_process('hello', creds)
```
### Summary
1. The function takes a correlation id and a dictionary called 'creds'. The correlation id is a uuid that will be passed
to the app by Orchestra when the app is triggered, and corresponds to a pipeline element / task. The creds object is a
dictionary that contains an 'apikey' key that should be securely stored and accessed via your app e.g. using a local config
file, environment variables, but ideally by interacting with a key vault
2. To kick off the process, add the code ```orcProcessInstance = orcProcess(correlation_id, creds)```. This lets Orchestra
know the process has started
3. Use try/except/finally logic to ensure Orchestra knows when parts of the pipeline element / task fail or succeed.
Use the methods ```sendFailure``` and ```sendCompletion``` to indicate whether stages fail. Note; arbitrary data can be
appended to status updates using the "data" parameter. You do not need to include status codes or times in the "data"
parameter; Orchestra calculates these by default.
Note; it's feasible that the above structure requires relatively deep nesting of try/except logic which can lead to
much lengthier code. It is perfectly fine to have a structure like below:
```
def run_something_complicated():
pass
def action_on_app_trigger(correlation_id, creds):
orcProcessInstance = orcProcess(correlation_id, creds)
try:
run_something_complicated()
except BaseException as e:
orcProcessInstance.sendFailure(message = str(e), data={'some':'arbitrary stuff'})
finally:
print('Completed')
orcProcessInstance.sendCompletion(message = 'Completed')
```
Which ensures that no matter what code is excecuted in ```run_something_complicated``` , Orchestra will know what happens
to the job.
## Quick-start; Fast API
For Production usage, you should have an App Service that accepts HTTP requests and contains all the logic you need
to carry out the pipeline/element task.
Firstly, ensure there is an endpoint that can be hit / triggered by Orchestra for the task you wish to carry out:
```
from fastapi import Depends, FastAPI
from services.data_ingestion import dataIngester
from services.creds_manager import credsManager
from orchestra_logger import orcProcess
from routers import base
app = FastAPI()
app.include_router(base.router)
@app.post("/kickOffIngestion")
async def root(body= Body()):
creds = credsManager.get_creds()
orcProcessInstance = orcProcess(body.correlation_id, creds)
ingester__ = dataIngester()
try:
ingester__.run()
except BaseException as e:
orcProcessInstance.sendFailure(message = str(e), data={'some':'arbitrary stuff'})
finally:
print('Completed')
orcProcessInstance.sendCompletion(message = 'Completed')
return {"message": "Process kicked off successfully"}
```
There are a few things happenning here:
1. You have a credentials service that securely fetches your Orchestra API Key, which is being imported and handles the
generation of the ```creds``` object
2. You instantiate the ```orcProcess``` by using the creds object in (1) but also by accessing the correlation ID, which
is passed to you in the body of the request usnig FastAPI's ```Body()``` function
3. You nest the function call ```ingeter__.run()``` within try / except / finally to ensure Orchestra has the relevant
information to know how the task is performing
4. Bonus: the code above is not actually asynchronous although the function is defined with async def, and will _only_
return the response once ```ingester.run()``` has finished running. It is good API practice to make the code execute
and return a response straightaway for Post-like requests
Raw data
{
"_id": null,
"home_page": "",
"name": "orchestra-logger",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "python,logging,data,orchestration",
"author": "Orchestra (Hugo Lu)",
"author_email": "<hugo@getorchestra.io>",
"download_url": "https://files.pythonhosted.org/packages/c9/ef/5b3f87ab5ebf5d02eae5fc6a513a5fab68eb64c8000cfb6738f34b2f034f/orchestra-logger-0.0.13.tar.gz",
"platform": null,
"description": "\r\n# orchestra-logger\r\r\n\r\r\nThis is a lightweight sdk that allows Orchestra (getorchestra.io) to interact with self-hosted applications.\r\r\n\r\r\nThe basic premise is for your self-hosted application to be triggerable via Orchestra, so accepts a\r\r\nwebhook or HTTP request. Within the application, the orchestra-logger is added to send checkpoint information\r\r\nto orchestra, such as whether the request was successful, whether it failed, why etc. That's it!\r\r\n\r\r\n## Quickstart - local testing\r\r\n\r\r\nFirstly, run \r\r\n```\r\r\npip install orchestra-logger==0.x.x\r\r\n```\r\r\nWith your chosen version. The current version is 0.0.8.\r\r\n\r\r\nNext, head to orchestra_logger/example/example.py:\r\r\n```\r\r\nfrom orchestra_logger.orcProcess.orcProcess import orcProcess\r\r\n\r\r\ndef run_example_process(correlation_id, creds):\r\r\n orcProcessInstance = orcProcess(correlation_id, creds)\r\r\n print('Starting complicated process')\r\r\n try:\r\r\n print('Trying something complicated')\r\r\n raise Exception\r\r\n except Exception as e:\r\r\n print('Failed to do something complicated')\r\r\n orcProcessInstance.sendFailure(message = str(e), data={'some':'arbitrary stuff'})\r\r\n finally:\r\r\n print('Completed')\r\r\n orcProcessInstance.sendCompletion(message = 'Completed')\r\r\n\r\r\ncreds = {'apikey':'my_api_key'}\r\r\nrun_example_process('hello', creds)\r\r\n```\r\r\n\r\r\n### Summary\r\r\n\r\r\n1. The function takes a correlation id and a dictionary called 'creds'. The correlation id is a uuid that will be passed\r\r\n to the app by Orchestra when the app is triggered, and corresponds to a pipeline element / task. The creds object is a\r\r\n dictionary that contains an 'apikey' key that should be securely stored and accessed via your app e.g. using a local config \r\r\n file, environment variables, but ideally by interacting with a key vault\r\r\n2. To kick off the process, add the code ```orcProcessInstance = orcProcess(correlation_id, creds)```. This lets Orchestra\r\r\n know the process has started\r\r\n3. Use try/except/finally logic to ensure Orchestra knows when parts of the pipeline element / task fail or succeed.\r\r\n Use the methods ```sendFailure``` and ```sendCompletion``` to indicate whether stages fail. Note; arbitrary data can be\r\r\n appended to status updates using the \"data\" parameter. You do not need to include status codes or times in the \"data\"\r\r\n parameter; Orchestra calculates these by default.\r\r\n\r\r\nNote; it's feasible that the above structure requires relatively deep nesting of try/except logic which can lead to\r\r\nmuch lengthier code. It is perfectly fine to have a structure like below:\r\r\n\r\r\n```\r\r\ndef run_something_complicated():\r\r\n pass\r\r\n\r\r\ndef action_on_app_trigger(correlation_id, creds):\r\r\n orcProcessInstance = orcProcess(correlation_id, creds)\r\r\n try:\r\r\n run_something_complicated()\r\r\n except BaseException as e:\r\r\n orcProcessInstance.sendFailure(message = str(e), data={'some':'arbitrary stuff'})\r\r\n finally:\r\r\n print('Completed')\r\r\n orcProcessInstance.sendCompletion(message = 'Completed')\r\r\n```\r\r\n \r\r\nWhich ensures that no matter what code is excecuted in ```run_something_complicated``` , Orchestra will know what happens\r\r\nto the job.\r\r\n\r\r\n## Quick-start; Fast API\r\r\n\r\r\nFor Production usage, you should have an App Service that accepts HTTP requests and contains all the logic you need\r\r\nto carry out the pipeline/element task.\r\r\n\r\r\nFirstly, ensure there is an endpoint that can be hit / triggered by Orchestra for the task you wish to carry out:\r\r\n\r\r\n```\r\r\nfrom fastapi import Depends, FastAPI\r\r\nfrom services.data_ingestion import dataIngester\r\r\nfrom services.creds_manager import credsManager\r\r\nfrom orchestra_logger import orcProcess\r\r\nfrom routers import base\r\r\n\r\r\napp = FastAPI()\r\r\n\r\r\napp.include_router(base.router)\r\r\n\r\r\n@app.post(\"/kickOffIngestion\")\r\r\nasync def root(body= Body()):\r\r\n creds = credsManager.get_creds()\r\r\n orcProcessInstance = orcProcess(body.correlation_id, creds)\r\r\n ingester__ = dataIngester()\r\r\n try:\r\r\n ingester__.run()\r\r\n except BaseException as e:\r\r\n orcProcessInstance.sendFailure(message = str(e), data={'some':'arbitrary stuff'})\r\r\n finally:\r\r\n print('Completed')\r\r\n orcProcessInstance.sendCompletion(message = 'Completed')\r\r\n return {\"message\": \"Process kicked off successfully\"}\r\r\n```\r\r\nThere are a few things happenning here:\r\r\n1. You have a credentials service that securely fetches your Orchestra API Key, which is being imported and handles the \r\r\n generation of the ```creds``` object\r\r\n2. You instantiate the ```orcProcess``` by using the creds object in (1) but also by accessing the correlation ID, which\r\r\n is passed to you in the body of the request usnig FastAPI's ```Body()``` function\r\r\n3. You nest the function call ```ingeter__.run()``` within try / except / finally to ensure Orchestra has the relevant\r\r\n information to know how the task is performing\r\r\n4. Bonus: the code above is not actually asynchronous although the function is defined with async def, and will _only_\r\r\n return the response once ```ingester.run()``` has finished running. It is good API practice to make the code execute\r\r\n and return a response straightaway for Post-like requests\r\r\n\r\r\n\r\r\n \r\n",
"bugtrack_url": null,
"license": "",
"summary": "Logging module to enable orchestration on self-hosted applications",
"version": "0.0.13",
"split_keywords": [
"python",
"logging",
"data",
"orchestration"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c9ef5b3f87ab5ebf5d02eae5fc6a513a5fab68eb64c8000cfb6738f34b2f034f",
"md5": "7b039284307f1ab45f339957034c4418",
"sha256": "6c0851ae4f41a5464d36b258255b45d9d80b2b8b2dbe416d23d702e0cab1b2ea"
},
"downloads": -1,
"filename": "orchestra-logger-0.0.13.tar.gz",
"has_sig": false,
"md5_digest": "7b039284307f1ab45f339957034c4418",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 8955,
"upload_time": "2023-04-13T10:51:37",
"upload_time_iso_8601": "2023-04-13T10:51:37.224985Z",
"url": "https://files.pythonhosted.org/packages/c9/ef/5b3f87ab5ebf5d02eae5fc6a513a5fab68eb64c8000cfb6738f34b2f034f/orchestra-logger-0.0.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-13 10:51:37",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "orchestra-logger"
}
JSON