| Name | otelib JSON |
| Version |
0.4.1
JSON |
| download |
| home_page | |
| Summary | Open Translation Environment (OTE) REST API client library. |
| upload_time | 2023-10-25 13:17:49 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | ~=3.9 |
| license | |
| keywords |
ote
ote-api
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# OTELib
[](https://pypi.org/project/otelib)
[](https://pypi.org/project/otelib)
[](https://app.codecov.io/gh/EMMC-ASBL/otelib)
[](https://github.com/EMMC-ASBL/otelib/actions/workflows/ci_tests.yml?query=branch%3Amaster)
[](https://github.com/EMMC-ASBL/otelib/pulse)
[](https://github.com/EMMC-ASBL/otelib/pulse)
[](https://zenodo.org/badge/latestdoi/447285057)
OTELib is a small Python library on top of the OTEAPI, that provides a simple and pythonic interface to the REST services.
It makes it very simple to configure, set up and run a pipeline based on the pipes and filter design pattern.
## Content
* [Overview](#overview)
* [How to use OTELib](#how-to-use-otelib)
* [Session](#session)
* [How to run the pore fraction example](#how-to-run-the-pore-fraction-example)
* [License](#license)
## Overview
The OTEAPI provides a REST API for connecting a sequence of microservices into a pipeline for connecting simulation tools to data sources in a very flexible and reusable way.
OTELib provides a simple Python interface to OTEAPI that makes it easy to utilise OTEAPI from user applications and Python scripts/notebooks as illustrated in the figure below.
_**Figure:** Overview showing how the OTELib can be used by user applications to access data and simulation tools via a simple Python interface._
OTEAPI utilises the [pipes and filter](https://en.wikipedia.org/wiki/Pipeline_(software)) software design pattern to separate the process of connecting to external data resources, mapping it to a common ontology (for interoperability), data processing and running simulations into a set of reusable components.
This design provides a very high degree of flexibility and is easy to maintain due to the modularity, making it well suited for use in EU projects with distributed and loosely coupled teams.
The pipes and filters themselves are implemented as microservices that can be fetched from a docker repository.
In OTELib the pipes and filters are represented by classes - one Pipe class and currently 4 generic filter classes.
It uses the [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) software design pattern is used to specify the underlying microservice.
* **Pipe:** represent data with a type and connects the output from one filter into the input of another filter.
Apart from connecting filters, pipes may also have some additional features, including (not implemented yet):
* Provide access to data and querying.
* Performing authorisation checks.
* Provide caching to avoid unnecessary repeating previous operations.
In order to avoid unauthorised access to the cache, the cache may utilise access tokens for authorisation.
If needed different caching strategies may be supported.
Pipes will be created transparently by OTELib, so as an user, you will normally not interact with pipes.
* **DataResource:** is a filter intended to connect to external data resources and allow the user to perform operations like query, read from or write to the resource.
Where it make sense, some data resource filters may implement full CRUD (create, read, update, and delete) support.
If the DataResource filter reads from a resource, its `get()` method will use a given data model (which may be generated on the fly) and store the content acquired data according to that.
It will return a reference to the internal data representation.
* **Mapping:** is a filter intended to map the data model describing the input to concepts in a common ontology.
The mappings themselves may either be provided during configuration of the filter or fetched from a triplestore.
* **DataFilter:** is a filter intended for fast data processing, like simple data conversions.
The `get()` method will not return before the output is generated.
* **Transformation:** is a filter intended for more time-consuming data processing, like simulations or demanding data conversions.
It supports polling and waiting for the transformation to finish.
## How to use OTELib
Lets assume that you want to set up a simple pipeline as shown in the following figure.
_**Figure:** Generic pipeline connecting three filters, a DataResource filter, a Mapping filter and a Transformation filter via pipes._
* The DataResource filter connects to a data resource and represents selected content from it using a data model.
* The Mapping filter maps the data model to ontologies, making the data read from the data resource fully semantically.
* The Transformation filter performs a transformation of the data, which could involve a simulation.
The result will typically be an instance of another data model.
To set up a pipeline, you will first have to connect to a running OTE server using the `OTEClient`:
```python
from otelib import OTEClient
client = OTEClient('http://localhost:8080')
```
Then you must configure the instances of the filters you want to use:
```python
data_resource = client.create_dataresource(
downloadUrl="https://jpeg.org/images/jpegsystems-home.jpg",
mediaType="image/jpeg",
)
mapping = client.create_mapping(mappingType="triples")
transformation = client.create_transformation(transformation_type="celery/remote")
```
Before you easily and elegant can combine them into a pipeline:
```python
pipeline = data_resource >> mapping >> transformation
```
Note, this also transparently creates the pipes connecting the filters.
To execute the pipeline, simply call its `get()` method:
```python
pipeline.get()
```
which will return a reference to the result of the last transformation.
Note that no data will be read before the `get()` method of the pipeline is called.
A pipeline can be reused and forked:
```python
filter2 = client.create_filter(
filterType="filter/crop",
configuration={"crop": [0, 0, 200, 200]},
)
filter3 = client.create_filter(filterType="filter/blur")
pipeline2 = pipeline >> filter2
pipeline3 = pipeline >> filter3
```
> **Note**: The `filter/blur` filter strategy does not exist in the core OTEAPI package.
> This strategy should come from another plugin package.
As well as merged:
```python
pipeline4 = pipeline2 + pipeline3
```
### Session
A pipeline is executed by calling its `get()` method which will call the `get()` method of the last filter, which will call the `get()` method of its input pipe and so forth.
Hence, a pipeline is executed upstream, from the back to the front, while the data is communicated downstream.
The purpose of a _session_ is to allow the user or downstream filters to provide configurations consumed by filters further upstream.
It is implemented as a common dict shared by all pipes and filters in a pipeline.
If a session is not provided when you call the `get()` method, a new _session_ will be created and passed upstream.
## License
OTELib is released under the [MIT license](LICENSE) with copyright © SINTEF.
## Acknowledgment
OTEAPI Core has been supported by the following projects:
* **OntoTrans** (2020-2024) that receives funding from the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement no. 862136.
* **VIPCOAT** (2021-2025) receives funding from the European Union’s Horizon 2020 Research and Innovation Programme - DT-NMBP-11-2020 Open Innovation Platform for Materials Modelling, under Grant Agreement no: 952903.
* **OpenModel** (2021-2025) receives funding from the European Union’s Horizon 2020 Research and Innovation Programme - DT-NMBP-11-2020 Open Innovation Platform for Materials Modelling, under Grant Agreement no: 953167.
Raw data
{
"_id": null,
"home_page": "",
"name": "otelib",
"maintainer": "",
"docs_url": null,
"requires_python": "~=3.9",
"maintainer_email": "",
"keywords": "OTE,OTE-API",
"author": "",
"author_email": "SINTEF <TEAM4.0@SINTEF.no>",
"download_url": "https://files.pythonhosted.org/packages/7a/53/01140b6c82f42cf6d03f2dc158dd960faa1c2fd6cc1e6a466783bb8beb64/otelib-0.4.1.tar.gz",
"platform": null,
"description": "# OTELib\n\n[](https://pypi.org/project/otelib)\n[](https://pypi.org/project/otelib)\n[](https://app.codecov.io/gh/EMMC-ASBL/otelib)\n[](https://github.com/EMMC-ASBL/otelib/actions/workflows/ci_tests.yml?query=branch%3Amaster)\n[](https://github.com/EMMC-ASBL/otelib/pulse)\n[](https://github.com/EMMC-ASBL/otelib/pulse)\n[](https://zenodo.org/badge/latestdoi/447285057)\n\n\n\nOTELib is a small Python library on top of the OTEAPI, that provides a simple and pythonic interface to the REST services.\n\nIt makes it very simple to configure, set up and run a pipeline based on the pipes and filter design pattern.\n\n## Content\n\n* [Overview](#overview)\n* [How to use OTELib](#how-to-use-otelib)\n* [Session](#session)\n* [How to run the pore fraction example](#how-to-run-the-pore-fraction-example)\n* [License](#license)\n\n## Overview\n\nThe OTEAPI provides a REST API for connecting a sequence of microservices into a pipeline for connecting simulation tools to data sources in a very flexible and reusable way.\nOTELib provides a simple Python interface to OTEAPI that makes it easy to utilise OTEAPI from user applications and Python scripts/notebooks as illustrated in the figure below.\n\n\n\n_**Figure:** Overview showing how the OTELib can be used by user applications to access data and simulation tools via a simple Python interface._\n\nOTEAPI utilises the [pipes and filter](https://en.wikipedia.org/wiki/Pipeline_(software)) software design pattern to separate the process of connecting to external data resources, mapping it to a common ontology (for interoperability), data processing and running simulations into a set of reusable components.\nThis design provides a very high degree of flexibility and is easy to maintain due to the modularity, making it well suited for use in EU projects with distributed and loosely coupled teams.\nThe pipes and filters themselves are implemented as microservices that can be fetched from a docker repository.\n\nIn OTELib the pipes and filters are represented by classes - one Pipe class and currently 4 generic filter classes.\nIt uses the [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) software design pattern is used to specify the underlying microservice.\n\n\n\n* **Pipe:** represent data with a type and connects the output from one filter into the input of another filter.\n Apart from connecting filters, pipes may also have some additional features, including (not implemented yet):\n\n * Provide access to data and querying.\n * Performing authorisation checks.\n * Provide caching to avoid unnecessary repeating previous operations.\n In order to avoid unauthorised access to the cache, the cache may utilise access tokens for authorisation.\n If needed different caching strategies may be supported.\n\n Pipes will be created transparently by OTELib, so as an user, you will normally not interact with pipes.\n* **DataResource:** is a filter intended to connect to external data resources and allow the user to perform operations like query, read from or write to the resource.\n Where it make sense, some data resource filters may implement full CRUD (create, read, update, and delete) support.\n\n If the DataResource filter reads from a resource, its `get()` method will use a given data model (which may be generated on the fly) and store the content acquired data according to that.\n It will return a reference to the internal data representation.\n* **Mapping:** is a filter intended to map the data model describing the input to concepts in a common ontology.\n The mappings themselves may either be provided during configuration of the filter or fetched from a triplestore.\n* **DataFilter:** is a filter intended for fast data processing, like simple data conversions.\n The `get()` method will not return before the output is generated.\n* **Transformation:** is a filter intended for more time-consuming data processing, like simulations or demanding data conversions.\n It supports polling and waiting for the transformation to finish.\n\n## How to use OTELib\n\nLets assume that you want to set up a simple pipeline as shown in the following figure.\n\n\n\n_**Figure:** Generic pipeline connecting three filters, a DataResource filter, a Mapping filter and a Transformation filter via pipes._\n\n* The DataResource filter connects to a data resource and represents selected content from it using a data model.\n* The Mapping filter maps the data model to ontologies, making the data read from the data resource fully semantically.\n* The Transformation filter performs a transformation of the data, which could involve a simulation.\n The result will typically be an instance of another data model.\n\nTo set up a pipeline, you will first have to connect to a running OTE server using the `OTEClient`:\n\n```python\nfrom otelib import OTEClient\n\nclient = OTEClient('http://localhost:8080')\n```\n\nThen you must configure the instances of the filters you want to use:\n\n```python\ndata_resource = client.create_dataresource(\n downloadUrl=\"https://jpeg.org/images/jpegsystems-home.jpg\",\n mediaType=\"image/jpeg\",\n)\nmapping = client.create_mapping(mappingType=\"triples\")\ntransformation = client.create_transformation(transformation_type=\"celery/remote\")\n```\n\nBefore you easily and elegant can combine them into a pipeline:\n\n```python\npipeline = data_resource >> mapping >> transformation\n```\n\nNote, this also transparently creates the pipes connecting the filters.\n\nTo execute the pipeline, simply call its `get()` method:\n\n```python\npipeline.get()\n```\n\nwhich will return a reference to the result of the last transformation.\nNote that no data will be read before the `get()` method of the pipeline is called.\n\nA pipeline can be reused and forked:\n\n```python\nfilter2 = client.create_filter(\n filterType=\"filter/crop\",\n configuration={\"crop\": [0, 0, 200, 200]},\n)\nfilter3 = client.create_filter(filterType=\"filter/blur\")\npipeline2 = pipeline >> filter2\npipeline3 = pipeline >> filter3\n```\n\n> **Note**: The `filter/blur` filter strategy does not exist in the core OTEAPI package.\n> This strategy should come from another plugin package.\n\nAs well as merged:\n\n```python\npipeline4 = pipeline2 + pipeline3\n```\n\n### Session\n\nA pipeline is executed by calling its `get()` method which will call the `get()` method of the last filter, which will call the `get()` method of its input pipe and so forth.\nHence, a pipeline is executed upstream, from the back to the front, while the data is communicated downstream.\n\nThe purpose of a _session_ is to allow the user or downstream filters to provide configurations consumed by filters further upstream.\nIt is implemented as a common dict shared by all pipes and filters in a pipeline.\nIf a session is not provided when you call the `get()` method, a new _session_ will be created and passed upstream.\n\n## License\n\nOTELib is released under the [MIT license](LICENSE) with copyright © SINTEF.\n\n## Acknowledgment\n\nOTEAPI Core has been supported by the following projects:\n\n* **OntoTrans** (2020-2024) that receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme, under Grant Agreement no. 862136.\n\n* **VIPCOAT** (2021-2025) receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme - DT-NMBP-11-2020 Open Innovation Platform for Materials Modelling, under Grant Agreement no: 952903.\n\n* **OpenModel** (2021-2025) receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme - DT-NMBP-11-2020 Open Innovation Platform for Materials Modelling, under Grant Agreement no: 953167.\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Open Translation Environment (OTE) REST API client library.",
"version": "0.4.1",
"project_urls": {
"Changelog": "https://github.com/EMMC-ASBL/otelib/blob/master/CHANGELOG.md",
"Documentation": "https://EMMC-ASBL.github.io/otelib",
"Home": "https://github.com/EMMC-ASBL/otelib",
"Issue Tracker": "https://github.com/EMMC-ASBL/otelib/issues",
"Package": "https://pypi.org/project/otelib",
"Source": "https://github.com/EMMC-ASBL/otelib"
},
"split_keywords": [
"ote",
"ote-api"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b638ced20b0676b122b941c391b274c6ff73471349fdf2ab9f2da7f053974379",
"md5": "d4a8d1d847a12e15eedf82ccce9d6855",
"sha256": "62badde58311dff440bbe9d09f9f6b7da2bbaa9bbdf4f1638cc4d29a1b0b06f8"
},
"downloads": -1,
"filename": "otelib-0.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d4a8d1d847a12e15eedf82ccce9d6855",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "~=3.9",
"size": 21941,
"upload_time": "2023-10-25T13:17:47",
"upload_time_iso_8601": "2023-10-25T13:17:47.636428Z",
"url": "https://files.pythonhosted.org/packages/b6/38/ced20b0676b122b941c391b274c6ff73471349fdf2ab9f2da7f053974379/otelib-0.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7a5301140b6c82f42cf6d03f2dc158dd960faa1c2fd6cc1e6a466783bb8beb64",
"md5": "acc337e05ebef06b6dfc32d9ef56612f",
"sha256": "0e13ebf1e9f13675d0b4e9f1da67a856c629bcd3e269c06f3876c90ae771f5a6"
},
"downloads": -1,
"filename": "otelib-0.4.1.tar.gz",
"has_sig": false,
"md5_digest": "acc337e05ebef06b6dfc32d9ef56612f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "~=3.9",
"size": 97787,
"upload_time": "2023-10-25T13:17:49",
"upload_time_iso_8601": "2023-10-25T13:17:49.743953Z",
"url": "https://files.pythonhosted.org/packages/7a/53/01140b6c82f42cf6d03f2dc158dd960faa1c2fd6cc1e6a466783bb8beb64/otelib-0.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-25 13:17:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "EMMC-ASBL",
"github_project": "otelib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "otelib"
}
