# General
This package provides an Interface that can be implemented to then be turned into a job.
The used URI interface (`from uai_pre_transform_interface import URI`) is basically a [pathlib.Path](https://docs.python.org/3/library/pathlib.html#pathlib.Path) with some missing functionality.
# WARNING: URI is not a local file
**Beware**: Inside understand.ai we will hand cloud paths as URIs. So using the string of a path to open it will fail in these cases!
E.g. Do **not** do things like this:
```python
from uai_pre_import_transform_interface import PreImportTransformInterface, URI
from PIL import Image
class PreImportTransformer(PreImportTransformInterface):
def transform(self, input_path: URI, output_path: URI):
path_to_image = input_path / "images" / "00001.png"
with Image.open(str(path_to_image)) as im: # This will fail if input path is "gs://dataset/clip1/"
im.show()
```
instead you could do
```python
from uai_pre_import_transform_interface import PreImportTransformInterface, URI
from PIL import Image
from tempfile import NamedTemporaryFile
class PreImportTransformer(PreImportTransformInterface):
def transform(self, input_path: URI, output_path: URI):
path_to_image = input_path / "images" / "00001.png"
# example for when your method takes readable bytes
with Image.open(path_to_image.open("rb")) as im:
im.show()
# example for when your method needs just a path to the file
with NamedTemporaryFile() as tmp:
tmp.write(path_to_image.open("rb").read())
Image.open(tmp.name)
```
**Unit tests do not catch this, unfortunately**. One idea to be more save against this could be to not use pathlib.Path directly for debugging but to instead extend path with an implementation that raises an error on usage of `__str__`.
```python
from pathlib import Path
class DebuPath(Path):
def __str__(self):
raise Exception("it is not a good idea to use strings for paths for anything but logging as these paths could point to remote resources.")
```
# Implementation
Every implementation should provide a python package named by you. Let's use `package_name` as an example. From this package the following import has to work:
```python
from package_name import PreImportTransformer
```
This should give your implementation of the interface. To achieve this the `__init__.py` oyour package should contain something like this (depending on how you name things):
```python
from .my_interface_implementation import PreImportTransformer
__all__: Sequence[str] = ["PreImportTransformer"]
```
This is how we will automatically bind your library into our system. 
Raw data
{
"_id": null,
"home_page": "",
"name": "uai-pre-import-transform-interface",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "understand.ai",
"author_email": "postmaster@understand.ai",
"download_url": "https://files.pythonhosted.org/packages/63/55/ce772ca1f5d9e62b61edacff9a00ee752f68ff7ea13b55ccf069cf14cbaf/uai_pre_import_transform_interface-1.1.2.tar.gz",
"platform": null,
"description": "# General\nThis package provides an Interface that can be implemented to then be turned into a job.\nThe used URI interface (`from uai_pre_transform_interface import URI`) is basically a [pathlib.Path](https://docs.python.org/3/library/pathlib.html#pathlib.Path) with some missing functionality.\n\n# WARNING: URI is not a local file\n**Beware**: Inside understand.ai we will hand cloud paths as URIs. So using the string of a path to open it will fail in these cases!\nE.g. Do **not** do things like this:\n```python\nfrom uai_pre_import_transform_interface import PreImportTransformInterface, URI\nfrom PIL import Image\n\nclass PreImportTransformer(PreImportTransformInterface):\n def transform(self, input_path: URI, output_path: URI):\n path_to_image = input_path / \"images\" / \"00001.png\"\n with Image.open(str(path_to_image)) as im: # This will fail if input path is \"gs://dataset/clip1/\"\n im.show()\n```\ninstead you could do\n```python\nfrom uai_pre_import_transform_interface import PreImportTransformInterface, URI\nfrom PIL import Image\nfrom tempfile import NamedTemporaryFile\n\nclass PreImportTransformer(PreImportTransformInterface):\n def transform(self, input_path: URI, output_path: URI):\n path_to_image = input_path / \"images\" / \"00001.png\"\n \n # example for when your method takes readable bytes\n with Image.open(path_to_image.open(\"rb\")) as im:\n im.show()\n\n # example for when your method needs just a path to the file\n with NamedTemporaryFile() as tmp:\n tmp.write(path_to_image.open(\"rb\").read())\n Image.open(tmp.name)\n```\n\n**Unit tests do not catch this, unfortunately**. One idea to be more save against this could be to not use pathlib.Path directly for debugging but to instead extend path with an implementation that raises an error on usage of `__str__`.\n```python\nfrom pathlib import Path\n\nclass DebuPath(Path):\n def __str__(self):\n raise Exception(\"it is not a good idea to use strings for paths for anything but logging as these paths could point to remote resources.\")\n```\n\n# Implementation\nEvery implementation should provide a python package named by you. Let's use `package_name` as an example. From this package the following import has to work:\n```python\nfrom package_name import PreImportTransformer\n```\nThis should give your implementation of the interface. To achieve this the `__init__.py` oyour package should contain something like this (depending on how you name things):\n```python\nfrom .my_interface_implementation import PreImportTransformer\n__all__: Sequence[str] = [\"PreImportTransformer\"]\n```\n\nThis is how we will automatically bind your library into our system. \n",
"bugtrack_url": null,
"license": "MIT",
"summary": "",
"version": "1.1.2",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "3536dc0808ebbe7b481123049ff7a16c",
"sha256": "55c51ee3097c73f0ca81e0571fd42021fd4ed20ef9c16d89342383a4fb192fcd"
},
"downloads": -1,
"filename": "uai_pre_import_transform_interface-1.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3536dc0808ebbe7b481123049ff7a16c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 4010,
"upload_time": "2022-12-20T10:14:42",
"upload_time_iso_8601": "2022-12-20T10:14:42.194992Z",
"url": "https://files.pythonhosted.org/packages/1b/22/1250fc726a11f195c06fce3092e323e4ab928c11fbc74bea8fc82872ff9e/uai_pre_import_transform_interface-1.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "f1ce1e92e14223a7df7c6c92a8ad867b",
"sha256": "eca2fcb5e1de5227953329b6da07e1fa35f6161f1368887683ee7decc1a81ed5"
},
"downloads": -1,
"filename": "uai_pre_import_transform_interface-1.1.2.tar.gz",
"has_sig": false,
"md5_digest": "f1ce1e92e14223a7df7c6c92a8ad867b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 3327,
"upload_time": "2022-12-20T10:14:43",
"upload_time_iso_8601": "2022-12-20T10:14:43.315476Z",
"url": "https://files.pythonhosted.org/packages/63/55/ce772ca1f5d9e62b61edacff9a00ee752f68ff7ea13b55ccf069cf14cbaf/uai_pre_import_transform_interface-1.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-20 10:14:43",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "uai-pre-import-transform-interface"
}
JSON