Informatics Matters Data Manager Job Decoder
============================================
.. image:: https://badge.fury.io/py/im-data-manager-job-decoder.svg
:target: https://badge.fury.io/py/im-data-manager-job-decoder
:alt: PyPI package (latest)
.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/build.yaml/badge.svg
:target: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/build.yaml
:alt: Build
.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/publish.yaml/badge.svg
:target: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/publish.yaml
:alt: Publish
A package that simplifies the decoding of encoded text strings and validation
of Job Definitions and Manifests.
Given an encoded string the ``decode()`` method
returns the decoded value or an error.
For example, given the following `jinja2`_ encoded string
``'{{ foo }}, bar={{ bar }}, baz={{ baz }}'`` and variable map
``{'foo': 1, 'bar': 2, 'baz': 3}`` the decoder returns
the string ``'foo=1, bar=2, baz=3'``.
The following encoding/decoding formats are supported: -
- jinja2 (3.0)
The package also provides ``validate_job_schema()`` and
``validate_manifest_schema()`` functions, which can (should) be used to
validate the Manifests and Job definitions against the
built-in schemas.
If you have a Job Definition or Manifest file you can add a reference to the
corresponding schema in this package to your file to help you identify schema
violations prior to deployment. To do this you just need to add a line starting
``# yaml-language-server:`` at the top of your file. For example,
you can link to the **v2.3.0** Job Definition schema by adding the following line::
# yaml-language-server: $schema=https://raw.githubusercontent.com/InformaticsMatters/squonk2-data-manager-job-decoder/refs/tags/2.3.0/decoder/job-definition-schema.yaml
.. _jinja2: https://jinja.palletsprojects.com/en/3.0.x/
Installation (Python)
=====================
The Job decoder is published on `PyPI`_ and can be installed from
there::
pip install im-data-manager-job-decoder
Once installed you can validate the definition (expected to be a dictionary
formed from the definition YAML file) with::
>>> from decoder import decoder
>>> error: Optional[str] = decoder.validate_manifest_schema(manifest)
>>> error: Optional[str] = decoder.validate_job_schema(job_definition)
And you can decode encoded fields within the job definition.
Here we're decoding the job's 'command' using a map of variables and their
values::
>>> decoded, success = decoder.decode(definition['command'],
variables,
'command',
decoder.TextEncoding.JINJA2_3_0)
A method also exists to generate a Data Manger Job documentation URL
for the supported repository types. For example, to get the
auto-generated documentation URL for a job definition hosted in a GitHub
repository you can do this::
>>> doc_url: str = decoder.get_job_doc_url("github",
collection
job_id,
job_definition,
manifest_url)
We support ``github`` and ``gitlab`` as repository types.
.. _PyPI: https://pypi.org/project/im-data-manager-job-decoder
Get in touch
============
- Report bugs, suggest features or view the source code `on GitHub`_.
.. _on GitHub: https://github.com/informaticsmatters/squonk2-data-manager-job-decoder
Raw data
{
"_id": null,
"home_page": "https://github.com/informaticsmatters/squonk2-data-manager-job-decoder",
"name": "im-data-manager-job-decoder",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "jinja2 decoder",
"author": "Alan Christie",
"author_email": "achristie@informaticsmatters.com",
"download_url": "https://files.pythonhosted.org/packages/50/b7/de1a3ef7cf31de2d3afedb69fdb931a77ec3057c32fc108ada4c6cfc2b8c/im_data_manager_job_decoder-2.5.0.tar.gz",
"platform": "any",
"description": "Informatics Matters Data Manager Job Decoder\n============================================\n\n.. image:: https://badge.fury.io/py/im-data-manager-job-decoder.svg\n :target: https://badge.fury.io/py/im-data-manager-job-decoder\n :alt: PyPI package (latest)\n\n.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/build.yaml/badge.svg\n :target: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/build.yaml\n :alt: Build\n\n.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/publish.yaml/badge.svg\n :target: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/publish.yaml\n :alt: Publish\n\nA package that simplifies the decoding of encoded text strings and validation\nof Job Definitions and Manifests.\n\nGiven an encoded string the ``decode()`` method\nreturns the decoded value or an error.\n\nFor example, given the following `jinja2`_ encoded string\n``'{{ foo }}, bar={{ bar }}, baz={{ baz }}'`` and variable map\n``{'foo': 1, 'bar': 2, 'baz': 3}`` the decoder returns\nthe string ``'foo=1, bar=2, baz=3'``.\n\nThe following encoding/decoding formats are supported: -\n\n- jinja2 (3.0)\n\nThe package also provides ``validate_job_schema()`` and\n``validate_manifest_schema()`` functions, which can (should) be used to\nvalidate the Manifests and Job definitions against the\nbuilt-in schemas.\n\nIf you have a Job Definition or Manifest file you can add a reference to the\ncorresponding schema in this package to your file to help you identify schema\nviolations prior to deployment. To do this you just need to add a line starting\n``# yaml-language-server:`` at the top of your file. For example,\nyou can link to the **v2.3.0** Job Definition schema by adding the following line::\n\n # yaml-language-server: $schema=https://raw.githubusercontent.com/InformaticsMatters/squonk2-data-manager-job-decoder/refs/tags/2.3.0/decoder/job-definition-schema.yaml\n\n.. _jinja2: https://jinja.palletsprojects.com/en/3.0.x/\n\nInstallation (Python)\n=====================\n\nThe Job decoder is published on `PyPI`_ and can be installed from\nthere::\n\n pip install im-data-manager-job-decoder\n\nOnce installed you can validate the definition (expected to be a dictionary\nformed from the definition YAML file) with::\n\n >>> from decoder import decoder\n >>> error: Optional[str] = decoder.validate_manifest_schema(manifest)\n >>> error: Optional[str] = decoder.validate_job_schema(job_definition)\n\nAnd you can decode encoded fields within the job definition.\nHere we're decoding the job's 'command' using a map of variables and their\nvalues::\n\n >>> decoded, success = decoder.decode(definition['command'],\n variables,\n 'command',\n decoder.TextEncoding.JINJA2_3_0)\n\nA method also exists to generate a Data Manger Job documentation URL\nfor the supported repository types. For example, to get the\nauto-generated documentation URL for a job definition hosted in a GitHub\nrepository you can do this::\n\n >>> doc_url: str = decoder.get_job_doc_url(\"github\",\n collection\n job_id,\n job_definition,\n manifest_url)\n\nWe support ``github`` and ``gitlab`` as repository types.\n\n.. _PyPI: https://pypi.org/project/im-data-manager-job-decoder\n\nGet in touch\n============\n\n- Report bugs, suggest features or view the source code `on GitHub`_.\n\n.. _on GitHub: https://github.com/informaticsmatters/squonk2-data-manager-job-decoder\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Job decoding logic",
"version": "2.5.0",
"project_urls": {
"Homepage": "https://github.com/informaticsmatters/squonk2-data-manager-job-decoder"
},
"split_keywords": [
"jinja2",
"decoder"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "0f9e7105bf8e872842ac802d917b7396efd179f336d968ca806ce6736aa99d72",
"md5": "3a3740bdf0bd8da21151ca502815d4e7",
"sha256": "d177a37083b73c82d71c137cd36ab3bf54de0a4ab5ab55e5aec49acb238b86f6"
},
"downloads": -1,
"filename": "im_data_manager_job_decoder-2.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3a3740bdf0bd8da21151ca502815d4e7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 13542,
"upload_time": "2025-09-01T11:21:14",
"upload_time_iso_8601": "2025-09-01T11:21:14.427617Z",
"url": "https://files.pythonhosted.org/packages/0f/9e/7105bf8e872842ac802d917b7396efd179f336d968ca806ce6736aa99d72/im_data_manager_job_decoder-2.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "50b7de1a3ef7cf31de2d3afedb69fdb931a77ec3057c32fc108ada4c6cfc2b8c",
"md5": "54a3492f5762c1fb1845d187d6e658a0",
"sha256": "1a0523ccead3ad851dcf6a450ec1792be1830d20a938d3ddfdf04ffcdf915a47"
},
"downloads": -1,
"filename": "im_data_manager_job_decoder-2.5.0.tar.gz",
"has_sig": false,
"md5_digest": "54a3492f5762c1fb1845d187d6e658a0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 17445,
"upload_time": "2025-09-01T11:21:15",
"upload_time_iso_8601": "2025-09-01T11:21:15.891951Z",
"url": "https://files.pythonhosted.org/packages/50/b7/de1a3ef7cf31de2d3afedb69fdb931a77ec3057c32fc108ada4c6cfc2b8c/im_data_manager_job_decoder-2.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-01 11:21:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "informaticsmatters",
"github_project": "squonk2-data-manager-job-decoder",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "jinja2",
"specs": [
[
"==",
"3.1.3"
]
]
},
{
"name": "jsonschema",
"specs": [
[
">=",
"4.21.1"
],
[
"<",
"5.0"
]
]
},
{
"name": "pyyaml",
"specs": [
[
"<",
"7.0"
],
[
">=",
"5.3.1"
]
]
}
],
"lcname": "im-data-manager-job-decoder"
}
JSON
