| Name | dkist-processing-common JSON |
| Version |
11.2.0
JSON |
| download |
| home_page | None |
| Summary | Common task classes used by the DKIST science data processing pipelines |
| upload_time | 2025-07-18 15:49:30 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.11 |
| license | BSD-3-Clause |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
dkist-processing-common
=======================
|codecov|
This repository works in concert with `dkist-processing-core <https://pypi.org/project/dkist-processing-core/>`_ and `dkist-processing-*instrument*` to
form the DKIST calibration processing stack.
Usage
-----
The classes in this repository should be used as the base of any DKIST processing pipeline tasks. Science tasks should subclass `ScienceTaskL0ToL1Base`.
Each class is built on an abstract base class with the `run` method left for a developer to fill out with the required steps that the task should take.
This class is then used as the callable object for the workflow and scheduling engine.
Example
-------
.. code-block:: python
from dkist_processing_common.tasks.base import ScienceTaskL0ToL1Base
class RemoveArtifacts(ScienceTaskL0ToL1Base):
def run(self):
# task code here
total = 2 + 5
Deployment
----------
dkist-processing-common is deployed to `PyPI <https://pypi.org/project/dkist-processing-common/>`_
Development
-----------
There are two prerequisites for test execution on a local machine:
* Redis. A running instance of redis on the local machine is required. The tests will use the default host ip of localhost and port of 6379 to connect to the database.
* RabbitMQ. A running instance of rabbitmq on the local machine is required. The tests will use the default host of localhost and a port of 5672 to connect to the interservice bus.
To run the tests locally, clone the repository and install the package in editable mode with the test extras.
.. code-block:: bash
git clone git@bitbucket.org:dkistdc/dkist-processing-common.git
cd dkist-processing-common
pre-commit install
pip install -e .[test]
# Redis must be running
pytest -v --cov dkist_processing_common
Changelog
#########
When you make **any** change to this repository it **MUST** be accompanied by a changelog file.
The changelog for this repository uses the `towncrier <https://github.com/twisted/towncrier>`__ package.
Entries in the changelog for the next release are added as individual files (one per change) to the ``changelog/`` directory.
Writing a Changelog Entry
^^^^^^^^^^^^^^^^^^^^^^^^^
A changelog entry accompanying a change should be added to the ``changelog/`` directory.
The name of a file in this directory follows a specific template::
<PULL REQUEST NUMBER>.<TYPE>[.<COUNTER>].rst
The fields have the following meanings:
* ``<PULL REQUEST NUMBER>``: This is the number of the pull request, so people can jump from the changelog entry to the diff on BitBucket.
* ``<TYPE>``: This is the type of the change and must be one of the values described below.
* ``<COUNTER>``: This is an optional field, if you make more than one change of the same type you can append a counter to the subsequent changes, i.e. ``100.bugfix.rst`` and ``100.bugfix.1.rst`` for two bugfix changes in the same PR.
The list of possible types is defined the the towncrier section of ``pyproject.toml``, the types are:
* ``feature``: This change is a new code feature.
* ``bugfix``: This is a change which fixes a bug.
* ``doc``: A documentation change.
* ``removal``: A deprecation or removal of public API.
* ``misc``: Any small change which doesn't fit anywhere else, such as a change to the package infrastructure.
Rendering the Changelog at Release Time
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When you are about to tag a release first you must run ``towncrier`` to render the changelog.
The steps for this are as follows:
* Run `towncrier build --version vx.y.z` using the version number you want to tag.
* Agree to have towncrier remove the fragments.
* Add and commit your changes.
* Tag the release.
**NOTE:** If you forget to add a Changelog entry to a tagged release (either manually or automatically with ``towncrier``)
then the Bitbucket pipeline will fail. To be able to use the same tag you must delete it locally and on the remote branch:
.. code-block:: bash
# First, actually update the CHANGELOG and commit the update
git commit
# Delete tags
git tag -d vWHATEVER.THE.VERSION
git push --delete origin vWHATEVER.THE.VERSION
# Re-tag with the same version
git tag vWHATEVER.THE.VERSION
git push --tags origin main
.. |codecov| image:: https://codecov.io/bb/dkistdc/dkist-processing-common/graph/badge.svg?token=3QSLGSEF3O
:target: https://codecov.io/bb/dkistdc/dkist-processing-common
Raw data
{
"_id": null,
"home_page": null,
"name": "dkist-processing-common",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "NSO / AURA <dkistdc@nso.edu>",
"download_url": "https://files.pythonhosted.org/packages/00/b2/6b9b2fec2ab30c50bd28e2f4ee2fcbf42716196f01ad2dbd137241d7f1cb/dkist_processing_common-11.2.0.tar.gz",
"platform": null,
"description": "dkist-processing-common\n=======================\n\n|codecov|\n\nThis repository works in concert with `dkist-processing-core <https://pypi.org/project/dkist-processing-core/>`_ and `dkist-processing-*instrument*` to\nform the DKIST calibration processing stack.\n\nUsage\n-----\n\nThe classes in this repository should be used as the base of any DKIST processing pipeline tasks. Science tasks should subclass `ScienceTaskL0ToL1Base`.\n\nEach class is built on an abstract base class with the `run` method left for a developer to fill out with the required steps that the task should take.\nThis class is then used as the callable object for the workflow and scheduling engine.\n\nExample\n-------\n\n.. code-block:: python\n\n from dkist_processing_common.tasks.base import ScienceTaskL0ToL1Base\n\n\n class RemoveArtifacts(ScienceTaskL0ToL1Base):\n def run(self):\n # task code here\n total = 2 + 5\n\nDeployment\n----------\n\ndkist-processing-common is deployed to `PyPI <https://pypi.org/project/dkist-processing-common/>`_\n\nDevelopment\n-----------\n\nThere are two prerequisites for test execution on a local machine:\n\n\n* Redis. A running instance of redis on the local machine is required. The tests will use the default host ip of localhost and port of 6379 to connect to the database.\n\n* RabbitMQ. A running instance of rabbitmq on the local machine is required. The tests will use the default host of localhost and a port of 5672 to connect to the interservice bus.\n\n\nTo run the tests locally, clone the repository and install the package in editable mode with the test extras.\n\n\n.. code-block:: bash\n\n git clone git@bitbucket.org:dkistdc/dkist-processing-common.git\n cd dkist-processing-common\n pre-commit install\n pip install -e .[test]\n # Redis must be running\n pytest -v --cov dkist_processing_common\n\nChangelog\n#########\n\nWhen you make **any** change to this repository it **MUST** be accompanied by a changelog file.\nThe changelog for this repository uses the `towncrier <https://github.com/twisted/towncrier>`__ package.\nEntries in the changelog for the next release are added as individual files (one per change) to the ``changelog/`` directory.\n\nWriting a Changelog Entry\n^^^^^^^^^^^^^^^^^^^^^^^^^\n\nA changelog entry accompanying a change should be added to the ``changelog/`` directory.\nThe name of a file in this directory follows a specific template::\n\n <PULL REQUEST NUMBER>.<TYPE>[.<COUNTER>].rst\n\nThe fields have the following meanings:\n\n* ``<PULL REQUEST NUMBER>``: This is the number of the pull request, so people can jump from the changelog entry to the diff on BitBucket.\n* ``<TYPE>``: This is the type of the change and must be one of the values described below.\n* ``<COUNTER>``: This is an optional field, if you make more than one change of the same type you can append a counter to the subsequent changes, i.e. ``100.bugfix.rst`` and ``100.bugfix.1.rst`` for two bugfix changes in the same PR.\n\nThe list of possible types is defined the the towncrier section of ``pyproject.toml``, the types are:\n\n* ``feature``: This change is a new code feature.\n* ``bugfix``: This is a change which fixes a bug.\n* ``doc``: A documentation change.\n* ``removal``: A deprecation or removal of public API.\n* ``misc``: Any small change which doesn't fit anywhere else, such as a change to the package infrastructure.\n\n\nRendering the Changelog at Release Time\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWhen you are about to tag a release first you must run ``towncrier`` to render the changelog.\nThe steps for this are as follows:\n\n* Run `towncrier build --version vx.y.z` using the version number you want to tag.\n* Agree to have towncrier remove the fragments.\n* Add and commit your changes.\n* Tag the release.\n\n**NOTE:** If you forget to add a Changelog entry to a tagged release (either manually or automatically with ``towncrier``)\nthen the Bitbucket pipeline will fail. To be able to use the same tag you must delete it locally and on the remote branch:\n\n.. code-block:: bash\n\n # First, actually update the CHANGELOG and commit the update\n git commit\n\n # Delete tags\n git tag -d vWHATEVER.THE.VERSION\n git push --delete origin vWHATEVER.THE.VERSION\n\n # Re-tag with the same version\n git tag vWHATEVER.THE.VERSION\n git push --tags origin main\n\n.. |codecov| image:: https://codecov.io/bb/dkistdc/dkist-processing-common/graph/badge.svg?token=3QSLGSEF3O\n :target: https://codecov.io/bb/dkistdc/dkist-processing-common\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Common task classes used by the DKIST science data processing pipelines",
"version": "11.2.0",
"project_urls": {
"Documentation": "https://docs.dkist.nso.edu/projects/common",
"Help": "https://nso.atlassian.net/servicedesk/customer/portal/5",
"Homepage": "https://nso.edu/dkist/data-center/",
"Repository": "https://bitbucket.org/dkistdc/dkist-processing-common/"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "f924d5e359729ef666eb65676286e49ae5efa5934f9b9d779c25791949eeabc2",
"md5": "aef20b97fe6c02b2b0661abc973cd1a5",
"sha256": "b7fe301ef64e783c4dfecf9eef62bb0137df942c2fdf22f97864a1ea9c098e44"
},
"downloads": -1,
"filename": "dkist_processing_common-11.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "aef20b97fe6c02b2b0661abc973cd1a5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 541387,
"upload_time": "2025-07-18T15:49:29",
"upload_time_iso_8601": "2025-07-18T15:49:29.380696Z",
"url": "https://files.pythonhosted.org/packages/f9/24/d5e359729ef666eb65676286e49ae5efa5934f9b9d779c25791949eeabc2/dkist_processing_common-11.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "00b26b9b2fec2ab30c50bd28e2f4ee2fcbf42716196f01ad2dbd137241d7f1cb",
"md5": "cd80f0da5f1159097bcf1ac71c2ef82e",
"sha256": "6f5f625c6001a2a0d272e2fc1bf163de454c0d001acb3822603040289c7bfad1"
},
"downloads": -1,
"filename": "dkist_processing_common-11.2.0.tar.gz",
"has_sig": false,
"md5_digest": "cd80f0da5f1159097bcf1ac71c2ef82e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 512990,
"upload_time": "2025-07-18T15:49:30",
"upload_time_iso_8601": "2025-07-18T15:49:30.990709Z",
"url": "https://files.pythonhosted.org/packages/00/b2/6b9b2fec2ab30c50bd28e2f4ee2fcbf42716196f01ad2dbd137241d7f1cb/dkist_processing_common-11.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-18 15:49:30",
"github": false,
"gitlab": false,
"bitbucket": true,
"codeberg": false,
"bitbucket_user": "dkistdc",
"bitbucket_project": "dkist-processing-common",
"lcname": "dkist-processing-common"
}
