======================
arch-release-promotion
======================
This project allows for promotion and synchronization of existing releases of a
project in Arch Linux's Gitlab instance.
Releases of a project (e.g. ``project``) may consist of several release types
(e.g. ``image_a`` and ``image_b``), which are addressed separately.
A promotion encompasses - per release type - PGP signatures for relevant
artifacts (optional), a torrent file (optional) and a JSON payload which can be
used by `archweb <https://github.com/archlinux/archweb>`_ to display
information about each release type.
Synchronization with a local directory can be achieved for a configurable
maximum amount of release versions (each consisting of their respective
configured release types) of a project.
Requirements
============
Arch-release-promotion is Python based. All language specific requirements are
specified in its `pyproject.toml <pyproject.toml>`_.
Additionally, ``arch-release-promotion`` requires `gnupg <https://gnupg.org/>`_
to handle detached PGP signatures.
Use
===
After installation, refer to the output of ``arch-release-promotion -h`` and
``arch-release-sync -h``.
Configuration
=============
The command-line tools ``arch-release-promotion`` and ``arch-release-sync``
make use of two sources of configuration:
* `makepkg.conf <https://man.archlinux.org/man/makepkg.conf.5>`_ is read from
any of its locations in the same priority as `makepkg
<https://man.archlinux.org/man/makepkg.8>`_ does.
All of the below can also be passed to the tool via environment variables:
* ``GPGKEY`` is recognized for establishing which PGP key to use for signing
* ``PACKAGER`` is recognized for establishing who is doing the signature and
is important for `WKD
<https://wiki.archlinux.org/title/GnuPG#Web_Key_Directory>`_ lookup
* ``MIRRORLIST_URL`` (not used by makepkg) is used during the generation of
torrent files to add webseeds (defaults to
``"https://archlinux.org/mirrorlist/?country=all&protocol=http&protocol=https"``)
* ``GITLAB_URL`` (not used by makepkg) is used to connect to a GitLab
instance to select, download and promote releases of a project (defaults to
``"https://gitlab.archlinux.org"``)
* ``PRIVATE_TOKEN`` (not used by makepkg) is used to authenticate against the
GitLab instance. The `personal access token
<https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html>`_
needs to provide write access for the target project.
* ``projects.toml`` is a configuration file that provides the configuration for a
project and its releases. Configuration files are read and merged with
descending priority from ``/etc/arch-release-promotion/projects.toml`` and
``$XDG_CONFIG_HOME/arch-release-promotion/projects.toml`` (which defaults to
``$HOME/.config/arch-release-promotion/projects.toml`` if
``$XDG_CONFIG_HOME`` is unset).
Please refer to `examples/example.toml <examples/example.toml>`_ for further
reference in regards to the available options
Openmetrics
-----------
If the upstream project offers an `openmetrics <https://openmetrics.io/>`_
based metrics file, the data from it can be used as additional information in
the JSON payload.
The following metrics are considered.
Version metrics
^^^^^^^^^^^^^^^
Description and version information about e.g. packages can be derived from
``version_info`` metrics of type ``info``, that define a ``name``,
``description`` and ``version`` label.
For the metrics to be considered, they have to be configured by adding a
``version_metrics`` list (a list of names to look for) to a release of a
project.
.. code::
# TYPE version_info info
# HELP version_info Package description and version information
version_info{name="my-package",description="Version of my-package used for build",version="1.0.0-1"} 1
The above metrics entry would result in the following JSON representation:
.. code:: json
"version_metrics": [
{
"name": "my-package",
"description": "Version of my-package used for build",
"version": "1.0.0-1"
}
]
Size metrics
^^^^^^^^^^^^
Artifact size information in MebiBytes (MiB) and description can be derived
from ``artifact_bytes`` metrics of type ``gauge``, that define a ``name`` and a
``description`` label.
For the metrics to be considered, they have to be configured by adding a
``size_metrics`` list (a list of names to look for) to a release of a
project.
.. code::
# TYPE artifact_bytes gauge
# HELP artifact_bytes Artifact sizes in Bytes
artifact_bytes{name="foo",description="Size of foo in MiB"} 832
The above metrics entry would result in the following JSON representation:
.. code:: json
"size_metrics": [
{
"name": "foo",
"description": "Size of foo in MiB",
"size": 832
}
]
Amount metrics
^^^^^^^^^^^^^^
Information on the amount of something (e.g. packages) and description can be
derived from ``data_count`` metrics of type ``summary``, that define a ``name``
and a ``description`` label.
For the metrics to be considered, they have to be configured by adding a
``amount_metrics`` list (a list of names to look for) to a release of a
project.
.. code::
# TYPE data_count summary
# HELP data_count The amount of something used in some context
data_count{name="foo",description="The amount of packages in foo"} 369
The above metrics entry would result in the following JSON representation:
.. code:: json
"amount_metrics": [
{
"name": "foo",
"description": "The amount of packages in foo",
"amount": 369
}
]
Promotion artifact
==================
The promotion artifact is a ZIP compressed file (``promotion.zip``), that is
uploaded to the project before its link is added to the release that it is
promoting.
The file contains one directory for each release type that the project offers.
In each release type directory there are is a **JSON payload**
(``<release_type>-<version>.json``), a directory
(``<release_type>-<version>/``) containing signatures for any files that have
been setup for detached signatures and optionally a torrent file
(``<release_type>-<version>.json``) that is created for the release type's
build artifacts *and* the detached signatures contained in the promotion
artifact.
.. code::
example
├── example-0.1.0
│ └── artifact.tar.gz.sig
├── example-0.1.0.json
└── example-0.1.0.torrent
JSON payload
------------
The promotion of a release encompasses one or more JSON payloads, that describe
each release type in the release.
.. code:: json
{
"amount_metrics": [
{
"name": "foo",
"description": "The amount of packages in foo",
"size": 369
}
],
"developer": "Foobar McFooface <foobar@mcfooface.com>",
"files": ["something.txt", "something.txt.sig"],
"name": "foo",
"pgp_public_key": "SOMEONESPGPKEYID",
"size_metrics": [
{
"name": "foo",
"description": "Size of foo in MiB",
"size": 832
}
],
"torrent_file": "foo-0.1.0.torrent",
"version_metrics": [
{
"name": "my-package",
"description": "Version of my-package used for build",
"version": "1.0.0-1"
}
],
"version": "0.1.0"
}
* ``amount_metrics``: A list of objects that describe the amount of something
(optional). The list depends on whether the project's configuration defines
``amount_metrics`` and whether those metrics are available in the specific
release.
* ``developer``: The full uid of the person promoting (and optionally signing
artifacts in) the release type.
* ``files``: A list of files in the release type.
* ``name``: The name of the release type.
* ``pgp_public_key``: The PGP key ID of the key signing files in the release
type.
* ``size_metrics``: A list of objects that describe the size of something
(optional). The list depends on whether the project's configuration defines
``size_metrics`` and whether those metrics are available in the specific
release.
* ``torrent_file`` (optional): The name of a torrent file created for the
release type. The value depends on whether the configuration for the release
type sets ``create_torrent`` to ``True``.
* ``version_metrics``: A list of objects that describe the version of something
(optional). The list depends on whether the project's configuration defines
``version_metrics`` and whether those metrics are available in the specific
release.
* ``version``: The version of the release type.
Synchronization
===============
The synchronization of releases works by retrieving the list of promoted
releases of the project from the remote. For each promoted release version, the
promotion artifact is downloaded and used to establish whether all of the
configured release types are fully synchronized.
Location and cleanup
--------------------
All release types for each release version are synchronized to a local
directory. The directory and and the maximum amount of synchronized release
versions are configurable globally or per project.
.. code::
sync_dir
├── example_a
│ ├── example_a-0.1.0
│ │ ├── foo.txt
│ │ └── foo.txt.sig
│ ├── example_a-0.1.0.json
│ ├── example_a-0.1.0.torrent
│ └── latest -> example_a-0.1.0
└── example_b
├── example_b-0.1.0
│ ├── bar.txt
│ └── bar.txt.sig
├── example_b-0.1.0.json
├── example_b-0.1.0.torrent
└── latest -> example_b-0.1.0
A ``latest`` symlink is created to point at the currently latest version of
each release type.
Any files and directories that are not owned by versions of release types of
the currently synchronized release versions are removed from the
synchronization directory.
If changes are introduced to files in the target directory (due to a
synchronization action), it is possible to write a Unix timestamp to a file
that is configurable globally or per project (the directory in which the file
resides in has to exist).
System integration
------------------
For systemd based systems there are example systemd system service and timer
files that are provided in `examples/systemd/ <examples/systemd/>`_.
The provided service file relies on the user ``arch-release-sync`` which may be
created using the `sysusers.d
<https://man.archlinux.org/man/core/systemd/sysusers.d.5.en>`_ integration
provided in `examples/sysusers.d/ <examples/sysusers.d/>`_.
License
=======
Arch-release-promotion is licensed under the terms of the **GPL-3.0-or-later**
(see `LICENSE <LICENSE>`_).
Raw data
{
"_id": null,
"home_page": "https://gitlab.archlinux.org/archlinux/arch-release-promotion",
"name": "arch-release-promotion",
"maintainer": "",
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": "",
"keywords": "arch linux releases pgp signature synchronization torrent",
"author": "",
"author_email": "David Runge <dvzrv@archlinux.org>",
"download_url": "https://files.pythonhosted.org/packages/73/e6/3186f3d8cb5e252321031eb32e10125a33b4eb0ec1537decc23a2dd4841b/arch_release_promotion-0.3.0.post0.tar.gz",
"platform": null,
"description": "======================\narch-release-promotion\n======================\n\nThis project allows for promotion and synchronization of existing releases of a\nproject in Arch Linux's Gitlab instance.\n\nReleases of a project (e.g. ``project``) may consist of several release types\n(e.g. ``image_a`` and ``image_b``), which are addressed separately.\n\nA promotion encompasses - per release type - PGP signatures for relevant\nartifacts (optional), a torrent file (optional) and a JSON payload which can be\nused by `archweb <https://github.com/archlinux/archweb>`_ to display\ninformation about each release type.\n\nSynchronization with a local directory can be achieved for a configurable\nmaximum amount of release versions (each consisting of their respective\nconfigured release types) of a project.\n\nRequirements\n============\n\nArch-release-promotion is Python based. All language specific requirements are\nspecified in its `pyproject.toml <pyproject.toml>`_.\n\nAdditionally, ``arch-release-promotion`` requires `gnupg <https://gnupg.org/>`_\nto handle detached PGP signatures.\n\nUse\n===\n\nAfter installation, refer to the output of ``arch-release-promotion -h`` and\n``arch-release-sync -h``.\n\nConfiguration\n=============\n\nThe command-line tools ``arch-release-promotion`` and ``arch-release-sync``\nmake use of two sources of configuration:\n\n* `makepkg.conf <https://man.archlinux.org/man/makepkg.conf.5>`_ is read from\n any of its locations in the same priority as `makepkg\n <https://man.archlinux.org/man/makepkg.8>`_ does.\n All of the below can also be passed to the tool via environment variables:\n\n * ``GPGKEY`` is recognized for establishing which PGP key to use for signing\n * ``PACKAGER`` is recognized for establishing who is doing the signature and\n is important for `WKD\n <https://wiki.archlinux.org/title/GnuPG#Web_Key_Directory>`_ lookup\n * ``MIRRORLIST_URL`` (not used by makepkg) is used during the generation of\n torrent files to add webseeds (defaults to\n ``\"https://archlinux.org/mirrorlist/?country=all&protocol=http&protocol=https\"``)\n * ``GITLAB_URL`` (not used by makepkg) is used to connect to a GitLab\n instance to select, download and promote releases of a project (defaults to\n ``\"https://gitlab.archlinux.org\"``)\n * ``PRIVATE_TOKEN`` (not used by makepkg) is used to authenticate against the\n GitLab instance. The `personal access token\n <https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html>`_\n needs to provide write access for the target project.\n\n* ``projects.toml`` is a configuration file that provides the configuration for a\n project and its releases. Configuration files are read and merged with\n descending priority from ``/etc/arch-release-promotion/projects.toml`` and\n ``$XDG_CONFIG_HOME/arch-release-promotion/projects.toml`` (which defaults to\n ``$HOME/.config/arch-release-promotion/projects.toml`` if\n ``$XDG_CONFIG_HOME`` is unset).\n Please refer to `examples/example.toml <examples/example.toml>`_ for further\n reference in regards to the available options\n\nOpenmetrics\n-----------\n\nIf the upstream project offers an `openmetrics <https://openmetrics.io/>`_\nbased metrics file, the data from it can be used as additional information in\nthe JSON payload.\n\nThe following metrics are considered.\n\nVersion metrics\n^^^^^^^^^^^^^^^\n\nDescription and version information about e.g. packages can be derived from\n``version_info`` metrics of type ``info``, that define a ``name``,\n``description`` and ``version`` label.\n\nFor the metrics to be considered, they have to be configured by adding a\n``version_metrics`` list (a list of names to look for) to a release of a\nproject.\n\n.. code::\n\n # TYPE version_info info\n # HELP version_info Package description and version information\n version_info{name=\"my-package\",description=\"Version of my-package used for build\",version=\"1.0.0-1\"} 1\n\nThe above metrics entry would result in the following JSON representation:\n\n.. code:: json\n\n \"version_metrics\": [\n {\n \"name\": \"my-package\",\n \"description\": \"Version of my-package used for build\",\n \"version\": \"1.0.0-1\"\n }\n ]\n\nSize metrics\n^^^^^^^^^^^^\n\nArtifact size information in MebiBytes (MiB) and description can be derived\nfrom ``artifact_bytes`` metrics of type ``gauge``, that define a ``name`` and a\n``description`` label.\n\nFor the metrics to be considered, they have to be configured by adding a\n``size_metrics`` list (a list of names to look for) to a release of a\nproject.\n\n.. code::\n\n # TYPE artifact_bytes gauge\n # HELP artifact_bytes Artifact sizes in Bytes\n artifact_bytes{name=\"foo\",description=\"Size of foo in MiB\"} 832\n\nThe above metrics entry would result in the following JSON representation:\n\n.. code:: json\n\n \"size_metrics\": [\n {\n \"name\": \"foo\",\n \"description\": \"Size of foo in MiB\",\n \"size\": 832\n }\n ]\n\nAmount metrics\n^^^^^^^^^^^^^^\n\nInformation on the amount of something (e.g. packages) and description can be\nderived from ``data_count`` metrics of type ``summary``, that define a ``name``\nand a ``description`` label.\n\nFor the metrics to be considered, they have to be configured by adding a\n``amount_metrics`` list (a list of names to look for) to a release of a\nproject.\n\n.. code::\n\n # TYPE data_count summary\n # HELP data_count The amount of something used in some context\n data_count{name=\"foo\",description=\"The amount of packages in foo\"} 369\n\nThe above metrics entry would result in the following JSON representation:\n\n.. code:: json\n\n \"amount_metrics\": [\n {\n \"name\": \"foo\",\n \"description\": \"The amount of packages in foo\",\n \"amount\": 369\n }\n ]\n\nPromotion artifact\n==================\n\nThe promotion artifact is a ZIP compressed file (``promotion.zip``), that is\nuploaded to the project before its link is added to the release that it is\npromoting.\n\nThe file contains one directory for each release type that the project offers.\nIn each release type directory there are is a **JSON payload**\n(``<release_type>-<version>.json``), a directory\n(``<release_type>-<version>/``) containing signatures for any files that have\nbeen setup for detached signatures and optionally a torrent file\n(``<release_type>-<version>.json``) that is created for the release type's\nbuild artifacts *and* the detached signatures contained in the promotion\nartifact.\n\n.. code::\n\n example\n \u251c\u2500\u2500 example-0.1.0\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 artifact.tar.gz.sig\n \u251c\u2500\u2500 example-0.1.0.json\n \u2514\u2500\u2500 example-0.1.0.torrent\n\nJSON payload\n------------\n\nThe promotion of a release encompasses one or more JSON payloads, that describe\neach release type in the release.\n\n.. code:: json\n\n {\n \"amount_metrics\": [\n {\n \"name\": \"foo\",\n \"description\": \"The amount of packages in foo\",\n \"size\": 369\n }\n ],\n \"developer\": \"Foobar McFooface <foobar@mcfooface.com>\",\n \"files\": [\"something.txt\", \"something.txt.sig\"],\n \"name\": \"foo\",\n \"pgp_public_key\": \"SOMEONESPGPKEYID\",\n \"size_metrics\": [\n {\n \"name\": \"foo\",\n \"description\": \"Size of foo in MiB\",\n \"size\": 832\n }\n ],\n \"torrent_file\": \"foo-0.1.0.torrent\",\n \"version_metrics\": [\n {\n \"name\": \"my-package\",\n \"description\": \"Version of my-package used for build\",\n \"version\": \"1.0.0-1\"\n }\n ],\n \"version\": \"0.1.0\"\n }\n\n* ``amount_metrics``: A list of objects that describe the amount of something\n (optional). The list depends on whether the project's configuration defines\n ``amount_metrics`` and whether those metrics are available in the specific\n release.\n* ``developer``: The full uid of the person promoting (and optionally signing\n artifacts in) the release type.\n* ``files``: A list of files in the release type.\n* ``name``: The name of the release type.\n* ``pgp_public_key``: The PGP key ID of the key signing files in the release\n type.\n* ``size_metrics``: A list of objects that describe the size of something\n (optional). The list depends on whether the project's configuration defines\n ``size_metrics`` and whether those metrics are available in the specific\n release.\n* ``torrent_file`` (optional): The name of a torrent file created for the\n release type. The value depends on whether the configuration for the release\n type sets ``create_torrent`` to ``True``.\n* ``version_metrics``: A list of objects that describe the version of something\n (optional). The list depends on whether the project's configuration defines\n ``version_metrics`` and whether those metrics are available in the specific\n release.\n* ``version``: The version of the release type.\n\nSynchronization\n===============\n\nThe synchronization of releases works by retrieving the list of promoted\nreleases of the project from the remote. For each promoted release version, the\npromotion artifact is downloaded and used to establish whether all of the\nconfigured release types are fully synchronized.\n\nLocation and cleanup\n--------------------\n\nAll release types for each release version are synchronized to a local\ndirectory. The directory and and the maximum amount of synchronized release\nversions are configurable globally or per project.\n\n.. code::\n\n sync_dir\n \u251c\u2500\u2500 example_a\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 example_a-0.1.0\n \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 foo.txt\n \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 foo.txt.sig\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 example_a-0.1.0.json\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 example_a-0.1.0.torrent\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 latest -> example_a-0.1.0\n \u2514\u2500\u2500 example_b\n \u251c\u2500\u2500 example_b-0.1.0\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 bar.txt\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 bar.txt.sig\n \u251c\u2500\u2500 example_b-0.1.0.json\n \u251c\u2500\u2500 example_b-0.1.0.torrent\n \u2514\u2500\u2500 latest -> example_b-0.1.0\n\nA ``latest`` symlink is created to point at the currently latest version of\neach release type.\n\nAny files and directories that are not owned by versions of release types of\nthe currently synchronized release versions are removed from the\nsynchronization directory.\n\nIf changes are introduced to files in the target directory (due to a\nsynchronization action), it is possible to write a Unix timestamp to a file\nthat is configurable globally or per project (the directory in which the file\nresides in has to exist).\n\nSystem integration\n------------------\n\nFor systemd based systems there are example systemd system service and timer\nfiles that are provided in `examples/systemd/ <examples/systemd/>`_.\nThe provided service file relies on the user ``arch-release-sync`` which may be\ncreated using the `sysusers.d\n<https://man.archlinux.org/man/core/systemd/sysusers.d.5.en>`_ integration\nprovided in `examples/sysusers.d/ <examples/sysusers.d/>`_.\n\nLicense\n=======\n\nArch-release-promotion is licensed under the terms of the **GPL-3.0-or-later**\n(see `LICENSE <LICENSE>`_).\n\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Promote official Arch Linux releases and synchronize them",
"version": "0.3.0.post0",
"project_urls": {
"Bug tracker": "https://gitlab.archlinux.org/archlinux/arch-release-promotion/-/issues/",
"Documentation": "https://gitlab.archlinux.org/archlinux/arch-release-promotion/-/blob/master/README.rst",
"Homepage": "https://gitlab.archlinux.org/archlinux/arch-release-promotion",
"Repository": "https://gitlab.archlinux.org/archlinux/arch-release-promotion"
},
"split_keywords": [
"arch",
"linux",
"releases",
"pgp",
"signature",
"synchronization",
"torrent"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8ef71f732e85675dd9fc923921fcdfc2bdf13de007caa9a404c4d183b4ba78c5",
"md5": "6f2307f05f899fe598eb1c045b1651fe",
"sha256": "66fb546736fea271f6dfe9c66d2d8dc24e5bf4e221fd3d4686ae964b970e7a8b"
},
"downloads": -1,
"filename": "arch_release_promotion-0.3.0.post0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6f2307f05f899fe598eb1c045b1651fe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 33877,
"upload_time": "2023-07-18T06:51:29",
"upload_time_iso_8601": "2023-07-18T06:51:29.090154Z",
"url": "https://files.pythonhosted.org/packages/8e/f7/1f732e85675dd9fc923921fcdfc2bdf13de007caa9a404c4d183b4ba78c5/arch_release_promotion-0.3.0.post0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "73e63186f3d8cb5e252321031eb32e10125a33b4eb0ec1537decc23a2dd4841b",
"md5": "2f12c66daf62da286e130a94ecfda603",
"sha256": "8a5666b76c0c3133991fd8886d3fbd3103294f81f9526974083e524b513d864f"
},
"downloads": -1,
"filename": "arch_release_promotion-0.3.0.post0.tar.gz",
"has_sig": false,
"md5_digest": "2f12c66daf62da286e130a94ecfda603",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 54075,
"upload_time": "2023-07-18T06:51:30",
"upload_time_iso_8601": "2023-07-18T06:51:30.923994Z",
"url": "https://files.pythonhosted.org/packages/73/e6/3186f3d8cb5e252321031eb32e10125a33b4eb0ec1537decc23a2dd4841b/arch_release_promotion-0.3.0.post0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-18 06:51:30",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "arch-release-promotion"
}
JSON