Fedora Badges Message consumer
==============================
This repo contains the consumer and the command necessary to hook the
badges stack (Tahrir, Tahrir-API, Tahrir-REST) into Fedora Messaging.
It is the process that runs in the background, monitoring activity of Fedora
contributors, and is responsible for awarding badges for activity as it happens.
It is separate from and sometimes confused with the *frontend* of the badges system
called `tahrir <https://github.com/fedora-infra/tahrir>`_.
This project (fedbadges) writes to a database that the web frontend (tahrir) reads
from.
The *actual badge rules* that we act on in Fedora Infrastructure can be
found `here <https://pagure.io/Fedora-Badges>`.
Architecture
------------
fedbadges is a callback class for the Fedora Messaging consumer.
When started, it will load some initial configuration
and a set of ``BadgeRules`` (more on that later) and then sit quietly
listening to the Fedora Messaging bus. Each rule (composed of some metadata,
a ``trigger``, an optional ``condition`` and an optional way of counting previous
messages) is defined on disk as a yaml file.
* When a new message comes along, our callback looks to see if it matches
any of the ``BadgeRules`` it has registered.
* Each BadgeRule must define a ``trigger`` -- a *lightweight* check.
When processing a message, this is the first thing that is checked. It
defines a *pattern* that the message must match. If the message does not
match, then the current BadgeRule is discarded and processing moves to
the next.
A ``trigger`` is typically something like "any bodhi message"
or "messages only from the failure of a koji build". More on their
specification below.
* BadgeRules can also define a ``previous`` value, as a way to count similar
messages that went through the bus in the past. This typically involves a
more expensive query to the
`datanommer <https://github.com/fedora-infra/datanommer>`_ database.
A BadgeRule ``previous`` query may read something like "updates pushed to
stable by the candidate" or "IRC meetings chaired by the candidate".
**Aside:** Although datanommer is the only currently supported backend, we
can implement other queryable backend in the future as needed like FAS
(to see if the user is in X number of groups) or even off-site services
like libravatar (to award a badge if the user is a user of the AGPL web
service).
* BadgeRule can define a ``condition`` that the number of messages returned by
the ``previous`` query must match. This can be something like
``greater than or equal to: 50``. If unset, the default condition is
``greater than or equal to 1``.
* If no ``previous`` query is set, then the rule only considers the current
incoming message (it's like the ``previous`` result is always ``1``). This
is relevant for rules that award a badge on the first action. Those rules
don't need to set a ``condition`` either, the default one will do.
* If a badge's ``trigger`` and ``condition`` both match, then the badge is
awarded. If the BadgeRule doesn't specify, we award the badge to the
author of the action using the message's ``agent_name`` property.
That is usually correct -- but sometimes, a BadgeRule needs to specify
that one particular user should be recipient of the badge.
In this case, the BadgeRule may define a ``recipient``
in dot-notation that instructs the ``Consumer`` how to extract the
recipient's username from the received message.
The badge is awarded to our deserving user via the `tahrir_api
<https://github.com/fedora-infra/tahrir-api>`_. At the end of the day,
this amounts to adding a row in a database table for the `Tahrir
<https://github.com/fedora-infra/tahrir>`_ application.
There are some optimizations in place omitted above for clarity.
For instance, after the trigger has matched we first check if the user
that *would* be awarded the badge already has it. If they do, we stop
processing the badge rule immediately to avoid making an unnecessary
expensive check against the datanommer db.
Configuration - Global
----------------------
fedbadges needs three major pieces of global configuration.
All configuration is loaded in the standard Fedora Messaging way, from
the ``[consumer_config]`` section of the configuration file. See
`fedbadges.toml.example
<https://github.com/fedora-infra/fedbadges/blob/develop/fedbadges.toml.example>`_
in the git repo for an example.
fedbadges also emits its own messages. In the Fedora Infrastructure, the
``topic_prefix`` will be ``org.fedoraproject.prod``.
Configuration - BadgeRule specification
---------------------------------------
BadgeRules are specified in `YAML <http://www.yaml.org/>`_ on the file system.
Triggers
~~~~~~~~
Every BadgeRule must carry the following minimum set of metadata::
# This is some metadata about the badge
name: Like a Rock
description: You have pushed 500 or more bodhi updates to stable status.
creator: ralph
# This is a link to the discussion about adopting this as a for-real badge.
discussion: http://github.com/fedora-infra/badges/pull/SOME_NUMBER
# A link to the image for the badge
image_url: http://somelink.org/to-an-image.png
Here's a simple example of a ``trigger``::
trigger:
category: bodhi
The above will match any bodhi message on any of the topics that come
from the bodhi update system.
Triggers may employ a little bit of logic to make more complex
filters. The following trigger will match any message that comes from
*either* the bodhi update system or the fedora git package repos::
trigger:
category:
any:
- bodhi
- git
At present triggers may directly compare themselves against only the
`category` or the `topic` of a message. In the future we'd like to add
more comparisons.. in the meantime, here's an example of comparing against
the fully qualified message topic. This will match any message
that is specifically for editing a wiki page::
trigger:
topic: org.fedoraproject.prod.wiki.article.edit
----
There is one additional way you can specify a trigger. If you need more
flexibility than ``topic`` and
``category`` allow, you may specify a custom filter expression with a
``lambda`` filter. For example::
trigger:
lambda: "a string of interest" in json.dumps(message.body)
The above trigger will match if the string ``"a string of interest"`` appears
anywhere in the incoming message. fedbadges takes the expression you provide
it and compiles it into a python callable on initialization. Our callable
here serializes the message to a JSON string before doing its comparison.
Powerful!
Previous
~~~~~~~~
As mentioned above in the architecture section, we currently only support
datanommer as a queryable backend for ``previous`` queries. We hope to expand
that in the future.
Datanommer queries are composed of two things:
- A **filter** limits the scope of the query to datanommer.
- An **operation** defines what we want to do with the filtered query.
Currently, we can *count* the results or run them through a ``lambda``
function that will return an integer (the number of matched messages).
Here's an example of a simple previous definition::
previous:
filter:
topics:
- message.topic
operation: count
The above ``previous`` query will return the number of messages in datanommer
with the same topic as the incoming message being handled. Here,
``message.topic`` is a ``lambda function`` that has the incoming ``message``
in scope.
----
The above example doesn't make much sense -- we'd never use it for a real
badge. The ``previous`` query would be true if there were two of *any* message
kicked off by *any* user at any time in the past. Pretty generic.
Here's a more interesting ``previous`` query::
previous:
filter:
topics:
- org.fedoraproject.prod.git.receive
users:
- message.body["commit"]["username"]
operation: count
This ``previous`` query would return the number of messages of the topic
``"org.fedoraproject.prod.git.receive"`` that were also kicked off by whatever
user is listed in the ``message.body['commit']['username']`` field of the
message being currently processed. In other words, this query would return
the number of pushes to the fedora git repos by the user.
Condition
~~~~~~~~~
You can do some fancy things with the **condition** field.
Here's a list of the possible comparisons you can make:
- ``"is greater than or equal to"`` or alternatively
``"greater than or equal to"``
- ``"greater than"``
- ``"is less than or equal to"`` or alternatively
``"less than or equal to"``
- ``"less than"``
- ``"equal to"`` or alternatively ``"is equal to"``
- ``"is not"`` or alternatively ``"is not equal to"``
As you can see, some of them are synonyms for each other.
----
If any of those don't meet your needs, you can specify a custom expression
by using the ``lambda`` condition whereby fedbadges will compile whatever
statement you provide into a callable and use that at runtime. For example::
condition:
lambda: value != 0 and ((value & (value - 1)) == 0)
Who knows why you would want to do this, but the above condition check will
succeed if the number of messages that matched in the past is exactly a power
of 2.
Specifying Recipients
~~~~~~~~~~~~~~~~~~~~~
By default, if the trigger and condition match, fedbadges will award badges
to the user returned by the message's ``agent_name`` property.
This *usually* corresponds with "which user is responsible" for this message.
That is *usually* what we want to award badges for.
There are some instances for which that is not what we want.
Take the `org.fedoraproject.prod.bodhi.update.comment
<https://fedora-messaging.readthedocs.io/en/stable/user-guide/schemas.html#bodhi>`_
message for example. When user A comments on user B's update, user A is returned
by the message's ``agent_name`` property.
Imagine we have a "Received Comments" badge that's awarded to packagers that
received comments on their updates. We don't want to inadvertently award that
badge to the person who *commented*, only to the one who *created the update*.
To allow for this scenario, badges may optionally define a ``recipient``
in dotted notation that tells fedbadges where to find the username of the
recipient in the originating message. For instance, the following would
handle the fas case we described above::
trigger:
topic: org.fedoraproject.prod.bodhi.update.comment
condition:
greater than or equal to: 1
previous:
filter:
topics:
- message.topic
users:
- recipient
operation: count
recipient: message.body["update"]["user"]["name"]
Raw data
{
"_id": null,
"home_page": "https://github.com/fedora-infra/fedbadges",
"name": "fedbadges",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "fedora",
"author": "Fedora Infrastructure",
"author_email": "admin@fedoraproject.org",
"download_url": "https://files.pythonhosted.org/packages/f6/84/45120fbc1a4fa015cfcc9816253227ee3b428a46e4f2244cecd5a49e1be7/fedbadges-2.2.0.tar.gz",
"platform": null,
"description": "Fedora Badges Message consumer\n==============================\n\nThis repo contains the consumer and the command necessary to hook the\nbadges stack (Tahrir, Tahrir-API, Tahrir-REST) into Fedora Messaging.\nIt is the process that runs in the background, monitoring activity of Fedora\ncontributors, and is responsible for awarding badges for activity as it happens.\nIt is separate from and sometimes confused with the *frontend* of the badges system\ncalled `tahrir <https://github.com/fedora-infra/tahrir>`_.\nThis project (fedbadges) writes to a database that the web frontend (tahrir) reads\nfrom.\n\nThe *actual badge rules* that we act on in Fedora Infrastructure can be\nfound `here <https://pagure.io/Fedora-Badges>`.\n\nArchitecture\n------------\n\nfedbadges is a callback class for the Fedora Messaging consumer.\nWhen started, it will load some initial configuration\nand a set of ``BadgeRules`` (more on that later) and then sit quietly\nlistening to the Fedora Messaging bus. Each rule (composed of some metadata,\na ``trigger``, an optional ``condition`` and an optional way of counting previous\nmessages) is defined on disk as a yaml file.\n\n* When a new message comes along, our callback looks to see if it matches\n any of the ``BadgeRules`` it has registered.\n\n* Each BadgeRule must define a ``trigger`` -- a *lightweight* check.\n When processing a message, this is the first thing that is checked. It\n defines a *pattern* that the message must match. If the message does not\n match, then the current BadgeRule is discarded and processing moves to\n the next.\n\n A ``trigger`` is typically something like \"any bodhi message\"\n or \"messages only from the failure of a koji build\". More on their\n specification below.\n\n* BadgeRules can also define a ``previous`` value, as a way to count similar\n messages that went through the bus in the past. This typically involves a\n more expensive query to the\n `datanommer <https://github.com/fedora-infra/datanommer>`_ database.\n\n A BadgeRule ``previous`` query may read something like \"updates pushed to\n stable by the candidate\" or \"IRC meetings chaired by the candidate\".\n\n **Aside:** Although datanommer is the only currently supported backend, we\n can implement other queryable backend in the future as needed like FAS\n (to see if the user is in X number of groups) or even off-site services\n like libravatar (to award a badge if the user is a user of the AGPL web\n service).\n\n* BadgeRule can define a ``condition`` that the number of messages returned by\n the ``previous`` query must match. This can be something like\n ``greater than or equal to: 50``. If unset, the default condition is\n ``greater than or equal to 1``.\n\n* If no ``previous`` query is set, then the rule only considers the current\n incoming message (it's like the ``previous`` result is always ``1``). This\n is relevant for rules that award a badge on the first action. Those rules\n don't need to set a ``condition`` either, the default one will do.\n\n* If a badge's ``trigger`` and ``condition`` both match, then the badge is\n awarded. If the BadgeRule doesn't specify, we award the badge to the\n author of the action using the message's ``agent_name`` property.\n\n That is usually correct -- but sometimes, a BadgeRule needs to specify\n that one particular user should be recipient of the badge.\n In this case, the BadgeRule may define a ``recipient``\n in dot-notation that instructs the ``Consumer`` how to extract the\n recipient's username from the received message.\n\n The badge is awarded to our deserving user via the `tahrir_api\n <https://github.com/fedora-infra/tahrir-api>`_. At the end of the day,\n this amounts to adding a row in a database table for the `Tahrir\n <https://github.com/fedora-infra/tahrir>`_ application.\n\nThere are some optimizations in place omitted above for clarity.\nFor instance, after the trigger has matched we first check if the user\nthat *would* be awarded the badge already has it. If they do, we stop\nprocessing the badge rule immediately to avoid making an unnecessary\nexpensive check against the datanommer db.\n\nConfiguration - Global\n----------------------\n\nfedbadges needs three major pieces of global configuration.\nAll configuration is loaded in the standard Fedora Messaging way, from\nthe ``[consumer_config]`` section of the configuration file. See\n`fedbadges.toml.example\n<https://github.com/fedora-infra/fedbadges/blob/develop/fedbadges.toml.example>`_\nin the git repo for an example.\n\nfedbadges also emits its own messages. In the Fedora Infrastructure, the\n``topic_prefix`` will be ``org.fedoraproject.prod``.\n\nConfiguration - BadgeRule specification\n---------------------------------------\n\nBadgeRules are specified in `YAML <http://www.yaml.org/>`_ on the file system.\n\nTriggers\n~~~~~~~~\n\nEvery BadgeRule must carry the following minimum set of metadata::\n\n # This is some metadata about the badge\n name: Like a Rock\n description: You have pushed 500 or more bodhi updates to stable status.\n creator: ralph\n\n # This is a link to the discussion about adopting this as a for-real badge.\n discussion: http://github.com/fedora-infra/badges/pull/SOME_NUMBER\n\n # A link to the image for the badge\n image_url: http://somelink.org/to-an-image.png\n\nHere's a simple example of a ``trigger``::\n\n trigger:\n category: bodhi\n\nThe above will match any bodhi message on any of the topics that come\nfrom the bodhi update system.\n\nTriggers may employ a little bit of logic to make more complex\nfilters. The following trigger will match any message that comes from\n*either* the bodhi update system or the fedora git package repos::\n\n trigger:\n category:\n any:\n - bodhi\n - git\n\nAt present triggers may directly compare themselves against only the\n`category` or the `topic` of a message. In the future we'd like to add\nmore comparisons.. in the meantime, here's an example of comparing against\nthe fully qualified message topic. This will match any message\nthat is specifically for editing a wiki page::\n\n trigger:\n topic: org.fedoraproject.prod.wiki.article.edit\n\n----\n\nThere is one additional way you can specify a trigger. If you need more\nflexibility than ``topic`` and\n``category`` allow, you may specify a custom filter expression with a\n``lambda`` filter. For example::\n\n trigger:\n lambda: \"a string of interest\" in json.dumps(message.body)\n\nThe above trigger will match if the string ``\"a string of interest\"`` appears\nanywhere in the incoming message. fedbadges takes the expression you provide\nit and compiles it into a python callable on initialization. Our callable\nhere serializes the message to a JSON string before doing its comparison.\nPowerful!\n\nPrevious\n~~~~~~~~\n\nAs mentioned above in the architecture section, we currently only support\ndatanommer as a queryable backend for ``previous`` queries. We hope to expand\nthat in the future.\n\nDatanommer queries are composed of two things:\n\n- A **filter** limits the scope of the query to datanommer.\n- An **operation** defines what we want to do with the filtered query.\n Currently, we can *count* the results or run them through a ``lambda``\n function that will return an integer (the number of matched messages).\n\nHere's an example of a simple previous definition::\n\n previous:\n filter:\n topics:\n - message.topic\n operation: count\n\nThe above ``previous`` query will return the number of messages in datanommer\nwith the same topic as the incoming message being handled. Here,\n``message.topic`` is a ``lambda function`` that has the incoming ``message``\nin scope.\n\n----\n\nThe above example doesn't make much sense -- we'd never use it for a real\nbadge. The ``previous`` query would be true if there were two of *any* message\nkicked off by *any* user at any time in the past. Pretty generic.\nHere's a more interesting ``previous`` query::\n\n previous:\n filter:\n topics:\n - org.fedoraproject.prod.git.receive\n users:\n - message.body[\"commit\"][\"username\"]\n operation: count\n\nThis ``previous`` query would return the number of messages of the topic\n``\"org.fedoraproject.prod.git.receive\"`` that were also kicked off by whatever\nuser is listed in the ``message.body['commit']['username']`` field of the\nmessage being currently processed. In other words, this query would return\nthe number of pushes to the fedora git repos by the user.\n\nCondition\n~~~~~~~~~\n\nYou can do some fancy things with the **condition** field.\nHere's a list of the possible comparisons you can make:\n\n- ``\"is greater than or equal to\"`` or alternatively\n ``\"greater than or equal to\"``\n- ``\"greater than\"``\n- ``\"is less than or equal to\"`` or alternatively\n ``\"less than or equal to\"``\n- ``\"less than\"``\n- ``\"equal to\"`` or alternatively ``\"is equal to\"``\n- ``\"is not\"`` or alternatively ``\"is not equal to\"``\n\nAs you can see, some of them are synonyms for each other.\n\n----\n\nIf any of those don't meet your needs, you can specify a custom expression\nby using the ``lambda`` condition whereby fedbadges will compile whatever\nstatement you provide into a callable and use that at runtime. For example::\n\n\n condition:\n lambda: value != 0 and ((value & (value - 1)) == 0)\n\nWho knows why you would want to do this, but the above condition check will\nsucceed if the number of messages that matched in the past is exactly a power\nof 2.\n\nSpecifying Recipients\n~~~~~~~~~~~~~~~~~~~~~\n\nBy default, if the trigger and condition match, fedbadges will award badges\nto the user returned by the message's ``agent_name`` property.\nThis *usually* corresponds with \"which user is responsible\" for this message.\nThat is *usually* what we want to award badges for.\n\nThere are some instances for which that is not what we want.\n\nTake the `org.fedoraproject.prod.bodhi.update.comment\n<https://fedora-messaging.readthedocs.io/en/stable/user-guide/schemas.html#bodhi>`_\nmessage for example. When user A comments on user B's update, user A is returned\nby the message's ``agent_name`` property.\n\nImagine we have a \"Received Comments\" badge that's awarded to packagers that\nreceived comments on their updates. We don't want to inadvertently award that\nbadge to the person who *commented*, only to the one who *created the update*.\n\nTo allow for this scenario, badges may optionally define a ``recipient``\nin dotted notation that tells fedbadges where to find the username of the\nrecipient in the originating message. For instance, the following would\nhandle the fas case we described above::\n\n trigger:\n topic: org.fedoraproject.prod.bodhi.update.comment\n condition:\n greater than or equal to: 1\n previous:\n filter:\n topics:\n - message.topic\n users:\n - recipient\n operation: count\n recipient: message.body[\"update\"][\"user\"][\"name\"]\n\n",
"bugtrack_url": null,
"license": "GPL-2.0-or-later",
"summary": "Fedora Messaging consumer for awarding open badges",
"version": "2.2.0",
"project_urls": {
"Homepage": "https://github.com/fedora-infra/fedbadges",
"Repository": "https://github.com/fedora-infra/fedbadges"
},
"split_keywords": [
"fedora"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2fba522928051bd83c661ac436413ff9610f777069c1ba7eb5b4706988a3f0fa",
"md5": "881620ba8b5bcd43da148e732f59dc12",
"sha256": "6fad363791e56fcc03f4f32cb2ac498b070f12d3c9a8ddeae1b6454fe4ec5e48"
},
"downloads": -1,
"filename": "fedbadges-2.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "881620ba8b5bcd43da148e732f59dc12",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 33078,
"upload_time": "2024-07-04T08:54:22",
"upload_time_iso_8601": "2024-07-04T08:54:22.362149Z",
"url": "https://files.pythonhosted.org/packages/2f/ba/522928051bd83c661ac436413ff9610f777069c1ba7eb5b4706988a3f0fa/fedbadges-2.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f68445120fbc1a4fa015cfcc9816253227ee3b428a46e4f2244cecd5a49e1be7",
"md5": "58e9da474a286285948a281d675ec63d",
"sha256": "b966502b8d62ee1a9ab00f2b905c9ffb330cf6227ea44cf791ee17a2327524ef"
},
"downloads": -1,
"filename": "fedbadges-2.2.0.tar.gz",
"has_sig": false,
"md5_digest": "58e9da474a286285948a281d675ec63d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 41695,
"upload_time": "2024-07-04T08:54:24",
"upload_time_iso_8601": "2024-07-04T08:54:24.175708Z",
"url": "https://files.pythonhosted.org/packages/f6/84/45120fbc1a4fa015cfcc9816253227ee3b428a46e4f2244cecd5a49e1be7/fedbadges-2.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-04 08:54:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fedora-infra",
"github_project": "fedbadges",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "fedbadges"
}
JSON
