gha-utils


Namegha-utils JSON
Version 4.13.0 PyPI version JSON
download
home_pageNone
Summary⚙️ CLI helpers for GitHub Actions + reuseable workflows
upload_time2025-01-21 07:20:30
maintainerNone
docs_urlNone
authorNone
requires_python>=3.10
licenseNone
keywords build-automation changelog-formatter ci-cd cli formatting github-actions labels linting markdown mypy nuitka packaging pypi python release-automation sphinx sponsorship terminal typo workflow-reusable yaml
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # `gha-utils` CLI + reusable workflows

[![Last release](https://img.shields.io/pypi/v/gha-utils.svg)](https://pypi.python.org/pypi/gha-utils)
[![Python versions](https://img.shields.io/pypi/pyversions/gha-utils.svg)](https://pypi.python.org/pypi/gha-utils)
[![Type checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
[![Unittests status](https://github.com/kdeldycke/workflows/actions/workflows/tests.yaml/badge.svg?branch=main)](https://github.com/kdeldycke/workflows/actions/workflows/tests.yaml?query=branch%3Amain)
[![Coverage status](https://codecov.io/gh/kdeldycke/workflows/branch/main/graph/badge.svg)](https://app.codecov.io/gh/kdeldycke/workflows)

Thanks to this project, I am able to **release Python packages multiple times a day with only 2-clicks**.

This repository contains a collection of reusable workflows and its companion CLI called `gha-utils` (which stands for *GitHub action workflows utilities*).

It is designed for `uv`-based Python projects (and Awesome List projects as a bonus).

It takes care of:

- Version bumping
- Formatting autofix for: Python, Markdown, JSON, typos
- Linting: Python types with `mypy`, YAML, `zsh`, GitHub actions, links, Awesome lists, secrets
- Compiling of Python binaries for Linux / macOS / Windows on `x86_64` & `arm64`
- Building of Python packages and upload to PyPi
- Git version tagging and GitHub release creation
- Synchronization of: `uv.lock`, `.gitignore`, `.mailmap` and Mermaid dependency graph
- Auto-locking of inactive closed issues
- Static image optimization
- Sphinx documentation building & deployment, and `autodoc` updates
- Label management, with file-based and content-based rules

Nothing is done behind your back. A PR is created every time a change is proposed, so you can inspect it, ala dependabot.

## `gha-utils` CLI

### Ad-hoc execution

Thanks to `uv`, you can install and run `gha-utils` in one command, without polluting your system:

```shell-session
$ uvx gha-utils
Installed 45 packages in 45ms
Usage: gha-utils [OPTIONS] COMMAND [ARGS]...

Options:
  --time / --no-time        Measure and print elapsed execution time.  [default:
                            no-time]
  --color, --ansi / --no-color, --no-ansi
                            Strip out all colors and all ANSI codes from output.
                            [default: color]
  -C, --config CONFIG_PATH  Location of the configuration file. Supports glob
                            pattern of local path and remote URL.  [default:
                            ~/Library/Application Support/gha-
                            utils/*.{toml,yaml,yml,json,ini,xml}]
  --show-params             Show all CLI parameters, their provenance, defaults
                            and value, then exit.
  -v, --verbosity LEVEL     Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
                            [default: WARNING]
  --version                 Show the version and exit.
  -h, --help                Show this message and exit.

Commands:
  changelog     Maintain a Markdown-formatted changelog
  mailmap-sync  Update Git's .mailmap file with missing contributors
  metadata      Output project metadata
```

```shell-session
$ uvx gha-utils --version
gha-utils, version 4.9.0
```

That's the best way to get started with `gha-utils`, and experiment with its features.

### Executables

To ease deployment, standalone executables of `gha-utils`'s latest version are available as direct downloads for several platforms and architectures:

| Platform    | `x86_64`                                                                                                                          | `arm64`                                                                                                                           |
| ----------- | --------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Linux**   | [Download `gha-utils-linux-x64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-linux-x64.bin)     | [Download `gha-utils-linux-arm64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-linux-arm64.bin) |
| **macOS**   | [Download `gha-utils-macos-x64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-macos-x64.bin)     | [Download `gha-utils-macos-arm64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-macos-arm64.bin) |
| **Windows** | [Download `gha-utils-windows-x64.exe`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-windows-x64.exe) |                                                                                                                                   |

### Development version

To play with the latest development version of `gha-utils`, you can install it directly from the repository:

```shell-session
$ git clone https://github.com/kdeldycke/workflows
$ cd workflows
$ python -m pip install uv
$ uv venv
$ source .venv/bin/activate
$ uv sync
$ uv run -- gha-utils
```

## Reusable workflows collection

This repository contains workflows to automate most of the boring tasks.

These workflows are mostly used for Python projects and their documentation, but not only. They're all [reusable GitHub actions workflows](https://docs.github.com/en/actions/learn-github-actions/reusing-workflows).

Reasons for a centralized workflow repository:

- reusability of course: no need to update dozens of repository where 95% of workflows are the same
- centralize all dependencies pertaining to automation: think of the point-release of an action that triggers dependabot upgrade to all your repositories depending on it

### Guidelines

I don't want to copy-n-past, keep in sync and maintain another `N`th CI/CD file at the root of my repositories.

So my policy is: move every repository-specific config in a `pyproject.toml` file, or hide the gory details in a reused workflow.

### `.github/workflows/docs.yaml` jobs

- Autofix typos

- Optimize images

- Keep `.mailmap` up to date

- Update dependency graph of Python projects

  - **Requires**:
    - Python package with a `pyproject.toml` file

- Build Sphinx-based documentation and publish it to GitHub Pages

  - **Requires**:
    - Python package with a `pyproject.toml` file
    - All Sphinx dependencies in a `docs` [extra dependency group](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements):
      ```toml
      [project.optional-dependencies]
      docs = [
          "furo == 2024.1.29",
          "myst-parser ~= 3.0.0",
          "sphinx >= 6",
          ...
      ]
      ```
    - Sphinx configuration file at `docs/conf.py`

- Sync awesome projects from `awesome-template` repository

### Why all these `requirements/*.txt` files?

Let's look for example at the `lint-yaml` job from [`.github/workflows/lint.yaml`](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/lint.yaml#L126). Here we only need the `yamllint` CLI. This CLI is [distributed on PyPi](https://pypi.org/project/yamllint/). So before executing it, we could have simply run the following step:

```yaml
  - name: Install yamllint
    run: |
      pip install yamllint
```

Instead, we install it via the [`requirements/yamllint.txt` file](https://github.com/kdeldycke/workflows/blob/main/requirements/yamllint.txt).

Why? Because I want the version of `yamllint` to be pinned. By pinning it, I make the workflow stable, predictable and reproducible.

So why use a dedicated requirements file? Why don't we simply add the version? Like this:

```yaml
  - name: Install yamllint
    run: |
      pip install yamllint==1.35.1
```

That would indeed pin the version. But it requires the maintainer (me) to keep track of new release and update manually the version string. That's a lot of work. And I'm lazy. So this should be automated.

To automate that, the only practical way I found was to rely on dependabot. But dependabot cannot update arbitrary versions in `run:` YAML blocks. It [only supports `requirements.txt` and `pyproject.toml`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#pip-and-pip-compile) files for Python projects.

So to keep track of new versions of dependencies while keeping them stable, we've hard-coded all Python libraries and CLIs in the `requirements/*.txt` files. All with pinned versions.

And for the case we need to install all dependencies in one go, we have a [`requirements.txt` file at the root](https://github.com/kdeldycke/workflows/blob/main/requirements.txt) that is referencing all files from the `requirements/` subfolder.

### Permissions and token

This repository updates itself via GitHub actions. It particularly updates its own YAML files in `.github/workflows`. That's forbidden by default. So we need extra permissions.

Usually, to grant special permissions to some jobs, you use the [`permissions` parameter in workflow](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions) files. It looks like this:

```yaml
on: (...)

jobs:

  my-job:
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write

    steps: (...)
```

But the `contents: write` permission doesn't allow write access to the workflow files in the `.github` subfolder. There is `actions: write`, but it only covers workflow runs, not their YAML source file. Even a `permissions: write-all` doesn't work. So you cannot use the `permissions` parameter to allow a repository's workflow update its own workflow files.

You will always end up with this kind or errors:

```text
   ! [remote rejected] branch_xxx -> branch_xxx (refusing to allow a GitHub App to create or update workflow `.github/workflows/my_workflow.yaml` without `workflows` permission)

  error: failed to push some refs to 'https://github.com/kdeldycke/my-repo'
```

> [!NOTE]
> That's also why the Settings > Actions > General > Workflow permissions parameter on your repository has no effect on this issue, even with the `Read and write permissions` set:
> ![](docs/assets/repo-workflow-permissions.png)

To bypass the limitation, we rely on a custom access token. By convention, we call it `WORKFLOW_UPDATE_GITHUB_PAT`. It will be used, [in place of the default `secrets.GITHUB_TOKEN`](https://github.com/search?q=repo%3Akdeldycke%2Fworkflows%20WORKFLOW_UPDATE_GITHUB_PAT&type=code), in steps in which we need to change the workflow YAML files.

To create this custom `WORKFLOW_UPDATE_GITHUB_PAT`:

- From your GitHub user, go to `Settings` > `Developer Settings` > `Personal Access Tokens` > `Fine-grained tokens`
- Click on the `Generate new token` button
- Choose a good token name like `workflow-self-update` to make your intention clear
- Choose `Only select repositories` and the list the repositories in needs of updating their workflow YAML files
- In the `Repository permissions` drop-down, sets:
  - `Contents`: `Access: **Read and Write**`
  - `Metadata` (mandatory): `Access: **Read-only**`
  - `Pull Requests`: `Access: **Read and Write**`
  - `Workflows`: `Access: **Read and Write**`
    > [!NOTE]
    > This is the only place where I can have control over the `Workflows` permission, which is not supported by the `permissions:` parameter in YAML files.
- Now save these parameters and copy the `github_pat_XXXX` secret token
- Got to your repo > `Settings` > `Security` > `Secrets and variables` > `Actions` > `Secrets` > `Repository secrets` and click `New repository secrets`
- Name your secret `WORKFLOW_UPDATE_GITHUB_PAT` and copy the `github_pat_XXXX` token in the `Secret` field

Now re-run your actions and they should be able to update the workflow files in `.github` folder without the `refusing to allow a GitHub App to create or update workflow` error.

### Release management

It turns out [Release Engineering is a full-time job, and full of edge-cases](https://blog.axo.dev/2023/02/cargo-dist).

Things have improved a lot in the Python ecosystem with `uv`. But there are still a lot of manual steps to do to release.

So I made up this [`release.yaml` workflow](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/release.yaml), which:

1. Extracts project metadata from `pyproject.toml`
1. Generates a build matrix of all commits / os / arch / CLI entry points
1. Builds Python wheels with `uv`
1. Compiles binaries of all CLI with Nuitka
1. Tag the release commit in Git
1. Produces attestations of released artefacts
1. Publish new version to PyPi
1. Publish a GitHub release
1. Attach and rename build artifacts to the GitHub release

## Changelog

A [detailed changelog](changelog.md) is available.

## Used in

Check these projects to get real-life examples of usage and inspiration:

- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-falsehood?label=%E2%AD%90&style=flat-square) [Awesome Falsehood](https://github.com/kdeldycke/awesome-falsehood#readme) - Falsehoods Programmers Believe in.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-engineering-team-management?label=%E2%AD%90&style=flat-square) [Awesome Engineering Team Management](https://github.com/kdeldycke/awesome-engineering-team-management#readme) - How to transition from software development to engineering management.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-iam?label=%E2%AD%90&style=flat-square) [Awesome IAM](https://github.com/kdeldycke/awesome-iam#readme) - Identity and Access Management knowledge for cloud platforms.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-billing?label=%E2%AD%90&style=flat-square) [Awesome Billing](https://github.com/kdeldycke/awesome-billing#readme) - Billing & Payments knowledge for cloud platforms.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/meta-package-manager?label=%E2%AD%90&style=flat-square) [Meta Package Manager](https://github.com/kdeldycke/meta-package-manager#readme) - A unifying CLI for multiple package managers.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/mail-deduplicate?label=%E2%AD%90&style=flat-square) [Mail Deduplicate](https://github.com/kdeldycke/mail-deduplicate#readme) - A CLI to deduplicate similar emails.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/dotfiles?label=%E2%AD%90&style=flat-square) [dotfiles](https://github.com/kdeldycke/dotfiles#readme) - macOS dotfiles for Python developers.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/click-extra?label=%E2%AD%90&style=flat-square) [Click Extra](https://github.com/kdeldycke/click-extra#readme) - Extra colorization and configuration loading for Click.
- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/wikibot?label=%E2%AD%90&style=flat-square) [Wiki bot](https://github.com/themagicalmammal/wikibot#readme) - A bot which provides features from Wikipedia like summary, title searches, location API etc.
- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/workflows?label=%E2%AD%90&style=flat-square) [workflows](https://github.com/kdeldycke/workflows#readme) - Itself. Eat your own dog-food.
- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/stock-analyser?label=%E2%AD%90&style=flat-square) [Stock Analysis](https://github.com/themagicalmammal/stock-analyser#readme) - Simple to use interfaces for basic technical analysis of stocks.
- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/genetictabler?label=%E2%AD%90&style=flat-square) [GeneticTabler](https://github.com/themagicalmammal/genetictabler#readme) - Time Table Scheduler using Genetic Algorithms.
- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/excel-write?label=%E2%AD%90&style=flat-square) [Excel Write](https://github.com/themagicalmammal/excel-write#readme) - Optimised way to write in excel files.

Feel free to send a PR to add your project in this list if you are relying on these scripts.

## Release process

All steps of the release process and version management are automated in the
[`changelog.yaml`](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/changelog.yaml)
and
[`release.yaml`](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/release.yaml)
workflows.

All there's left to do is to:

- [check the open draft `prepare-release` PR](https://github.com/kdeldycke/workflows/pulls?q=is%3Apr+is%3Aopen+head%3Aprepare-release)
  and its changes,
- click the `Ready for review` button,
- click the `Rebase and merge` button,
- let the workflows tag the release and set back the `main` branch into a
  development state.

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "gha-utils",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "build-automation, changelog-formatter, ci-cd, cli, formatting, github-actions, labels, linting, markdown, mypy, nuitka, packaging, pypi, python, release-automation, sphinx, sponsorship, terminal, typo, workflow-reusable, yaml",
    "author": null,
    "author_email": "Kevin Deldycke <kevin@deldycke.com>",
    "download_url": "https://files.pythonhosted.org/packages/91/53/763f5b4d4c978db30674be4c655f3753373982aee66a1214bb7a9004af29/gha_utils-4.13.0.tar.gz",
    "platform": null,
    "description": "# `gha-utils` CLI + reusable workflows\n\n[![Last release](https://img.shields.io/pypi/v/gha-utils.svg)](https://pypi.python.org/pypi/gha-utils)\n[![Python versions](https://img.shields.io/pypi/pyversions/gha-utils.svg)](https://pypi.python.org/pypi/gha-utils)\n[![Type checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)\n[![Unittests status](https://github.com/kdeldycke/workflows/actions/workflows/tests.yaml/badge.svg?branch=main)](https://github.com/kdeldycke/workflows/actions/workflows/tests.yaml?query=branch%3Amain)\n[![Coverage status](https://codecov.io/gh/kdeldycke/workflows/branch/main/graph/badge.svg)](https://app.codecov.io/gh/kdeldycke/workflows)\n\nThanks to this project, I am able to **release Python packages multiple times a day with only 2-clicks**.\n\nThis repository contains a collection of reusable workflows and its companion CLI called `gha-utils` (which stands for *GitHub action workflows utilities*).\n\nIt is designed for `uv`-based Python projects (and Awesome List projects as a bonus).\n\nIt takes care of:\n\n- Version bumping\n- Formatting autofix for: Python, Markdown, JSON, typos\n- Linting: Python types with `mypy`, YAML, `zsh`, GitHub actions, links, Awesome lists, secrets\n- Compiling of Python binaries for Linux / macOS / Windows on `x86_64` & `arm64`\n- Building of Python packages and upload to PyPi\n- Git version tagging and GitHub release creation\n- Synchronization of: `uv.lock`, `.gitignore`, `.mailmap` and Mermaid dependency graph\n- Auto-locking of inactive closed issues\n- Static image optimization\n- Sphinx documentation building & deployment, and `autodoc` updates\n- Label management, with file-based and content-based rules\n\nNothing is done behind your back. A PR is created every time a change is proposed, so you can inspect it, ala dependabot.\n\n## `gha-utils` CLI\n\n### Ad-hoc execution\n\nThanks to `uv`, you can install and run `gha-utils` in one command, without polluting your system:\n\n```shell-session\n$ uvx gha-utils\nInstalled 45 packages in 45ms\nUsage: gha-utils [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n  --time / --no-time        Measure and print elapsed execution time.  [default:\n                            no-time]\n  --color, --ansi / --no-color, --no-ansi\n                            Strip out all colors and all ANSI codes from output.\n                            [default: color]\n  -C, --config CONFIG_PATH  Location of the configuration file. Supports glob\n                            pattern of local path and remote URL.  [default:\n                            ~/Library/Application Support/gha-\n                            utils/*.{toml,yaml,yml,json,ini,xml}]\n  --show-params             Show all CLI parameters, their provenance, defaults\n                            and value, then exit.\n  -v, --verbosity LEVEL     Either CRITICAL, ERROR, WARNING, INFO, DEBUG.\n                            [default: WARNING]\n  --version                 Show the version and exit.\n  -h, --help                Show this message and exit.\n\nCommands:\n  changelog     Maintain a Markdown-formatted changelog\n  mailmap-sync  Update Git's .mailmap file with missing contributors\n  metadata      Output project metadata\n```\n\n```shell-session\n$ uvx gha-utils --version\ngha-utils, version 4.9.0\n```\n\nThat's the best way to get started with `gha-utils`, and experiment with its features.\n\n### Executables\n\nTo ease deployment, standalone executables of `gha-utils`'s latest version are available as direct downloads for several platforms and architectures:\n\n| Platform    | `x86_64`                                                                                                                          | `arm64`                                                                                                                           |\n| ----------- | --------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |\n| **Linux**   | [Download `gha-utils-linux-x64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-linux-x64.bin)     | [Download `gha-utils-linux-arm64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-linux-arm64.bin) |\n| **macOS**   | [Download `gha-utils-macos-x64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-macos-x64.bin)     | [Download `gha-utils-macos-arm64.bin`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-macos-arm64.bin) |\n| **Windows** | [Download `gha-utils-windows-x64.exe`](https://github.com/kdeldycke/workflows/releases/latest/download/gha-utils-windows-x64.exe) |                                                                                                                                   |\n\n### Development version\n\nTo play with the latest development version of `gha-utils`, you can install it directly from the repository:\n\n```shell-session\n$ git clone https://github.com/kdeldycke/workflows\n$ cd workflows\n$ python -m pip install uv\n$ uv venv\n$ source .venv/bin/activate\n$ uv sync\n$ uv run -- gha-utils\n```\n\n## Reusable workflows collection\n\nThis repository contains workflows to automate most of the boring tasks.\n\nThese workflows are mostly used for Python projects and their documentation, but not only. They're all [reusable GitHub actions workflows](https://docs.github.com/en/actions/learn-github-actions/reusing-workflows).\n\nReasons for a centralized workflow repository:\n\n- reusability of course: no need to update dozens of repository where 95% of workflows are the same\n- centralize all dependencies pertaining to automation: think of the point-release of an action that triggers dependabot upgrade to all your repositories depending on it\n\n### Guidelines\n\nI don't want to copy-n-past, keep in sync and maintain another `N`th CI/CD file at the root of my repositories.\n\nSo my policy is: move every repository-specific config in a `pyproject.toml` file, or hide the gory details in a reused workflow.\n\n### `.github/workflows/docs.yaml` jobs\n\n- Autofix typos\n\n- Optimize images\n\n- Keep `.mailmap` up to date\n\n- Update dependency graph of Python projects\n\n  - **Requires**:\n    - Python package with a `pyproject.toml` file\n\n- Build Sphinx-based documentation and publish it to GitHub Pages\n\n  - **Requires**:\n    - Python package with a `pyproject.toml` file\n    - All Sphinx dependencies in a `docs` [extra dependency group](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements):\n      ```toml\n      [project.optional-dependencies]\n      docs = [\n          \"furo == 2024.1.29\",\n          \"myst-parser ~= 3.0.0\",\n          \"sphinx >= 6\",\n          ...\n      ]\n      ```\n    - Sphinx configuration file at `docs/conf.py`\n\n- Sync awesome projects from `awesome-template` repository\n\n### Why all these `requirements/*.txt` files?\n\nLet's look for example at the `lint-yaml` job from [`.github/workflows/lint.yaml`](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/lint.yaml#L126). Here we only need the `yamllint` CLI. This CLI is [distributed on PyPi](https://pypi.org/project/yamllint/). So before executing it, we could have simply run the following step:\n\n```yaml\n  - name: Install yamllint\n    run: |\n      pip install yamllint\n```\n\nInstead, we install it via the [`requirements/yamllint.txt` file](https://github.com/kdeldycke/workflows/blob/main/requirements/yamllint.txt).\n\nWhy? Because I want the version of `yamllint` to be pinned. By pinning it, I make the workflow stable, predictable and reproducible.\n\nSo why use a dedicated requirements file? Why don't we simply add the version? Like this:\n\n```yaml\n  - name: Install yamllint\n    run: |\n      pip install yamllint==1.35.1\n```\n\nThat would indeed pin the version. But it requires the maintainer (me) to keep track of new release and update manually the version string. That's a lot of work. And I'm lazy. So this should be automated.\n\nTo automate that, the only practical way I found was to rely on dependabot. But dependabot cannot update arbitrary versions in `run:` YAML blocks. It [only supports `requirements.txt` and `pyproject.toml`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#pip-and-pip-compile) files for Python projects.\n\nSo to keep track of new versions of dependencies while keeping them stable, we've hard-coded all Python libraries and CLIs in the `requirements/*.txt` files. All with pinned versions.\n\nAnd for the case we need to install all dependencies in one go, we have a [`requirements.txt` file at the root](https://github.com/kdeldycke/workflows/blob/main/requirements.txt) that is referencing all files from the `requirements/` subfolder.\n\n### Permissions and token\n\nThis repository updates itself via GitHub actions. It particularly updates its own YAML files in `.github/workflows`. That's forbidden by default. So we need extra permissions.\n\nUsually, to grant special permissions to some jobs, you use the [`permissions` parameter in workflow](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions) files. It looks like this:\n\n```yaml\non: (...)\n\njobs:\n\n  my-job:\n    runs-on: ubuntu-latest\n    permissions:\n      contents: write\n      pull-requests: write\n\n    steps: (...)\n```\n\nBut the `contents: write` permission doesn't allow write access to the workflow files in the `.github` subfolder. There is `actions: write`, but it only covers workflow runs, not their YAML source file. Even a `permissions: write-all` doesn't work. So you cannot use the `permissions` parameter to allow a repository's workflow update its own workflow files.\n\nYou will always end up with this kind or errors:\n\n```text\n   ! [remote rejected] branch_xxx -> branch_xxx (refusing to allow a GitHub App to create or update workflow `.github/workflows/my_workflow.yaml` without `workflows` permission)\n\n  error: failed to push some refs to 'https://github.com/kdeldycke/my-repo'\n```\n\n> [!NOTE]\n> That's also why the Settings > Actions > General > Workflow permissions parameter on your repository has no effect on this issue, even with the `Read and write permissions` set:\n> ![](docs/assets/repo-workflow-permissions.png)\n\nTo bypass the limitation, we rely on a custom access token. By convention, we call it `WORKFLOW_UPDATE_GITHUB_PAT`. It will be used, [in place of the default `secrets.GITHUB_TOKEN`](https://github.com/search?q=repo%3Akdeldycke%2Fworkflows%20WORKFLOW_UPDATE_GITHUB_PAT&type=code), in steps in which we need to change the workflow YAML files.\n\nTo create this custom `WORKFLOW_UPDATE_GITHUB_PAT`:\n\n- From your GitHub user, go to `Settings` > `Developer Settings` > `Personal Access Tokens` > `Fine-grained tokens`\n- Click on the `Generate new token` button\n- Choose a good token name like `workflow-self-update` to make your intention clear\n- Choose `Only select repositories` and the list the repositories in needs of updating their workflow YAML files\n- In the `Repository permissions` drop-down, sets:\n  - `Contents`: `Access: **Read and Write**`\n  - `Metadata` (mandatory): `Access: **Read-only**`\n  - `Pull Requests`: `Access: **Read and Write**`\n  - `Workflows`: `Access: **Read and Write**`\n    > [!NOTE]\n    > This is the only place where I can have control over the `Workflows` permission, which is not supported by the `permissions:` parameter in YAML files.\n- Now save these parameters and copy the `github_pat_XXXX` secret token\n- Got to your repo > `Settings` > `Security` > `Secrets and variables` > `Actions` > `Secrets` > `Repository secrets` and click `New repository secrets`\n- Name your secret `WORKFLOW_UPDATE_GITHUB_PAT` and copy the `github_pat_XXXX` token in the `Secret` field\n\nNow re-run your actions and they should be able to update the workflow files in `.github` folder without the `refusing to allow a GitHub App to create or update workflow` error.\n\n### Release management\n\nIt turns out [Release Engineering is a full-time job, and full of edge-cases](https://blog.axo.dev/2023/02/cargo-dist).\n\nThings have improved a lot in the Python ecosystem with `uv`. But there are still a lot of manual steps to do to release.\n\nSo I made up this [`release.yaml` workflow](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/release.yaml), which:\n\n1. Extracts project metadata from `pyproject.toml`\n1. Generates a build matrix of all commits / os / arch / CLI entry points\n1. Builds Python wheels with `uv`\n1. Compiles binaries of all CLI with Nuitka\n1. Tag the release commit in Git\n1. Produces attestations of released artefacts\n1. Publish new version to PyPi\n1. Publish a GitHub release\n1. Attach and rename build artifacts to the GitHub release\n\n## Changelog\n\nA [detailed changelog](changelog.md) is available.\n\n## Used in\n\nCheck these projects to get real-life examples of usage and inspiration:\n\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-falsehood?label=%E2%AD%90&style=flat-square) [Awesome Falsehood](https://github.com/kdeldycke/awesome-falsehood#readme) - Falsehoods Programmers Believe in.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-engineering-team-management?label=%E2%AD%90&style=flat-square) [Awesome Engineering Team Management](https://github.com/kdeldycke/awesome-engineering-team-management#readme) - How to transition from software development to engineering management.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-iam?label=%E2%AD%90&style=flat-square) [Awesome IAM](https://github.com/kdeldycke/awesome-iam#readme) - Identity and Access Management knowledge for cloud platforms.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/awesome-billing?label=%E2%AD%90&style=flat-square) [Awesome Billing](https://github.com/kdeldycke/awesome-billing#readme) - Billing & Payments knowledge for cloud platforms.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/meta-package-manager?label=%E2%AD%90&style=flat-square) [Meta Package Manager](https://github.com/kdeldycke/meta-package-manager#readme) - A unifying CLI for multiple package managers.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/mail-deduplicate?label=%E2%AD%90&style=flat-square) [Mail Deduplicate](https://github.com/kdeldycke/mail-deduplicate#readme) - A CLI to deduplicate similar emails.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/dotfiles?label=%E2%AD%90&style=flat-square) [dotfiles](https://github.com/kdeldycke/dotfiles#readme) - macOS dotfiles for Python developers.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/click-extra?label=%E2%AD%90&style=flat-square) [Click Extra](https://github.com/kdeldycke/click-extra#readme) - Extra colorization and configuration loading for Click.\n- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/wikibot?label=%E2%AD%90&style=flat-square) [Wiki bot](https://github.com/themagicalmammal/wikibot#readme) - A bot which provides features from Wikipedia like summary, title searches, location API etc.\n- ![GitHub stars](https://img.shields.io/github/stars/kdeldycke/workflows?label=%E2%AD%90&style=flat-square) [workflows](https://github.com/kdeldycke/workflows#readme) - Itself. Eat your own dog-food.\n- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/stock-analyser?label=%E2%AD%90&style=flat-square) [Stock Analysis](https://github.com/themagicalmammal/stock-analyser#readme) - Simple to use interfaces for basic technical analysis of stocks.\n- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/genetictabler?label=%E2%AD%90&style=flat-square) [GeneticTabler](https://github.com/themagicalmammal/genetictabler#readme) - Time Table Scheduler using Genetic Algorithms.\n- ![GitHub stars](https://img.shields.io/github/stars/themagicalmammal/excel-write?label=%E2%AD%90&style=flat-square) [Excel Write](https://github.com/themagicalmammal/excel-write#readme) - Optimised way to write in excel files.\n\nFeel free to send a PR to add your project in this list if you are relying on these scripts.\n\n## Release process\n\nAll steps of the release process and version management are automated in the\n[`changelog.yaml`](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/changelog.yaml)\nand\n[`release.yaml`](https://github.com/kdeldycke/workflows/blob/main/.github/workflows/release.yaml)\nworkflows.\n\nAll there's left to do is to:\n\n- [check the open draft `prepare-release` PR](https://github.com/kdeldycke/workflows/pulls?q=is%3Apr+is%3Aopen+head%3Aprepare-release)\n  and its changes,\n- click the `Ready for review` button,\n- click the `Rebase and merge` button,\n- let the workflows tag the release and set back the `main` branch into a\n  development state.\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "\u2699\ufe0f CLI helpers for GitHub Actions + reuseable workflows",
    "version": "4.13.0",
    "project_urls": {
        "Changelog": "https://github.com/kdeldycke/workflows/blob/main/changelog.md",
        "Funding": "https://github.com/sponsors/kdeldycke",
        "Homepage": "https://github.com/kdeldycke/workflows",
        "Issues": "https://github.com/kdeldycke/workflows/issues",
        "Repository": "https://github.com/kdeldycke/workflows"
    },
    "split_keywords": [
        "build-automation",
        " changelog-formatter",
        " ci-cd",
        " cli",
        " formatting",
        " github-actions",
        " labels",
        " linting",
        " markdown",
        " mypy",
        " nuitka",
        " packaging",
        " pypi",
        " python",
        " release-automation",
        " sphinx",
        " sponsorship",
        " terminal",
        " typo",
        " workflow-reusable",
        " yaml"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "5b1a4e135914b498e86e2bcb3b3c72c48d2b868dfc945c63e3267e7bfc70697f",
                "md5": "3fe8b8586d621a39b764f677cdcbb951",
                "sha256": "245b829960d322e08b697e137dfab394b1cbee16388842718e3d88a922372eb1"
            },
            "downloads": -1,
            "filename": "gha_utils-4.13.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "3fe8b8586d621a39b764f677cdcbb951",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 30474,
            "upload_time": "2025-01-21T07:20:29",
            "upload_time_iso_8601": "2025-01-21T07:20:29.082696Z",
            "url": "https://files.pythonhosted.org/packages/5b/1a/4e135914b498e86e2bcb3b3c72c48d2b868dfc945c63e3267e7bfc70697f/gha_utils-4.13.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "9153763f5b4d4c978db30674be4c655f3753373982aee66a1214bb7a9004af29",
                "md5": "48f65bd7d803dd76155f01a3e7154189",
                "sha256": "77dbd918dd05fd999b8b4ff118d04655722e4fc0e030a3a6fbac8d5bb2c14780"
            },
            "downloads": -1,
            "filename": "gha_utils-4.13.0.tar.gz",
            "has_sig": false,
            "md5_digest": "48f65bd7d803dd76155f01a3e7154189",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 39683,
            "upload_time": "2025-01-21T07:20:30",
            "upload_time_iso_8601": "2025-01-21T07:20:30.511639Z",
            "url": "https://files.pythonhosted.org/packages/91/53/763f5b4d4c978db30674be4c655f3753373982aee66a1214bb7a9004af29/gha_utils-4.13.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-01-21 07:20:30",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "kdeldycke",
    "github_project": "workflows",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "requirements": [],
    "lcname": "gha-utils"
}
        
Elapsed time: 0.44491s