charmed-analytics-ci


Namecharmed-analytics-ci JSON
Version 1.0.0 PyPI version JSON
download
home_pageNone
SummaryA CLI tool for automating CI tasks across Charmed Operator repositories
upload_time2025-07-18 11:54:52
maintainerNone
docs_urlNone
authorCharmed Kubeflow
requires_python<4.0,>=3.12
licenseApache-2.0
keywords charm ci rock automation github charmed-kubeflow
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # charmed-analytics-ci

A CLI tool to automate CI-driven integration of updated **rock images** into consumer **Charmed Operator** repositories.

This tool is part of Canonical's Charmed Kubeflow stack and enables automated pull request creation after a rock image is built and published. It eliminates manual effort, reduces human error, and supports scalable, reproducible CI/CD pipelines.

---

## โœจ Features

- โœ… Automatically clones target charm repositories
- ๐Ÿ” Updates image references in YAML or JSON configuration files
- โš™๏ธ Optionally modifies service-spec fields like `user` and `command`
- ๐Ÿ”ง Validates metadata schemas for correctness before modification
- ๐Ÿค– Opens pull requests with deterministic branches and templated descriptions
- ๐Ÿ” Supports GitHub authentication via token or environment variable
- ๐Ÿ”— Optionally links back to triggering PR
- ๐Ÿ“ฆ Installable via PyPI and usable from CI pipelines
- ๐Ÿงช Supports dry-run mode for previewing changes

---

## ๐Ÿš€ Installation

Install from PyPI:

```bash
pip install charmed-analytics-ci
```

Or install for development:

```bash
git clone https://github.com/canonical/charmed-analytics-ci.git
cd charmed-analytics-ci
poetry install
```

---

## ๐Ÿงช CLI Usage

After installing, the CLI provides a single command:

```bash
chaci integrate-rock METADATA_FILE BASE_BRANCH ROCK_IMAGE [OPTIONS]
```

### Example:

```bash
export GH_TOKEN="ghp_abc123..."  # or pass explicitly with --github-token

chaci integrate-rock rock-ci-metadata.yaml main ghcr.io/canonical/my-rock:1.0.0 --dry-run
```

### Arguments:

| Argument            | Description                                                                 |
|---------------------|-----------------------------------------------------------------------------|
| `METADATA_FILE`     | Path to `rock-ci-metadata.yaml` describing integration targets              |
| `BASE_BRANCH`       | Target branch for PRs (e.g. `main` or `develop`)                            |
| `ROCK_IMAGE`        | Full rock image string (e.g. `ghcr.io/org/my-rock:1.0.0`)                   |

### Options:

| Option                  | Description                                                                                     |
|--------------------------|-------------------------------------------------------------------------------------------------|
| `--github-token`         | Optional. GitHub token. Falls back to `$GH_TOKEN` environment variable if not provided.         |
| `--github-username`      | Optional. GitHub username. Defaults to `"__token__"` if not provided.                           |
| `--clone-dir PATH`       | Optional. Directory where target repos will be cloned (default: `/tmp`).                        |
| `--dry-run`              | Optional. If set, changes are simulated but not committed or pushed. Logs changes to console.   |
| `--triggering-pr URL`    | Optional. Link to the PR which triggered this run. Included in the PR body if present.          |

---

## ๐Ÿ“„ rock-ci-metadata.yaml Format

```yaml
integrations:
  - consumer-repository: canonical/my-charm
    replace-image:
      - file: "metadata.yaml"
        path: "resources.my-rock.upstream-source"
      - file: "src/images.json"
        path: "config.batcher"
    service-spec:
      - file: "service-spec.json"
        user:
          path: "containers[0].user"
          value: "1001"
        command:
          path: "containers[0].command[1]"
          value: "/start"
```

- All file paths are **relative to the repo root**
- Paths can use `dot` and `bracket` notation for navigating YAML/JSON

---

## ๐Ÿงช Testing

### Unit tests

```bash
tox -e unit
```

### ๐Ÿ” Integration tests

> โš ๏ธ These tests **interact with a real GitHub repository** and require a **fine-grained GitHub token** with appropriate permissions.

#### Required GitHub token permissions

The token must be a **fine-grained personal access token** (PAT) with:

- **Repository access**: Select the repository you're testing against
- **Permissions**:
  - `Contents: Read and write`
  - `Pull requests: Read and write`

These are needed to:
- Clone the repo
- Push branches
- Open and manage pull requests

---

#### Setup and run:

```bash
export CHACI_TEST_TOKEN=<your_token>
export CHACI_TEST_REPO="org/repo-name"
export CHACI_TEST_BASE_BRANCH="target-branch"

tox -e integration
```

> The integration tests will:
> - Clone the specified repository
> - Create a temporary branch and pull request
> - Validate the PR contents
> - Clean up the branch and PR after execution

---

## ๐Ÿงฐ Development & Contributing

This project uses:
- [tox](https://tox.readthedocs.io/) for test environments
- [pytest](https://docs.pytest.org/) for testing
- [black](https://black.readthedocs.io/) + [ruff](https://docs.astral.sh/ruff/) for linting

To run all checks locally:

```bash
tox -e lint,unit,integration
```

---

## ๐Ÿ“ Project Structure

| File                          | Purpose                                      |
|-------------------------------|----------------------------------------------|
| `rock_integrator.py`          | Core logic for modifying files with images   |
| `git_client.py`               | Git and GitHub abstraction for PR workflow   |
| `rock_metadata_handler.py`    | Orchestration for multi-repo integration     |
| `rock_ci_metadata_models.py` | Pydantic model for metadata schema validation|
| `main.py`                     | CLI entrypoint via `click`                   |
| `templates/pr_body.md.j2`     | Jinja2 template for pull request bodies      |

---

## ๐Ÿ”’ License

This project is licensed under the [Apache 2.0 License](LICENSE).

---

## โœ๏ธ Authors

Built by the [Canonical Charmed Kubeflow team](https://github.com/canonical).

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "charmed-analytics-ci",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4.0,>=3.12",
    "maintainer_email": null,
    "keywords": "charm, ci, rock, automation, github, charmed-kubeflow",
    "author": "Charmed Kubeflow",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/ee/fa/b72524d0ffb03db69e6537ffb1e05e0143d8a1746fa5d0d814f0b4d3499b/charmed_analytics_ci-1.0.0.tar.gz",
    "platform": null,
    "description": "# charmed-analytics-ci\n\nA CLI tool to automate CI-driven integration of updated **rock images** into consumer **Charmed Operator** repositories.\n\nThis tool is part of Canonical's Charmed Kubeflow stack and enables automated pull request creation after a rock image is built and published. It eliminates manual effort, reduces human error, and supports scalable, reproducible CI/CD pipelines.\n\n---\n\n## \u2728 Features\n\n- \u2705 Automatically clones target charm repositories\n- \ud83d\udd01 Updates image references in YAML or JSON configuration files\n- \u2699\ufe0f Optionally modifies service-spec fields like `user` and `command`\n- \ud83d\udd27 Validates metadata schemas for correctness before modification\n- \ud83e\udd16 Opens pull requests with deterministic branches and templated descriptions\n- \ud83d\udd10 Supports GitHub authentication via token or environment variable\n- \ud83d\udd17 Optionally links back to triggering PR\n- \ud83d\udce6 Installable via PyPI and usable from CI pipelines\n- \ud83e\uddea Supports dry-run mode for previewing changes\n\n---\n\n## \ud83d\ude80 Installation\n\nInstall from PyPI:\n\n```bash\npip install charmed-analytics-ci\n```\n\nOr install for development:\n\n```bash\ngit clone https://github.com/canonical/charmed-analytics-ci.git\ncd charmed-analytics-ci\npoetry install\n```\n\n---\n\n## \ud83e\uddea CLI Usage\n\nAfter installing, the CLI provides a single command:\n\n```bash\nchaci integrate-rock METADATA_FILE BASE_BRANCH ROCK_IMAGE [OPTIONS]\n```\n\n### Example:\n\n```bash\nexport GH_TOKEN=\"ghp_abc123...\"  # or pass explicitly with --github-token\n\nchaci integrate-rock rock-ci-metadata.yaml main ghcr.io/canonical/my-rock:1.0.0 --dry-run\n```\n\n### Arguments:\n\n| Argument            | Description                                                                 |\n|---------------------|-----------------------------------------------------------------------------|\n| `METADATA_FILE`     | Path to `rock-ci-metadata.yaml` describing integration targets              |\n| `BASE_BRANCH`       | Target branch for PRs (e.g. `main` or `develop`)                            |\n| `ROCK_IMAGE`        | Full rock image string (e.g. `ghcr.io/org/my-rock:1.0.0`)                   |\n\n### Options:\n\n| Option                  | Description                                                                                     |\n|--------------------------|-------------------------------------------------------------------------------------------------|\n| `--github-token`         | Optional. GitHub token. Falls back to `$GH_TOKEN` environment variable if not provided.         |\n| `--github-username`      | Optional. GitHub username. Defaults to `\"__token__\"` if not provided.                           |\n| `--clone-dir PATH`       | Optional. Directory where target repos will be cloned (default: `/tmp`).                        |\n| `--dry-run`              | Optional. If set, changes are simulated but not committed or pushed. Logs changes to console.   |\n| `--triggering-pr URL`    | Optional. Link to the PR which triggered this run. Included in the PR body if present.          |\n\n---\n\n## \ud83d\udcc4 rock-ci-metadata.yaml Format\n\n```yaml\nintegrations:\n  - consumer-repository: canonical/my-charm\n    replace-image:\n      - file: \"metadata.yaml\"\n        path: \"resources.my-rock.upstream-source\"\n      - file: \"src/images.json\"\n        path: \"config.batcher\"\n    service-spec:\n      - file: \"service-spec.json\"\n        user:\n          path: \"containers[0].user\"\n          value: \"1001\"\n        command:\n          path: \"containers[0].command[1]\"\n          value: \"/start\"\n```\n\n- All file paths are **relative to the repo root**\n- Paths can use `dot` and `bracket` notation for navigating YAML/JSON\n\n---\n\n## \ud83e\uddea Testing\n\n### Unit tests\n\n```bash\ntox -e unit\n```\n\n### \ud83d\udd01 Integration tests\n\n> \u26a0\ufe0f These tests **interact with a real GitHub repository** and require a **fine-grained GitHub token** with appropriate permissions.\n\n#### Required GitHub token permissions\n\nThe token must be a **fine-grained personal access token** (PAT) with:\n\n- **Repository access**: Select the repository you're testing against\n- **Permissions**:\n  - `Contents: Read and write`\n  - `Pull requests: Read and write`\n\nThese are needed to:\n- Clone the repo\n- Push branches\n- Open and manage pull requests\n\n---\n\n#### Setup and run:\n\n```bash\nexport CHACI_TEST_TOKEN=<your_token>\nexport CHACI_TEST_REPO=\"org/repo-name\"\nexport CHACI_TEST_BASE_BRANCH=\"target-branch\"\n\ntox -e integration\n```\n\n> The integration tests will:\n> - Clone the specified repository\n> - Create a temporary branch and pull request\n> - Validate the PR contents\n> - Clean up the branch and PR after execution\n\n---\n\n## \ud83e\uddf0 Development & Contributing\n\nThis project uses:\n- [tox](https://tox.readthedocs.io/) for test environments\n- [pytest](https://docs.pytest.org/) for testing\n- [black](https://black.readthedocs.io/) + [ruff](https://docs.astral.sh/ruff/) for linting\n\nTo run all checks locally:\n\n```bash\ntox -e lint,unit,integration\n```\n\n---\n\n## \ud83d\udcc1 Project Structure\n\n| File                          | Purpose                                      |\n|-------------------------------|----------------------------------------------|\n| `rock_integrator.py`          | Core logic for modifying files with images   |\n| `git_client.py`               | Git and GitHub abstraction for PR workflow   |\n| `rock_metadata_handler.py`    | Orchestration for multi-repo integration     |\n| `rock_ci_metadata_models.py` | Pydantic model for metadata schema validation|\n| `main.py`                     | CLI entrypoint via `click`                   |\n| `templates/pr_body.md.j2`     | Jinja2 template for pull request bodies      |\n\n---\n\n## \ud83d\udd12 License\n\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n\n---\n\n## \u270d\ufe0f Authors\n\nBuilt by the [Canonical Charmed Kubeflow team](https://github.com/canonical).\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "A CLI tool for automating CI tasks across Charmed Operator repositories",
    "version": "1.0.0",
    "project_urls": {
        "Homepage": "https://github.com/canonical/charmed-analytics-ci",
        "Repository": "https://github.com/canonical/charmed-analytics-ci"
    },
    "split_keywords": [
        "charm",
        " ci",
        " rock",
        " automation",
        " github",
        " charmed-kubeflow"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "1430c57b5217c2dc3c2df3e54106503199a66609d07d104e23b3a969dd6d80af",
                "md5": "e3b78afe30297df626cd5dd7b8498862",
                "sha256": "9b55ca88e360956d0b6591c696d3569b1d7fbee4028366024d0a3f0c94a096f3"
            },
            "downloads": -1,
            "filename": "charmed_analytics_ci-1.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e3b78afe30297df626cd5dd7b8498862",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.12",
            "size": 20182,
            "upload_time": "2025-07-18T11:54:51",
            "upload_time_iso_8601": "2025-07-18T11:54:51.723593Z",
            "url": "https://files.pythonhosted.org/packages/14/30/c57b5217c2dc3c2df3e54106503199a66609d07d104e23b3a969dd6d80af/charmed_analytics_ci-1.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "eefab72524d0ffb03db69e6537ffb1e05e0143d8a1746fa5d0d814f0b4d3499b",
                "md5": "c684a33c30a62aff0c85821391047c71",
                "sha256": "905a1c7e7659801466bcf2734349ad7edca30057662fc077ae5075645c9d530a"
            },
            "downloads": -1,
            "filename": "charmed_analytics_ci-1.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "c684a33c30a62aff0c85821391047c71",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.12",
            "size": 18129,
            "upload_time": "2025-07-18T11:54:52",
            "upload_time_iso_8601": "2025-07-18T11:54:52.751903Z",
            "url": "https://files.pythonhosted.org/packages/ee/fa/b72524d0ffb03db69e6537ffb1e05e0143d8a1746fa5d0d814f0b4d3499b/charmed_analytics_ci-1.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-07-18 11:54:52",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "canonical",
    "github_project": "charmed-analytics-ci",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "tox": true,
    "lcname": "charmed-analytics-ci"
}
        
Elapsed time: 0.42904s