# charmed-analytics-ci
A CLI tool to automate CI-driven integration of updated **rock images** into consumer **Charmed Operator** repositories.
This tool is part of Canonical's Charmed Kubeflow stack and enables automated pull request creation after a rock image is built and published. It eliminates manual effort, reduces human error, and supports scalable, reproducible CI/CD pipelines.
---
## ✨ Features
- ✅ Automatically clones target charm repositories
- 🔁 Updates image references in YAML or JSON configuration files
- ⚙️ Optionally modifies service-spec fields like `user` and `command`
- 🔧 Validates metadata schemas for correctness before modification
- 🤖 Opens pull requests with deterministic branches and templated descriptions
- 🔐 Supports GitHub authentication via token or environment variable
- 🔗 Optionally links back to triggering PR
- 📦 Installable via PyPI and usable from CI pipelines
- 🧪 Supports dry-run mode for previewing changes
---
## 🚀 Installation
Install from PyPI:
```bash
pip install charmed-analytics-ci
```
Or install for development:
```bash
git clone https://github.com/canonical/charmed-analytics-ci.git
cd charmed-analytics-ci
poetry install
```
---
## 🧪 CLI Usage
After installing, the CLI provides a single command:
```bash
chaci integrate-rock METADATA_FILE BASE_BRANCH ROCK_IMAGE [OPTIONS]
```
### Example:
```bash
export GH_TOKEN="ghp_abc123..." # or pass explicitly with --github-token
chaci integrate-rock rock-ci-metadata.yaml main ghcr.io/canonical/my-rock:1.0.0 --dry-run
```
### Arguments:
| Argument | Description |
|---------------------|-----------------------------------------------------------------------------|
| `METADATA_FILE` | Path to `rock-ci-metadata.yaml` describing integration targets |
| `BASE_BRANCH` | Target branch for PRs (e.g. `main` or `develop`) |
| `ROCK_IMAGE` | Full rock image string (e.g. `ghcr.io/org/my-rock:1.0.0`) |
### Options:
| Option | Description |
|--------------------------|-------------------------------------------------------------------------------------------------|
| `--github-token` | Optional. GitHub token. Falls back to `$GH_TOKEN` environment variable if not provided. |
| `--github-username` | Optional. GitHub username. Defaults to `"__token__"` if not provided. |
| `--clone-dir PATH` | Optional. Directory where target repos will be cloned (default: `/tmp`). |
| `--dry-run` | Optional. If set, changes are simulated but not committed or pushed. Logs changes to console. |
| `--triggering-pr URL` | Optional. Link to the PR which triggered this run. Included in the PR body if present. |
---
## 📄 rock-ci-metadata.yaml Format
```yaml
integrations:
- consumer-repository: canonical/my-charm
replace-image:
- file: "metadata.yaml"
path: "resources.my-rock.upstream-source"
- file: "src/images.json"
path: "config.batcher"
service-spec:
- file: "service-spec.json"
user:
path: "containers[0].user"
value: "1001"
command:
path: "containers[0].command[1]"
value: "/start"
```
- All file paths are **relative to the repo root**
- Paths can use `dot` and `bracket` notation for navigating YAML/JSON
---
## 🧪 Testing
### Unit tests
```bash
tox -e unit
```
### 🔁 Integration tests
> ⚠️ These tests **interact with a real GitHub repository** and require a **fine-grained GitHub token** with appropriate permissions.
#### Required GitHub token permissions
The token must be a **fine-grained personal access token** (PAT) with:
- **Repository access**: Select the repository you're testing against
- **Permissions**:
- `Contents: Read and write`
- `Pull requests: Read and write`
These are needed to:
- Clone the repo
- Push branches
- Open and manage pull requests
---
#### Setup and run:
```bash
export CHACI_TEST_TOKEN=<your_token>
export CHACI_TEST_REPO="org/repo-name"
export CHACI_TEST_BASE_BRANCH="target-branch"
tox -e integration
```
> The integration tests will:
> - Clone the specified repository
> - Create a temporary branch and pull request
> - Validate the PR contents
> - Clean up the branch and PR after execution
---
## 🧰 Development & Contributing
This project uses:
- [tox](https://tox.readthedocs.io/) for test environments
- [pytest](https://docs.pytest.org/) for testing
- [black](https://black.readthedocs.io/) + [ruff](https://docs.astral.sh/ruff/) for linting
To run all checks locally:
```bash
tox -e lint,unit,integration
```
---
## 📁 Project Structure
| File | Purpose |
|-------------------------------|----------------------------------------------|
| `rock_integrator.py` | Core logic for modifying files with images |
| `git_client.py` | Git and GitHub abstraction for PR workflow |
| `rock_metadata_handler.py` | Orchestration for multi-repo integration |
| `rock_ci_metadata_models.py` | Pydantic model for metadata schema validation|
| `main.py` | CLI entrypoint via `click` |
| `templates/pr_body.md.j2` | Jinja2 template for pull request bodies |
---
## 🔒 License
This project is licensed under the [Apache 2.0 License](LICENSE).
---
## ✍️ Authors
Built by the [Canonical Charmed Kubeflow team](https://github.com/canonical).
Raw data
{
"_id": null,
"home_page": null,
"name": "charmed-analytics-ci",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": "charm, ci, rock, automation, github, charmed-kubeflow",
"author": "Charmed Kubeflow",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/ee/fa/b72524d0ffb03db69e6537ffb1e05e0143d8a1746fa5d0d814f0b4d3499b/charmed_analytics_ci-1.0.0.tar.gz",
"platform": null,
"description": "# charmed-analytics-ci\n\nA CLI tool to automate CI-driven integration of updated **rock images** into consumer **Charmed Operator** repositories.\n\nThis tool is part of Canonical's Charmed Kubeflow stack and enables automated pull request creation after a rock image is built and published. It eliminates manual effort, reduces human error, and supports scalable, reproducible CI/CD pipelines.\n\n---\n\n## \u2728 Features\n\n- \u2705 Automatically clones target charm repositories\n- \ud83d\udd01 Updates image references in YAML or JSON configuration files\n- \u2699\ufe0f Optionally modifies service-spec fields like `user` and `command`\n- \ud83d\udd27 Validates metadata schemas for correctness before modification\n- \ud83e\udd16 Opens pull requests with deterministic branches and templated descriptions\n- \ud83d\udd10 Supports GitHub authentication via token or environment variable\n- \ud83d\udd17 Optionally links back to triggering PR\n- \ud83d\udce6 Installable via PyPI and usable from CI pipelines\n- \ud83e\uddea Supports dry-run mode for previewing changes\n\n---\n\n## \ud83d\ude80 Installation\n\nInstall from PyPI:\n\n```bash\npip install charmed-analytics-ci\n```\n\nOr install for development:\n\n```bash\ngit clone https://github.com/canonical/charmed-analytics-ci.git\ncd charmed-analytics-ci\npoetry install\n```\n\n---\n\n## \ud83e\uddea CLI Usage\n\nAfter installing, the CLI provides a single command:\n\n```bash\nchaci integrate-rock METADATA_FILE BASE_BRANCH ROCK_IMAGE [OPTIONS]\n```\n\n### Example:\n\n```bash\nexport GH_TOKEN=\"ghp_abc123...\" # or pass explicitly with --github-token\n\nchaci integrate-rock rock-ci-metadata.yaml main ghcr.io/canonical/my-rock:1.0.0 --dry-run\n```\n\n### Arguments:\n\n| Argument | Description |\n|---------------------|-----------------------------------------------------------------------------|\n| `METADATA_FILE` | Path to `rock-ci-metadata.yaml` describing integration targets |\n| `BASE_BRANCH` | Target branch for PRs (e.g. `main` or `develop`) |\n| `ROCK_IMAGE` | Full rock image string (e.g. `ghcr.io/org/my-rock:1.0.0`) |\n\n### Options:\n\n| Option | Description |\n|--------------------------|-------------------------------------------------------------------------------------------------|\n| `--github-token` | Optional. GitHub token. Falls back to `$GH_TOKEN` environment variable if not provided. |\n| `--github-username` | Optional. GitHub username. Defaults to `\"__token__\"` if not provided. |\n| `--clone-dir PATH` | Optional. Directory where target repos will be cloned (default: `/tmp`). |\n| `--dry-run` | Optional. If set, changes are simulated but not committed or pushed. Logs changes to console. |\n| `--triggering-pr URL` | Optional. Link to the PR which triggered this run. Included in the PR body if present. |\n\n---\n\n## \ud83d\udcc4 rock-ci-metadata.yaml Format\n\n```yaml\nintegrations:\n - consumer-repository: canonical/my-charm\n replace-image:\n - file: \"metadata.yaml\"\n path: \"resources.my-rock.upstream-source\"\n - file: \"src/images.json\"\n path: \"config.batcher\"\n service-spec:\n - file: \"service-spec.json\"\n user:\n path: \"containers[0].user\"\n value: \"1001\"\n command:\n path: \"containers[0].command[1]\"\n value: \"/start\"\n```\n\n- All file paths are **relative to the repo root**\n- Paths can use `dot` and `bracket` notation for navigating YAML/JSON\n\n---\n\n## \ud83e\uddea Testing\n\n### Unit tests\n\n```bash\ntox -e unit\n```\n\n### \ud83d\udd01 Integration tests\n\n> \u26a0\ufe0f These tests **interact with a real GitHub repository** and require a **fine-grained GitHub token** with appropriate permissions.\n\n#### Required GitHub token permissions\n\nThe token must be a **fine-grained personal access token** (PAT) with:\n\n- **Repository access**: Select the repository you're testing against\n- **Permissions**:\n - `Contents: Read and write`\n - `Pull requests: Read and write`\n\nThese are needed to:\n- Clone the repo\n- Push branches\n- Open and manage pull requests\n\n---\n\n#### Setup and run:\n\n```bash\nexport CHACI_TEST_TOKEN=<your_token>\nexport CHACI_TEST_REPO=\"org/repo-name\"\nexport CHACI_TEST_BASE_BRANCH=\"target-branch\"\n\ntox -e integration\n```\n\n> The integration tests will:\n> - Clone the specified repository\n> - Create a temporary branch and pull request\n> - Validate the PR contents\n> - Clean up the branch and PR after execution\n\n---\n\n## \ud83e\uddf0 Development & Contributing\n\nThis project uses:\n- [tox](https://tox.readthedocs.io/) for test environments\n- [pytest](https://docs.pytest.org/) for testing\n- [black](https://black.readthedocs.io/) + [ruff](https://docs.astral.sh/ruff/) for linting\n\nTo run all checks locally:\n\n```bash\ntox -e lint,unit,integration\n```\n\n---\n\n## \ud83d\udcc1 Project Structure\n\n| File | Purpose |\n|-------------------------------|----------------------------------------------|\n| `rock_integrator.py` | Core logic for modifying files with images |\n| `git_client.py` | Git and GitHub abstraction for PR workflow |\n| `rock_metadata_handler.py` | Orchestration for multi-repo integration |\n| `rock_ci_metadata_models.py` | Pydantic model for metadata schema validation|\n| `main.py` | CLI entrypoint via `click` |\n| `templates/pr_body.md.j2` | Jinja2 template for pull request bodies |\n\n---\n\n## \ud83d\udd12 License\n\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n\n---\n\n## \u270d\ufe0f Authors\n\nBuilt by the [Canonical Charmed Kubeflow team](https://github.com/canonical).\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A CLI tool for automating CI tasks across Charmed Operator repositories",
"version": "1.0.0",
"project_urls": {
"Homepage": "https://github.com/canonical/charmed-analytics-ci",
"Repository": "https://github.com/canonical/charmed-analytics-ci"
},
"split_keywords": [
"charm",
" ci",
" rock",
" automation",
" github",
" charmed-kubeflow"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1430c57b5217c2dc3c2df3e54106503199a66609d07d104e23b3a969dd6d80af",
"md5": "e3b78afe30297df626cd5dd7b8498862",
"sha256": "9b55ca88e360956d0b6591c696d3569b1d7fbee4028366024d0a3f0c94a096f3"
},
"downloads": -1,
"filename": "charmed_analytics_ci-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e3b78afe30297df626cd5dd7b8498862",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 20182,
"upload_time": "2025-07-18T11:54:51",
"upload_time_iso_8601": "2025-07-18T11:54:51.723593Z",
"url": "https://files.pythonhosted.org/packages/14/30/c57b5217c2dc3c2df3e54106503199a66609d07d104e23b3a969dd6d80af/charmed_analytics_ci-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "eefab72524d0ffb03db69e6537ffb1e05e0143d8a1746fa5d0d814f0b4d3499b",
"md5": "c684a33c30a62aff0c85821391047c71",
"sha256": "905a1c7e7659801466bcf2734349ad7edca30057662fc077ae5075645c9d530a"
},
"downloads": -1,
"filename": "charmed_analytics_ci-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "c684a33c30a62aff0c85821391047c71",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 18129,
"upload_time": "2025-07-18T11:54:52",
"upload_time_iso_8601": "2025-07-18T11:54:52.751903Z",
"url": "https://files.pythonhosted.org/packages/ee/fa/b72524d0ffb03db69e6537ffb1e05e0143d8a1746fa5d0d814f0b4d3499b/charmed_analytics_ci-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-18 11:54:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "canonical",
"github_project": "charmed-analytics-ci",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "charmed-analytics-ci"
}
JSON
