<h1 align="center">
<a href="https://otterdog.eclipse.org">
<img style="width: 350px;" src="https://raw.githubusercontent.com/eclipse-csi/.github/refs/heads/main/artwork/eclipse-otterdog/Logo%20Color%20-%20Transparent%20Bg.png">
</a>
</h1>
<p align="center">
<a href="https://pypi.org/project/otterdog"><img alt="PyPI" src="https://img.shields.io/pypi/v/otterdog.svg?color=blue&maxAge=600" /></a>
<a href="https://pypi.org/project/otterdog"><img alt="PyPI - Python Versions" src="https://img.shields.io/pypi/pyversions/otterdog.svg?maxAge=600" /></a>
<a href="https://github.com/eclipse-csi/otterdog/blob/main/LICENSE"><img alt="EPLv2 License" src="https://img.shields.io/github/license/eclipse-csi/otterdog" /></a>
<a href="https://github.com/eclipse-csi/otterdog/actions/workflows/build.yml?query=branch%3Amain"><img alt="Build Status on GitHub" src="https://github.com/eclipse-csi/otterdog/actions/workflows/build.yml/badge.svg?branch:main&workflow:Build" /></a>
<a href="https://otterdog.readthedocs.io"><img alt="Documentation Status" src="https://readthedocs.org/projects/otterdog/badge/?version=latest" /></a><br>
<a href="https://scorecard.dev/viewer/?uri=github.com/eclipse-csi/otterdog"><img alt="OpenSSF Scorecard" src="https://api.securityscorecards.dev/projects/github.com/eclipse-csi/otterdog/badge" /></a>
<a href="https://www.bestpractices.dev/projects/9624"><img alt="OpenSSF Best Practices" src="https://www.bestpractices.dev/projects/9624/badge" /></a>
<a href="https://slsa.dev"><img alt="OpenSSF SLSA Level 3" src="https://slsa.dev/images/gh-badge-level3.svg" /></a>
</p>
# Eclipse Otterdog
## Introduction
Otterdog is a tool to manage GitHub organizations at scale using a configuration as code approach.
It is actively developed by the Eclipse Foundation and used to manage its numerous projects hosted on GitHub.
## Quickstart
To install and use the cli part of otterdog you have to install the following:
* git (mandatory): install using `apt install git`
* otterdog (mandatory): install using `pipx install otterdog`
* bitwarden cli tool (optional): install using `snap install bw`
* pass cli tool (optional): install using `apt install pass`
[Otterdog Presentation @ Open Source Summit 2023](https://docs.google.com/presentation/d/1lLqbhDQf9s5U2A2TkcoFYA39qtODcSot2308vnKbkbA/edit?usp=sharing)
[Default Configuration used @ Eclipse Foundation](https://github.com/EclipseFdn/otterdog-defaults/)
## Documentation
The documentation is available at [otterdog.readthedocs.io](https://otterdog.readthedocs.io).
## Build instructions
### System requirements:
* python3.11+ (mandatory): e.g. install using `apt install python3` or use `pyenv install 3.12`
* git (mandatory): install using `apt install git`
* poetry (mandatory): install using `pipx install poetry`
* dynamic versioning plugin (mandatory): install using `pipx inject poetry "poetry-dynamic-versioning[plugin]"`
* bitwarden cli tool (optional): install using `snap install bw`
* pass cli tool (optional): install using `apt install pass`
### Building Steps
* Create a virtual python environment and install necessary python dependencies using poetry:
```console
$ make init
```
Running `make init` will also install `poetry` and the `dynamic versioning plugin` if it is not installed yet.
* Testing build
```console
$ ./otterdog.sh -h
```
## Quick Setup
To start using the cli part of `otterdog` right away on a specific organization you have to set up the following:
- define a default configuration to use, you can use the following [default config](https://github.com/eclipse-csi/otterdog/blob/main/examples/template/otterdog-defaults.libsonnet) as a starting point
- create a `otterdog.json` file that contains the list of organizations you want to manage and some customizations
- start managing your organizations using the cli
### Default configuration
You can define your own default configuration or use the following base template right away: `https://github.com/eclipse-csi/otterdog#examples/template/otterdog-defaults.libsonnet@main`.
### Otterdog configuration
Create a `otterdog.json` file with the following content (replace bracketed values according to your setup):
```json
{
"defaults": {
"jsonnet": {
"base_template": "https://github.com/eclipse-csi/otterdog#examples/template/otterdog-defaults.libsonnet@main",
"config_dir": "orgs"
},
"github": {
"config_repo": ".otterdog"
}
},
"organizations": [
{
"name": "<project-name>",
"github_id": "<github-id>",
"credentials": {
"provider": "bitwarden",
"item_id": "<bitwarden item id>"
}
}
]
}
```
The name of the configuration file can be freely chosen (can be specified with the __-c__ flag).
However, when named `otterdog.json`, the cli tool will automatically detect and use that file if it is in the current working directory.
### Credentials
Otterdog needs certain credentials to access information from an organization and its repositories on GitHub:
* username / password / 2FA seed
* API token
The login / username / 2FA seed are required to access the web interface of GitHub in order to retrieve certain
settings that are not accessible via its rest / graphql API.
The GitHub api token needs to have the following scopes enabled:
* repo
* workflow
* admin:org
* admin:org_hook
* delete_repo
The credentials can be stored in different providers (bitwarden, pass).
#### Bitwarden
When using **bitwarden** to store the credentials, you need to enter a valid __item id__ as additional credential data:
```json
{
"organizations": [
{
"name": "<org name>",
"github_id": "<github org id>",
"credentials": {
"provider": "bitwarden",
"item_id" : "<bitwarden item id>"
}
}
]
}
```
The item stored in bitwarden needs to contain the following information (a sample json output of such an item):
```json
{
"object": "item",
"id": "<bitwarden item id>",
"name": "<item name>",
"fields": [
{
"name": "api_token_admin",
"value": "<github API token>"
}
],
"login": {
"username": "<github username>",
"password": "<github password>",
"totp": "<github TOTP text code>"
}
}
```
Mandatory items:
* Field with name "api_token_admin" and as value the GitHub token to access the organization
* __login.username__ of a user that can access the organization with enabled 2FA
* __login.password__ the password of that user
* __login.totp__ the TOTP text code
#### Pass
When using **pass** to store the credentials, you need to enter fully qualified pass names to access the various
required credential data:
```json
{
"organizations": [
{
"name": "<org name>",
"github_id": "<github org id>",
"credentials": {
"provider": "pass",
"api_token": "<path/to/api_token>",
"username": "<path/to/username>",
"password": "<path/to/password>",
"2fa_seed": "<path/to/2fa_seed>"
}
}
]
}
```
In case your password storage dir is not located at the default location, you can
configurate that in the `defaults`:
```json
{
"defaults": {
"pass": {
"password_store_dir": "path/to/storage/dir"
}
}
}
```
As the `password_store_dir` might be different on different machines, you can also customize that in a separate `.otterdog-defaults.json` file:
```json
{
"pass": {
"password_store_dir": "path/to/storage/dir"
}
}
```
## Typical Workflow
In general, all operations act on the local configuration stored in `<cwd>/<config-dir>/<organization>/<organization>.jsonnet`.
A typical workflow to handle changes to an organization are as follows:
1. (first time) run an initial `import` of the organization
2. (first time) run an `apply` operation to create all resources already inherited from the default config (e.g. config repo)
3. (regular) fetch the latest config from the config repo using `fetch-config`
4. (optional) make any local changes to the configuration
5. (optional) run the `validate` operation to see if the configuration is syntactically and semantically correct
6. (optional) run the `plan` operation to see which changes would be applied taking the current live configuration into account
7. (regular) run the `apply` operation to actually apply the changes (also runs `validate` and `plan`, so steps 6 & 7 are redundant)
8. (regular) push the local configuration to the config repo using the `push-config` operation
Raw data
{
"_id": null,
"home_page": "https://github.com/eclipse-csi/otterdog",
"name": "otterdog",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "infrastructure-as-code, supply-chain-security, github, gitops",
"author": "Thomas Neidhart",
"author_email": "thomas.neidhart@eclipse-foundation.org",
"download_url": "https://files.pythonhosted.org/packages/51/9a/6e71c183da9e866b2656658cf5b259834b7ed466f98d8bf9306310af55a8/otterdog-0.10.0.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">\n\n<a href=\"https://otterdog.eclipse.org\">\n <img style=\"width: 350px;\" src=\"https://raw.githubusercontent.com/eclipse-csi/.github/refs/heads/main/artwork/eclipse-otterdog/Logo%20Color%20-%20Transparent%20Bg.png\">\n</a>\n\n</h1>\n\n<p align=\"center\">\n <a href=\"https://pypi.org/project/otterdog\"><img alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/otterdog.svg?color=blue&maxAge=600\" /></a>\n <a href=\"https://pypi.org/project/otterdog\"><img alt=\"PyPI - Python Versions\" src=\"https://img.shields.io/pypi/pyversions/otterdog.svg?maxAge=600\" /></a>\n <a href=\"https://github.com/eclipse-csi/otterdog/blob/main/LICENSE\"><img alt=\"EPLv2 License\" src=\"https://img.shields.io/github/license/eclipse-csi/otterdog\" /></a>\n <a href=\"https://github.com/eclipse-csi/otterdog/actions/workflows/build.yml?query=branch%3Amain\"><img alt=\"Build Status on GitHub\" src=\"https://github.com/eclipse-csi/otterdog/actions/workflows/build.yml/badge.svg?branch:main&workflow:Build\" /></a>\n <a href=\"https://otterdog.readthedocs.io\"><img alt=\"Documentation Status\" src=\"https://readthedocs.org/projects/otterdog/badge/?version=latest\" /></a><br>\n <a href=\"https://scorecard.dev/viewer/?uri=github.com/eclipse-csi/otterdog\"><img alt=\"OpenSSF Scorecard\" src=\"https://api.securityscorecards.dev/projects/github.com/eclipse-csi/otterdog/badge\" /></a>\n <a href=\"https://www.bestpractices.dev/projects/9624\"><img alt=\"OpenSSF Best Practices\" src=\"https://www.bestpractices.dev/projects/9624/badge\" /></a>\n <a href=\"https://slsa.dev\"><img alt=\"OpenSSF SLSA Level 3\" src=\"https://slsa.dev/images/gh-badge-level3.svg\" /></a>\n</p>\n\n# Eclipse Otterdog\n\n## Introduction\n\nOtterdog is a tool to manage GitHub organizations at scale using a configuration as code approach.\nIt is actively developed by the Eclipse Foundation and used to manage its numerous projects hosted on GitHub.\n\n## Quickstart\n\nTo install and use the cli part of otterdog you have to install the following:\n\n* git (mandatory): install using `apt install git`\n* otterdog (mandatory): install using `pipx install otterdog`\n* bitwarden cli tool (optional): install using `snap install bw`\n* pass cli tool (optional): install using `apt install pass`\n\n[Otterdog Presentation @ Open Source Summit 2023](https://docs.google.com/presentation/d/1lLqbhDQf9s5U2A2TkcoFYA39qtODcSot2308vnKbkbA/edit?usp=sharing)\n\n[Default Configuration used @ Eclipse Foundation](https://github.com/EclipseFdn/otterdog-defaults/)\n\n## Documentation\n\nThe documentation is available at [otterdog.readthedocs.io](https://otterdog.readthedocs.io).\n\n## Build instructions\n\n### System requirements:\n\n* python3.11+ (mandatory): e.g. install using `apt install python3` or use `pyenv install 3.12`\n* git (mandatory): install using `apt install git`\n* poetry (mandatory): install using `pipx install poetry`\n* dynamic versioning plugin (mandatory): install using `pipx inject poetry \"poetry-dynamic-versioning[plugin]\"`\n* bitwarden cli tool (optional): install using `snap install bw`\n* pass cli tool (optional): install using `apt install pass`\n\n### Building Steps\n\n* Create a virtual python environment and install necessary python dependencies using poetry:\n\n```console\n$ make init\n```\n\nRunning `make init` will also install `poetry` and the `dynamic versioning plugin` if it is not installed yet.\n\n* Testing build\n\n```console\n$ ./otterdog.sh -h\n```\n\n## Quick Setup\n\nTo start using the cli part of `otterdog` right away on a specific organization you have to set up the following:\n\n- define a default configuration to use, you can use the following [default config](https://github.com/eclipse-csi/otterdog/blob/main/examples/template/otterdog-defaults.libsonnet) as a starting point\n- create a `otterdog.json` file that contains the list of organizations you want to manage and some customizations\n- start managing your organizations using the cli\n\n### Default configuration\n\nYou can define your own default configuration or use the following base template right away: `https://github.com/eclipse-csi/otterdog#examples/template/otterdog-defaults.libsonnet@main`.\n\n### Otterdog configuration\n\nCreate a `otterdog.json` file with the following content (replace bracketed values according to your setup):\n\n```json\n{\n \"defaults\": {\n \"jsonnet\": {\n \"base_template\": \"https://github.com/eclipse-csi/otterdog#examples/template/otterdog-defaults.libsonnet@main\",\n \"config_dir\": \"orgs\"\n },\n \"github\": {\n \"config_repo\": \".otterdog\"\n }\n },\n \"organizations\": [\n {\n \"name\": \"<project-name>\",\n \"github_id\": \"<github-id>\",\n \"credentials\": {\n \"provider\": \"bitwarden\",\n \"item_id\": \"<bitwarden item id>\"\n }\n }\n ]\n}\n```\n\nThe name of the configuration file can be freely chosen (can be specified with the __-c__ flag).\nHowever, when named `otterdog.json`, the cli tool will automatically detect and use that file if it is in the current working directory.\n\n### Credentials\n\nOtterdog needs certain credentials to access information from an organization and its repositories on GitHub:\n\n* username / password / 2FA seed\n* API token\n\nThe login / username / 2FA seed are required to access the web interface of GitHub in order to retrieve certain\nsettings that are not accessible via its rest / graphql API.\n\nThe GitHub api token needs to have the following scopes enabled:\n\n* repo\n* workflow\n* admin:org\n* admin:org_hook\n* delete_repo\n\nThe credentials can be stored in different providers (bitwarden, pass).\n\n#### Bitwarden\n\nWhen using **bitwarden** to store the credentials, you need to enter a valid __item id__ as additional credential data:\n\n```json\n{\n \"organizations\": [\n {\n \"name\": \"<org name>\",\n \"github_id\": \"<github org id>\",\n \"credentials\": {\n \"provider\": \"bitwarden\",\n \"item_id\" : \"<bitwarden item id>\"\n }\n }\n ]\n}\n```\n\nThe item stored in bitwarden needs to contain the following information (a sample json output of such an item):\n\n```json\n{\n \"object\": \"item\",\n \"id\": \"<bitwarden item id>\",\n \"name\": \"<item name>\",\n \"fields\": [\n {\n \"name\": \"api_token_admin\",\n \"value\": \"<github API token>\"\n }\n ],\n \"login\": {\n \"username\": \"<github username>\",\n \"password\": \"<github password>\",\n \"totp\": \"<github TOTP text code>\"\n }\n}\n```\n\nMandatory items:\n\n* Field with name \"api_token_admin\" and as value the GitHub token to access the organization\n* __login.username__ of a user that can access the organization with enabled 2FA\n* __login.password__ the password of that user\n* __login.totp__ the TOTP text code\n\n#### Pass\n\nWhen using **pass** to store the credentials, you need to enter fully qualified pass names to access the various\nrequired credential data:\n\n```json\n{\n \"organizations\": [\n {\n \"name\": \"<org name>\",\n \"github_id\": \"<github org id>\",\n \"credentials\": {\n \"provider\": \"pass\",\n \"api_token\": \"<path/to/api_token>\",\n \"username\": \"<path/to/username>\",\n \"password\": \"<path/to/password>\",\n \"2fa_seed\": \"<path/to/2fa_seed>\"\n }\n }\n ]\n}\n```\n\nIn case your password storage dir is not located at the default location, you can\nconfigurate that in the `defaults`:\n\n```json\n{\n \"defaults\": {\n \"pass\": {\n \"password_store_dir\": \"path/to/storage/dir\"\n }\n }\n}\n```\n\nAs the `password_store_dir` might be different on different machines, you can also customize that in a separate `.otterdog-defaults.json` file:\n\n```json\n{\n \"pass\": {\n \"password_store_dir\": \"path/to/storage/dir\"\n }\n}\n```\n\n## Typical Workflow\n\nIn general, all operations act on the local configuration stored in `<cwd>/<config-dir>/<organization>/<organization>.jsonnet`.\n\nA typical workflow to handle changes to an organization are as follows:\n\n1. (first time) run an initial `import` of the organization\n2. (first time) run an `apply` operation to create all resources already inherited from the default config (e.g. config repo)\n3. (regular) fetch the latest config from the config repo using `fetch-config`\n4. (optional) make any local changes to the configuration\n5. (optional) run the `validate` operation to see if the configuration is syntactically and semantically correct\n6. (optional) run the `plan` operation to see which changes would be applied taking the current live configuration into account\n7. (regular) run the `apply` operation to actually apply the changes (also runs `validate` and `plan`, so steps 6 & 7 are redundant)\n8. (regular) push the local configuration to the config repo using the `push-config` operation\n\n",
"bugtrack_url": null,
"license": "Eclipse Public License 2.0 (EPL-2.0)",
"summary": "Tool to manage GitHub organizations and their repositories.",
"version": "0.10.0",
"project_urls": {
"Documentation": "https://otterdog.readthedocs.io",
"Homepage": "https://github.com/eclipse-csi/otterdog",
"Repository": "https://github.com/eclipse-csi/otterdog"
},
"split_keywords": [
"infrastructure-as-code",
" supply-chain-security",
" github",
" gitops"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f7de7f4c0516a234262f155b5536eb2cc64c4245f418993bbdca658fd73a3cf7",
"md5": "d4115737087007c1ffb3723c255fe165",
"sha256": "abfb2588216361ba4eb3146ad0f213c59972620f8a2e3edf76de9dbdd1bfa056"
},
"downloads": -1,
"filename": "otterdog-0.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d4115737087007c1ffb3723c255fe165",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 211562,
"upload_time": "2024-12-20T14:04:07",
"upload_time_iso_8601": "2024-12-20T14:04:07.245291Z",
"url": "https://files.pythonhosted.org/packages/f7/de/7f4c0516a234262f155b5536eb2cc64c4245f418993bbdca658fd73a3cf7/otterdog-0.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "519a6e71c183da9e866b2656658cf5b259834b7ed466f98d8bf9306310af55a8",
"md5": "bf1c044f84279736d617ccc33ef3834a",
"sha256": "a7834431a003fe0da9ff0db6028cba2e21afb246c66f836d3d0b974dc6c6b226"
},
"downloads": -1,
"filename": "otterdog-0.10.0.tar.gz",
"has_sig": false,
"md5_digest": "bf1c044f84279736d617ccc33ef3834a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 290591,
"upload_time": "2024-12-20T14:04:09",
"upload_time_iso_8601": "2024-12-20T14:04:09.319328Z",
"url": "https://files.pythonhosted.org/packages/51/9a/6e71c183da9e866b2656658cf5b259834b7ed466f98d8bf9306310af55a8/otterdog-0.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-20 14:04:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eclipse-csi",
"github_project": "otterdog",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "otterdog"
}
JSON
