# Gitlab dumper
A simple CLI tool for creating a dump of all projects in your Gitlab instance.
Supports dump as raw repo and archives.
## Preflight
Go to `https://<YOUR_GITLAB_ENDPOINT>/-/profile/personal_access_tokens` and create personal token with permissions:
- api
- read_api
- read_repository
If you want to see all repositories, it may make sense to grant administrator permissions for the access token.
### Configuration
Create `.env` file and put credentials:
```bash
LOG_LEVEL="info"
GITLAB_URL="https://<YOUR_GITLAB_ENDPOINT>"
GITLAB_PERSONAL_TOKEN="xxxxxx" # also, GITLAB_OAUTH_TOKEN supports
```
Or set environment variables directly:
```bash
export LOG_LEVEL="info"
export GITLAB_URL="https://<YOUR_GITLAB_ENDPOINT>"
export GITLAB_PERSONAL_TOKEN="xxxxxx"
```
**Requirements:**
- Python >= 3.10
- The git command line tool must be installed on your PC
- The utility clones over SSH, so make sure that your public key is added to the GitLab profile, **[read the doc](https://docs.gitlab.com/ee/user/ssh.html)**
- If you are having problems with the SSH key, just run the dump command with the `--as-archive` flag, it does not require SSH.
### Install
Using pip:
```shell
python3 -m venv venv
source venv/bin/activate
pip install gitlab-dumper
gitlab-dumper --help
```
Installing from source:
```shell
git clone https://github.com/akimrx/gitlab-dumper --recursive
cd gitlab-dumper
python3 -m venv venv
source venv/bin/activate
make install # or: python3 setup.py install
gitlab-dumper --help
```
## Usage
```
Commands:
groups Operations with Gitlab groups.
projects Operations with Gitlab projects.
tree Show groups, subgroups and projects as tree.
Groups commands:
list Show available Gitlab groups.
projects List projects in group or subgroup.
Projects commands:
dump Download, clone or re-pull all available projects.
list Show available Gitlab projects.
```
#### Dump
Before really dump use `--dry-run` flag, for example:
```shell
gitlab-dumper projects dump --namespaces foo,bar --no-personal --dry-run
```
For full dump just use `gitlab-dumper projects dump`
Available optional flags:
```
--dumps-dir TEXT Directory for dumps (default: ./dumps).
--delay INTEGER Delay between clones in seconds (default 0).
--skip-empty Ignore empty projects.
--no-personal Ignore personal user projects.
--as-archive Download projects as tar.gz archive instead clone.
--dry-run Safe simulate dump without download/clone.
--namespaces TEXT Comma-separated namespaces to operate.
--exclude TEXT Comma-separated projects (slug) to exclude.
--help Show this message and exit.
```
### Development run
```shell
python3 -m 'src.main' --help
```
Raw data
{
"_id": null,
"home_page": "https://github.com/akimrx/gitlab-dumper",
"name": "gitlab-dumper",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "gitlab",
"author": "akimrx",
"author_email": "akimstrong@yandex.ru",
"download_url": "https://files.pythonhosted.org/packages/74/1a/40925d5b895707129389403d0274c876eb19802b6cf2c24aceb823f3bf69/gitlab-dumper-0.1.1.tar.gz",
"platform": "osx",
"description": "# Gitlab dumper\n\nA simple CLI tool for creating a dump of all projects in your Gitlab instance.\n \nSupports dump as raw repo and archives.\n\n## Preflight\n\nGo to `https://<YOUR_GITLAB_ENDPOINT>/-/profile/personal_access_tokens` and create personal token with permissions:\n- api\n- read_api\n- read_repository\n\nIf you want to see all repositories, it may make sense to grant administrator permissions for the access token.\n\n### Configuration\n\nCreate `.env` file and put credentials:\n```bash\n\nLOG_LEVEL=\"info\"\nGITLAB_URL=\"https://<YOUR_GITLAB_ENDPOINT>\"\nGITLAB_PERSONAL_TOKEN=\"xxxxxx\" # also, GITLAB_OAUTH_TOKEN supports\n```\n\nOr set environment variables directly:\n```bash\nexport LOG_LEVEL=\"info\"\nexport GITLAB_URL=\"https://<YOUR_GITLAB_ENDPOINT>\"\nexport GITLAB_PERSONAL_TOKEN=\"xxxxxx\"\n```\n\n**Requirements:**\n- Python >= 3.10\n- The git command line tool must be installed on your PC\n- The utility clones over SSH, so make sure that your public key is added to the GitLab profile, **[read the doc](https://docs.gitlab.com/ee/user/ssh.html)**\n - If you are having problems with the SSH key, just run the dump command with the `--as-archive` flag, it does not require SSH.\n\n\n### Install\n\nUsing pip:\n```shell\npython3 -m venv venv\nsource venv/bin/activate\npip install gitlab-dumper\n\ngitlab-dumper --help\n```\n\n\nInstalling from source:\n```shell\ngit clone https://github.com/akimrx/gitlab-dumper --recursive\ncd gitlab-dumper\npython3 -m venv venv\nsource venv/bin/activate\nmake install # or: python3 setup.py install\n\ngitlab-dumper --help\n```\n\n\n## Usage\n\n```\nCommands:\n groups Operations with Gitlab groups.\n projects Operations with Gitlab projects.\n tree Show groups, subgroups and projects as tree.\n\nGroups commands:\n list Show available Gitlab groups.\n projects List projects in group or subgroup.\n\nProjects commands:\n dump Download, clone or re-pull all available projects.\n list Show available Gitlab projects.\n```\n\n#### Dump\n\nBefore really dump use `--dry-run` flag, for example:\n```shell\ngitlab-dumper projects dump --namespaces foo,bar --no-personal --dry-run\n```\n \nFor full dump just use `gitlab-dumper projects dump`\n\n\nAvailable optional flags: \n```\n --dumps-dir TEXT Directory for dumps (default: ./dumps).\n --delay INTEGER Delay between clones in seconds (default 0).\n --skip-empty Ignore empty projects.\n --no-personal Ignore personal user projects.\n --as-archive Download projects as tar.gz archive instead clone.\n --dry-run Safe simulate dump without download/clone.\n --namespaces TEXT Comma-separated namespaces to operate.\n --exclude TEXT Comma-separated projects (slug) to exclude.\n --help Show this message and exit.\n```\n\n### Development run\n\n```shell\npython3 -m 'src.main' --help\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "CLI tool for dump Gitlab projects",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/akimrx/gitlab-dumper"
},
"split_keywords": [
"gitlab"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "79fc46ef6f3141aaa5a24e74745042c9311dcc4c8f0a0353b94c59eddd1c5720",
"md5": "608c31f3ec845b10bc782c9b193fdcdd",
"sha256": "302d6d3f642188f008900c20819e998b53fd33d84b705c8f28d6d333b1e3ff0d"
},
"downloads": -1,
"filename": "gitlab_dumper-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "608c31f3ec845b10bc782c9b193fdcdd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 11894,
"upload_time": "2024-03-17T07:30:11",
"upload_time_iso_8601": "2024-03-17T07:30:11.995876Z",
"url": "https://files.pythonhosted.org/packages/79/fc/46ef6f3141aaa5a24e74745042c9311dcc4c8f0a0353b94c59eddd1c5720/gitlab_dumper-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "741a40925d5b895707129389403d0274c876eb19802b6cf2c24aceb823f3bf69",
"md5": "e027ae63a444fd82a7a1c35bc09fd02b",
"sha256": "bc58fd0e0b0a85a4f1aa2f71444ae0053f0aa4bee98ef9170778da23457eb391"
},
"downloads": -1,
"filename": "gitlab-dumper-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "e027ae63a444fd82a7a1c35bc09fd02b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 9957,
"upload_time": "2024-03-17T07:30:13",
"upload_time_iso_8601": "2024-03-17T07:30:13.792213Z",
"url": "https://files.pythonhosted.org/packages/74/1a/40925d5b895707129389403d0274c876eb19802b6cf2c24aceb823f3bf69/gitlab-dumper-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-17 07:30:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "akimrx",
"github_project": "gitlab-dumper",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "gitlab-dumper"
}
JSON
