# git-dumper
A tool to dump a git repository from a website.
## Install
This can be installed easily with pip:
```
pip install git-dumper
```
## Usage
```
usage: git-dumper [options] URL DIR
Dump a git repository from a website.
positional arguments:
URL url
DIR output directory
optional arguments:
-h, --help show this help message and exit
--proxy PROXY use the specified proxy
-j JOBS, --jobs JOBS number of simultaneous requests
-r RETRY, --retry RETRY
number of request attempts before giving up
-t TIMEOUT, --timeout TIMEOUT
maximum time in seconds before giving up
-u USER_AGENT, --user-agent USER_AGENT
user-agent to use for requests
-H HEADER, --header HEADER
additional http headers, e.g `NAME=VALUE`
--client-cert-p12 CLIENT_CERT_P12
client certificate in PKCS#12 format
--client-cert-p12-password CLIENT_CERT_P12_PASSWORD
password for the client certificate
```
### Example
```
git-dumper http://website.com/.git ~/website
```
### Disclaimer
**Use this software at your own risk!**
You should know that if the repository you are downloading is controlled by an attacker,
this could lead to remote code execution on your machine.
## Build from source
Simply install the dependencies with pip:
```
pip install -r requirements.txt
```
Then, simply use:
```
./git_dumper.py http://website.com/.git ~/website
```
## How does it work?
The tool will first check if directory listing is available. If it is, then it will just recursively download the .git directory (what you would do with `wget`).
If directory listing is not available, it will use several methods to find as many files as possible. Step by step, git-dumper will:
* Fetch all common files (`.gitignore`, `.git/HEAD`, `.git/index`, etc.);
* Find as many refs as possible (such as `refs/heads/master`, `refs/remotes/origin/HEAD`, etc.) by analyzing `.git/HEAD`, `.git/logs/HEAD`, `.git/config`, `.git/packed-refs` and so on;
* Find as many objects (sha1) as possible by analyzing `.git/packed-refs`, `.git/index`, `.git/refs/*` and `.git/logs/*`;
* Fetch all objects recursively, analyzing each commits to find their parents;
* Run `git checkout .` to recover the current working tree
Raw data
{
"_id": null,
"home_page": "https://github.com/arthaud/git-dumper",
"name": "git-dumper",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.0",
"maintainer_email": null,
"keywords": "dump, git, repository, security, vulnerability, ctf",
"author": "Maxime Arthaud",
"author_email": "maxime@arthaud.me",
"download_url": "https://files.pythonhosted.org/packages/24/02/ebad601361797096ebb074e27bcdddddcef84be43aca321c4c2055d78c7d/git_dumper-1.0.8.tar.gz",
"platform": null,
"description": "# git-dumper\n\nA tool to dump a git repository from a website.\n\n## Install\n\nThis can be installed easily with pip:\n```\npip install git-dumper\n```\n\n## Usage\n\n```\nusage: git-dumper [options] URL DIR\n\nDump a git repository from a website.\n\npositional arguments:\n URL url\n DIR output directory\n\noptional arguments:\n -h, --help show this help message and exit\n --proxy PROXY use the specified proxy\n -j JOBS, --jobs JOBS number of simultaneous requests\n -r RETRY, --retry RETRY\n number of request attempts before giving up\n -t TIMEOUT, --timeout TIMEOUT\n maximum time in seconds before giving up\n -u USER_AGENT, --user-agent USER_AGENT\n user-agent to use for requests\n -H HEADER, --header HEADER\n additional http headers, e.g `NAME=VALUE`\n --client-cert-p12 CLIENT_CERT_P12\n client certificate in PKCS#12 format\n --client-cert-p12-password CLIENT_CERT_P12_PASSWORD\n password for the client certificate\n```\n\n### Example\n\n```\ngit-dumper http://website.com/.git ~/website\n```\n\n### Disclaimer\n\n**Use this software at your own risk!**\n\nYou should know that if the repository you are downloading is controlled by an attacker,\nthis could lead to remote code execution on your machine.\n\n## Build from source\n\nSimply install the dependencies with pip:\n```\npip install -r requirements.txt\n```\n\nThen, simply use:\n```\n./git_dumper.py http://website.com/.git ~/website\n```\n\n## How does it work?\n\nThe tool will first check if directory listing is available. If it is, then it will just recursively download the .git directory (what you would do with `wget`).\n\nIf directory listing is not available, it will use several methods to find as many files as possible. Step by step, git-dumper will:\n* Fetch all common files (`.gitignore`, `.git/HEAD`, `.git/index`, etc.);\n* Find as many refs as possible (such as `refs/heads/master`, `refs/remotes/origin/HEAD`, etc.) by analyzing `.git/HEAD`, `.git/logs/HEAD`, `.git/config`, `.git/packed-refs` and so on;\n* Find as many objects (sha1) as possible by analyzing `.git/packed-refs`, `.git/index`, `.git/refs/*` and `.git/logs/*`;\n* Fetch all objects recursively, analyzing each commits to find their parents;\n* Run `git checkout .` to recover the current working tree\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A tool to dump a git repository from a website",
"version": "1.0.8",
"project_urls": {
"Homepage": "https://github.com/arthaud/git-dumper"
},
"split_keywords": [
"dump",
" git",
" repository",
" security",
" vulnerability",
" ctf"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "634f7d9aad6ea0f6be1037b31c1d2792fe1c75e564af52b0851c2c1166d899f6",
"md5": "58484400a503e6edce086259bd28dbe3",
"sha256": "83aab0dfa054ff18a858fdeb24f1a8c0d9648f0ab5e1acbeb1493cf5e4438bce"
},
"downloads": -1,
"filename": "git_dumper-1.0.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "58484400a503e6edce086259bd28dbe3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.0",
"size": 9418,
"upload_time": "2024-11-23T18:40:36",
"upload_time_iso_8601": "2024-11-23T18:40:36.635876Z",
"url": "https://files.pythonhosted.org/packages/63/4f/7d9aad6ea0f6be1037b31c1d2792fe1c75e564af52b0851c2c1166d899f6/git_dumper-1.0.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2402ebad601361797096ebb074e27bcdddddcef84be43aca321c4c2055d78c7d",
"md5": "6c71e48ab780d0d27be23d3ef8510c0a",
"sha256": "31510d4ed965729697d9b6f1bb1e9ad502b9b026c2e48b788efa84ce6dab868a"
},
"downloads": -1,
"filename": "git_dumper-1.0.8.tar.gz",
"has_sig": false,
"md5_digest": "6c71e48ab780d0d27be23d3ef8510c0a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.0",
"size": 9192,
"upload_time": "2024-11-23T18:40:38",
"upload_time_iso_8601": "2024-11-23T18:40:38.295692Z",
"url": "https://files.pythonhosted.org/packages/24/02/ebad601361797096ebb074e27bcdddddcef84be43aca321c4c2055d78c7d/git_dumper-1.0.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-23 18:40:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "arthaud",
"github_project": "git-dumper",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "PySocks",
"specs": []
},
{
"name": "requests",
"specs": []
},
{
"name": "beautifulsoup4",
"specs": []
},
{
"name": "dulwich",
"specs": []
},
{
"name": "requests-pkcs12",
"specs": []
}
],
"lcname": "git-dumper"
}
JSON
