[](https://github.com/EBI-Metagenomics/fetch_tool/actions/workflows/test.yml)
[](https://badge.fury.io/py/fetch-tool)
[](https://quay.io/repository/microbiome-informatics/fetch-tool)
# Microbiome Informatics ENA fetch tool
Set of tools which allows you to fetch RAW read and assembly files from the European Nucleotide Archive (ENA).
## Install fetch tool
Install from Pypi
```bash
$ pip install fetch-tool
```
Install from the git repo
```bash
$ pip install https://github.com/EBI-Metagenomics/fetch_tool/archive/master.zip
```
#### Configuration options
The tool has a number of options, with sensible defaults for the most common use cases.
Setup the configuration file, the template [fetchdata-config-template.json](config/fetchdata-config-template.json) for the configuration file.
The required fields are:
-
- ena_api_user
- ena_api_password
## Fetch read files (amplicon and WGS data)
### Usage
```bash
$ fetch-read-tool -h
usage: fetch-read-tool [-h] [-p PROJECTS [PROJECTS ...] | -l PROJECT_LIST] [-d DIR] [-v] [--version] [-f] [--ignore-errors] [--private] [-i] [-c CONFIG_FILE] [--fix-desc-file] [-ru RUNS [RUNS ...]
| --run-list RUN_LIST]
optional arguments:
-h, --help show this help message and exit
-p PROJECTS [PROJECTS ...], --projects PROJECTS [PROJECTS ...]
Whitespace separated list of project accession(s)
-l PROJECT_LIST, --project-list PROJECT_LIST
File containing line-separated project list
-d DIR, --dir DIR Base directory for downloads
-v, --verbose Verbose
--version Version
-f, --force Ignore download errors and force re-download all files
--ignore-errors Ignore download errors and continue
--private Use when fetching private data
-i, --interactive interactive mode - allows you to skip failed downloads.
-c CONFIG_FILE, --config-file CONFIG_FILE
Alternative config file
--fix-desc-file Fixed runs in project description file
-ru RUNS [RUNS ...], --runs RUNS [RUNS ...]
Run accession(s), whitespace separated. Use to download only certain project runs
--run-list RUN_LIST File containing line-separated run accessions
```
### Example
Download amplicon study:
```bash
$ fetch-read-tool -p SRP062869 -v -d /home/<user>/temp/
```
## Fetch assembly files
### Usage
```
fetch-assembly-tool -h
usage: fetch-assembly-tool [-h] [-p PROJECTS [PROJECTS ...] | -l PROJECT_LIST] [-d DIR] [-v] [--version] [-f] [--ignore-errors] [--private] [-i] [-c CONFIG_FILE] [--fix-desc-file]
[-as ASSEMBLIES [ASSEMBLIES ...]] [--assembly-type {primary metagenome,binned metagenome,metatranscriptome}] [--assembly-list ASSEMBLY_LIST]
optional arguments:
-h, --help show this help message and exit
-p PROJECTS [PROJECTS ...], --projects PROJECTS [PROJECTS ...]
Whitespace separated list of project accession(s)
-l PROJECT_LIST, --project-list PROJECT_LIST
File containing line-separated project list
-d DIR, --dir DIR Base directory for downloads
-v, --verbose Verbose
--version Version
-f, --force Ignore download errors and force re-download all files
--ignore-errors Ignore download errors and continue
--private Use when fetching private data
-i, --interactive interactive mode - allows you to skip failed downloads.
-c CONFIG_FILE, --config-file CONFIG_FILE
Alternative config file
--fix-desc-file Fixed runs in project description file
-as ASSEMBLIES [ASSEMBLIES ...], --assemblies ASSEMBLIES [ASSEMBLIES ...]
Assembly ERZ accession(s), whitespace separated. Use to download only certain project assemblies
--assembly-type {primary metagenome,binned metagenome,metatranscriptome}
Assembly type
--assembly-list ASSEMBLY_LIST
File containing line-separated assembly accessions
```
### Example
Download assembly study:
```bash
$ fetch-assembly-tool -p ERP111288 -v -d /home/<user>/temp/
```
# How to set up your development environment
We recommend you to use [miniconda|conda](https://docs.conda.io/en/latest/miniconda.html) to manage the environment.
Clone the repo and install the requirements.
```
$ git clone git@github.com:EBI-Metagenomics/fetch_tool.git
$ cd fetch_tool
$ # activate anv (conda activate xxx)
$ pip install .[dev]
```
## Pre-commit hooks
Setup the git [pre-commit hook](https://pre-commit.com/):
```bash
pre-commit install
```
*Why?*
pre-commit will run a set of pre-configured tools before allowing you to commit files. You can find the currently configure hooks and configurations in [.pre-commit-config.yaml](./.pre-commit-config.yaml)
## Tests
This repo uses [pytest](https://docs.pytest.org).
Raw data
{
"_id": null,
"home_page": null,
"name": "fetch-tool",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "bioinformatics, tool, metagenomics",
"author": null,
"author_email": "MGnify team <metagenomics-help@ebi.ac.uk>",
"download_url": "https://files.pythonhosted.org/packages/c2/5a/fff4758be5b66ee5fcaaee8b57f73daeaf57efe186cf2ae637c40b9e30ed/fetch_tool-1.0.3.tar.gz",
"platform": null,
"description": "[](https://github.com/EBI-Metagenomics/fetch_tool/actions/workflows/test.yml)\n[](https://badge.fury.io/py/fetch-tool)\n[![Docker Repository on Quay](https://quay.io/repository/microbiome-informatics/fetch-tool/status \"Docker Repository on Quay\")](https://quay.io/repository/microbiome-informatics/fetch-tool)\n\n# Microbiome Informatics ENA fetch tool\n\nSet of tools which allows you to fetch RAW read and assembly files from the European Nucleotide Archive (ENA).\n\n## Install fetch tool\n\nInstall from Pypi\n\n```bash\n$ pip install fetch-tool\n```\n\nInstall from the git repo\n\n```bash\n$ pip install https://github.com/EBI-Metagenomics/fetch_tool/archive/master.zip\n```\n\n#### Configuration options\n\nThe tool has a number of options, with sensible defaults for the most common use cases.\n\nSetup the configuration file, the template [fetchdata-config-template.json](config/fetchdata-config-template.json) for the configuration file.\n\nThe required fields are:\n-\n - ena_api_user\n - ena_api_password\n\n## Fetch read files (amplicon and WGS data)\n\n### Usage\n\n```bash\n$ fetch-read-tool -h\nusage: fetch-read-tool [-h] [-p PROJECTS [PROJECTS ...] | -l PROJECT_LIST] [-d DIR] [-v] [--version] [-f] [--ignore-errors] [--private] [-i] [-c CONFIG_FILE] [--fix-desc-file] [-ru RUNS [RUNS ...]\n | --run-list RUN_LIST]\n\noptional arguments:\n -h, --help show this help message and exit\n -p PROJECTS [PROJECTS ...], --projects PROJECTS [PROJECTS ...]\n Whitespace separated list of project accession(s)\n -l PROJECT_LIST, --project-list PROJECT_LIST\n File containing line-separated project list\n -d DIR, --dir DIR Base directory for downloads\n -v, --verbose Verbose\n --version Version\n -f, --force Ignore download errors and force re-download all files\n --ignore-errors Ignore download errors and continue\n --private Use when fetching private data\n -i, --interactive interactive mode - allows you to skip failed downloads.\n -c CONFIG_FILE, --config-file CONFIG_FILE\n Alternative config file\n --fix-desc-file Fixed runs in project description file\n -ru RUNS [RUNS ...], --runs RUNS [RUNS ...]\n Run accession(s), whitespace separated. Use to download only certain project runs\n --run-list RUN_LIST File containing line-separated run accessions\n```\n\n### Example\n\nDownload amplicon study:\n\n```bash\n$ fetch-read-tool -p SRP062869 -v -d /home/<user>/temp/\n```\n\n## Fetch assembly files\n\n### Usage\n\n```\nfetch-assembly-tool -h\nusage: fetch-assembly-tool [-h] [-p PROJECTS [PROJECTS ...] | -l PROJECT_LIST] [-d DIR] [-v] [--version] [-f] [--ignore-errors] [--private] [-i] [-c CONFIG_FILE] [--fix-desc-file]\n [-as ASSEMBLIES [ASSEMBLIES ...]] [--assembly-type {primary metagenome,binned metagenome,metatranscriptome}] [--assembly-list ASSEMBLY_LIST]\n\noptional arguments:\n -h, --help show this help message and exit\n -p PROJECTS [PROJECTS ...], --projects PROJECTS [PROJECTS ...]\n Whitespace separated list of project accession(s)\n -l PROJECT_LIST, --project-list PROJECT_LIST\n File containing line-separated project list\n -d DIR, --dir DIR Base directory for downloads\n -v, --verbose Verbose\n --version Version\n -f, --force Ignore download errors and force re-download all files\n --ignore-errors Ignore download errors and continue\n --private Use when fetching private data\n -i, --interactive interactive mode - allows you to skip failed downloads.\n -c CONFIG_FILE, --config-file CONFIG_FILE\n Alternative config file\n --fix-desc-file Fixed runs in project description file\n -as ASSEMBLIES [ASSEMBLIES ...], --assemblies ASSEMBLIES [ASSEMBLIES ...]\n Assembly ERZ accession(s), whitespace separated. Use to download only certain project assemblies\n --assembly-type {primary metagenome,binned metagenome,metatranscriptome}\n Assembly type\n --assembly-list ASSEMBLY_LIST\n File containing line-separated assembly accessions\n```\n\n### Example\n\nDownload assembly study:\n\n```bash\n$ fetch-assembly-tool -p ERP111288 -v -d /home/<user>/temp/\n```\n\n# How to set up your development environment\n\nWe recommend you to use [miniconda|conda](https://docs.conda.io/en/latest/miniconda.html) to manage the environment.\n\nClone the repo and install the requirements.\n\n```\n$ git clone git@github.com:EBI-Metagenomics/fetch_tool.git\n$ cd fetch_tool\n$ # activate anv (conda activate xxx)\n$ pip install .[dev]\n```\n\n## Pre-commit hooks\n\nSetup the git [pre-commit hook](https://pre-commit.com/):\n\n```bash\npre-commit install\n```\n\n*Why?*\n\npre-commit will run a set of pre-configured tools before allowing you to commit files. You can find the currently configure hooks and configurations in [.pre-commit-config.yaml](./.pre-commit-config.yaml)\n\n## Tests\n\nThis repo uses [pytest](https://docs.pytest.org).\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "Utility to fetch public and private RAW read and assembly files from the ENA",
"version": "1.0.3",
"project_urls": null,
"split_keywords": [
"bioinformatics",
" tool",
" metagenomics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8de7b638a81b1c6757db271f7a8b028f811a26d8bc6cf91d5054d1133ebc93b0",
"md5": "bce46a4da9bffa98023dff2bf650bc8f",
"sha256": "2f4f96b9da034e1f81ce19e12668e0895221bfcf051a9d168b816e3d077cadaf"
},
"downloads": -1,
"filename": "fetch_tool-1.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bce46a4da9bffa98023dff2bf650bc8f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 21427,
"upload_time": "2024-10-03T15:35:41",
"upload_time_iso_8601": "2024-10-03T15:35:41.997724Z",
"url": "https://files.pythonhosted.org/packages/8d/e7/b638a81b1c6757db271f7a8b028f811a26d8bc6cf91d5054d1133ebc93b0/fetch_tool-1.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c25afff4758be5b66ee5fcaaee8b57f73daeaf57efe186cf2ae637c40b9e30ed",
"md5": "b0e3a98630f975d0c0dfad86b1e19437",
"sha256": "6627d6138069cb2e4ab49db0c00400b0ae162d2d7dace4cfb90a29fb13eae220"
},
"downloads": -1,
"filename": "fetch_tool-1.0.3.tar.gz",
"has_sig": false,
"md5_digest": "b0e3a98630f975d0c0dfad86b1e19437",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 17811,
"upload_time": "2024-10-03T15:35:43",
"upload_time_iso_8601": "2024-10-03T15:35:43.753201Z",
"url": "https://files.pythonhosted.org/packages/c2/5a/fff4758be5b66ee5fcaaee8b57f73daeaf57efe186cf2ae637c40b9e30ed/fetch_tool-1.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-03 15:35:43",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "fetch-tool"
}
JSON