==========
GenomeHubs
==========
About
=====
GenomeHubs comprises a set of tools to parse index and search and display genomic metadata, assembly features and sequencing status for projects under the `Earth BioGenome Project <https://www.earthbiogenome.org>`_ umbrella that aim to sequence all described eukaryotic species over a period of 10 years.
Genomehubs builds on legacy code that supported taxon-oriented databases of butterflies & moths (`lepbase.org <https://lepbase.org>`_), molluscs (`molluscdb.org <https://molluscdb.org>`_), mealybugs (`mealybug.org <https://mealybug.org>`_) and more. Genomehubs is now search-oriented and positioned to scale to the challenges of mining data across almost 2 million species.
The first output from the new search-oriented GenomeHubs is Genomes on a Tree (GoaT, `goat.genomehubs.org <https://goat.genomehubs.org>`_), which has been opublised in: Challis *et al*. 2023, **Genomes on a Tree (GoaT): A versatile, scalable search engine for genomic and sequencing project metadata across the eukaryotic tree of life**. Wellcome Open Research, **8**:24 doi:`10.12688/wellcomeopenres.18658.1 <https://doi.org/10.12688/wellcomeopenres.18658.1>`_
The goat.genomehubs.org website is freely available with no logins or restrictions, and is being widely used by the academic community and especially by the Earth BioGenome Project to plan and coordinate efforts to sequence all described eukaryotic species.
The core GoaT/Genomehubs components are available as a set of Docker containers:
GoaT UI |goat-docker|
---------------------
A bundled web server to run a GoaT-specific instance of the GenomeHubs UI, as used at `goat.genomehubs.org <https://goat.genomehubs.org>`_.
Usage
^^^^^
.. code-block:: bash
docker pull genomehubs/goat:latest
docker run -d --restart always \
--net net-es -p 8880:8880 \
--user $UID:$GROUPS \
-e GH_CLIENT_PORT=8880 \
-e GH_API_URL=https://goat.genomehubs.org/api/v2 \
-e GH_SUGGESTED_TERM=Canidae \
--name goat-ui \
genomehubs/goat:latest
.. |goat-docker| image:: https://img.shields.io/docker/v/genomehubs/goat/latest?label=docker%20hub&style=flat-square
:alt: Docker image
:target: https://hub.docker.com/r/genomehubs/goat
Genomehubs UI |ui-docker|
-------------------------
A bundled web server to run an instance of the GenomeHubs UI, such as `goat.genomehubs.org <https://goat.genomehubs.org>`_.
Usage
^^^^^
.. code-block:: bash
docker pull genomehubs/genomehubs-ui:latest
docker run -d --restart always \
--net net-es -p 8880:8880 \
--user $UID:$GROUPS \
-e GH_CLIENT_PORT=8880 \
-e GH_API_URL=https://goat.genomehubs.org/api/v2 \
-e GH_SUGGESTED_TERM=Canidae \
--name gh-ui \
genomehubs/genomehubs-ui:latest
.. |ui-docker| image:: https://img.shields.io/docker/v/genomehubs/genomehubs-ui/latest?label=docker%20hub&style=flat-square
:alt: Docker image
:target: https://hub.docker.com/r/genomehubs/genomehubs-ui
Genomehubs API |api-docker|
---------------------------
A bundled web server to run an instance of the GenomeHubs API. The GenomeHubs API underpins all search functionality for Genomes on a Tree (GoaT) `goat.genomehubs.org <https://goat.genomehubs.org>`_. OpenAPI documentation for the GenomeHubs API instance used by GoaT is available at `goat.genomehubs.org/api-docs <https://goat.genomehubs.org/api-docs>`_.
Usage
^^^^^
.. code-block:: bash
docker pull genomehubs/genomehubs-api:latest
docker run -d \
--restart always \
--net net-es -p 3000:3000 \
--user $UID:$GROUPS \
-e GH_ORIGINS="https://goat.genomehubs.org null" \
-e GH_HUBNAME=goat \
-e GH_HUBPATH="/genomehubs/resources/" \
-e GH_NODE="http://es1:9200" \
-e GH_API_URL=https://goat.genomehubs.org/api/v2 \
-e GH_RELEASE=$RELEASE \
-e GH_SOURCE=https://github.com/genomehubs/goat-data \
-e GH_ACCESS_LOG=/genomehubs/logs/access.log \
-e GH_ERROR_LOG=/genomehubs/logs/error.log \
-v /volumes/docker/logs/$RELEASE:/genomehubs/logs \
-v /volumes/docker/resources:/genomehubs/resources \
--name goat-api \
genomehubs/genomehubs-api:latest;
.. |api-docker| image:: https://img.shields.io/docker/v/genomehubs/genomehubs-api/latest?label=docker%20hub&style=flat-square
:alt: Docker image
:target: https://hub.docker.com/r/genomehubs/genomehubs-api
Genomehubs CLI |genomehubs-docker|
----------------------------------
command line tool to process and index genomic metadata for GenomeHubs. Used to build and update GenomeHubs instances such as Genomes on a Tree `goat.genomehubs.org <https://goat.genomehubs.org>`_.
Usage
^^^^^
.. code-block:: bash
docker pull genomehubs/genomehubs:latest
Parse [NCBI datasets](https://www.ncbi.nlm.nih.gov/datasets/) genome assembly metadata:
.. code-block:: bash
docker run --rm --network=host \
-v `pwd`/sources:/genomehubs/sources \
genomehubs/genomehubs:latest bash -c \
"genomehubs parse \
--ncbi-datasets-genome sources/assembly-data \
--outfile sources/assembly-data/ncbi_datasets_eukaryota.tsv.gz"
Initialise a set of ElasticSearch indexes with [NCBI taxonomy](https://www.ncbi.nlm.nih.gov/taxonomy/) data for all eukaryotes:
.. code-block:: bash
docker run --rm --network=host \
-v `pwd`/sources:/genomehubs/sources \
genomehubs/genomehubs:latest bash -c \
"genomehubs init \
--es-host http://es1:9200 \
--taxonomy-source ncbi \
--config-file sources/goat.yaml \
--taxonomy-jsonl sources/ena-taxonomy/ena-taxonomy.extra.jsonl.gz \
--taxonomy-ncbi-root 2759 \
--taxon-preload"
Index assembly metadata:
.. code-block:: bash
docker run --rm --network=host \
-v `pwd`/sources:/genomehubs/sources \
genomehubs/genomehubs:latest bash -c \
"genomehubs index \
--es-host http://es1:9200 \
--taxonomy-source ncbi \
--config-file sources/goat.yaml \
--assembly-dir sources/assembly-data"
Fill taxon attribute values across the tree of life:
.. code-block:: bash
docker run --rm --network=host \
-v `pwd`/sources:/genomehubs/sources \
genomehubs/genomehubs:latest bash -c \
"genomehubs fill \
--es-host http://es1:9200 \
--taxonomy-source ncbi \
--config-file sources/goat.yaml \
--traverse-root 2759 \
--traverse-infer-both"
.. |genomehubs-docker| image:: https://img.shields.io/docker/v/genomehubs/genomehubs/latest?label=docker%20hub&style=flat-square
:alt: Docker image
:target: https://hub.docker.com/r/genomehubs/genomehubs
Related projects
================
Some GenomeHubs components are hosted in separate open source repositories (all under MIT licenses), including:
BlobToolKit |blobtoolkit-release|
---------------------------------
Interactive quality assessment of genome assemblies.
Explore analysed public assemblies at `blobtoolkit.genomehubs.org/view <https://blobtoolkit.genomehubs.org/view>`_
.. |blobtoolkit-release| image:: https://img.shields.io/github/v/tag/blobtoolkit/blobtoolkit?label=release&sort=semver&style=flat-square
:alt: GitHub release
:target: https://github.com/blobtoolkit/blobtoolkit
GoaT CLI |goat-cli-release|
----------------------------
A command line interface for GoaT.
The GoaT CLI builds URLs to query the Goat API, removing some of the complexity of the `GoaT API <https://goat.genomehubs.org/api-docs>`_. for the end user.
.. |goat-cli-release| image:: https://img.shields.io/github/v/tag/genomehubs/goat-cli?label=release&sort=semver&style=flat-square
:alt: GitHub release
:target: https://github.com/genomehubs/goat-cli
Changelog
=========
2.0.0 (2020-07-02)
------------------
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/genomehubs/genomehubs",
"name": "genomehubs",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6, <4",
"maintainer_email": "",
"keywords": "bioinformatics",
"author": "genomehubs",
"author_email": "genomehubs@genomehubs.org",
"download_url": "https://files.pythonhosted.org/packages/75/af/f763032f0c6111c7090bd647966ce66ab5e2ec5a1da9c6ca3e083a86ca66/genomehubs-2.8.7.tar.gz",
"platform": null,
"description": "==========\nGenomeHubs\n==========\n\n\n\n\nAbout\n=====\n\nGenomeHubs comprises a set of tools to parse index and search and display genomic metadata, assembly features and sequencing status for projects under the `Earth BioGenome Project <https://www.earthbiogenome.org>`_ umbrella that aim to sequence all described eukaryotic species over a period of 10 years.\n\nGenomehubs builds on legacy code that supported taxon-oriented databases of butterflies & moths (`lepbase.org <https://lepbase.org>`_), molluscs (`molluscdb.org <https://molluscdb.org>`_), mealybugs (`mealybug.org <https://mealybug.org>`_) and more. Genomehubs is now search-oriented and positioned to scale to the challenges of mining data across almost 2 million species.\n\nThe first output from the new search-oriented GenomeHubs is Genomes on a Tree (GoaT, `goat.genomehubs.org <https://goat.genomehubs.org>`_), which has been opublised in: Challis *et al*. 2023, **Genomes on a Tree (GoaT): A versatile, scalable search engine for genomic and sequencing project metadata across the eukaryotic tree of life**. Wellcome Open Research, **8**:24 doi:`10.12688/wellcomeopenres.18658.1 <https://doi.org/10.12688/wellcomeopenres.18658.1>`_\n\nThe goat.genomehubs.org website is freely available with no logins or restrictions, and is being widely used by the academic community and especially by the Earth BioGenome Project to plan and coordinate efforts to sequence all described eukaryotic species.\n\nThe core GoaT/Genomehubs components are available as a set of Docker containers:\n\nGoaT UI |goat-docker|\n---------------------\n\nA bundled web server to run a GoaT-specific instance of the GenomeHubs UI, as used at `goat.genomehubs.org <https://goat.genomehubs.org>`_.\n\nUsage\n^^^^^\n.. code-block:: bash\n\n docker pull genomehubs/goat:latest\n \n docker run -d --restart always \\\n --net net-es -p 8880:8880 \\\n --user $UID:$GROUPS \\\n -e GH_CLIENT_PORT=8880 \\\n -e GH_API_URL=https://goat.genomehubs.org/api/v2 \\\n -e GH_SUGGESTED_TERM=Canidae \\\n --name goat-ui \\\n genomehubs/goat:latest\n\n.. |goat-docker| image:: https://img.shields.io/docker/v/genomehubs/goat/latest?label=docker%20hub&style=flat-square\n :alt: Docker image\n :target: https://hub.docker.com/r/genomehubs/goat\n\n\nGenomehubs UI |ui-docker|\n-------------------------\n\nA bundled web server to run an instance of the GenomeHubs UI, such as `goat.genomehubs.org <https://goat.genomehubs.org>`_.\n\nUsage\n^^^^^\n.. code-block:: bash\n\n docker pull genomehubs/genomehubs-ui:latest\n \n docker run -d --restart always \\\n --net net-es -p 8880:8880 \\\n --user $UID:$GROUPS \\\n -e GH_CLIENT_PORT=8880 \\\n -e GH_API_URL=https://goat.genomehubs.org/api/v2 \\\n -e GH_SUGGESTED_TERM=Canidae \\\n --name gh-ui \\\n genomehubs/genomehubs-ui:latest\n\n\n.. |ui-docker| image:: https://img.shields.io/docker/v/genomehubs/genomehubs-ui/latest?label=docker%20hub&style=flat-square\n :alt: Docker image\n :target: https://hub.docker.com/r/genomehubs/genomehubs-ui\n\n\nGenomehubs API |api-docker|\n---------------------------\n\nA bundled web server to run an instance of the GenomeHubs API. The GenomeHubs API underpins all search functionality for Genomes on a Tree (GoaT) `goat.genomehubs.org <https://goat.genomehubs.org>`_. OpenAPI documentation for the GenomeHubs API instance used by GoaT is available at `goat.genomehubs.org/api-docs <https://goat.genomehubs.org/api-docs>`_.\n\n\nUsage\n^^^^^\n.. code-block:: bash\n\n docker pull genomehubs/genomehubs-api:latest\n \n docker run -d \\\n --restart always \\\n --net net-es -p 3000:3000 \\\n --user $UID:$GROUPS \\\n -e GH_ORIGINS=\"https://goat.genomehubs.org null\" \\\n -e GH_HUBNAME=goat \\\n -e GH_HUBPATH=\"/genomehubs/resources/\" \\\n -e GH_NODE=\"http://es1:9200\" \\\n -e GH_API_URL=https://goat.genomehubs.org/api/v2 \\\n -e GH_RELEASE=$RELEASE \\\n -e GH_SOURCE=https://github.com/genomehubs/goat-data \\\n -e GH_ACCESS_LOG=/genomehubs/logs/access.log \\\n -e GH_ERROR_LOG=/genomehubs/logs/error.log \\\n -v /volumes/docker/logs/$RELEASE:/genomehubs/logs \\\n -v /volumes/docker/resources:/genomehubs/resources \\\n --name goat-api \\\n genomehubs/genomehubs-api:latest;\n\n.. |api-docker| image:: https://img.shields.io/docker/v/genomehubs/genomehubs-api/latest?label=docker%20hub&style=flat-square\n :alt: Docker image\n :target: https://hub.docker.com/r/genomehubs/genomehubs-api\n\n\nGenomehubs CLI |genomehubs-docker|\n----------------------------------\n\ncommand line tool to process and index genomic metadata for GenomeHubs. Used to build and update GenomeHubs instances such as Genomes on a Tree `goat.genomehubs.org <https://goat.genomehubs.org>`_.\n\nUsage\n^^^^^\n.. code-block:: bash\n\n docker pull genomehubs/genomehubs:latest\n\nParse [NCBI datasets](https://www.ncbi.nlm.nih.gov/datasets/) genome assembly metadata:\n\n.. code-block:: bash\n\n docker run --rm --network=host \\\n -v `pwd`/sources:/genomehubs/sources \\\n genomehubs/genomehubs:latest bash -c \\\n \"genomehubs parse \\\n --ncbi-datasets-genome sources/assembly-data \\\n --outfile sources/assembly-data/ncbi_datasets_eukaryota.tsv.gz\"\n\n\nInitialise a set of ElasticSearch indexes with [NCBI taxonomy](https://www.ncbi.nlm.nih.gov/taxonomy/) data for all eukaryotes:\n\n.. code-block:: bash\n\n docker run --rm --network=host \\\n -v `pwd`/sources:/genomehubs/sources \\\n genomehubs/genomehubs:latest bash -c \\\n \"genomehubs init \\\n --es-host http://es1:9200 \\\n --taxonomy-source ncbi \\\n --config-file sources/goat.yaml \\\n --taxonomy-jsonl sources/ena-taxonomy/ena-taxonomy.extra.jsonl.gz \\\n --taxonomy-ncbi-root 2759 \\\n --taxon-preload\"\n\nIndex assembly metadata:\n\n.. code-block:: bash\n\n docker run --rm --network=host \\\n -v `pwd`/sources:/genomehubs/sources \\\n genomehubs/genomehubs:latest bash -c \\\n \"genomehubs index \\\n --es-host http://es1:9200 \\\n --taxonomy-source ncbi \\\n --config-file sources/goat.yaml \\\n --assembly-dir sources/assembly-data\"\n\nFill taxon attribute values across the tree of life:\n\n.. code-block:: bash\n\n docker run --rm --network=host \\\n -v `pwd`/sources:/genomehubs/sources \\\n genomehubs/genomehubs:latest bash -c \\\n \"genomehubs fill \\\n --es-host http://es1:9200 \\\n --taxonomy-source ncbi \\\n --config-file sources/goat.yaml \\\n --traverse-root 2759 \\\n --traverse-infer-both\"\n\n\n.. |genomehubs-docker| image:: https://img.shields.io/docker/v/genomehubs/genomehubs/latest?label=docker%20hub&style=flat-square\n :alt: Docker image\n :target: https://hub.docker.com/r/genomehubs/genomehubs\n\n\nRelated projects\n================\n\nSome GenomeHubs components are hosted in separate open source repositories (all under MIT licenses), including:\n\nBlobToolKit |blobtoolkit-release|\n---------------------------------\n\nInteractive quality assessment of genome assemblies.\n\nExplore analysed public assemblies at `blobtoolkit.genomehubs.org/view <https://blobtoolkit.genomehubs.org/view>`_\n\n.. |blobtoolkit-release| image:: https://img.shields.io/github/v/tag/blobtoolkit/blobtoolkit?label=release&sort=semver&style=flat-square\n :alt: GitHub release\n :target: https://github.com/blobtoolkit/blobtoolkit\n\nGoaT CLI |goat-cli-release|\n----------------------------\n\nA command line interface for GoaT.\n\nThe GoaT CLI builds URLs to query the Goat API, removing some of the complexity of the `GoaT API <https://goat.genomehubs.org/api-docs>`_. for the end user.\n\n.. |goat-cli-release| image:: https://img.shields.io/github/v/tag/genomehubs/goat-cli?label=release&sort=semver&style=flat-square\n :alt: GitHub release\n :target: https://github.com/genomehubs/goat-cli\n\n\nChangelog\n=========\n\n2.0.0 (2020-07-02)\n------------------\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": "",
"summary": "GenomeHubs",
"version": "2.8.7",
"project_urls": {
"Bug Reports": "https://github.com/genomehubs/genomehubs/issues",
"Homepage": "https://github.com/genomehubs/genomehubs",
"Source": "https://github.com/genomehubs/genomehubs"
},
"split_keywords": [
"bioinformatics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8f89d011019931a61a30bd7ce10a0ae53b8290b33b83ee737d78376d5fb995c7",
"md5": "a90e3c4e8f12cac2b0334a1ad2c1e3f0",
"sha256": "65899198c6f1c76d8dd22ed0004193ea6f0f75d438ffc10942b7b1c117ba280d"
},
"downloads": -1,
"filename": "genomehubs-2.8.7-py3-none-manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "a90e3c4e8f12cac2b0334a1ad2c1e3f0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6, <4",
"size": 146125,
"upload_time": "2024-03-14T16:23:22",
"upload_time_iso_8601": "2024-03-14T16:23:22.141570Z",
"url": "https://files.pythonhosted.org/packages/8f/89/d011019931a61a30bd7ce10a0ae53b8290b33b83ee737d78376d5fb995c7/genomehubs-2.8.7-py3-none-manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "75aff763032f0c6111c7090bd647966ce66ab5e2ec5a1da9c6ca3e083a86ca66",
"md5": "91c477f7eee392c9c6df39684e479ab0",
"sha256": "450570ebccd3925b294b151742906525a1350ca3aca37794574259663ada35e9"
},
"downloads": -1,
"filename": "genomehubs-2.8.7.tar.gz",
"has_sig": false,
"md5_digest": "91c477f7eee392c9c6df39684e479ab0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6, <4",
"size": 118018,
"upload_time": "2024-03-14T16:23:24",
"upload_time_iso_8601": "2024-03-14T16:23:24.823993Z",
"url": "https://files.pythonhosted.org/packages/75/af/f763032f0c6111c7090bd647966ce66ab5e2ec5a1da9c6ca3e083a86ca66/genomehubs-2.8.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-14 16:23:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "genomehubs",
"github_project": "genomehubs",
"travis_ci": true,
"coveralls": true,
"github_actions": true,
"requirements": [
{
"name": "pycodestyle",
"specs": [
[
">=",
"2.6.0"
]
]
},
{
"name": "pydocstyle",
"specs": [
[
">=",
"5.0.2"
]
]
},
{
"name": "pylint",
"specs": [
[
">=",
"2.5.3"
]
]
},
{
"name": "coverage",
"specs": [
[
">=",
"5.1"
]
]
},
{
"name": "coveralls",
"specs": [
[
">=",
"2.0.0"
]
]
},
{
"name": "mock",
"specs": [
[
">=",
"4.0.2"
]
]
},
{
"name": "pytest-cov",
"specs": [
[
">=",
"2.10.0"
]
]
},
{
"name": "pytest-isort",
"specs": [
[
">=",
"1.1.0"
]
]
},
{
"name": "pytest-mock",
"specs": [
[
">=",
"3.1.1"
]
]
},
{
"name": "pytest",
"specs": [
[
">=",
"6.0.0"
]
]
},
{
"name": "biopython",
"specs": [
[
">=",
"1.78"
]
]
},
{
"name": "docopt",
"specs": [
[
">=",
"0.6.2"
]
]
},
{
"name": "elasticsearch",
"specs": [
[
"==",
"8.7"
]
]
},
{
"name": "fastjsonschema",
"specs": [
[
">=",
"2.15.3"
]
]
},
{
"name": "filetype",
"specs": [
[
">=",
"1.0.7"
]
]
},
{
"name": "h3",
"specs": [
[
">=",
"3.7.4"
]
]
},
{
"name": "Pillow",
"specs": [
[
">=",
"8.0"
]
]
},
{
"name": "pyyaml",
"specs": []
},
{
"name": "sparqlwrapper",
"specs": [
[
">=",
"1.4.1"
]
]
},
{
"name": "tolkein",
"specs": [
[
">=",
"0.5.0"
]
]
},
{
"name": "ujson",
"specs": [
[
">=",
"3.0.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
">=",
"1.26.2"
],
[
"<",
"2"
]
]
}
],
"tox": true,
"lcname": "genomehubs"
}
JSON
