refget


Namerefget JSON
Version 0.8.0 PyPI version JSON
download
home_pagehttps://github.com/refgenie/refget
SummaryPython client for refget
upload_time2025-03-11 21:31:04
maintainerNone
docs_urlNone
authorNathan Sheffield, Michal Stolarczyk
requires_pythonNone
licenseBSD2
keywords genome assembly bioinformatics reference sequence
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Refget

![Run pytests](https://github.com/pepkit/looper/workflows/Run%20pytests/badge.svg)

User-facing documentation is hosted at [refgenie.org/refget](https://refgenie.org/refget/).

This repository includes:

1. `/refget`: The `refget` Python package, which provides a Python interface to both remote and local use of refget standards. It has clients and functions for both refget sequences and refget sequence collections (seqcol).
2. `/seqcolapi`: Sequence collections API software, a FastAPI wrapper built on top of the `refget` package. It provides a bare-bones Sequence Collections API service.
3. `/deployment`: Server configurations for demo instances and public deployed instances. There are also github workflows (in `.github/workflows`) that deploy the demo server instance from this repository.
4. `/test_fasta` and `/test_api`: Dummy data and a compliance test, to test external implementations of the Refget Sequence Collections API.
5. `/frontend`: a React seqcolapi front-end.

## Testing

### Local unit tests of refget package

- `pytest` to test `refget` package, local unit tests

## Development and deployment: Backend

### Easy-peasy way

In a moment I'll show you how to do these steps individually, but if you're in a hurry, the easy way get a development API running for testing is to just use my very simple shell script like this (no data persistence, just loads demo data):

```console
bash deployment/demo_up.sh
```

This will:
- populate env vars
- launch postgres container with docker
- run the refget service with uvicorn
- load up the demo data
- block the terminal until you press Ctrl+C, which will shut down all services.

### Setting up a database connection

First configure a database connection through environment variables. Choose one of these:

```
source deployment/local_demo/local_demo.env # local demo (see below to create the database using docker)
source deployment/seqcolapi.databio.org/production.env # connect to production database
```

If you're using the `local_demo`, then use docker to launch a local postgres database service like this:

```
docker run --rm --name refget-postgres -p 127.0.0.1:5432:5432 \
  -e POSTGRES_PASSWORD \
  -e POSTGRES_USER \
  -e POSTGRES_DB \
  -e POSTGRES_HOST \
  postgres:17.0
```

If you need to load test data into your server, then you have to install [gtars](https://docs.bedbase.org/gtars/) (with `pip install gtars`), a Python package for computing GA4GH digests. You can then load test data like this:

```
python data_loaders/load_demo_data.py
```

or:

```
refget add-fasta -p test_fasta/test_fasta_metadata.csv -r test_fasta
```


### Running the seqcolapi API backend

Run the demo `seqcolapi` service like this:

```
uvicorn seqcolapi.main:app --reload --port 8100
```

### Running with docker

To build the docker file, first build the image from the root of this repository:

```
docker build -f deployment/dockerhub/Dockerfile -t databio/seqcolapi seqcolapi
```

To run in container:

```
source deployment/seqcolapi.databio.org/production.env
docker run --rm -p 8000:80 --name seqcolapi \
  --env "POSTGRES_USER" \
  --env "POSTGRES_DB" \
  --env "POSTGRES_PASSWORD" \
  --env "POSTGRES_HOST" \
  databio/seqcolapi
```

### Deploying container to dockerhub

Use the github action in this repo which deploys on release, or through manual dispatch.

## Running the frontend

Once you have a backend running, you can run a frontend to interact with it

### Local client with local server

```
cd frontend
npm i
VITE_API_BASE="http://localhost:8100" npm run dev
```

### Local client with production server

```
cd frontend
npm i
VITE_API_BASE="https://seqcolapi.databio.org" npm run dev
```

## Deploy to AWS ECS

- Test locally first, using 1. native test; 2. local docker test.

### Deploying

1. Ensure the [refget](https://github.com/refgenie/refget/) package master branch is as you want it.
2. Deploy the updated [secqolapi](https://github.com/refgenie/seqcolapi/) app to dockerhub (using manual dispatch, or deploy on github release).
3. Finally, deploy the instance with manual dispatch using the included GitHub action.

## Developer notes

### Models

The objects and attributes are represented as SQLModel objects in `refget/models.py`. To add a new attribute:

1. create a new model. This will create a table for that model, etc.
2. change the function that creates the objects, to populate the new attribute.




## Example of loading reference fasta datasets:

```
refget add-fasta -p ref_fasta.csv -r $BRICKYARD/datasets_downloaded/pangenome_fasta/reference_fasta
```

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/refgenie/refget",
    "name": "refget",
    "maintainer": null,
    "docs_url": null,
    "requires_python": null,
    "maintainer_email": null,
    "keywords": "genome, assembly, bioinformatics, reference, sequence",
    "author": "Nathan Sheffield, Michal Stolarczyk",
    "author_email": "nathan@code.databio.org",
    "download_url": "https://files.pythonhosted.org/packages/ae/c1/a8b3d94b11d52664513caddfce13198c451365622f1082d78ee32b34fe41/refget-0.8.0.tar.gz",
    "platform": null,
    "description": "# Refget\n\n![Run pytests](https://github.com/pepkit/looper/workflows/Run%20pytests/badge.svg)\n\nUser-facing documentation is hosted at [refgenie.org/refget](https://refgenie.org/refget/).\n\nThis repository includes:\n\n1. `/refget`: The `refget` Python package, which provides a Python interface to both remote and local use of refget standards. It has clients and functions for both refget sequences and refget sequence collections (seqcol).\n2. `/seqcolapi`: Sequence collections API software, a FastAPI wrapper built on top of the `refget` package. It provides a bare-bones Sequence Collections API service.\n3. `/deployment`: Server configurations for demo instances and public deployed instances. There are also github workflows (in `.github/workflows`) that deploy the demo server instance from this repository.\n4. `/test_fasta` and `/test_api`: Dummy data and a compliance test, to test external implementations of the Refget Sequence Collections API.\n5. `/frontend`: a React seqcolapi front-end.\n\n## Testing\n\n### Local unit tests of refget package\n\n- `pytest` to test `refget` package, local unit tests\n\n## Development and deployment: Backend\n\n### Easy-peasy way\n\nIn a moment I'll show you how to do these steps individually, but if you're in a hurry, the easy way get a development API running for testing is to just use my very simple shell script like this (no data persistence, just loads demo data):\n\n```console\nbash deployment/demo_up.sh\n```\n\nThis will:\n- populate env vars\n- launch postgres container with docker\n- run the refget service with uvicorn\n- load up the demo data\n- block the terminal until you press Ctrl+C, which will shut down all services.\n\n### Setting up a database connection\n\nFirst configure a database connection through environment variables. Choose one of these:\n\n```\nsource deployment/local_demo/local_demo.env # local demo (see below to create the database using docker)\nsource deployment/seqcolapi.databio.org/production.env # connect to production database\n```\n\nIf you're using the `local_demo`, then use docker to launch a local postgres database service like this:\n\n```\ndocker run --rm --name refget-postgres -p 127.0.0.1:5432:5432 \\\n  -e POSTGRES_PASSWORD \\\n  -e POSTGRES_USER \\\n  -e POSTGRES_DB \\\n  -e POSTGRES_HOST \\\n  postgres:17.0\n```\n\nIf you need to load test data into your server, then you have to install [gtars](https://docs.bedbase.org/gtars/) (with `pip install gtars`), a Python package for computing GA4GH digests. You can then load test data like this:\n\n```\npython data_loaders/load_demo_data.py\n```\n\nor:\n\n```\nrefget add-fasta -p test_fasta/test_fasta_metadata.csv -r test_fasta\n```\n\n\n### Running the seqcolapi API backend\n\nRun the demo `seqcolapi` service like this:\n\n```\nuvicorn seqcolapi.main:app --reload --port 8100\n```\n\n### Running with docker\n\nTo build the docker file, first build the image from the root of this repository:\n\n```\ndocker build -f deployment/dockerhub/Dockerfile -t databio/seqcolapi seqcolapi\n```\n\nTo run in container:\n\n```\nsource deployment/seqcolapi.databio.org/production.env\ndocker run --rm -p 8000:80 --name seqcolapi \\\n  --env \"POSTGRES_USER\" \\\n  --env \"POSTGRES_DB\" \\\n  --env \"POSTGRES_PASSWORD\" \\\n  --env \"POSTGRES_HOST\" \\\n  databio/seqcolapi\n```\n\n### Deploying container to dockerhub\n\nUse the github action in this repo which deploys on release, or through manual dispatch.\n\n## Running the frontend\n\nOnce you have a backend running, you can run a frontend to interact with it\n\n### Local client with local server\n\n```\ncd frontend\nnpm i\nVITE_API_BASE=\"http://localhost:8100\" npm run dev\n```\n\n### Local client with production server\n\n```\ncd frontend\nnpm i\nVITE_API_BASE=\"https://seqcolapi.databio.org\" npm run dev\n```\n\n## Deploy to AWS ECS\n\n- Test locally first, using 1. native test; 2. local docker test.\n\n### Deploying\n\n1. Ensure the [refget](https://github.com/refgenie/refget/) package master branch is as you want it.\n2. Deploy the updated [secqolapi](https://github.com/refgenie/seqcolapi/) app to dockerhub (using manual dispatch, or deploy on github release).\n3. Finally, deploy the instance with manual dispatch using the included GitHub action.\n\n## Developer notes\n\n### Models\n\nThe objects and attributes are represented as SQLModel objects in `refget/models.py`. To add a new attribute:\n\n1. create a new model. This will create a table for that model, etc.\n2. change the function that creates the objects, to populate the new attribute.\n\n\n\n\n## Example of loading reference fasta datasets:\n\n```\nrefget add-fasta -p ref_fasta.csv -r $BRICKYARD/datasets_downloaded/pangenome_fasta/reference_fasta\n```\n",
    "bugtrack_url": null,
    "license": "BSD2",
    "summary": "Python client for refget",
    "version": "0.8.0",
    "project_urls": {
        "Homepage": "https://github.com/refgenie/refget"
    },
    "split_keywords": [
        "genome",
        " assembly",
        " bioinformatics",
        " reference",
        " sequence"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "ff9c6fbbd028d75a0c7294d773b9cb8bf5f1a3c8c61bfbf41c73550ace657970",
                "md5": "976d4c93aa2f3b29dd18700f587b52f6",
                "sha256": "bf6d485ceaa5bdbb7fcdf1e1b7c6c48572d2bc45109d36e810cea40ed12de2b5"
            },
            "downloads": -1,
            "filename": "refget-0.8.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "976d4c93aa2f3b29dd18700f587b52f6",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 28842,
            "upload_time": "2025-03-11T21:31:02",
            "upload_time_iso_8601": "2025-03-11T21:31:02.547186Z",
            "url": "https://files.pythonhosted.org/packages/ff/9c/6fbbd028d75a0c7294d773b9cb8bf5f1a3c8c61bfbf41c73550ace657970/refget-0.8.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "aec1a8b3d94b11d52664513caddfce13198c451365622f1082d78ee32b34fe41",
                "md5": "11cb8ff80f37ce6b34022236b8fbd606",
                "sha256": "251ad5d14b578eba7daa10c3a2159b70a70e31296995f9d2c80b0ea9efa1d5ab"
            },
            "downloads": -1,
            "filename": "refget-0.8.0.tar.gz",
            "has_sig": false,
            "md5_digest": "11cb8ff80f37ce6b34022236b8fbd606",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 29416,
            "upload_time": "2025-03-11T21:31:04",
            "upload_time_iso_8601": "2025-03-11T21:31:04.218602Z",
            "url": "https://files.pythonhosted.org/packages/ae/c1/a8b3d94b11d52664513caddfce13198c451365622f1082d78ee32b34fe41/refget-0.8.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-11 21:31:04",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "refgenie",
    "github_project": "refget",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "refget"
}
        
Elapsed time: 1.68211s