# Refget
User-facing documentation is hosted at [refgenie.org/refget](https://refgenie.org/refget/).
This repository includes:
1. `/refget`: The `refget` Python package, which provides a Python interface to both remote and local use of refget standards. It has clients and functions for both refget sequences and refget sequence collections (seqcol).
2. `/seqcolapi`: Sequence collections API software, a FastAPI wrapper built on top of the `refget` package. It provides a bare-bones Sequence Collections API service.
3. `/deployment`: Server configurations for demo instances and public deployed instances. There are also github workflows (in `.github/workflows`) that deploy the demo server instance from this repository.
4. `/test_fasta` and `/test_api`: Dummy data and a compliance test, to test external implementations of the Refget Sequence Collections API.
5. `/frontend`: a React seqcolapi front-end.
## Testing
### Local unit tests of refget package
- `pytest` to test `refget` package, local unit tests
## Development and deployment: Backend
### Easy-peasy way
In a moment I'll show you how to do these steps individually, but if you're in a hurry, the easy way get a development API running for testing is to just use my very simple shell script like this (no data persistence, just loads demo data):
```console
bash deployment/demo_up.sh
```
This will:
- populate env vars
- launch postgres container with docker
- run the refget service with uvicorn
- load up the demo data
- block the terminal until you press Ctrl+C, which will shut down all services.
### Setting up a database connection
First configure a database connection through environment variables. Choose one of these:
```
source deployment/local_demo/local_demo.env # local demo (see below to create the database using docker)
source deployment/seqcolapi.databio.org/production.env # connect to production database
```
If you're using the `local_demo`, then use docker to launch a local postgres database service like this:
```
docker run --rm --name refget-postgres -p 127.0.0.1:5432:5432 \
-e POSTGRES_PASSWORD \
-e POSTGRES_USER \
-e POSTGRES_DB \
-e POSTGRES_HOST \
postgres:17.0
```
If you need to load test data into your server, then you have to install [gtars](https://docs.bedbase.org/gtars/) (with `pip install gtars`), a Python package for computing GA4GH digests. You can then load test data like this:
```
python data_loaders/load_demo_data.py
```
or:
```
refget add-fasta -p test_fasta/test_fasta_metadata.csv -r test_fasta
```
### Running the seqcolapi API backend
Run the demo `seqcolapi` service like this:
```
uvicorn seqcolapi.main:app --reload --port 8100
```
### Running with docker
To build the docker file, first build the image from the root of this repository:
```
docker build -f deployment/dockerhub/Dockerfile -t databio/seqcolapi seqcolapi
```
To run in container:
```
source deployment/seqcolapi.databio.org/production.env
docker run --rm -p 8000:80 --name seqcolapi \
--env "POSTGRES_USER" \
--env "POSTGRES_DB" \
--env "POSTGRES_PASSWORD" \
--env "POSTGRES_HOST" \
databio/seqcolapi
```
### Deploying container to dockerhub
Use the github action in this repo which deploys on release, or through manual dispatch.
## Running the frontend
Once you have a backend running, you can run a frontend to interact with it
### Local client with local server
```
cd frontend
npm i
VITE_API_BASE="http://localhost:8100" npm run dev
```
### Local client with production server
```
cd frontend
npm i
VITE_API_BASE="https://seqcolapi.databio.org" npm run dev
```
## Deploy to AWS ECS
- Test locally first, using 1. native test; 2. local docker test.
### Deploying
1. Ensure the [refget](https://github.com/refgenie/refget/) package master branch is as you want it.
2. Deploy the updated [secqolapi](https://github.com/refgenie/seqcolapi/) app to dockerhub (using manual dispatch, or deploy on github release).
3. Finally, deploy the instance with manual dispatch using the included GitHub action.
## Developer notes
### Models
The objects and attributes are represented as SQLModel objects in `refget/models.py`. To add a new attribute:
1. create a new model. This will create a table for that model, etc.
2. change the function that creates the objects, to populate the new attribute.
## Example of loading reference fasta datasets:
```
refget add-fasta -p ref_fasta.csv -r $BRICKYARD/datasets_downloaded/pangenome_fasta/reference_fasta
```
Raw data
{
"_id": null,
"home_page": "https://github.com/refgenie/refget",
"name": "refget",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "genome, assembly, bioinformatics, reference, sequence",
"author": "Nathan Sheffield, Michal Stolarczyk",
"author_email": "nathan@code.databio.org",
"download_url": "https://files.pythonhosted.org/packages/ae/c1/a8b3d94b11d52664513caddfce13198c451365622f1082d78ee32b34fe41/refget-0.8.0.tar.gz",
"platform": null,
"description": "# Refget\n\n\n\nUser-facing documentation is hosted at [refgenie.org/refget](https://refgenie.org/refget/).\n\nThis repository includes:\n\n1. `/refget`: The `refget` Python package, which provides a Python interface to both remote and local use of refget standards. It has clients and functions for both refget sequences and refget sequence collections (seqcol).\n2. `/seqcolapi`: Sequence collections API software, a FastAPI wrapper built on top of the `refget` package. It provides a bare-bones Sequence Collections API service.\n3. `/deployment`: Server configurations for demo instances and public deployed instances. There are also github workflows (in `.github/workflows`) that deploy the demo server instance from this repository.\n4. `/test_fasta` and `/test_api`: Dummy data and a compliance test, to test external implementations of the Refget Sequence Collections API.\n5. `/frontend`: a React seqcolapi front-end.\n\n## Testing\n\n### Local unit tests of refget package\n\n- `pytest` to test `refget` package, local unit tests\n\n## Development and deployment: Backend\n\n### Easy-peasy way\n\nIn a moment I'll show you how to do these steps individually, but if you're in a hurry, the easy way get a development API running for testing is to just use my very simple shell script like this (no data persistence, just loads demo data):\n\n```console\nbash deployment/demo_up.sh\n```\n\nThis will:\n- populate env vars\n- launch postgres container with docker\n- run the refget service with uvicorn\n- load up the demo data\n- block the terminal until you press Ctrl+C, which will shut down all services.\n\n### Setting up a database connection\n\nFirst configure a database connection through environment variables. Choose one of these:\n\n```\nsource deployment/local_demo/local_demo.env # local demo (see below to create the database using docker)\nsource deployment/seqcolapi.databio.org/production.env # connect to production database\n```\n\nIf you're using the `local_demo`, then use docker to launch a local postgres database service like this:\n\n```\ndocker run --rm --name refget-postgres -p 127.0.0.1:5432:5432 \\\n -e POSTGRES_PASSWORD \\\n -e POSTGRES_USER \\\n -e POSTGRES_DB \\\n -e POSTGRES_HOST \\\n postgres:17.0\n```\n\nIf you need to load test data into your server, then you have to install [gtars](https://docs.bedbase.org/gtars/) (with `pip install gtars`), a Python package for computing GA4GH digests. You can then load test data like this:\n\n```\npython data_loaders/load_demo_data.py\n```\n\nor:\n\n```\nrefget add-fasta -p test_fasta/test_fasta_metadata.csv -r test_fasta\n```\n\n\n### Running the seqcolapi API backend\n\nRun the demo `seqcolapi` service like this:\n\n```\nuvicorn seqcolapi.main:app --reload --port 8100\n```\n\n### Running with docker\n\nTo build the docker file, first build the image from the root of this repository:\n\n```\ndocker build -f deployment/dockerhub/Dockerfile -t databio/seqcolapi seqcolapi\n```\n\nTo run in container:\n\n```\nsource deployment/seqcolapi.databio.org/production.env\ndocker run --rm -p 8000:80 --name seqcolapi \\\n --env \"POSTGRES_USER\" \\\n --env \"POSTGRES_DB\" \\\n --env \"POSTGRES_PASSWORD\" \\\n --env \"POSTGRES_HOST\" \\\n databio/seqcolapi\n```\n\n### Deploying container to dockerhub\n\nUse the github action in this repo which deploys on release, or through manual dispatch.\n\n## Running the frontend\n\nOnce you have a backend running, you can run a frontend to interact with it\n\n### Local client with local server\n\n```\ncd frontend\nnpm i\nVITE_API_BASE=\"http://localhost:8100\" npm run dev\n```\n\n### Local client with production server\n\n```\ncd frontend\nnpm i\nVITE_API_BASE=\"https://seqcolapi.databio.org\" npm run dev\n```\n\n## Deploy to AWS ECS\n\n- Test locally first, using 1. native test; 2. local docker test.\n\n### Deploying\n\n1. Ensure the [refget](https://github.com/refgenie/refget/) package master branch is as you want it.\n2. Deploy the updated [secqolapi](https://github.com/refgenie/seqcolapi/) app to dockerhub (using manual dispatch, or deploy on github release).\n3. Finally, deploy the instance with manual dispatch using the included GitHub action.\n\n## Developer notes\n\n### Models\n\nThe objects and attributes are represented as SQLModel objects in `refget/models.py`. To add a new attribute:\n\n1. create a new model. This will create a table for that model, etc.\n2. change the function that creates the objects, to populate the new attribute.\n\n\n\n\n## Example of loading reference fasta datasets:\n\n```\nrefget add-fasta -p ref_fasta.csv -r $BRICKYARD/datasets_downloaded/pangenome_fasta/reference_fasta\n```\n",
"bugtrack_url": null,
"license": "BSD2",
"summary": "Python client for refget",
"version": "0.8.0",
"project_urls": {
"Homepage": "https://github.com/refgenie/refget"
},
"split_keywords": [
"genome",
" assembly",
" bioinformatics",
" reference",
" sequence"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ff9c6fbbd028d75a0c7294d773b9cb8bf5f1a3c8c61bfbf41c73550ace657970",
"md5": "976d4c93aa2f3b29dd18700f587b52f6",
"sha256": "bf6d485ceaa5bdbb7fcdf1e1b7c6c48572d2bc45109d36e810cea40ed12de2b5"
},
"downloads": -1,
"filename": "refget-0.8.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "976d4c93aa2f3b29dd18700f587b52f6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 28842,
"upload_time": "2025-03-11T21:31:02",
"upload_time_iso_8601": "2025-03-11T21:31:02.547186Z",
"url": "https://files.pythonhosted.org/packages/ff/9c/6fbbd028d75a0c7294d773b9cb8bf5f1a3c8c61bfbf41c73550ace657970/refget-0.8.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "aec1a8b3d94b11d52664513caddfce13198c451365622f1082d78ee32b34fe41",
"md5": "11cb8ff80f37ce6b34022236b8fbd606",
"sha256": "251ad5d14b578eba7daa10c3a2159b70a70e31296995f9d2c80b0ea9efa1d5ab"
},
"downloads": -1,
"filename": "refget-0.8.0.tar.gz",
"has_sig": false,
"md5_digest": "11cb8ff80f37ce6b34022236b8fbd606",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 29416,
"upload_time": "2025-03-11T21:31:04",
"upload_time_iso_8601": "2025-03-11T21:31:04.218602Z",
"url": "https://files.pythonhosted.org/packages/ae/c1/a8b3d94b11d52664513caddfce13198c451365622f1082d78ee32b34fe41/refget-0.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-03-11 21:31:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "refgenie",
"github_project": "refget",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "refget"
}
JSON
