# Refget
User-facing documentation is hosted at [refgenie.org/refget](https://refgenie.org/refget/).
In this repository you will find:
1. `/refget`: The `refget` package, which provides a Python interface to both remote and local use of the refget protocol. It has clients and functions for both refget sequences and refget sequence collections (seqcol).
2. `/seqcolapi`: Sequence collections API software, a fastAPI wrapper, built on top of the `refget` package. It provides a bare-bones Sequence Collections API service.
3. `/actions`: GitHub Actions for demo server instance
4. `/deployment`: Server configurations for demo instances and public deployed instances.
5. `/test_fasta` and `/test_api`: Dummy data and a compliance test, to test external implementations of the Refget Sequence Collections API.
6. `/frontend`: a React seqcolapi front-end.
## Testing
### Local unit tests of refget package
- `pytest` to test `refget` package, local unit tests
### Compliance testing of Sequence Collections API
Under `/test_api` are compliance tests for a service implementing the sequence collections API. This will test your collection and comparison endpoints to make sure the comparison function is working.
- `pytest test_api` to tests API compliance
- `pytest test_api --api_root http://127.0.0.1:8100` to customize the API root URL to test
1. Load the fasta files from the `test_fasta` folder into your API database.
2. Run `pytest test_api --api_root <API_URL>`, pointing to your URL to test
For example, this will test a remote server instance:
```
pytest test_api --api_root https://seqcolapi.databio.org
```
## Development and deployment
### Setting up a database connection
First populate environment variables to configure a database connection. Choose one of these:
```
source deployment/local_demo/local_demo.env # local demo (see below to create the database using docker)
source deployment/seqcolapi.databio.org/production.env # connect to production database
```
If you're using the `local_demo`, then use docker to create a local postgres database like this:
```
docker run --rm --name refget-postgres -p 127.0.0.1:5432:5432 \
-e POSTGRES_PASSWORD \
-e POSTGRES_USER \
-e POSTGRES_DB \
-e POSTGRES_HOST \
postgres:16.3
```
If you need to load test data into your server, then you have to install either `gc_count` (fast) or `pyfaidx` (slow). You can load test data like this:
```
python load_demo_data.py
# refget add-fasta path/to/fasta.fa # This could be a way in the future...
# python load_pangenome_reference.py ../seqcolapi/analysis/data/demo.csv test_fasta # loads an entire pangenome
```
## Running the seqcolapi API backend
Run the demo `seqcolapi` service like this:
```
uvicorn seqcolapi.main:app --reload --port 8100
```
### Running with docker
To build the docker file, from the root of this repository:
First you build the general-purpose image
```
docker build -f deployment/dockerhub/Dockerfile -t databio/seqcolapi seqcolapi
```
Next you build the wrapped image (this just wraps the config into the app):
```
docker build -f deployment/seqcolapi.databio.org/Dockerfile -t seqcolapi.databio.org deployment/seqcolapi.databio.org
```
To run in a container:
```
source deployment/seqcolapi.databio.org/production.env
docker run --rm -p 8000:80 --name seqcolapi \
--env "POSTGRES_USER" \
--env "POSTGRES_DB" \
--env "POSTGRES_PASSWORD" \
--env "POSTGRES_HOST" \
seqcolapi.databio.org
```
### Alternative: Mount the config
Instead of building a bundle with the config, you could just mount it into the base image:
```
docker run --rm -p 8000:8000 --name sccon \
--env "POSTGRES_PASSWORD" \
--volume $CODE/seqcolapi.databio.org/config/seqcolapi.yaml:/config.yaml \
seqcolapi
```
### Deploying container to dockerhub
Use github action in this repo which deploys on release, or through manual dispatch.
## Running the frontend
### Local client with local server
```
cd frontend
npm i
VITE_API_BASE="http://localhost:8100" npm run dev
```
### Local client with production server
```
cd frontend
npm i
VITE_API_BASE="https://seqcolapi.databio.org" npm run dev
```
## Deploy to AWS ECS
- Test locally first, using 1. native test; 2. local docker test.
### Deploying
To upgrade the software:
Use config file located in `/servers/seqcolapi.databio.org`. This will use the image in docker.io://databio/seqcolapi, github repo: [refgenie/seqcolapi](https://github.com/refgenie/seqcolapi) as base, bundle it with the above config, and deploy to the shefflab ECS.
1. Ensure the [refget](https://github.com/refgenie/refget/) package master branch is as you want it.
2. Deploy the updated [secqolapi](https://github.com/refgenie/seqcolapi/) app to dockerhub (using manual dispatch, or deploy on github release).
3. Finally, deploy the instance with manual dispatch using the included GitHub action.
Raw data
{
"_id": null,
"home_page": "https://github.com/refgenie/refget",
"name": "refget",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "genome, assembly, bioinformatics, reference, sequence",
"author": "Nathan Sheffield, Michal Stolarczyk",
"author_email": "nathan@code.databio.org",
"download_url": "https://files.pythonhosted.org/packages/fb/c9/64bae0082450c562510cbd8999cfc2bc6755c5b69434b827204b0fd62dea/refget-0.6.0.tar.gz",
"platform": null,
"description": "# Refget\n\n\n\nUser-facing documentation is hosted at [refgenie.org/refget](https://refgenie.org/refget/).\n\nIn this repository you will find:\n\n1. `/refget`: The `refget` package, which provides a Python interface to both remote and local use of the refget protocol. It has clients and functions for both refget sequences and refget sequence collections (seqcol).\n2. `/seqcolapi`: Sequence collections API software, a fastAPI wrapper, built on top of the `refget` package. It provides a bare-bones Sequence Collections API service.\n3. `/actions`: GitHub Actions for demo server instance \n4. `/deployment`: Server configurations for demo instances and public deployed instances.\n5. `/test_fasta` and `/test_api`: Dummy data and a compliance test, to test external implementations of the Refget Sequence Collections API.\n6. `/frontend`: a React seqcolapi front-end.\n\n## Testing\n\n### Local unit tests of refget package\n\n- `pytest` to test `refget` package, local unit tests\n\n### Compliance testing of Sequence Collections API\n\nUnder `/test_api` are compliance tests for a service implementing the sequence collections API. This will test your collection and comparison endpoints to make sure the comparison function is working. \n\n- `pytest test_api` to tests API compliance\n- `pytest test_api --api_root http://127.0.0.1:8100` to customize the API root URL to test\n\n1. Load the fasta files from the `test_fasta` folder into your API database.\n2. Run `pytest test_api --api_root <API_URL>`, pointing to your URL to test\n\nFor example, this will test a remote server instance:\n\n```\npytest test_api --api_root https://seqcolapi.databio.org\n```\n\n## Development and deployment\n\n### Setting up a database connection\n\nFirst populate environment variables to configure a database connection. Choose one of these:\n\n```\nsource deployment/local_demo/local_demo.env # local demo (see below to create the database using docker)\nsource deployment/seqcolapi.databio.org/production.env # connect to production database\n```\n\nIf you're using the `local_demo`, then use docker to create a local postgres database like this:\n\n```\ndocker run --rm --name refget-postgres -p 127.0.0.1:5432:5432 \\\n -e POSTGRES_PASSWORD \\\n -e POSTGRES_USER \\\n -e POSTGRES_DB \\\n -e POSTGRES_HOST \\\n postgres:16.3\n```\n\nIf you need to load test data into your server, then you have to install either `gc_count` (fast) or `pyfaidx` (slow). You can load test data like this:\n\n```\npython load_demo_data.py\n# refget add-fasta path/to/fasta.fa # This could be a way in the future...\n# python load_pangenome_reference.py ../seqcolapi/analysis/data/demo.csv test_fasta # loads an entire pangenome\n```\n\n## Running the seqcolapi API backend\n\nRun the demo `seqcolapi` service like this:\n\n```\nuvicorn seqcolapi.main:app --reload --port 8100\n```\n\n### Running with docker\n\nTo build the docker file, from the root of this repository:\n\n\nFirst you build the general-purpose image\n\n```\ndocker build -f deployment/dockerhub/Dockerfile -t databio/seqcolapi seqcolapi\n```\n\nNext you build the wrapped image (this just wraps the config into the app):\n\n```\ndocker build -f deployment/seqcolapi.databio.org/Dockerfile -t seqcolapi.databio.org deployment/seqcolapi.databio.org\n```\n\nTo run in a container:\n```\nsource deployment/seqcolapi.databio.org/production.env\ndocker run --rm -p 8000:80 --name seqcolapi \\\n --env \"POSTGRES_USER\" \\\n --env \"POSTGRES_DB\" \\\n --env \"POSTGRES_PASSWORD\" \\\n --env \"POSTGRES_HOST\" \\\n seqcolapi.databio.org\n```\n\n### Alternative: Mount the config\n\nInstead of building a bundle with the config, you could just mount it into the base image:\n```\ndocker run --rm -p 8000:8000 --name sccon \\\n --env \"POSTGRES_PASSWORD\" \\\n --volume $CODE/seqcolapi.databio.org/config/seqcolapi.yaml:/config.yaml \\\n seqcolapi \n```\n\n### Deploying container to dockerhub\n\nUse github action in this repo which deploys on release, or through manual dispatch.\n\n\n## Running the frontend\n\n### Local client with local server\n\n```\ncd frontend\nnpm i\nVITE_API_BASE=\"http://localhost:8100\" npm run dev\n```\n\n### Local client with production server\n\n```\ncd frontend\nnpm i\nVITE_API_BASE=\"https://seqcolapi.databio.org\" npm run dev\n```\n\n\n\n## Deploy to AWS ECS\n\n- Test locally first, using 1. native test; 2. local docker test.\n\n### Deploying\n\nTo upgrade the software:\n\nUse config file located in `/servers/seqcolapi.databio.org`. This will use the image in docker.io://databio/seqcolapi, github repo: [refgenie/seqcolapi](https://github.com/refgenie/seqcolapi) as base, bundle it with the above config, and deploy to the shefflab ECS.\n\n1. Ensure the [refget](https://github.com/refgenie/refget/) package master branch is as you want it.\n2. Deploy the updated [secqolapi](https://github.com/refgenie/seqcolapi/) app to dockerhub (using manual dispatch, or deploy on github release).\n3. Finally, deploy the instance with manual dispatch using the included GitHub action.\n",
"bugtrack_url": null,
"license": "BSD2",
"summary": "Python client for refget",
"version": "0.6.0",
"project_urls": {
"Homepage": "https://github.com/refgenie/refget"
},
"split_keywords": [
"genome",
" assembly",
" bioinformatics",
" reference",
" sequence"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "828aadde8d35482645380900abc6cf2d5eef3e6c5f1ff756567db66641d4ed23",
"md5": "fcb294a9959ba4c5c1165b3eaef9ce74",
"sha256": "01110d8ac22872a2e8050ecc5dfbc3126b3f3a10baba53061af44adb84591bf1"
},
"downloads": -1,
"filename": "refget-0.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fcb294a9959ba4c5c1165b3eaef9ce74",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 26429,
"upload_time": "2024-08-09T02:27:57",
"upload_time_iso_8601": "2024-08-09T02:27:57.866825Z",
"url": "https://files.pythonhosted.org/packages/82/8a/adde8d35482645380900abc6cf2d5eef3e6c5f1ff756567db66641d4ed23/refget-0.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fbc964bae0082450c562510cbd8999cfc2bc6755c5b69434b827204b0fd62dea",
"md5": "841355888c4c065c49ada844a9929244",
"sha256": "ab1524f2389659826505f693bc2d37e5b73b3b7f5bfe6d9ed68afa21d4d87edb"
},
"downloads": -1,
"filename": "refget-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "841355888c4c065c49ada844a9929244",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 26480,
"upload_time": "2024-08-09T02:27:58",
"upload_time_iso_8601": "2024-08-09T02:27:58.800667Z",
"url": "https://files.pythonhosted.org/packages/fb/c9/64bae0082450c562510cbd8999cfc2bc6755c5b69434b827204b0fd62dea/refget-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-09 02:27:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "refgenie",
"github_project": "refget",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "refget"
}
JSON
