# vFind
[](https://pypi.org/project/vfind/)
*A simple variant finder for NGS data.*
1. [Introduction](#introduction)
2. [Installation](#installation)
3. [Examples](#examples)
4. [Contributing](#contributing)
5. [License](#license)
## Introduction
vFind is unlike a traditional [*variant caller*](https://gencore.bio.nyu.edu/variant-calling-pipeline-gatk4/).
It is actually using a simpler algorithm which is *usually* sufficient for
screening experiments. The main use case is finding variants from a library that
has constant adapter sequences flanking a variable region.
This simple algorithm is summarized as:
1. Define a pair of adapter sequences that flank the variable region.
2. For each fastq read, search for exact matches of these adapters.
3. If both adapters are found exactly, recover the variable region.
4. For each adapter without an exact match, perform semi-global alignment between the given adapter and read (optional see the [alignment parameters](#using-custom-alignment-parameters) section).
5. If the alignment score meets a set threshold, that adapter is considered to match.
6. If both adapters are exactly or partially matched, recover the variable region.
7. For exact matches of both adapters, recover the variable region. Otherwise, continue to the next read.
8. Finally, translate the variable region to its amino acid sequence and filter out any sequences with partial codons (Optional, see the [miscellaneuous](#miscellaneuous) section).
> [!WARNING]
> Note that vFind doesn't do any kind of preprocessing. For initial quality
> filtering, merging, and other common preprocessing operations, you might be
> interested in something like [fastp](https://github.com/OpenGene/fastp) or
> [ngmerge](https://github.com/jsh58/NGmerge). We generally recommend using
> fastp for quality filtering and merging fastq files before using vFind.
Installation details and usage examples are given below. For more usage details,
please see the [API reference](docs/api-reference.md)
## Installation
vFind is a Python package and can be installed via pip or nix. For a CLI version,
see the [vFind-cli](https://github.com/nsbuitrago/vfind-cli) repository.
### PyPI (Recommended for most)
The package is available on [PyPI](https://pypi.org/project/vfind) and can be installed via pip (or alternatives like [uv](https://github.com/astral-sh/uv)).
Below is an example using pip with Python3 in a new project.
```bash
# create a new virtual env
python3 -m venv .venv # create a new virtual env if haven't already
source .venv/bin/activate # activate the virtual env
python3 -m pip install vfind # install vfind
```
### Nix
vFind is also available on [NixPkgs](https://search.nixos.org/packages?). You can declare new
enviroments using [nix flakes](https://wiki.nixos.org/wiki/Flakes).
For something quick, you can use nix-shell. For example, the following will
create a new shell with Python 3.11, vFind, and polars installed.
```bash
nix-shell -p python311 python3Packages.vfind python3Packages.polars
```
## Examples
### Basic Usage
```python
from vfind import find_variants
import polars as pl # variants are returned in a polars dataframe
adapters = ("GGG", "CCC") # define the adapters
fq_path = "./path/to/your/fastq/file.fq.gz" # path to fq file
variants = find_variants(fq_path, adapters)
# print the number of unique sequences
print(variants.n_unique())
```
`find_variants` returns a polars dataframe with `sequence` and `count` columns.
`sequence` contains the amino acid sequence of the variable regions and
`count` contains the frequency of those variant.
We can then use [dataframe methods](https://docs.pola.rs/py-polars/html/reference/dataframe/index.html)
to further analyze the recovered variants. Some examples are shown below.
```python
# Get the top 5 most frequent variants
variants.sort("count", descending=True) # sort by the counts in descending order
print(variants.head(5)) # print the first 5 (most frequent) variants
# filter out sequences with less than 10 read counts
# also any sequences that have a pre-mature stop codon (i.e., * before the last residue)
filtered_variants = variants.filter(
variants["count"] > 10,
~variants["sequence"][::-2].str.contains("*")
)
# write the filtered variants to a csv file
filtered_variants.write_csv("filtered_variants.csv")
```
### Using Custom Alignment Parameters
By default, vFind uses semi-global alignment with the following parameters:
- match score = 3
- mismatch score = -2
- gap open penalty = 5
- gap extend penalty = 2
Note that the gap penalties are represented as positive integers. This is largely due to how the underlying
alignment library works.
To adjust these alignment parameters, use the `match_score`, `mismatch_score`,
`gap_open_penalty`, and `gap_extend_penalty` keyword arguments:
```python
from vfind import find_variants
# ... define adapters and fq_path
# use identity scoring with no gap penalties for alignments
variants = find_variants(
fq_path,
adapters,
match_score = 1,
mismatch_score = -1,
gap_open_penalty: 0,
gap_extend_penalty: 0,
)
```
Alignments are accepted if they produce a score above a set threshold. The threshold
for considering an acceptable alignment can be adjusted with the `accept_prefix_alignment`
and `accept_suffix_alignment` arguments. By default, both thresholds are set to 0.75.
The thresholds are represent a percentage of the maximum alignment score. So, a value of 0.75
means alignments producing scores that are greater than 75% the maximum theoretical score will be accepted. Thus, valid values are between 0 and 1.
Either an exact match or partial match (accepted alignment) must be made for both adapter sequences to recover a variant.
In order to skip alignment and only look for exact matches, set the `skip_alignment` argument to `True`.
### Miscellaneous
**Q:** I don't need the amino acid sequence. Can I just get the DNA sequence?
**A:** Yes. Just set `skip_translation` to True.
```python
# ...
dna_seqs = find_variants(fq_path, adapters, skip_translation=True)
```
---
**Q:** I don't want to use polars. Can I use pandas instead?
**A:** Yes. Use the [`to_pandas`](https://docs.pola.rs/py-polars/html/reference/dataframe/api/polars.DataFrame.to_pandas.html#polars.DataFrame.to_pandas) method on the dataframe.
---
**Q:** I have a lot of data and `find_variants` is slow. Is there anything I can do to speed it up?
**A:** Maybe. Try changing the number of threads or queue length the function uses.
```python
# ...
variants = find_variants(fq_path, adapters, n_threads=6, queue_len=4)
```
For more usage details, see the [API reference](docs/api-reference.md).
## Contributing
Feedback is a gift and contributions are more than welcome. Please submit an
issue or pull request for any bugs, suggestions, or feedback. Please see the
[developing](docs/developing) guide for more details on how to work on vFind.
## License
vFind is licensed under the [MIT license](LICENSE)
Raw data
{
"_id": null,
"home_page": null,
"name": "vfind",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "bioinformatics, sequence analysis, NGS, variant finding",
"author": null,
"author_email": "nsbuitrago <44626938+nsbuitrago@users.noreply.github.com>",
"download_url": "https://files.pythonhosted.org/packages/05/df/4a4291b8ecc76e257affa271d380efe2290dcc431d6a5fba5208a0fc6b12/vfind-0.1.1.tar.gz",
"platform": null,
"description": "# vFind\n\n[](https://pypi.org/project/vfind/)\n\n*A simple variant finder for NGS data.*\n\n1. [Introduction](#introduction)\n2. [Installation](#installation)\n3. [Examples](#examples)\n4. [Contributing](#contributing)\n5. [License](#license)\n\n## Introduction\n\nvFind is unlike a traditional [*variant caller*](https://gencore.bio.nyu.edu/variant-calling-pipeline-gatk4/).\nIt is actually using a simpler algorithm which is *usually* sufficient for \nscreening experiments. The main use case is finding variants from a library that\nhas constant adapter sequences flanking a variable region.\n\nThis simple algorithm is summarized as:\n\n1. Define a pair of adapter sequences that flank the variable region.\n2. For each fastq read, search for exact matches of these adapters.\n3. If both adapters are found exactly, recover the variable region.\n4. For each adapter without an exact match, perform semi-global alignment between the given adapter and read (optional see the [alignment parameters](#using-custom-alignment-parameters) section).\n5. If the alignment score meets a set threshold, that adapter is considered to match.\n6. If both adapters are exactly or partially matched, recover the variable region.\n7. For exact matches of both adapters, recover the variable region. Otherwise, continue to the next read.\n8. Finally, translate the variable region to its amino acid sequence and filter out any sequences with partial codons (Optional, see the [miscellaneuous](#miscellaneuous) section).\n\n> [!WARNING]\n> Note that vFind doesn't do any kind of preprocessing. For initial quality\n> filtering, merging, and other common preprocessing operations, you might be\n> interested in something like [fastp](https://github.com/OpenGene/fastp) or\n> [ngmerge](https://github.com/jsh58/NGmerge). We generally recommend using\n> fastp for quality filtering and merging fastq files before using vFind.\n\nInstallation details and usage examples are given below. For more usage details,\nplease see the [API reference](docs/api-reference.md)\n\n## Installation\n\nvFind is a Python package and can be installed via pip or nix. For a CLI version,\nsee the [vFind-cli](https://github.com/nsbuitrago/vfind-cli) repository.\n\n### PyPI (Recommended for most)\n\nThe package is available on [PyPI](https://pypi.org/project/vfind) and can be installed via pip (or alternatives like [uv](https://github.com/astral-sh/uv)).\n\nBelow is an example using pip with Python3 in a new project.\n\n```bash\n# create a new virtual env\npython3 -m venv .venv # create a new virtual env if haven't already\nsource .venv/bin/activate # activate the virtual env\n\npython3 -m pip install vfind # install vfind\n```\n\n### Nix\n\nvFind is also available on [NixPkgs](https://search.nixos.org/packages?). You can declare new\nenviroments using [nix flakes](https://wiki.nixos.org/wiki/Flakes).\n\nFor something quick, you can use nix-shell. For example, the following will\ncreate a new shell with Python 3.11, vFind, and polars installed.\n\n```bash\nnix-shell -p python311 python3Packages.vfind python3Packages.polars\n```\n\n## Examples\n\n### Basic Usage\n\n```python\nfrom vfind import find_variants\nimport polars as pl # variants are returned in a polars dataframe\n\nadapters = (\"GGG\", \"CCC\") # define the adapters\nfq_path = \"./path/to/your/fastq/file.fq.gz\" # path to fq file\n\nvariants = find_variants(fq_path, adapters)\n\n# print the number of unique sequences \nprint(variants.n_unique())\n```\n\n`find_variants` returns a polars dataframe with `sequence` and `count` columns.\n`sequence` contains the amino acid sequence of the variable regions and\n`count` contains the frequency of those variant.\n\nWe can then use [dataframe methods](https://docs.pola.rs/py-polars/html/reference/dataframe/index.html) \nto further analyze the recovered variants. Some examples are shown below.\n\n```python\n# Get the top 5 most frequent variants\nvariants.sort(\"count\", descending=True) # sort by the counts in descending order\nprint(variants.head(5)) # print the first 5 (most frequent) variants\n\n# filter out sequences with less than 10 read counts\n# also any sequences that have a pre-mature stop codon (i.e., * before the last residue)\n\nfiltered_variants = variants.filter(\n variants[\"count\"] > 10,\n ~variants[\"sequence\"][::-2].str.contains(\"*\")\n)\n\n# write the filtered variants to a csv file\nfiltered_variants.write_csv(\"filtered_variants.csv\")\n```\n\n### Using Custom Alignment Parameters\n\nBy default, vFind uses semi-global alignment with the following parameters:\n\n- match score = 3\n- mismatch score = -2\n- gap open penalty = 5\n- gap extend penalty = 2\n\nNote that the gap penalties are represented as positive integers. This is largely due to how the underlying\nalignment library works.\n\nTo adjust these alignment parameters, use the `match_score`, `mismatch_score`,\n`gap_open_penalty`, and `gap_extend_penalty` keyword arguments:\n\n```python\nfrom vfind import find_variants\n\n# ... define adapters and fq_path\n\n# use identity scoring with no gap penalties for alignments\nvariants = find_variants(\n fq_path,\n adapters,\n match_score = 1,\n mismatch_score = -1,\n gap_open_penalty: 0,\n gap_extend_penalty: 0,\n)\n```\n\nAlignments are accepted if they produce a score above a set threshold. The threshold\nfor considering an acceptable alignment can be adjusted with the `accept_prefix_alignment`\nand `accept_suffix_alignment` arguments. By default, both thresholds are set to 0.75.\n\nThe thresholds are represent a percentage of the maximum alignment score. So, a value of 0.75\nmeans alignments producing scores that are greater than 75% the maximum theoretical score will be accepted. Thus, valid values are between 0 and 1.\n\nEither an exact match or partial match (accepted alignment) must be made for both adapter sequences to recover a variant. \nIn order to skip alignment and only look for exact matches, set the `skip_alignment` argument to `True`.\n\n### Miscellaneous\n\n**Q:** I don't need the amino acid sequence. Can I just get the DNA sequence?\n\n**A:** Yes. Just set `skip_translation` to True.\n\n```python\n# ...\ndna_seqs = find_variants(fq_path, adapters, skip_translation=True)\n```\n\n---\n\n**Q:** I don't want to use polars. Can I use pandas instead?\n\n**A:** Yes. Use the [`to_pandas`](https://docs.pola.rs/py-polars/html/reference/dataframe/api/polars.DataFrame.to_pandas.html#polars.DataFrame.to_pandas) method on the dataframe.\n\n---\n\n**Q:** I have a lot of data and `find_variants` is slow. Is there anything I can do to speed it up?\n\n**A:** Maybe. Try changing the number of threads or queue length the function uses.\n\n```python\n# ...\nvariants = find_variants(fq_path, adapters, n_threads=6, queue_len=4)\n```\n\nFor more usage details, see the [API reference](docs/api-reference.md).\n\n## Contributing\n\nFeedback is a gift and contributions are more than welcome. Please submit an\nissue or pull request for any bugs, suggestions, or feedback. Please see the \n[developing](docs/developing) guide for more details on how to work on vFind.\n\n## License\n\nvFind is licensed under the [MIT license](LICENSE)\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A simple variant finder for NGS data",
"version": "0.1.1",
"project_urls": {
"Issues": "https://github.com/nsbuitrago/vfind/issues",
"Repository": "https://github.com/nsbuitrago/vfind"
},
"split_keywords": [
"bioinformatics",
" sequence analysis",
" ngs",
" variant finding"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "21b3638f55520f813c34721741a63e06a5bf123f43ccd42b9239ffea1d83f249",
"md5": "90a7016684e436e3e132e4bae87b07f7",
"sha256": "5975054db02dd05077e27e41d4ea0bd2d6caafc3d5b75de09b78ba1b1997541a"
},
"downloads": -1,
"filename": "vfind-0.1.1-cp310-cp310-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "90a7016684e436e3e132e4bae87b07f7",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.8",
"size": 7600881,
"upload_time": "2024-05-06T05:27:38",
"upload_time_iso_8601": "2024-05-06T05:27:38.817596Z",
"url": "https://files.pythonhosted.org/packages/21/b3/638f55520f813c34721741a63e06a5bf123f43ccd42b9239ffea1d83f249/vfind-0.1.1-cp310-cp310-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c06d580cb81704abc2f249699d8cf81a079954b5ed3db46a89066722f0159041",
"md5": "3859f7ef5cbedd524ea48d5113602e84",
"sha256": "25a8a1b214e03cf224fae66762643a9b38a6a8a9e8e2566e735c4a57862c3abb"
},
"downloads": -1,
"filename": "vfind-0.1.1-cp311-cp311-macosx_10_12_x86_64.whl",
"has_sig": false,
"md5_digest": "3859f7ef5cbedd524ea48d5113602e84",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.8",
"size": 3528094,
"upload_time": "2024-05-06T05:27:42",
"upload_time_iso_8601": "2024-05-06T05:27:42.775529Z",
"url": "https://files.pythonhosted.org/packages/c0/6d/580cb81704abc2f249699d8cf81a079954b5ed3db46a89066722f0159041/vfind-0.1.1-cp311-cp311-macosx_10_12_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e985231b54148e6cc35eb03f72aad849f1959936efee04f6180c1536d029b166",
"md5": "6e013e75436c2a23a056d1100208e8f2",
"sha256": "f7949fcd9595113514b72f2f9c3dd5d6b64da364081893f13919e0fcd06a0c97"
},
"downloads": -1,
"filename": "vfind-0.1.1-cp311-cp311-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "6e013e75436c2a23a056d1100208e8f2",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.8",
"size": 4012839,
"upload_time": "2024-05-06T05:27:46",
"upload_time_iso_8601": "2024-05-06T05:27:46.300669Z",
"url": "https://files.pythonhosted.org/packages/e9/85/231b54148e6cc35eb03f72aad849f1959936efee04f6180c1536d029b166/vfind-0.1.1-cp311-cp311-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0748d5595b536d9b533edce049b4d17c4b5967d8b8dd8258363fd508b7ee4bc5",
"md5": "1a9a2abd45e1caf7d0898290b57e2a72",
"sha256": "cbfd7186b94216bd433bbb742089a6fef8504001935a589ed1d8ffa3e5ea4486"
},
"downloads": -1,
"filename": "vfind-0.1.1-cp312-cp312-macosx_10_12_x86_64.whl",
"has_sig": false,
"md5_digest": "1a9a2abd45e1caf7d0898290b57e2a72",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.8",
"size": 3529253,
"upload_time": "2024-05-06T05:27:49",
"upload_time_iso_8601": "2024-05-06T05:27:49.867189Z",
"url": "https://files.pythonhosted.org/packages/07/48/d5595b536d9b533edce049b4d17c4b5967d8b8dd8258363fd508b7ee4bc5/vfind-0.1.1-cp312-cp312-macosx_10_12_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c76457ec9c39f731a7bf2ed61a798b17e5d0cadc7e236a6700664abc5a056469",
"md5": "752c9b79f229983a7ea9ea3226b57d1e",
"sha256": "c0345988e9fe46b6d2567acf8704499ea127d957442a19b4b48f3681a342aefc"
},
"downloads": -1,
"filename": "vfind-0.1.1-cp312-cp312-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "752c9b79f229983a7ea9ea3226b57d1e",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.8",
"size": 4012206,
"upload_time": "2024-05-06T05:27:53",
"upload_time_iso_8601": "2024-05-06T05:27:53.254976Z",
"url": "https://files.pythonhosted.org/packages/c7/64/57ec9c39f731a7bf2ed61a798b17e5d0cadc7e236a6700664abc5a056469/vfind-0.1.1-cp312-cp312-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "04cedeca03e6b5d3a5cacf10c6476c55e546854a892e366926539a18e099f0ff",
"md5": "b9ae4879092346f1320602d3ec54ec0a",
"sha256": "e780581b780573197be05c7d4187cb46e8649b6d14b7b7fcaf987ea97e30d366"
},
"downloads": -1,
"filename": "vfind-0.1.1-cp312-cp312-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "b9ae4879092346f1320602d3ec54ec0a",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.8",
"size": 7600060,
"upload_time": "2024-05-06T05:27:58",
"upload_time_iso_8601": "2024-05-06T05:27:58.007194Z",
"url": "https://files.pythonhosted.org/packages/04/ce/deca03e6b5d3a5cacf10c6476c55e546854a892e366926539a18e099f0ff/vfind-0.1.1-cp312-cp312-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "05df4a4291b8ecc76e257affa271d380efe2290dcc431d6a5fba5208a0fc6b12",
"md5": "0a4db8040a435f4ef691efd8324ce668",
"sha256": "55ec520af5b0d8e7940a2ad0f937d54a64428a1871cdc66ac7c4e5ced412cc07"
},
"downloads": -1,
"filename": "vfind-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "0a4db8040a435f4ef691efd8324ce668",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 32578,
"upload_time": "2024-05-06T05:28:00",
"upload_time_iso_8601": "2024-05-06T05:28:00.177914Z",
"url": "https://files.pythonhosted.org/packages/05/df/4a4291b8ecc76e257affa271d380efe2290dcc431d6a5fba5208a0fc6b12/vfind-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-06 05:28:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nsbuitrago",
"github_project": "vfind",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "vfind"
}
JSON
