# Project Links
This repository is part of the [CI-SpliceAI](https://ci-spliceai.com) software package published in [PLOS One](https://doi.org/10.1371/journal.pone.0269159).
This is the project for offline variant annotation using the final models. You may also be interested in the [code to train CI-SpliceAI](https://github.com/YStrauch/CI-SpliceAI__Train), [code comparing different tools on variant data](https://github.com/YStrauch/CI-SpliceAI__Comparison), and the [website providing online annotation of variants](https://ci-spliceai.com).
# Install
We strongly advise you to install [conda](https://docs.conda.io/en/latest/) and create a new conda environment. This keeps your projects and their dependencies separate. So go ahead and install conda first.
CI-SpliceAI can be run on your CPU or GPU. CPU is much easier to set up, but inference is much slower than using a GPU. If you only need to annotate a few variants (less than 50 in total), don't bother setting up GPU support.
If you don't need GPU support, you can skip the next paragraph.
## Setting up your GPU
For experienced users only. GPU is only supported for CUDA devices (i.e. NVIDIA graphic cards).
We suggest you install [conda](https://docs.conda.io/en/latest/), CUDA and tensorflow-gpu first and ensure that your GPU is being used by tensorflow (by running python and importing tensorflow). You can use a command similar to this:
```sh
conda create -n cis_use python=3.8 tensorflow-gpu=2.2.0 cudnn=7.6.5.32=hc0a50b0_1 keras=2.4.3 -c conda-forge
```
We used this command with CUDA/11.0.
Please activate the new environment and make sure that your GPU is being recognised:
```sh
conda activate cis_use
python -c "import keras; from tensorflow.python.client import device_lib; print([x.name for x in device_lib.list_local_devices()])"
```
If you set-up your GPU correctly, the last line generated by this command should list all your GPUs. If it does not, do not proceed with installation until fixed.
Once the GPU shows up, you can proceed with the next section and omit the [cpu] suffix.
## Installing the python module
If you did not set-up your GPU, you need to create the conda environment first:
```sh
conda create -n cis_use python=3.8
conda activate cis_use
```
Then, install cispliceai like this (remove [cpu] suffix if you set up a GPU in the previous step):
```sh
pip install "cispliceai[cpu]"
```
# Usage
```
cis-vcf [-h] [--annotation ANNOTATION] [--input INPUT] [--output OUTPUT] [--distance DISTANCE] [--batch BATCH] [--all] [--outside] [--mask] reference
```
<pre>
positional arguments:
reference path to the reference fasta file (*.fa, *.fa.gz)
optional arguments:
-h, --help show this help message and exit
--annotation ANNOTATION, -a ANNOTATION
annotation table with gene names, defaults to "grch38" (table included). You can specify "grch37" or a path to your own table of the same format.
--input INPUT, -i INPUT
input VCF; defaults to stdin
--output OUTPUT, -o OUTPUT
output VCF; defaults to stdout
--distance DISTANCE, -d DISTANCE
maximum distance from the variant; defaults to 1000
--batch BATCH, -b BATCH
maximum input batch size in MB. Be careful to leave enough space for the model and inference process. We recommend to increase this only for GPU processing. Defaults to 10
--all annotate all affected genes/regions, not only the most significant
--outside keep nucleotides outside of annotated transcript areas (defined in annotation table); by default outside nucleotides are encoded as N
--mask, -m mask events to disregard gains of splice sites and losses of non-splice sites
</pre>
## Example command
Annotates an `input.vcf` file into an `output.vcf` file. Uses GrCh37 coordinates, outputs all genes affected, and masks to canonical losses or non-canonical gains.
```sh
cis-vcf -i input.vcf -o output.vcf -a grch37 --all --mask
```
## Parameter `reference`
Please download a human reference genome file (i.e. the latest primary assembly used by GENCODE: [GRCh38](https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/latest_release/GRCh38.primary_assembly.genome.fa.gz), [GRCh37](https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/latest_release/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz)). You need to unzip it first!
If you want to save space (and decrease performance!), you can use a bgzipped version of the reference genome. To do so, install samtools and biopython (`conda install samtools biopython -c bioconda`), then run `bgzip -c <file>.fa > <file>.fa.gz`.
## Annotation table (`--annotation`)
By default, the module uses an annotation table of collapsed isoforms of GENCODE on GRCh38. You can change this to GRCh37 by specifying "-a grch37".
You can also provide your own annotation table. Please use a table provided within this module, i.e. [the GrCh38 one](cispliceai/data/grch38.csv), as a template. It's a CSV file with a (gene) ID, the chromosome, strand, start and end of the transcript, and comma-separated junction start end end positions. On the forward strand, junction start/end is equivalent to donor/acceptor sites, on the reverse-strand it's the other way round.
## In/Output (`--input` and `--output`)
You can specify the path to VCF files here. Or, you can omit the parameters pipe them to stdin and stdout `cat variants.vcf | cis-vcf <reference> > output.vcf`
## Maximum distance from the variant (`--distance`)
The algorithm predicts on your REF annotation plus some nucleotides around it. This parameter changes the number of nucleotides predicted to the left and right. (Note: An additional 5,000 nucleotides in either direction is extracted for context but not used for annotation).
## Batch size (`--batch`)
This parameter is only useful when using a GPU. We don't recommend changing it for CPU usage.
Each GPU has its own memory limit, so you need to find the optimal batch size yourself. If it's too small, your GPU won't be used efficiently, if it's too high, it will run out of memory.
We suggest finding the optimal value by measuring memory consumption using `nvidia-smi`, extrapolate usage to 100%, and try again. The parameter itself is a maximum value in MB, so it will work for different sequence lengths (depending on `--distance` and REF/ALT length)/
## All annotations per allele (`--all`)
By default, only the most significant annotation per ALT allele is written to VCF (i.e. the one with the highest delta position). You can add the '--all' flag to instead output all alleles.
## Retain sequences outside of annotated regions (`--outside`)
_This is an experimental feature_.
By default, all nucleotides outside of the overlapping annotated region (defined by the `--annotation` table) are replaced with N, except when the variant does not intercept with a region. You can instead always retain the sequences.
## Mask scores (`--mask`)
This parameter masks the delta score: It will remove (i.e. set to zero) all losses of non-splice sites and all gains of splice sites. Therefore only losses of existing splice sites and gains of novel splice sites are annotated.
# Output
The VCF output has an INFO column with the following annotations:
| ID | Description |
| -------- | ----------- |
| ALLELE | The ALT allele from your input |
| SYMBOL | Ensembl gene ID, or strand if no gene overlaps |
| DS_AG | Delta score (acceptor gain) |
| DS_AL | Delta score (acceptor loss) |
| DS_DG | Delta score (donor gain) |
| DS_DL | Delta score (donor loss) |
| DP_AG | Delta position (acceptor gain) |
| DP_AL | Delta position (acceptor loss) |
| DP_DG | Delta position (donor gain) |
| DP_DL | Delta position (donor loss) |
# Comparison between CI-SpliceAI and similar tools
There is a separate repository [here](https://github.com/YStrauch/splice-variant-comparison) comparing CI-SpliceAI with SpliceAI, MMSplice, SQUIRLS, and two MaxEntScan variants.
# Differences to SpliceAI
This tool is very similar to the [SpliceAI](https://github.com/Illumina/SpliceAI) annotation tool. These are the most important differences:
- Models are trained on collapsed GENCODE isoforms
- The final models are trained on all chromosomes; SpliceAI models are only trained on training chromosomes
- Models are optimised for inference and load/predict faster
- All overlapping genes are analysed, not only the first one
- When no genes overlap, the tool is run on the forward and backward strand
- *All* variants are annotated, even multi-nucleotide variants
- Better batch inference on GPU
- Masking (`--mask`) is done prior to annotation, allowing the next-highest annotation to be returned rather than capping the output to zero (see [this github issue](https://github.com/Illumina/SpliceAI/issues/27))
- Potential differences when filtering GENCODE annotations. SpliceAI's data pipeline was not released and their selection process not disclosed (see [this github issue](https://github.com/Illumina/SpliceAI/issues/87))
# Current limitations
The tool is only using one CPU/GPU max. Multi-processing is currently not supported.
# Prediction of custom sequences
Since version 1.1, you can predict the likelihood of each nucleotide being a splice site like so:
```python
from cispliceai.model import CISpliceAI
from cispliceai.data import DNAPreprocessor
model = CISpliceAI()
# you can run multiple sequences in a batch
# you can input sequences of varying length, but it's your responsibility to ensure that the batch fits on memory!
# The sequences here are only an example, you should upload much longer slices to get sensible results
predictions = model.predict([
DNAPreprocessor.onehot_and_pad('CTTCCTCTCCTCCTGCCCCACCTTCCTCTCCTCCTGCCCCACCAGAACCGGGGGCGGCTGGCAGACAAGAGGACAGTCGCCCTGCCTGCC'),
DNAPreprocessor.onehot_and_pad('CTCCTCCTGCCCCACCAGAACCGGGGGCGGCTG'),
])
# Get acceptor and donor scores for the two sequences
acceptor_prob_1 = predictions[0][:, 1]
donor_prob_1 = predictions[0][:, 2]
acceptor_prob_2 = predictions[1][:, 1]
donor_prob_2 = predictions[1][:, 2]
```
# Development
## To install from a local directory
```sh
pip install -e "./[cpu]"
```
## Build to release
```sh
pip install build
python -m build
```
## Pull requests / Bug reports
Pull requests are welcome. Please submit bug reports to the git issues system.
# License
<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
Raw data
{
"_id": null,
"home_page": "https://ci-spliceai.com",
"name": "cispliceai",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "splicing,rna,machine learning,SpliceAI",
"author": "Yaron Strauch",
"author_email": "git@yaron-strauch.com",
"download_url": "https://files.pythonhosted.org/packages/32/77/cfd98fb318dcde9a548417a6194826e8a321b1abfd38f79aeb7d9893e7bf/cispliceai-1.1.1.tar.gz",
"platform": null,
"description": "# Project Links\nThis repository is part of the [CI-SpliceAI](https://ci-spliceai.com) software package published in [PLOS One](https://doi.org/10.1371/journal.pone.0269159).\n\nThis is the project for offline variant annotation using the final models. You may also be interested in the [code to train CI-SpliceAI](https://github.com/YStrauch/CI-SpliceAI__Train), [code comparing different tools on variant data](https://github.com/YStrauch/CI-SpliceAI__Comparison), and the [website providing online annotation of variants](https://ci-spliceai.com).\n\n# Install\nWe strongly advise you to install [conda](https://docs.conda.io/en/latest/) and create a new conda environment. This keeps your projects and their dependencies separate. So go ahead and install conda first.\n\nCI-SpliceAI can be run on your CPU or GPU. CPU is much easier to set up, but inference is much slower than using a GPU. If you only need to annotate a few variants (less than 50 in total), don't bother setting up GPU support.\n\nIf you don't need GPU support, you can skip the next paragraph.\n\n## Setting up your GPU\nFor experienced users only. GPU is only supported for CUDA devices (i.e. NVIDIA graphic cards).\n\nWe suggest you install [conda](https://docs.conda.io/en/latest/), CUDA and tensorflow-gpu first and ensure that your GPU is being used by tensorflow (by running python and importing tensorflow). You can use a command similar to this:\n\n```sh\nconda create -n cis_use python=3.8 tensorflow-gpu=2.2.0 cudnn=7.6.5.32=hc0a50b0_1 keras=2.4.3 -c conda-forge\n```\n\nWe used this command with CUDA/11.0.\n\nPlease activate the new environment and make sure that your GPU is being recognised:\n```sh\nconda activate cis_use\npython -c \"import keras; from tensorflow.python.client import device_lib; print([x.name for x in device_lib.list_local_devices()])\"\n```\nIf you set-up your GPU correctly, the last line generated by this command should list all your GPUs. If it does not, do not proceed with installation until fixed.\nOnce the GPU shows up, you can proceed with the next section and omit the [cpu] suffix.\n\n## Installing the python module\nIf you did not set-up your GPU, you need to create the conda environment first:\n```sh\nconda create -n cis_use python=3.8\nconda activate cis_use\n```\n\nThen, install cispliceai like this (remove [cpu] suffix if you set up a GPU in the previous step):\n```sh\npip install \"cispliceai[cpu]\"\n```\n\n# Usage\n\n```\ncis-vcf [-h] [--annotation ANNOTATION] [--input INPUT] [--output OUTPUT] [--distance DISTANCE] [--batch BATCH] [--all] [--outside] [--mask] reference\n```\n<pre>\npositional arguments:\n reference path to the reference fasta file (*.fa, *.fa.gz)\n\noptional arguments:\n -h, --help show this help message and exit\n\n --annotation ANNOTATION, -a ANNOTATION\n annotation table with gene names, defaults to \"grch38\" (table included). You can specify \"grch37\" or a path to your own table of the same format.\n\n --input INPUT, -i INPUT\n input VCF; defaults to stdin\n\n --output OUTPUT, -o OUTPUT\n output VCF; defaults to stdout\n\n --distance DISTANCE, -d DISTANCE\n maximum distance from the variant; defaults to 1000\n\n --batch BATCH, -b BATCH\n maximum input batch size in MB. Be careful to leave enough space for the model and inference process. We recommend to increase this only for GPU processing. Defaults to 10\n\n --all annotate all affected genes/regions, not only the most significant\n\n --outside keep nucleotides outside of annotated transcript areas (defined in annotation table); by default outside nucleotides are encoded as N\n\n --mask, -m mask events to disregard gains of splice sites and losses of non-splice sites\n</pre>\n\n## Example command\nAnnotates an `input.vcf` file into an `output.vcf` file. Uses GrCh37 coordinates, outputs all genes affected, and masks to canonical losses or non-canonical gains.\n\n```sh\ncis-vcf -i input.vcf -o output.vcf -a grch37 --all --mask\n```\n## Parameter `reference`\nPlease download a human reference genome file (i.e. the latest primary assembly used by GENCODE: [GRCh38](https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/latest_release/GRCh38.primary_assembly.genome.fa.gz), [GRCh37](https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/latest_release/GRCh37_mapping/GRCh37.primary_assembly.genome.fa.gz)). You need to unzip it first!\n\nIf you want to save space (and decrease performance!), you can use a bgzipped version of the reference genome. To do so, install samtools and biopython (`conda install samtools biopython -c bioconda`), then run `bgzip -c <file>.fa > <file>.fa.gz`.\n\n## Annotation table (`--annotation`)\nBy default, the module uses an annotation table of collapsed isoforms of GENCODE on GRCh38. You can change this to GRCh37 by specifying \"-a grch37\".\n\nYou can also provide your own annotation table. Please use a table provided within this module, i.e. [the GrCh38 one](cispliceai/data/grch38.csv), as a template. It's a CSV file with a (gene) ID, the chromosome, strand, start and end of the transcript, and comma-separated junction start end end positions. On the forward strand, junction start/end is equivalent to donor/acceptor sites, on the reverse-strand it's the other way round.\n\n## In/Output (`--input` and `--output`)\nYou can specify the path to VCF files here. Or, you can omit the parameters pipe them to stdin and stdout `cat variants.vcf | cis-vcf <reference> > output.vcf`\n\n## Maximum distance from the variant (`--distance`)\nThe algorithm predicts on your REF annotation plus some nucleotides around it. This parameter changes the number of nucleotides predicted to the left and right. (Note: An additional 5,000 nucleotides in either direction is extracted for context but not used for annotation).\n\n## Batch size (`--batch`)\nThis parameter is only useful when using a GPU. We don't recommend changing it for CPU usage.\n\nEach GPU has its own memory limit, so you need to find the optimal batch size yourself. If it's too small, your GPU won't be used efficiently, if it's too high, it will run out of memory.\n\nWe suggest finding the optimal value by measuring memory consumption using `nvidia-smi`, extrapolate usage to 100%, and try again. The parameter itself is a maximum value in MB, so it will work for different sequence lengths (depending on `--distance` and REF/ALT length)/\n\n## All annotations per allele (`--all`)\nBy default, only the most significant annotation per ALT allele is written to VCF (i.e. the one with the highest delta position). You can add the '--all' flag to instead output all alleles.\n\n## Retain sequences outside of annotated regions (`--outside`)\n_This is an experimental feature_.\n\nBy default, all nucleotides outside of the overlapping annotated region (defined by the `--annotation` table) are replaced with N, except when the variant does not intercept with a region. You can instead always retain the sequences.\n\n## Mask scores (`--mask`)\nThis parameter masks the delta score: It will remove (i.e. set to zero) all losses of non-splice sites and all gains of splice sites. Therefore only losses of existing splice sites and gains of novel splice sites are annotated.\n\n# Output\nThe VCF output has an INFO column with the following annotations:\n\n| ID | Description |\n| -------- | ----------- |\n| ALLELE | The ALT allele from your input |\n| SYMBOL | Ensembl gene ID, or strand if no gene overlaps |\n| DS_AG | Delta score (acceptor gain) |\n| DS_AL | Delta score (acceptor loss) |\n| DS_DG | Delta score (donor gain) |\n| DS_DL | Delta score (donor loss) |\n| DP_AG | Delta position (acceptor gain) |\n| DP_AL | Delta position (acceptor loss) |\n| DP_DG | Delta position (donor gain) |\n| DP_DL | Delta position (donor loss) |\n\n# Comparison between CI-SpliceAI and similar tools\nThere is a separate repository [here](https://github.com/YStrauch/splice-variant-comparison) comparing CI-SpliceAI with SpliceAI, MMSplice, SQUIRLS, and two MaxEntScan variants.\n\n# Differences to SpliceAI\nThis tool is very similar to the [SpliceAI](https://github.com/Illumina/SpliceAI) annotation tool. These are the most important differences:\n\n- Models are trained on collapsed GENCODE isoforms\n- The final models are trained on all chromosomes; SpliceAI models are only trained on training chromosomes\n- Models are optimised for inference and load/predict faster\n- All overlapping genes are analysed, not only the first one\n- When no genes overlap, the tool is run on the forward and backward strand\n- *All* variants are annotated, even multi-nucleotide variants\n- Better batch inference on GPU\n- Masking (`--mask`) is done prior to annotation, allowing the next-highest annotation to be returned rather than capping the output to zero (see [this github issue](https://github.com/Illumina/SpliceAI/issues/27))\n- Potential differences when filtering GENCODE annotations. SpliceAI's data pipeline was not released and their selection process not disclosed (see [this github issue](https://github.com/Illumina/SpliceAI/issues/87))\n\n# Current limitations\nThe tool is only using one CPU/GPU max. Multi-processing is currently not supported.\n\n# Prediction of custom sequences\nSince version 1.1, you can predict the likelihood of each nucleotide being a splice site like so:\n```python\nfrom cispliceai.model import CISpliceAI\nfrom cispliceai.data import DNAPreprocessor\n\nmodel = CISpliceAI()\n\n# you can run multiple sequences in a batch\n# you can input sequences of varying length, but it's your responsibility to ensure that the batch fits on memory!\n# The sequences here are only an example, you should upload much longer slices to get sensible results\npredictions = model.predict([\n DNAPreprocessor.onehot_and_pad('CTTCCTCTCCTCCTGCCCCACCTTCCTCTCCTCCTGCCCCACCAGAACCGGGGGCGGCTGGCAGACAAGAGGACAGTCGCCCTGCCTGCC'),\n DNAPreprocessor.onehot_and_pad('CTCCTCCTGCCCCACCAGAACCGGGGGCGGCTG'),\n])\n\n# Get acceptor and donor scores for the two sequences\nacceptor_prob_1 = predictions[0][:, 1]\ndonor_prob_1 = predictions[0][:, 2]\n\nacceptor_prob_2 = predictions[1][:, 1]\ndonor_prob_2 = predictions[1][:, 2]\n```\n\n# Development\n## To install from a local directory\n```sh\npip install -e \"./[cpu]\"\n```\n\n## Build to release\n```sh\npip install build\npython -m build\n```\n\n## Pull requests / Bug reports\nPull requests are welcome. Please submit bug reports to the git issues system.\n\n# License\n<a rel=\"license\" href=\"http://creativecommons.org/licenses/by/4.0/\"><img alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by/4.0/88x31.png\" /></a><br />This work is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by/4.0/\">Creative Commons Attribution 4.0 International License</a>.\n",
"bugtrack_url": null,
"license": "CC BY 4.0",
"summary": "CI-SpliceAI: Predicting splicing disruptions, trained on all isoforms",
"version": "1.1.1",
"split_keywords": [
"splicing",
"rna",
"machine learning",
"spliceai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "10fc599a445f0d1894a4b72382aeb3e51107fab98a7c50fb5bd7635a088792c9",
"md5": "711a0970ab5a62ea8848aff5e32d4706",
"sha256": "b78d4c3a034016c4c464b88f5ed291f14e49a537e5a3e9af7800ab9fd04f8f09"
},
"downloads": -1,
"filename": "cispliceai-1.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "711a0970ab5a62ea8848aff5e32d4706",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 16996019,
"upload_time": "2023-01-25T19:26:19",
"upload_time_iso_8601": "2023-01-25T19:26:19.455078Z",
"url": "https://files.pythonhosted.org/packages/10/fc/599a445f0d1894a4b72382aeb3e51107fab98a7c50fb5bd7635a088792c9/cispliceai-1.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3277cfd98fb318dcde9a548417a6194826e8a321b1abfd38f79aeb7d9893e7bf",
"md5": "1381a485f6305c1d4f4ff134a2a266ff",
"sha256": "98d0db50111c5d0f8b17582329319080765c188346f156bb5205cb2ff2691e7d"
},
"downloads": -1,
"filename": "cispliceai-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "1381a485f6305c1d4f4ff134a2a266ff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 16989893,
"upload_time": "2023-01-25T19:26:46",
"upload_time_iso_8601": "2023-01-25T19:26:46.030617Z",
"url": "https://files.pythonhosted.org/packages/32/77/cfd98fb318dcde9a548417a6194826e8a321b1abfd38f79aeb7d9893e7bf/cispliceai-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-25 19:26:46",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "cispliceai"
}
JSON