| Name | PyamilySeq JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | https://github.com/NickJD/PyamilySeq |
| Summary | PyamilySeq - A a tool to investigate sequence-based gene groups identified by clustering methods such as CD-HIT, DIAMOND, BLAST or MMseqs2. |
| upload_time | 2024-11-20 06:06:34 |
| maintainer | None |
| docs_url | None |
| author | Nicholas Dimonaco |
| requires_python | >=3.6 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# PyamilySeq
**PyamilySeq** is a Python tool for clustering gene sequences into groups based on sequence similarity identified by tools such as CD-HIT, BLAST, DIAMOND or MMseqs2.
This work is an extension of the gene family / pangenome tool developed for the StORF-Reporter publication in NAR (https://doi.org/10.1093/nar/gkad814).
## Features
- **End-to-End**: PyamilySeq can take a directory of GFF+FASTA files, run CD-HIT for clustering and process the results.
- **Clustering input**: Supports input from CD-HIT formatted files as well as CSV and TSV node-edge lists (MMseqs2 and -outfmt 6 from BLAST/DIAMOND).
- **Reclustering**: Allows for the addition of new sequences post-initial clustering - Ensures continuity of contemporary clustering results and highlights impact of novel gene predictions.
- **'Genus Mode'**: Unlike other 'pangenome' tools, PyamilySeq can identify gene groups found across multiple genera as unique entities (see below).
- **Output**: Generates a 'Roary/Panaroo' formatted presence-absence CSV formatted file for downstream analysis.
- User-define species-/genus-wide gene groups - User has control over grouping parameters (core = 99/95% or min 6 genera etc).
- Aligns representative sequences using MAFFT.
- Output concatenated aligned sequences for tree building.
- Optionally output sequences of each separate identified gene group.
- Group-Splitter tool to split multi-copy gene groups.
- Numerous additional tools to assist in the pre- and post-processing of data.
## Installation
PyamilySeq probably requires Python 3.6 or higher and the levenshtein library (https://pypi.org/project/Levenshtein/) - \
If levenshtein is not available, a Python implementation is utilised which is significantly slower.
#### Install using pip:
```bash
pip install PyamilySeq [optionally add -U]
```
PyamilySeq is currently still under active development so expect 'regular' updates with bugfixes and new features. \
To update to the newest version add '-U' to end of the pip install command.
## Example usage: Below are two examples of running PyamilySeq in its two main modes.
```commandline
usage: PyamilySeq.py [-h] {Full,Partial} ...
PyamilySeq v1.0.1: A tool for gene clustering and analysis.
positional arguments:
{Full,Partial} Choose a mode: 'Full' or 'Partial'.
Full Full mode: PyamilySeq to cluster with CD-HIT and process output.
Partial Partial mode: PyamilySeq to process pre-clustered data.
options:
-h, --help show this help message and exit
```
### 'Full Mode': Will conduct clustering of sequences with CD-HIT as part of PyamilySeq run
```
PyamilySeq Full -output_dir .../PyamilySeq_10_AA_90_80_Full_GFFs -input_type combined -input_dir .../genomes/ -name_split _combined.gff3
```
### 'Partial Mode': Will process the output of a sequence clustering from MMseqs, BLAST, DIAMOND etc.
```
PyamilySeq Partial -clustering_format CD-HIT -cluster_file .../all_10_combined_pep_CD-HIT_90_80.clstr -original_fasta .../all_10_combined_pep.fasta -output_dir .../PyamilySeq_10_AA_90_80_Partial-w 99 -a
```
#### Note: using a '-clustering_format' other than the default CD-HIT, requires input to be two in two columns as below (Same format as MMseqs2 tsv and BLAST outfmt 6) - Genome name and sequence name are separated by '|'.
```
Escherichia_coli_110957|ENSB:lL-zIKt-gh0oSno Escherichia_coli_110957|ENSB:lL-zIKt-gh0oSno
Escherichia_coli_110957|ENSB:lL-zIKt-gh0oSno Escherichia_coli_113290|ENSB:2fj4rJ8e8Z9PNdX
Escherichia_coli_110957|ENSB:lL-zIKt-gh0oSno Escherichia_coli_b185|ENSB:G_PVe28-ej8q-3S
Escherichia_coli_110957|ENSB:TIZS9kbTvShDvyX Escherichia_coli_110957|ENSB:TIZS9kbTvShDvyX
```
### Example output:
```
Running PyamilySeq v1.0.1
Calculating Groups
Number of Genomes: 10
Gene Groups
First_core_99: 2994
First_core_95: 0
First_core_15: 3266
First_core_0: 5466
Total Number of First Gene Groups (Including Singletons): 11726
Outputting gene_presence_absence file
Outputting gene group FASTA files
Combined FASTA file saved to: ../combined_group_sequences_dna.fasta
Processing gene group alignment
Thank you for using PyamilySeq -- A detailed user manual can be found at https://github.com/NickJD/PyamilySeq
Please report any issues to: https://github.com/NickJD/PyamilySeq/issues
```
[//]: # (## Genus mode: )
[//]: # (### In addition to "Species mode" (see above) which reports gene groups the same as pangenome tools such as Roary and Panaroo, Genus mode reports gene groups identified across multiple genera.)
[//]: # (#### Example:)
[//]: # (```)
[//]: # (PyamilySeq -run_mode Partial -group_mode Genus -clustering_format CD-HIT -output_dir .../test_data/genus/testing/)
[//]: # ( -cluster_file .../test_data/genus/CD-HIT/combined_cds_cd-hit_80_60.clstr -gpa )
[//]: # (```)
[//]: # (```commandline)
[//]: # (Running PyamilySeq v1.0.1)
[//]: # (Calculating Groups)
[//]: # (Genus Groups:)
[//]: # (First_genera_1: 28549)
[//]: # (First_genera_2: 12)
[//]: # (First_genera_3: 0)
[//]: # (First_genera_>: 0)
[//]: # (Total Number of First Gene Groups (Including Singletons): 28561)
[//]: # (Outputting gene_presence_absence file)
[//]: # (Thank you for using PyamilySeq -- A detailed user manual can be found at https://github.com/NickJD/PyamilySeq)
[//]: # (Please report any issues to: https://github.com/NickJD/PyamilySeq/issues)
[//]: # (#####)
[//]: # (```)
## Reclustering:
### Reclustering can be used to see where additional sequences/genes lay in relation to a contemporary pangenome/gene grouping.
```
PyamilySeq Partial -clustering_format CD-HIT -cluster_file .../all_10_combined_pep_CD-HIT_90_80.clstr -reclustered .../all_10_combined_pep_CD-HIT_90_80_AND_StORFs_CD-HIT_90_80.clstr -original_fasta .../all_10_combined_pep_AND_StORFs.fasta -output_dir .../PyamilySeq_10_AA_90_80_Partial_Reclustered_StORFs -w 99 -a
```
#### As can be seen below, the additional sequences recovered by the StORF-Reporter annotation tool have 'extended' contemporary or created entirely new gene groups. 'First' corresponds to the groups identified from the first clustering round and 'Second' for the second. In 'reclustering' mode, First_core_# groups are unaffected thus retaining the initial grouping information.
```commandline
Number of Genomes: 10
Gene Groups
First_core_99: 2994
First_core_95: 0
First_core_15: 3266
First_core_0: 5466
extended_core_99: 3
extended_core_95: 0
extended_core_15: 49
extended_core_0: 0
combined_core_99: 0
combined_core_95: 0
combined_core_15: 3
combined_core_0: 0
Second_core_99: 0
Second_core_95: 0
Second_core_15: 20
Second_core_0: 39
only_Second_core_99: 768
only_Second_core_95: 0
only_Second_core_15: 4472
only_Second_core_0: 8395
Total Number of First Gene Groups (Including Singletons): 11726
Total Number of Second Gene Groups (Including Singletons): 25359
Total Number of First Gene Groups That Had Additional Second Sequences But Not New Genomes: 5
```
## PyamilySeq is separated into two main 'run modes', Full and Partial. They each have their own set of required and optional arguments.
### PyamilySeq - Full Menu:
```
usage: PyamilySeq.py Full [-h] -output_dir OUTPUT_DIR -input_type {separate,combined,fasta} [-input_dir INPUT_DIR]
[-input_fasta INPUT_FASTA] [-name_split NAME_SPLIT] [-sequence_type {AA,DNA}] [-gene_ident GENE_IDENT]
[-c PIDENT] [-s LEN_DIFF] [-fast_mode] [-group_mode {Species,Genus}] [-species_groups SPECIES_GROUPS]
[-genus_groups GENUS_GROUPS] [-w WRITE_GROUPS] [-wi] [-a] [-align_aa] [-no_gpa] [-M MEM] [-T THREADS]
[-verbose] [-v]
options:
-h, --help show this help message and exit
-output_dir OUTPUT_DIR
Directory for all output files.
-input_type {separate,combined,fasta}
Type of input files: 'separate' for matching FASTA and GFF files, 'combined' for GFF+FASTA, or 'fasta'
for a prepared FASTA file.
-input_dir INPUT_DIR Directory containing GFF/FASTA files - Use with -input_type separate/combined.
-input_fasta INPUT_FASTA
Input FASTA file - Use with - input_type fasta.
-name_split NAME_SPLIT
Substring to split filenames and extract genome names (e.g., '_combined.gff3') - Use with -input_type
separate/combined.
-sequence_type {AA,DNA}
Clustering mode: 'DNA' or 'AA'.
-gene_ident GENE_IDENT
Gene identifiers to extract sequences (e.g., 'CDS, tRNA').
-c PIDENT Sequence identity threshold for clustering (default: 0.90) - CD-HIT parameter '-c'.
-s LEN_DIFF Length difference threshold for clustering (default: 0.80) - CD-HIT parameter '-s'.
-fast_mode Enable fast mode for CD-HIT (not recommended) - CD-HIT parameter '-g'.
-group_mode {Species,Genus}
Grouping mode: 'Species' or 'Genus'.
-species_groups SPECIES_GROUPS
Gene groupings for 'Species' mode (default: '99,95,15').
-genus_groups GENUS_GROUPS
Gene groupings for 'Genus' mode (default: '1-10').
-w WRITE_GROUPS Output gene groups as a single FASTA file (specify levels: e.g., '-w 99,95').
-wi Output individual FASTA files for each group.
-a Align and concatenate sequences for 'core' groups.
-align_aa Align sequences as amino acids.
-no_gpa Skip creation of gene_presence_absence.csv.
-M MEM Memory allocation for clustering (MB) - CD-HIT parameter '-M'.
-T THREADS Number of threads for clustering/alignment - CD-HIT parameter '-T' | MAFFT parameter '--thread'.
-verbose Print verbose output.
-v, --version Print version number and exit.
```
### PyamilySeq - Partial Menu:
```commandline
usage: PyamilySeq.py Partial [-h] -clustering_format {CD-HIT,MMseqs,BLAST} -cluster_file CLUSTER_FILE -original_fasta ORIGINAL_FASTA -output_dir OUTPUT_DIR
[-reclustered RECLUSTERED] [-seq_tag SEQUENCE_TAG] [-group_mode {Species,Genus}] [-species_groups SPECIES_GROUPS] [-genus_groups GENUS_GROUPS]
[-w WRITE_GROUPS] [-wi] [-a] [-align_aa] [-no_gpa] [-M MEM] [-T THREADS] [-verbose] [-v]
options:
-h, --help show this help message and exit
-clustering_format {CD-HIT,MMseqs,BLAST}
Clustering format used: CD-HIT, MMseqs2, or BLAST.
-cluster_file CLUSTER_FILE
Cluster file containing pre-clustered groups from CD-HIT, MMseqs, BLAST etc.
-original_fasta ORIGINAL_FASTA
FASTA file used in pre-clustering (Provide sequences in DNA form).
-output_dir OUTPUT_DIR
Directory for all output files.
-reclustered RECLUSTERED
Clustering output file from a second round of clustering.
-seq_tag SEQUENCE_TAG
Tag for distinguishing reclustered sequences.
-group_mode {Species,Genus}
Grouping mode: 'Species' or 'Genus'.
-species_groups SPECIES_GROUPS
Gene groupings for 'Species' mode (default: '99,95,15').
-genus_groups GENUS_GROUPS
Gene groupings for 'Genus' mode (default: '1-10').
-w WRITE_GROUPS Output gene groups as a single FASTA file (specify levels: e.g., '-w 99,95').
-wi Output individual FASTA files for each group.
-a Align and concatenate sequences for 'core' groups.
-align_aa Align sequences as amino acids.
-no_gpa Skip creation of gene_presence_absence.csv.
-M MEM Memory allocation for clustering (MB) - CD-HIT parameter '-M'.
-T THREADS Number of threads for clustering/alignment - CD-HIT parameter '-T' | MAFFT parameter '--thread'.
-verbose Print verbose output.
-v, --version Print version number and exit.
```
## Seq-Combiner: This tool is provided to enable the pre-processing of multiple GFF/FASTA files together ready to be clustered by the user.
### Example:
```bash
Seq-Combiner -input_dir .../test_data/genomes -name_split .gff3 -output_dir .../test_data/genomes -output_name combine_fasta_seqs.fa -input_type combined
```
### Seq-Combiner Menu:
```
usage: Seq_Combiner.py [-h] -input_dir INPUT_DIR -input_type {separate,combined,fasta} -name_split NAME_SPLIT -output_dir OUTPUT_DIR -output_name
OUTPUT_FILE [-gene_ident GENE_IDENT] [-translate] [-v]
PyamilySeq v1.0.1: Seq-Combiner - A tool to extract sequences from GFF/FASTA files and prepare them for PyamilySeq.
options:
-h, --help show this help message and exit
Required Arguments:
-input_dir INPUT_DIR Directory location where the files are located.
-input_type {separate,combined,fasta}
Type of input files: "separate" for separate FASTA and GFF files, "combined" for GFF files with embedded FASTA sequences and "fasta"
for combining multiple FASTA files together.
-name_split NAME_SPLIT
substring used to split the filename and extract the genome name ('_combined.gff3' or '.gff').
-output_dir OUTPUT_DIR
Directory for all output files.
-output_name OUTPUT_FILE
Output file name.
Optional Arguments:
-gene_ident GENE_IDENT
Default - "CDS": Identifier used for extraction of sequences such as
"misc_RNA,gene,mRNA,CDS,rRNA,tRNA,tmRNA,CRISPR,ncRNA,regulatory_region,oriC,pseudo" - Not compatible with "fasta" input mode.
-translate Default - False: Translate extracted sequences to their AA counterpart?
Misc Arguments:
-v, --version Print out version number and exit
```
## Group-Splitter: This tool can split multi-copy gene groups using CD-HIT after initial PyamilySeq analysis.
### Example:
```bash
Group-Splitter -genome_num 74 -input_fasta .../test/species/ -output_dir .../test/species/ -sequence_type AA
```
### Group-Splitter Menu:
```
usage: Group_Splitter.py [-h] -input_fasta INPUT_FASTA -sequence_type {AA,DNA}
-genome_num GENOME_NUM -output_dir OUTPUT_DIR
[-groups GROUPS] [-group_threshold GROUP_THRESHOLD]
[-c PIDENT] [-s LEN_DIFF] [-T CLUSTERING_THREADS]
[-M CLUSTERING_MEMORY] [-no_delete_temp_files]
[-verbose] [-v]
PyamilySeq v1.0.1: Group-Splitter - A tool to split multi-copy gene groups
identified by PyamilySeq.
options:
-h, --help show this help message and exit
Required Parameters:
-input_fasta INPUT_FASTA
Input FASTA file containing gene groups.
-sequence_type {AA,DNA}
Default - DNA: Are groups "DNA" or "AA" sequences?
-genome_num GENOME_NUM
The total number of genomes must be provide
-output_dir OUTPUT_DIR
Output directory.
Regrouping Parameters:
-groups GROUPS Default - auto: Detect groups to be split (see
-group_threshold). Provide "-groups 1,2,3,4" with
group IDs to split specific groups.
-group_threshold GROUP_THRESHOLD
Minimum percentage of genomes with multi-copy
(default: 80.0) - Does not work with "-groups"
CD-HIT Reclustering Parameters:
-c PIDENT Sequence identity threshold (default: 0.8) - Probably
should be higher than what was used in initial
clustering.
-s LEN_DIFF Length difference cutoff (default: 0.20) - Often the
most impactful parameter to split 'multi-copy' gene
groups.
-T CLUSTERING_THREADS
Number of threads for clustering (default: 4)
-M CLUSTERING_MEMORY Memory limit in MB for clustering (default: 2000)
Misc Parameters:
-no_delete_temp_files
Default: Delete all temporary files after processing.
-verbose Print verbose output.
-v, --version Print out version number and exit
```
## Cluster-Summary menu: This tool can be used to summarise CD-HIT .clstr files:
### Example:
```bash
Cluster-Summary -genome_num 74 -input_clstr .../test_data/species/E-coli/E-coli_extracted_pep_cd-hit_80.clstr -output_tsv .../test_data/species/E-coli/E-coli_extracted_pep_cd-hit_80_Summary.tsv
```
### Cluster-Summary Menu:
```
usage: Cluster_Summary.py [-h] -input_clstr INPUT_CLSTR -output OUTPUT -genome_num GENOME_NUM
[-output_dir OUTPUT_DIR] [-verbose] [-v]
PyamilySeq v1.0.1: Cluster-Summary - A tool to summarise CD-HIT clustering files.
options:
-h, --help show this help message and exit
Required Parameters:
-input_clstr INPUT_CLSTR
Input CD-HIT .clstr file
-output OUTPUT Output TSV file to store cluster summaries - Will add '.tsv' if not
provided by user
-genome_num GENOME_NUM
The total number of genomes must be provide
Optional Arguments:
-output_dir OUTPUT_DIR
Default: Same as input file
Misc Parameters:
-verbose Print verbose output.
-v, --version Print out version number and exit
```
### All example input and output data can be found in the 'test_data' directory.
Raw data
{
"_id": null,
"home_page": "https://github.com/NickJD/PyamilySeq",
"name": "PyamilySeq",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Nicholas Dimonaco",
"author_email": "nicholas@dimonaco.co.uk",
"download_url": "https://files.pythonhosted.org/packages/88/74/dde572ec7c54b01bfda5021e2ed8999626cd137377ba599872e9506a27b1/pyamilyseq-1.0.1.tar.gz",
"platform": null,
"description": "# PyamilySeq \n**PyamilySeq** is a Python tool for clustering gene sequences into groups based on sequence similarity identified by tools such as CD-HIT, BLAST, DIAMOND or MMseqs2.\nThis work is an extension of the gene family / pangenome tool developed for the StORF-Reporter publication in NAR (https://doi.org/10.1093/nar/gkad814).\n\n## Features\n- **End-to-End**: PyamilySeq can take a directory of GFF+FASTA files, run CD-HIT for clustering and process the results.\n- **Clustering input**: Supports input from CD-HIT formatted files as well as CSV and TSV node-edge lists (MMseqs2 and -outfmt 6 from BLAST/DIAMOND).\n- **Reclustering**: Allows for the addition of new sequences post-initial clustering - Ensures continuity of contemporary clustering results and highlights impact of novel gene predictions.\n- **'Genus Mode'**: Unlike other 'pangenome' tools, PyamilySeq can identify gene groups found across multiple genera as unique entities (see below). \n- **Output**: Generates a 'Roary/Panaroo' formatted presence-absence CSV formatted file for downstream analysis.\n - User-define species-/genus-wide gene groups - User has control over grouping parameters (core = 99/95% or min 6 genera etc).\n - Aligns representative sequences using MAFFT.\n - Output concatenated aligned sequences for tree building.\n - Optionally output sequences of each separate identified gene group.\n - Group-Splitter tool to split multi-copy gene groups.\n - Numerous additional tools to assist in the pre- and post-processing of data.\n\n## Installation\nPyamilySeq probably requires Python 3.6 or higher and the levenshtein library (https://pypi.org/project/Levenshtein/) - \\\nIf levenshtein is not available, a Python implementation is utilised which is significantly slower. \n#### Install using pip:\n\n```bash\npip install PyamilySeq [optionally add -U]\n```\nPyamilySeq is currently still under active development so expect 'regular' updates with bugfixes and new features. \\\nTo update to the newest version add '-U' to end of the pip install command.\n## Example usage: Below are two examples of running PyamilySeq in its two main modes.\n```commandline\nusage: PyamilySeq.py [-h] {Full,Partial} ...\n\nPyamilySeq v1.0.1: A tool for gene clustering and analysis.\n\npositional arguments:\n {Full,Partial} Choose a mode: 'Full' or 'Partial'.\n Full Full mode: PyamilySeq to cluster with CD-HIT and process output.\n Partial Partial mode: PyamilySeq to process pre-clustered data.\n\noptions:\n -h, --help show this help message and exit\n\n```\n### 'Full Mode': Will conduct clustering of sequences with CD-HIT as part of PyamilySeq run\n```\nPyamilySeq Full -output_dir .../PyamilySeq_10_AA_90_80_Full_GFFs -input_type combined -input_dir .../genomes/ -name_split _combined.gff3\n```\n### 'Partial Mode': Will process the output of a sequence clustering from MMseqs, BLAST, DIAMOND etc.\n```\nPyamilySeq Partial -clustering_format CD-HIT -cluster_file .../all_10_combined_pep_CD-HIT_90_80.clstr -original_fasta .../all_10_combined_pep.fasta -output_dir .../PyamilySeq_10_AA_90_80_Partial-w 99 -a \n```\n\n\n#### Note: using a '-clustering_format' other than the default CD-HIT, requires input to be two in two columns as below (Same format as MMseqs2 tsv and BLAST outfmt 6) - Genome name and sequence name are separated by '|'.\n```\nEscherichia_coli_110957|ENSB:lL-zIKt-gh0oSno\tEscherichia_coli_110957|ENSB:lL-zIKt-gh0oSno\nEscherichia_coli_110957|ENSB:lL-zIKt-gh0oSno\tEscherichia_coli_113290|ENSB:2fj4rJ8e8Z9PNdX\nEscherichia_coli_110957|ENSB:lL-zIKt-gh0oSno\tEscherichia_coli_b185|ENSB:G_PVe28-ej8q-3S\nEscherichia_coli_110957|ENSB:TIZS9kbTvShDvyX\tEscherichia_coli_110957|ENSB:TIZS9kbTvShDvyX\n```\n### Example output:\n```\nRunning PyamilySeq v1.0.1\nCalculating Groups\nNumber of Genomes: 10\nGene Groups\nFirst_core_99: 2994\nFirst_core_95: 0\nFirst_core_15: 3266\nFirst_core_0: 5466\nTotal Number of First Gene Groups (Including Singletons): 11726\nOutputting gene_presence_absence file\nOutputting gene group FASTA files\nCombined FASTA file saved to: ../combined_group_sequences_dna.fasta\nProcessing gene group alignment\nThank you for using PyamilySeq -- A detailed user manual can be found at https://github.com/NickJD/PyamilySeq\nPlease report any issues to: https://github.com/NickJD/PyamilySeq/issues\n```\n\n[//]: # (## Genus mode: )\n\n[//]: # (### In addition to \"Species mode\" (see above) which reports gene groups the same as pangenome tools such as Roary and Panaroo, Genus mode reports gene groups identified across multiple genera.)\n\n[//]: # (#### Example:)\n\n[//]: # (```)\n\n[//]: # (PyamilySeq -run_mode Partial -group_mode Genus -clustering_format CD-HIT -output_dir .../test_data/genus/testing/)\n\n[//]: # ( -cluster_file .../test_data/genus/CD-HIT/combined_cds_cd-hit_80_60.clstr -gpa )\n\n[//]: # (```)\n\n[//]: # (```commandline)\n\n[//]: # (Running PyamilySeq v1.0.1)\n\n[//]: # (Calculating Groups)\n\n[//]: # (Genus Groups:)\n[//]: # (First_genera_1:\t28549)\n\n[//]: # (First_genera_2:\t12)\n\n[//]: # (First_genera_3:\t0)\n\n[//]: # (First_genera_>:\t0)\n\n[//]: # (Total Number of First Gene Groups (Including Singletons): 28561)\n\n[//]: # (Outputting gene_presence_absence file)\n\n[//]: # (Thank you for using PyamilySeq -- A detailed user manual can be found at https://github.com/NickJD/PyamilySeq)\n\n[//]: # (Please report any issues to: https://github.com/NickJD/PyamilySeq/issues)\n\n[//]: # (#####)\n\n[//]: # (```)\n\n## Reclustering:\n### Reclustering can be used to see where additional sequences/genes lay in relation to a contemporary pangenome/gene grouping.\n```\nPyamilySeq Partial -clustering_format CD-HIT -cluster_file .../all_10_combined_pep_CD-HIT_90_80.clstr -reclustered .../all_10_combined_pep_CD-HIT_90_80_AND_StORFs_CD-HIT_90_80.clstr -original_fasta .../all_10_combined_pep_AND_StORFs.fasta -output_dir .../PyamilySeq_10_AA_90_80_Partial_Reclustered_StORFs -w 99 -a \n```\n#### As can be seen below, the additional sequences recovered by the StORF-Reporter annotation tool have 'extended' contemporary or created entirely new gene groups. 'First' corresponds to the groups identified from the first clustering round and 'Second' for the second. In 'reclustering' mode, First_core_# groups are unaffected thus retaining the initial grouping information. \n```commandline\nNumber of Genomes: 10\nGene Groups\nFirst_core_99: 2994\nFirst_core_95: 0\nFirst_core_15: 3266\nFirst_core_0: 5466\nextended_core_99: 3\nextended_core_95: 0\nextended_core_15: 49\nextended_core_0: 0\ncombined_core_99: 0\ncombined_core_95: 0\ncombined_core_15: 3\ncombined_core_0: 0\nSecond_core_99: 0\nSecond_core_95: 0\nSecond_core_15: 20\nSecond_core_0: 39\nonly_Second_core_99: 768\nonly_Second_core_95: 0\nonly_Second_core_15: 4472\nonly_Second_core_0: 8395\nTotal Number of First Gene Groups (Including Singletons): 11726\nTotal Number of Second Gene Groups (Including Singletons): 25359\nTotal Number of First Gene Groups That Had Additional Second Sequences But Not New Genomes: 5\n```\n\n## PyamilySeq is separated into two main 'run modes', Full and Partial. They each have their own set of required and optional arguments.\n### PyamilySeq - Full Menu: \n```\nusage: PyamilySeq.py Full [-h] -output_dir OUTPUT_DIR -input_type {separate,combined,fasta} [-input_dir INPUT_DIR]\n [-input_fasta INPUT_FASTA] [-name_split NAME_SPLIT] [-sequence_type {AA,DNA}] [-gene_ident GENE_IDENT]\n [-c PIDENT] [-s LEN_DIFF] [-fast_mode] [-group_mode {Species,Genus}] [-species_groups SPECIES_GROUPS]\n [-genus_groups GENUS_GROUPS] [-w WRITE_GROUPS] [-wi] [-a] [-align_aa] [-no_gpa] [-M MEM] [-T THREADS]\n [-verbose] [-v]\n\noptions:\n -h, --help show this help message and exit\n -output_dir OUTPUT_DIR\n Directory for all output files.\n -input_type {separate,combined,fasta}\n Type of input files: 'separate' for matching FASTA and GFF files, 'combined' for GFF+FASTA, or 'fasta'\n for a prepared FASTA file.\n -input_dir INPUT_DIR Directory containing GFF/FASTA files - Use with -input_type separate/combined.\n -input_fasta INPUT_FASTA\n Input FASTA file - Use with - input_type fasta.\n -name_split NAME_SPLIT\n Substring to split filenames and extract genome names (e.g., '_combined.gff3') - Use with -input_type\n separate/combined.\n -sequence_type {AA,DNA}\n Clustering mode: 'DNA' or 'AA'.\n -gene_ident GENE_IDENT\n Gene identifiers to extract sequences (e.g., 'CDS, tRNA').\n -c PIDENT Sequence identity threshold for clustering (default: 0.90) - CD-HIT parameter '-c'.\n -s LEN_DIFF Length difference threshold for clustering (default: 0.80) - CD-HIT parameter '-s'.\n -fast_mode Enable fast mode for CD-HIT (not recommended) - CD-HIT parameter '-g'.\n -group_mode {Species,Genus}\n Grouping mode: 'Species' or 'Genus'.\n -species_groups SPECIES_GROUPS\n Gene groupings for 'Species' mode (default: '99,95,15').\n -genus_groups GENUS_GROUPS\n Gene groupings for 'Genus' mode (default: '1-10').\n -w WRITE_GROUPS Output gene groups as a single FASTA file (specify levels: e.g., '-w 99,95').\n -wi Output individual FASTA files for each group.\n -a Align and concatenate sequences for 'core' groups.\n -align_aa Align sequences as amino acids.\n -no_gpa Skip creation of gene_presence_absence.csv.\n -M MEM Memory allocation for clustering (MB) - CD-HIT parameter '-M'.\n -T THREADS Number of threads for clustering/alignment - CD-HIT parameter '-T' | MAFFT parameter '--thread'.\n -verbose Print verbose output.\n -v, --version Print version number and exit.\n```\n### PyamilySeq - Partial Menu: \n```commandline\nusage: PyamilySeq.py Partial [-h] -clustering_format {CD-HIT,MMseqs,BLAST} -cluster_file CLUSTER_FILE -original_fasta ORIGINAL_FASTA -output_dir OUTPUT_DIR\n [-reclustered RECLUSTERED] [-seq_tag SEQUENCE_TAG] [-group_mode {Species,Genus}] [-species_groups SPECIES_GROUPS] [-genus_groups GENUS_GROUPS]\n [-w WRITE_GROUPS] [-wi] [-a] [-align_aa] [-no_gpa] [-M MEM] [-T THREADS] [-verbose] [-v]\n\noptions:\n -h, --help show this help message and exit\n -clustering_format {CD-HIT,MMseqs,BLAST}\n Clustering format used: CD-HIT, MMseqs2, or BLAST.\n -cluster_file CLUSTER_FILE\n Cluster file containing pre-clustered groups from CD-HIT, MMseqs, BLAST etc.\n -original_fasta ORIGINAL_FASTA\n FASTA file used in pre-clustering (Provide sequences in DNA form).\n -output_dir OUTPUT_DIR\n Directory for all output files.\n -reclustered RECLUSTERED\n Clustering output file from a second round of clustering.\n -seq_tag SEQUENCE_TAG\n Tag for distinguishing reclustered sequences.\n -group_mode {Species,Genus}\n Grouping mode: 'Species' or 'Genus'.\n -species_groups SPECIES_GROUPS\n Gene groupings for 'Species' mode (default: '99,95,15').\n -genus_groups GENUS_GROUPS\n Gene groupings for 'Genus' mode (default: '1-10').\n -w WRITE_GROUPS Output gene groups as a single FASTA file (specify levels: e.g., '-w 99,95').\n -wi Output individual FASTA files for each group.\n -a Align and concatenate sequences for 'core' groups.\n -align_aa Align sequences as amino acids.\n -no_gpa Skip creation of gene_presence_absence.csv.\n -M MEM Memory allocation for clustering (MB) - CD-HIT parameter '-M'.\n -T THREADS Number of threads for clustering/alignment - CD-HIT parameter '-T' | MAFFT parameter '--thread'.\n -verbose Print verbose output.\n -v, --version Print version number and exit.\n\n```\n\n## Seq-Combiner: This tool is provided to enable the pre-processing of multiple GFF/FASTA files together ready to be clustered by the user.\n### Example:\n```bash\nSeq-Combiner -input_dir .../test_data/genomes -name_split .gff3 -output_dir .../test_data/genomes -output_name combine_fasta_seqs.fa -input_type combined\n```\n### Seq-Combiner Menu:\n```\nusage: Seq_Combiner.py [-h] -input_dir INPUT_DIR -input_type {separate,combined,fasta} -name_split NAME_SPLIT -output_dir OUTPUT_DIR -output_name\n OUTPUT_FILE [-gene_ident GENE_IDENT] [-translate] [-v]\n\nPyamilySeq v1.0.1: Seq-Combiner - A tool to extract sequences from GFF/FASTA files and prepare them for PyamilySeq.\n\noptions:\n -h, --help show this help message and exit\n\nRequired Arguments:\n -input_dir INPUT_DIR Directory location where the files are located.\n -input_type {separate,combined,fasta}\n Type of input files: \"separate\" for separate FASTA and GFF files, \"combined\" for GFF files with embedded FASTA sequences and \"fasta\"\n for combining multiple FASTA files together.\n -name_split NAME_SPLIT\n substring used to split the filename and extract the genome name ('_combined.gff3' or '.gff').\n -output_dir OUTPUT_DIR\n Directory for all output files.\n -output_name OUTPUT_FILE\n Output file name.\n\nOptional Arguments:\n -gene_ident GENE_IDENT\n Default - \"CDS\": Identifier used for extraction of sequences such as\n \"misc_RNA,gene,mRNA,CDS,rRNA,tRNA,tmRNA,CRISPR,ncRNA,regulatory_region,oriC,pseudo\" - Not compatible with \"fasta\" input mode.\n -translate Default - False: Translate extracted sequences to their AA counterpart?\n\nMisc Arguments:\n -v, --version Print out version number and exit\n\n```\n\n## Group-Splitter: This tool can split multi-copy gene groups using CD-HIT after initial PyamilySeq analysis.\n### Example:\n```bash\nGroup-Splitter -genome_num 74 -input_fasta .../test/species/ -output_dir .../test/species/ -sequence_type AA\n```\n### Group-Splitter Menu:\n```\nusage: Group_Splitter.py [-h] -input_fasta INPUT_FASTA -sequence_type {AA,DNA}\n -genome_num GENOME_NUM -output_dir OUTPUT_DIR\n [-groups GROUPS] [-group_threshold GROUP_THRESHOLD]\n [-c PIDENT] [-s LEN_DIFF] [-T CLUSTERING_THREADS]\n [-M CLUSTERING_MEMORY] [-no_delete_temp_files]\n [-verbose] [-v]\n\nPyamilySeq v1.0.1: Group-Splitter - A tool to split multi-copy gene groups\nidentified by PyamilySeq.\n\noptions:\n -h, --help show this help message and exit\n\nRequired Parameters:\n -input_fasta INPUT_FASTA\n Input FASTA file containing gene groups.\n -sequence_type {AA,DNA}\n Default - DNA: Are groups \"DNA\" or \"AA\" sequences?\n -genome_num GENOME_NUM\n The total number of genomes must be provide\n -output_dir OUTPUT_DIR\n Output directory.\n\nRegrouping Parameters:\n -groups GROUPS Default - auto: Detect groups to be split (see\n -group_threshold). Provide \"-groups 1,2,3,4\" with\n group IDs to split specific groups.\n -group_threshold GROUP_THRESHOLD\n Minimum percentage of genomes with multi-copy\n (default: 80.0) - Does not work with \"-groups\"\n\nCD-HIT Reclustering Parameters:\n -c PIDENT Sequence identity threshold (default: 0.8) - Probably\n should be higher than what was used in initial\n clustering.\n -s LEN_DIFF Length difference cutoff (default: 0.20) - Often the\n most impactful parameter to split 'multi-copy' gene\n groups.\n -T CLUSTERING_THREADS\n Number of threads for clustering (default: 4)\n -M CLUSTERING_MEMORY Memory limit in MB for clustering (default: 2000)\n\nMisc Parameters:\n -no_delete_temp_files\n Default: Delete all temporary files after processing.\n -verbose Print verbose output.\n -v, --version Print out version number and exit\n\n```\n\n## Cluster-Summary menu: This tool can be used to summarise CD-HIT .clstr files:\n### Example:\n```bash\nCluster-Summary -genome_num 74 -input_clstr .../test_data/species/E-coli/E-coli_extracted_pep_cd-hit_80.clstr -output_tsv .../test_data/species/E-coli/E-coli_extracted_pep_cd-hit_80_Summary.tsv\n```\n### Cluster-Summary Menu: \n```\nusage: Cluster_Summary.py [-h] -input_clstr INPUT_CLSTR -output OUTPUT -genome_num GENOME_NUM\n [-output_dir OUTPUT_DIR] [-verbose] [-v]\n\nPyamilySeq v1.0.1: Cluster-Summary - A tool to summarise CD-HIT clustering files.\n\noptions:\n -h, --help show this help message and exit\n\nRequired Parameters:\n -input_clstr INPUT_CLSTR\n Input CD-HIT .clstr file\n -output OUTPUT Output TSV file to store cluster summaries - Will add '.tsv' if not\n provided by user\n -genome_num GENOME_NUM\n The total number of genomes must be provide\n\nOptional Arguments:\n -output_dir OUTPUT_DIR\n Default: Same as input file\n\nMisc Parameters:\n -verbose Print verbose output.\n -v, --version Print out version number and exit\n\n```\n\n### All example input and output data can be found in the 'test_data' directory.\n",
"bugtrack_url": null,
"license": null,
"summary": "PyamilySeq - A a tool to investigate sequence-based gene groups identified by clustering methods such as CD-HIT, DIAMOND, BLAST or MMseqs2.",
"version": "1.0.1",
"project_urls": {
"Bug Tracker": "https://github.com/NickJD/PyamilySeq/issues",
"Homepage": "https://github.com/NickJD/PyamilySeq"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ef1dd56780d8c3c728c6d3479ef29d9ae7359dfda83fae013dbe4f0d5f1d9dec",
"md5": "519a56e8a485569ab926e0b10de425ef",
"sha256": "09b837099063f1da960eb1547c7656fae8306ad762a7b9389fc460aa01d97c09"
},
"downloads": -1,
"filename": "PyamilySeq-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "519a56e8a485569ab926e0b10de425ef",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 50882,
"upload_time": "2024-11-20T06:06:33",
"upload_time_iso_8601": "2024-11-20T06:06:33.132119Z",
"url": "https://files.pythonhosted.org/packages/ef/1d/d56780d8c3c728c6d3479ef29d9ae7359dfda83fae013dbe4f0d5f1d9dec/PyamilySeq-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8874dde572ec7c54b01bfda5021e2ed8999626cd137377ba599872e9506a27b1",
"md5": "c5986c8629c3b72d9102e724f145731e",
"sha256": "bbc406f83a95ed5790de0c2bfd7e93f95fc079e05d27bfd1172a5abab1c4f81b"
},
"downloads": -1,
"filename": "pyamilyseq-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "c5986c8629c3b72d9102e724f145731e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 48319,
"upload_time": "2024-11-20T06:06:34",
"upload_time_iso_8601": "2024-11-20T06:06:34.251610Z",
"url": "https://files.pythonhosted.org/packages/88/74/dde572ec7c54b01bfda5021e2ed8999626cd137377ba599872e9506a27b1/pyamilyseq-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-20 06:06:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NickJD",
"github_project": "PyamilySeq",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyamilyseq"
}
