# relecov-tools
[](https://github.com/BU-ISCIII/relecov-tools/actions/workflows/python_lint.yml)
[](https://github.com/psf/black)
> THIS REPO IS UNDER ACTIVE DEVELOPMENT.
relecov-tools is a set of helper tools for the assembly of the different elements in the RELECOV platform (Spanish Network for genomic surveillance of SARS-Cov-2) as data download, processing, validation and upload to public databases, as well as analysis runs and database storage.
## Table of contents
- [relecov-tools](#relecov-tools)
- [Table of contents](#table-of-contents)
- [Installation](#installation)
- [Bioconda](#bioconda)
- [Pip](#pip)
- [Development version](#development-version)
- [Usage](#usage)
- [Command-line](#command-line)
- [download](#download)
- [read-lab-metadata](#read-lab-metadata)
- [read-bioinfo-metadata](#read-bioinfo-metadata)
- [validate](#validate)
- [map](#map)
- [upload-to-ena](#upload-to-ena)
- [upload-to-gisaid](#upload-to-gisaid)
- [update-db](#update-db)
- [pipeline-manager](#pipeline-manager)
- [wrapper](#wrapper)
- [logs-to-excel](#logs-to-excel)
- [build-schema](#build-schema)
- [Mandatory Fields](#mandatory-fields)
- [custom logs](#custom-logs)
- [Python package mode](#python-package-mode)
- [Acknowledgements](#acknowledgements)
## Installation
### Bioconda
Soon
### Pip
relecov-tools is available in Pypi and can be installed via pip:
```
pip install relecov-tools
```
### Development version
If you want to install the latest code in the repository:
```
conda create -n relecov_dev pip
pip install --force-reinstall --upgrade git+https://github.com/bu-isciii/relecov-tools.git@develop
```
## Usage
### Command-line
relecov-tools provides a command-line version with help descriptions and params prompt if needed.
```
$ relecov-tools --help
___ ___ ___ ___ ___
\ |--| | \ | | | | | | \ /
\ \ / |__ / |__ | |___ | | | \ /
/ / \ | \ | | | | | | \ /
/ |--| | \ |___ |___ |___ |___ |___| \/
RELECOV-tools version 1.2.0
Usage: relecov-tools [OPTIONS] COMMAND [ARGS]...
Options:
--version Show the version and exit.
-v, --verbose Print verbose output to the console.
-l, --log-file <filename> Save a verbose log to a file.
--help Show this message and exit.
Commands:
download Download files located in sftp server.
read-lab-metadata Create the json compliant to the relecov schema...
validate Validate json file against schema.
map Convert data between phage plus schema to ENA,...
upload-to-ena parse data to create xml files to upload to ena
upload-to-gisaid parsed data to create files to upload to gisaid
update-db upload the information included in json file to...
read-bioinfo-metadata Create the json compliant from the Bioinfo...
metadata-homogeneizer Parse institution metadata lab to the one used...
pipeline-manager Create the symbolic links for the samples which...
wrapper Execute download, read-lab-metadata and validate...
build-schema Generates and updates JSON Schema files from...
logs-to-excel Creates a merged xlsx report from all the log...
```
#### download
The command `download` connects to a transfer protocol (currently sftp) and downloads all files in the different available folders in the passed credentials. In addition, it checks if the files in the current folder match the files in the metadata file and also checks if there are md5sum for each file. Else, it creates one before storing in the final repository.
```
$ relecov-tools download --help
Usage: relecov-tools download [OPTIONS]
Download files located in sftp server.
Options:
-u, --user User name for login to sftp server
-p, --password Password for the user to login
-d, --download_option Select the download option: [download_only, download_clean, delete_only].
download_only will only download the files
download_clean will remove files from sftp after download
delete_only will only delete the files
-o, --output_location Flag: Select location for downloaded files, overrides config file location
-t, --target_folders Flag: Select which sftp folders will be targeted giving [paths] or via prompt
-f, --conf_file Configuration file in yaml format (no params file)
--help Show this message and exit.
```
Configuration can be passed in several formats:
- if no config_file is passed, default values are fetched from conf/configuration.json, and user and password are asked in prompt.
- Default values can be overwritten using a yml config file, so you can input user, password, sftp_server, etc.
Config file example with all available options:
```
sftp_server: "sftprelecov.isciii.es"
sftp_port: "22"
sftp_user : "user"
sftp_passwd : "pass"
storage_local_folder: "/tmp/relecov"
tmp_folder_for_metadata: "/tmp/relecov/tmp"
allowed_sample_extensions:
- .fastq.gz
- .fasta
```
#### read-lab-metadata
`read-lab-metadata` command reads the excel file with laboratory metadata and processes it adding additional needed fields.
```
$ relecov-tools read-lab-metadata --help
Usage: relecov-tools read-metadata [OPTIONS]
Create the json compliant to the relecov schema from the Metadata file.
Options:
-m, --metadata_file PATH file containing metadata in xlsx format.
-s, --sample_list_file PATH Json with the additional metadata to add to the
received user metadata.
-o, --metadata-out PATH Path to save output metadata file in json format.
--help Show this message and exit.
```
An example for the metadata excel file can be found [here](./relecov_tools/example_data/METADATA_LAB_TEST.xlsx)
#### read-bioinfo-metadata
`read-bioinfo-metadata` Include the results from the Bioinformatics analysis into the Json previously created with read-lab-metadata module.
```
$ relecov-tools read-bioinfo-metadata --help
Usage: relecov-tools read-bioinfo-metadata [OPTIONS]
Create the json compliant to the relecov schema with Bioinfo Metadata.
Options:
-j, --json_file Json file containing lab metadata
-i, --input_folder Path to folder containing analysis results
-s, --software_name Name of the software employed in the bioinformatics analysis (default: viralrecon).
-o, --out_dir Path to save output file"
```
- Note: Software-specific configurations are available in [bioinfo_config.json](./relecov_tools/conf/bioinfo_config.json).
#### validate
`validate` commands validate the data in json format outputted by `read-metadata` command against a json schema, in this case the relecov [schema specification](./relecov_tools/schema/relecov_schema.json). It also creates a summary of the errors and warnings found in excel format as a report to the users.
```
$ relecov-tools validate --help
Usage: relecov-tools validate [OPTIONS]
Validate json file against schema.
Options:
-j, --json_file TEXT Json file to validate
-s, --json_schema TEXT Json schema (default: relecov-schema)
-m, --metadata PATH Origin file containing metadata
-o, --out_folder TEXT Path to save validate json file
--help Show this message and exit.
```
#### map
The command `map` converts a data in json format from relecov data model to ena or gisaid data model using their own schemas acording to their annotated ontology terms.
```
$ relecov-tools map --help
Usage: relecov-tools map [OPTIONS]
Convert data between phage plus schema to ENA, GISAID, or any other schema
Options:
-p, --origin_schema TEXT File with the origin (relecov) schema
-j, --json_data TEXT File with the json data to convert
-d, --destination_schema [ENA|GSAID|other]
schema to be mapped
-f, --schema_file TEXT file with the custom schema
-o, --output TEXT File name and path to store the mapped json
--help Show this message and exit.
```
#### upload-to-ena
`upload-to-ena` command uses json data mapped to ena schema to use the [ena_upload_cli](https://github.com/usegalaxy-eu/ena-upload-cli) package to upload raw data and metadata to ENA db.
```
Usage: relecov-tools upload-to-ena [OPTIONS]
parsed data to create xml files to upload to ena
Options:
-u, --user user name for login to ena
-p, --password password for the user to login
-c, --center center name
-e, --ena_json where the validated json is
-t, --template_path path to folder containing ENA xml templates
-a, --action select one of the available options: [add|modify|cancel|release]
--dev Flag: Test submission
--upload_fastq Flag: Upload fastq files. Mandatory for "add" action
-m", --metadata_types List of metadata xml types to submit [study,experiment,run,sample]
-o, --output_path TEXT output folder for the xml generated files
--help Show this message and exit.
```
#### upload-to-gisaid
`upload-to-gisaid` uses the json mapped to gisaid schema to upload raw data and metadata to GISAID db
```
Usage: relecov-tools upload-to-gisaid [OPTIONS]
parsed data to create xml files to upload to ena
Options:
-u, --user user name for login
-p, --password password for the user to login
-c, --client_id client-ID provided by clisupport@gisaid.org
-t, --token path to athentication token
-e, --gisaid_json path to validated json mapped to GISAID
-i, --input_path path to fastas folder or multifasta file
-f, --frameshift frameshift notification: ["catch_all", "catch_none", "catch_novel"]
-x, --proxy_config introduce your proxy credentials as: username:password@proxy:port
--single Flag: input is a folder with several fasta files.
--gzip Flag: input fasta is gziped.
```
#### update-db
-u, --user user name for login
-p, --password password for the user to login
-t, --type Select the type of information to upload to database [sample,bioinfodata,variantdata]
-d, --databaseServer Name of the database server receiving the data [iskylims,relecov]
#### pipeline-manager
Create the folder structure to execute the given pipeline for the latest sample batches after executing download, read-lab-metadata and validate modules. This module will create symbolic links for each sample and generate the necessary files for pipeline execution using the information from validated_BATCH-NAME_DATE.json.
```
Usage: relecov-tools pipeline-manager [OPTIONS]
Create the symbolic links for the samples which are validated to prepare for
bioinformatics pipeline execution.
Options:
-i, --input PATH Path to the input folder where sample files are located
-t, --template PATH Path to the pipeline template folder to be copied in the output folder
-c, --config PATH Path to the the template config file
-o, --out_dir PATH Path to output folder
--help Show this message and exit.
```
#### wrapper
Execute download, read-lab-metadata and validate sequentially using a config file to fill the arguments for each one. It also creates a global report with all the logs for the three processes in a user-friendly .xlsx format. The config file should include the name of each module that is executed, along with the necessary parameters in YAML format.
```
Usage: relecov-tools wrapper [OPTIONS]
Executes the modules in config file sequentially
Options:
-c, --config_file PATH Path to config file in yaml format [required]
-o, --output_folder PATH Path to folder where global results are saved [required]
--help Show this message and exit.
```
#### logs-to-excel
Creates an xlsx file with all the entries found for a specified laboratory in a given set of log_summary.json files (from log-summary module). The laboratory name must match the name of one of the keys in the provided logs to work.
```
Usage: relecov-tools logs-to-excel [OPTIONS]
Creates a merged xlsx report from all the log summary jsons given as input
Options:
-l, --lab_name Name for target laboratory in log-summary.json files
-o, --output_folder Path to output folder where xlsx file is saved
-f, --files Paths to log_summary.json files to merge into xlsx file, called once per file
```
### build-schema
The `build-schema` module provides functionality to generate and manage JSON Schema files based on database definitions from Excel spreadsheets. It automates the creation of JSON Schemas, including validation, drafting, and comparison with existing schemas.
```
Usage: relecov-tools build-schema [OPTIONS]
Generates and updates JSON Schema files from Excel-based database
definitions.
Options:
-i, --input_file PATH Path to the Excel document containing the database
definition. This file must have a .xlsx extension.
[required]
-s, --schema_base PATH Path to the base schema file. This file is used as
a reference to compare it with the schema
generated using this module. (Default: installed
schema in 'relecov-
tools/relecov_tools/schema/relecov_schema.json')
-v, --draft_version TEXT Version of the JSON schema specification to be
used. Example: '2020-12'. See: https://json-
schema.org/specification-links
-d, --diff BOOLEAN Prints a changelog/diff between the base and
incoming versions of the schema.
-o, --out_dir PATH Path to save output file/s
--help Show this message and exit.
```
#### Mandatory Fields
Ensure that the fields below are properly defined as headers in your Excel sheet (database definition):
```
enum: List of possible values for enumeration.
examples: Example values for the property.
ontology_id: Identifier for ontology.
type: Data type of the property (e.g., string, integer).
description: Description of the property.
classification: Classification or category of the property.
label_name: Label or name for the property.
fill_mode: Mode for filling in the property (e.g., required, optional).
required (Y/N): Indicates if the property is required (Y) or optional (N).
complex_field (Y/N): Indicates if the property is a complex (nested) field (Y) or a standard field (N).
```
#### custom logs
After executing each of these modules, you may find a custom log report in json format named "DATE_EXECUTED-MODULE_log_summary.json. These custom log summaries can be useful to detect errors in metadata in order to fix them and/or notify the users.
### Python package mode
relecov-tools is designed in a way that you can use import the different modules and use them in your own scripts, for example:
```
import relecov_tools.sftp_handle
user="admin"
passwd="1234"
conf_file="/path/to/conf"
sftp_connection = relecov_tools.sftp_handle.SftpHandle(
user, password, conf_file
)
sftp_connection.download()
```
DOCs soon!!
## Acknowledgements
Python package idea and design is really inspired in [nf-core/tools](https://github.com/nf-core/tools).
Raw data
{
"_id": null,
"home_page": "https://github.com/BU-ISCIII/relecov-tools",
"name": "relecov-tools",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "buisciii, bioinformatics, pipeline, sequencing, NGS, next generation sequencing",
"author": "Sara Monzon",
"author_email": "smonzon@isciii.es",
"download_url": "https://files.pythonhosted.org/packages/13/c0/06529da7a867d3a412cc5700a2f447d5414ed69dc5ef4dd60e89bb871ff4/relecov_tools-1.2.0.tar.gz",
"platform": null,
"description": "# relecov-tools\n[](https://github.com/BU-ISCIII/relecov-tools/actions/workflows/python_lint.yml)\n[](https://github.com/psf/black)\n\n> THIS REPO IS UNDER ACTIVE DEVELOPMENT.\n\nrelecov-tools is a set of helper tools for the assembly of the different elements in the RELECOV platform (Spanish Network for genomic surveillance of SARS-Cov-2) as data download, processing, validation and upload to public databases, as well as analysis runs and database storage.\n\n## Table of contents\n\n- [relecov-tools](#relecov-tools)\n - [Table of contents](#table-of-contents)\n - [Installation](#installation)\n - [Bioconda](#bioconda)\n - [Pip](#pip)\n - [Development version](#development-version)\n - [Usage](#usage)\n - [Command-line](#command-line)\n - [download](#download)\n - [read-lab-metadata](#read-lab-metadata)\n - [read-bioinfo-metadata](#read-bioinfo-metadata)\n - [validate](#validate)\n - [map](#map)\n - [upload-to-ena](#upload-to-ena)\n - [upload-to-gisaid](#upload-to-gisaid)\n - [update-db](#update-db)\n - [pipeline-manager](#pipeline-manager)\n - [wrapper](#wrapper)\n - [logs-to-excel](#logs-to-excel)\n - [build-schema](#build-schema)\n - [Mandatory Fields](#mandatory-fields)\n - [custom logs](#custom-logs)\n - [Python package mode](#python-package-mode)\n - [Acknowledgements](#acknowledgements)\n\n## Installation\n\n### Bioconda\nSoon\n\n### Pip\nrelecov-tools is available in Pypi and can be installed via pip:\n```\npip install relecov-tools\n```\n\n### Development version\nIf you want to install the latest code in the repository:\n\n```\nconda create -n relecov_dev pip\npip install --force-reinstall --upgrade git+https://github.com/bu-isciii/relecov-tools.git@develop\n```\n\n## Usage\n\n### Command-line\nrelecov-tools provides a command-line version with help descriptions and params prompt if needed.\n\n```\n$ relecov-tools --help\n ___ ___ ___ ___ ___\n\\ |--| | \\ | | | | | | \\ /\n\\ \\ / |__ / |__ | |___ | | | \\ /\n/ / \\ | \\ | | | | | | \\ /\n/ |--| | \\ |___ |___ |___ |___ |___| \\/\nRELECOV-tools version 1.2.0\nUsage: relecov-tools [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n--version Show the version and exit.\n-v, --verbose Print verbose output to the console.\n-l, --log-file <filename> Save a verbose log to a file.\n--help Show this message and exit.\n\nCommands:\n download Download files located in sftp server.\n read-lab-metadata Create the json compliant to the relecov schema...\n validate Validate json file against schema.\n map Convert data between phage plus schema to ENA,...\n upload-to-ena parse data to create xml files to upload to ena\n upload-to-gisaid parsed data to create files to upload to gisaid\n update-db upload the information included in json file to...\n read-bioinfo-metadata Create the json compliant from the Bioinfo...\n metadata-homogeneizer Parse institution metadata lab to the one used...\n pipeline-manager Create the symbolic links for the samples which...\n wrapper Execute download, read-lab-metadata and validate...\n build-schema Generates and updates JSON Schema files from...\n logs-to-excel Creates a merged xlsx report from all the log...\n```\n#### download\nThe command `download` connects to a transfer protocol (currently sftp) and downloads all files in the different available folders in the passed credentials. In addition, it checks if the files in the current folder match the files in the metadata file and also checks if there are md5sum for each file. Else, it creates one before storing in the final repository.\n\n```\n$ relecov-tools download --help\nUsage: relecov-tools download [OPTIONS]\n\n Download files located in sftp server.\n\n Options:\n -u, --user User name for login to sftp server\n -p, --password Password for the user to login\n -d, --download_option Select the download option: [download_only, download_clean, delete_only].\n download_only will only download the files\n download_clean will remove files from sftp after download\n delete_only will only delete the files\n -o, --output_location Flag: Select location for downloaded files, overrides config file location\n -t, --target_folders Flag: Select which sftp folders will be targeted giving [paths] or via prompt\n -f, --conf_file Configuration file in yaml format (no params file)\n --help Show this message and exit.\n```\n\nConfiguration can be passed in several formats:\n- if no config_file is passed, default values are fetched from conf/configuration.json, and user and password are asked in prompt.\n- Default values can be overwritten using a yml config file, so you can input user, password, sftp_server, etc.\n\nConfig file example with all available options:\n```\nsftp_server: \"sftprelecov.isciii.es\"\nsftp_port: \"22\"\nsftp_user : \"user\"\nsftp_passwd : \"pass\"\nstorage_local_folder: \"/tmp/relecov\"\ntmp_folder_for_metadata: \"/tmp/relecov/tmp\"\nallowed_sample_extensions:\n - .fastq.gz\n - .fasta\n```\n\n#### read-lab-metadata\n`read-lab-metadata` command reads the excel file with laboratory metadata and processes it adding additional needed fields.\n\n```\n$ relecov-tools read-lab-metadata --help\nUsage: relecov-tools read-metadata [OPTIONS]\n\n Create the json compliant to the relecov schema from the Metadata file.\n\n Options:\n -m, --metadata_file PATH file containing metadata in xlsx format.\n -s, --sample_list_file PATH Json with the additional metadata to add to the\n received user metadata.\n -o, --metadata-out PATH Path to save output metadata file in json format.\n --help Show this message and exit.\n```\n\n\nAn example for the metadata excel file can be found [here](./relecov_tools/example_data/METADATA_LAB_TEST.xlsx)\n\n#### read-bioinfo-metadata\n`read-bioinfo-metadata` Include the results from the Bioinformatics analysis into the Json previously created with read-lab-metadata module.\n\n```\n$ relecov-tools read-bioinfo-metadata --help\nUsage: relecov-tools read-bioinfo-metadata [OPTIONS]\n\n Create the json compliant to the relecov schema with Bioinfo Metadata.\n\n Options:\n -j, --json_file Json file containing lab metadata\n -i, --input_folder Path to folder containing analysis results\n -s, --software_name Name of the software employed in the bioinformatics analysis (default: viralrecon).\n -o, --out_dir Path to save output file\"\n```\n- Note: Software-specific configurations are available in [bioinfo_config.json](./relecov_tools/conf/bioinfo_config.json).\n\n#### validate\n`validate` commands validate the data in json format outputted by `read-metadata` command against a json schema, in this case the relecov [schema specification](./relecov_tools/schema/relecov_schema.json). It also creates a summary of the errors and warnings found in excel format as a report to the users.\n\n```\n$ relecov-tools validate --help\nUsage: relecov-tools validate [OPTIONS]\n\n Validate json file against schema.\n\n Options:\n -j, --json_file TEXT Json file to validate\n -s, --json_schema TEXT Json schema (default: relecov-schema)\n -m, --metadata PATH Origin file containing metadata\n -o, --out_folder TEXT Path to save validate json file\n --help Show this message and exit.\n\n```\n\n#### map\nThe command `map` converts a data in json format from relecov data model to ena or gisaid data model using their own schemas acording to their annotated ontology terms.\n\n```\n$ relecov-tools map --help\nUsage: relecov-tools map [OPTIONS]\n\n Convert data between phage plus schema to ENA, GISAID, or any other schema\n\n Options:\n -p, --origin_schema TEXT File with the origin (relecov) schema\n -j, --json_data TEXT File with the json data to convert\n -d, --destination_schema [ENA|GSAID|other]\n schema to be mapped\n -f, --schema_file TEXT file with the custom schema\n -o, --output TEXT File name and path to store the mapped json\n --help Show this message and exit.\n```\n\n#### upload-to-ena\n`upload-to-ena` command uses json data mapped to ena schema to use the [ena_upload_cli](https://github.com/usegalaxy-eu/ena-upload-cli) package to upload raw data and metadata to ENA db.\n\n```\nUsage: relecov-tools upload-to-ena [OPTIONS]\n\n parsed data to create xml files to upload to ena\n\n Options:\n -u, --user user name for login to ena\n -p, --password password for the user to login\n -c, --center center name\n -e, --ena_json where the validated json is\n -t, --template_path path to folder containing ENA xml templates\n -a, --action select one of the available options: [add|modify|cancel|release]\n --dev Flag: Test submission\n --upload_fastq Flag: Upload fastq files. Mandatory for \"add\" action\n -m\", --metadata_types List of metadata xml types to submit [study,experiment,run,sample]\n -o, --output_path TEXT output folder for the xml generated files\n --help Show this message and exit.\n\n```\n\n#### upload-to-gisaid\n`upload-to-gisaid` uses the json mapped to gisaid schema to upload raw data and metadata to GISAID db\n\n```\nUsage: relecov-tools upload-to-gisaid [OPTIONS]\n\n parsed data to create xml files to upload to ena\n\n Options:\n -u, --user user name for login\n -p, --password password for the user to login\n -c, --client_id client-ID provided by clisupport@gisaid.org\n -t, --token path to athentication token\n -e, --gisaid_json path to validated json mapped to GISAID\n -i, --input_path path to fastas folder or multifasta file\n -f, --frameshift frameshift notification: [\"catch_all\", \"catch_none\", \"catch_novel\"]\n -x, --proxy_config introduce your proxy credentials as: username:password@proxy:port\n --single Flag: input is a folder with several fasta files.\n --gzip Flag: input fasta is gziped.\n```\n\n#### update-db\n -u, --user user name for login\n -p, --password password for the user to login\n -t, --type Select the type of information to upload to database [sample,bioinfodata,variantdata]\n -d, --databaseServer Name of the database server receiving the data [iskylims,relecov]\n\n#### pipeline-manager\nCreate the folder structure to execute the given pipeline for the latest sample batches after executing download, read-lab-metadata and validate modules. This module will create symbolic links for each sample and generate the necessary files for pipeline execution using the information from validated_BATCH-NAME_DATE.json.\n```\nUsage: relecov-tools pipeline-manager [OPTIONS]\n\n Create the symbolic links for the samples which are validated to prepare for\n bioinformatics pipeline execution.\n\nOptions:\n -i, --input PATH Path to the input folder where sample files are located\n -t, --template PATH Path to the pipeline template folder to be copied in the output folder\n -c, --config PATH Path to the the template config file\n -o, --out_dir PATH Path to output folder\n --help Show this message and exit.\n```\n\n#### wrapper\nExecute download, read-lab-metadata and validate sequentially using a config file to fill the arguments for each one. It also creates a global report with all the logs for the three processes in a user-friendly .xlsx format. The config file should include the name of each module that is executed, along with the necessary parameters in YAML format.\n```\nUsage: relecov-tools wrapper [OPTIONS]\n\n Executes the modules in config file sequentially\n\nOptions:\n -c, --config_file PATH Path to config file in yaml format [required]\n -o, --output_folder PATH Path to folder where global results are saved [required]\n --help Show this message and exit.\n```\n\n#### logs-to-excel\nCreates an xlsx file with all the entries found for a specified laboratory in a given set of log_summary.json files (from log-summary module). The laboratory name must match the name of one of the keys in the provided logs to work.\n```\nUsage: relecov-tools logs-to-excel [OPTIONS]\n\n Creates a merged xlsx report from all the log summary jsons given as input\n\nOptions:\n -l, --lab_name Name for target laboratory in log-summary.json files\n -o, --output_folder Path to output folder where xlsx file is saved\n -f, --files Paths to log_summary.json files to merge into xlsx file, called once per file\n```\n\n### build-schema\nThe `build-schema` module provides functionality to generate and manage JSON Schema files based on database definitions from Excel spreadsheets. It automates the creation of JSON Schemas, including validation, drafting, and comparison with existing schemas.\n\n```\nUsage: relecov-tools build-schema [OPTIONS]\n\n Generates and updates JSON Schema files from Excel-based database\n definitions.\n\nOptions:\n -i, --input_file PATH Path to the Excel document containing the database\n definition. This file must have a .xlsx extension.\n [required]\n -s, --schema_base PATH Path to the base schema file. This file is used as\n a reference to compare it with the schema\n generated using this module. (Default: installed\n schema in 'relecov-\n tools/relecov_tools/schema/relecov_schema.json')\n -v, --draft_version TEXT Version of the JSON schema specification to be\n used. Example: '2020-12'. See: https://json-\n schema.org/specification-links\n -d, --diff BOOLEAN Prints a changelog/diff between the base and\n incoming versions of the schema.\n -o, --out_dir PATH Path to save output file/s\n --help Show this message and exit.\n```\n\n#### Mandatory Fields\nEnsure that the fields below are properly defined as headers in your Excel sheet (database definition):\n\n```\nenum: List of possible values for enumeration.\nexamples: Example values for the property.\nontology_id: Identifier for ontology.\ntype: Data type of the property (e.g., string, integer).\ndescription: Description of the property.\nclassification: Classification or category of the property.\nlabel_name: Label or name for the property.\nfill_mode: Mode for filling in the property (e.g., required, optional).\nrequired (Y/N): Indicates if the property is required (Y) or optional (N).\ncomplex_field (Y/N): Indicates if the property is a complex (nested) field (Y) or a standard field (N).\n```\n\n#### custom logs\nAfter executing each of these modules, you may find a custom log report in json format named \"DATE_EXECUTED-MODULE_log_summary.json. These custom log summaries can be useful to detect errors in metadata in order to fix them and/or notify the users.\n\n### Python package mode\nrelecov-tools is designed in a way that you can use import the different modules and use them in your own scripts, for example:\n\n```\nimport relecov_tools.sftp_handle\nuser=\"admin\"\npasswd=\"1234\"\nconf_file=\"/path/to/conf\"\n\nsftp_connection = relecov_tools.sftp_handle.SftpHandle(\n user, password, conf_file\n)\nsftp_connection.download()\n```\n\nDOCs soon!!\n\n\n## Acknowledgements\nPython package idea and design is really inspired in [nf-core/tools](https://github.com/nf-core/tools).\n",
"bugtrack_url": null,
"license": "GNU GENERAL PUBLIC LICENSE v.3",
"summary": "Tools for managing and resolution of buisciii services.",
"version": "1.2.0",
"project_urls": {
"Homepage": "https://github.com/BU-ISCIII/relecov-tools"
},
"split_keywords": [
"buisciii",
" bioinformatics",
" pipeline",
" sequencing",
" ngs",
" next generation sequencing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "50590454acb7bd2299c4f36bca6f77cc1737e1c0f207faffe32afa23d0ca9c8a",
"md5": "379c8f3b6210c9c39337846732936321",
"sha256": "1e8aa0f4f6d91c505624175843477b347456d0462b5034cc88b98b24e9013d77"
},
"downloads": -1,
"filename": "relecov_tools-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "379c8f3b6210c9c39337846732936321",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 446949,
"upload_time": "2024-10-11T15:59:26",
"upload_time_iso_8601": "2024-10-11T15:59:26.478618Z",
"url": "https://files.pythonhosted.org/packages/50/59/0454acb7bd2299c4f36bca6f77cc1737e1c0f207faffe32afa23d0ca9c8a/relecov_tools-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "13c006529da7a867d3a412cc5700a2f447d5414ed69dc5ef4dd60e89bb871ff4",
"md5": "a9fdc3f5532179ba5dc5d35c31577d58",
"sha256": "9563109368fdda60ab1f6563526eedd5b91ed276ae6378ac65b88a98efa7af2d"
},
"downloads": -1,
"filename": "relecov_tools-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "a9fdc3f5532179ba5dc5d35c31577d58",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 436438,
"upload_time": "2024-10-11T15:59:28",
"upload_time_iso_8601": "2024-10-11T15:59:28.888754Z",
"url": "https://files.pythonhosted.org/packages/13/c0/06529da7a867d3a412cc5700a2f447d5414ed69dc5ef4dd60e89bb871ff4/relecov_tools-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-11 15:59:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "BU-ISCIII",
"github_project": "relecov-tools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "click",
"specs": []
},
{
"name": "questionary",
"specs": []
},
{
"name": "jsonschema",
"specs": []
},
{
"name": "packaging",
"specs": []
},
{
"name": "prompt_toolkit",
"specs": [
[
">=",
"3.0.3"
]
]
},
{
"name": "rich",
"specs": [
[
">=",
"10.0.0"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.27.1"
]
]
},
{
"name": "paramiko",
"specs": [
[
">=",
"2.10.1"
]
]
},
{
"name": "pyyaml",
"specs": [
[
"==",
"6.0.1"
]
]
},
{
"name": "openpyxl",
"specs": [
[
">=",
"3.1.2"
]
]
},
{
"name": "ena-upload-cli",
"specs": []
},
{
"name": "bio",
"specs": [
[
"==",
"1.4.0"
]
]
},
{
"name": "xlsxwriter",
"specs": [
[
"==",
"3.2.0"
]
]
},
{
"name": "bs4",
"specs": [
[
"==",
"0.0.2"
]
]
},
{
"name": "tabulate",
"specs": []
},
{
"name": "pandas",
"specs": []
}
],
"tox": true,
"lcname": "relecov-tools"
}
JSON
