# DeepRank2
| Badges | |
| :------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **fairness** | [](https://fair-software.eu) [](https://bestpractices.coreinfrastructure.org/projects/6403) |
| **package** | [](https://badge.fury.io/py/deeprank2) [](https://app.codacy.com/gh/DeepRank/deeprank2/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade) |
| **docs** | [](https://deeprank2.readthedocs.io/en/latest/?badge=latest) [](https://research-software-directory.org/software/deeprankcore) [](https://zenodo.org/badge/latestdoi/450496579) [](https://doi.org/10.21105/joss.05983) |
| **tests** | [](https://github.com/DeepRank/deeprank2/actions)  [](https://coveralls.io/github/DeepRank/deeprank2?branch=main)  <!---  --> |
| **running on** |  |
| **license** | [](https://opensource.org/license/apache-2-0/) |
## Overview
<p align="center">
<img src="./deeprank2.png" width="700">
</p>
DeepRank2 is an open-source deep learning (DL) framework for data mining of protein-protein interfaces (PPIs) or single-residue variants (SRVs). This package is an improved and unified version of three previously developed packages: [DeepRank](https://github.com/DeepRank/deeprank), [DeepRank-GNN](https://github.com/DeepRank/Deeprank-GNN), and [DeepRank-Mut](https://github.com/DeepRank/DeepRank-Mut).
As input, DeepRank2 takes [PDB-formatted](https://www.cgl.ucsf.edu/chimera/docs/UsersGuide/tutorials/pdbintro.html) atomic structures, and map them to graphs, where nodes can represent either residues or atoms, as chosen by the user, and edges represent the interactions between them. DeepRank2 has the option to choose between two types of queries as input for the featurization phase:
- PPIs, for mining interaction patterns within protein-protein complexes, implemented by the `ProteinProteinInterfaceQuery` class;
- SRVs, for mining mutation phenotypes within protein structures, implemented by the `SingleResidueVariantQuery` class.
The physico-chemical and geometrical features are then computed and assigned to each node and edge. The user can choose which features to generate from several pre-existing options defined in the package, or define custom features modules, as explained in the documentation. The graphs can then be mapped to 3D-grids as well. The generated data can be used for training neural networks. DeepRank2 also offers a pre-implemented training pipeline, using either [CNNs](https://en.wikipedia.org/wiki/Convolutional_neural_network) (for 3D-grids) or [GNNs](https://en.wikipedia.org/wiki/Graph_neural_network) (for graphs), as well as output exporters for evaluating performances.
Main features:
- Predefined atom-level and residue-level feature types
- e.g. atom/residue type, charge, size, potential energy
- All features' documentation is available [here](https://deeprank2.readthedocs.io/en/latest/features.html)
- Predefined target types
- binary class, CAPRI categories, DockQ, RMSD, and FNAT
- Detailed docking scores documentation is available [here](https://deeprank2.readthedocs.io/en/latest/docking.html)
- Flexible definition of both new features and targets
- Features generation for both graphs and 3D-grids
- Efficient data storage in HDF5 format
- Support for both classification and regression (based on [PyTorch](https://pytorch.org/) and [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/latest/))
📚 [Documentation](https://deeprank2.rtfd.io/)
📣 [Discussions](https://github.com/DeepRank/deeprank2/discussions)
## Table of Contents
- [DeepRank2](#deeprank2)
- [Overview](#overview)
- [Table of Contents](#table-of-contents)
- [Installation](#installation)
- [Containerized Installation](#containerized-installation)
- [Pull and Run the Pre-build Docker Image (Recommended)](#pull-and-run-the-pre-build-docker-image-recommended)
- [Build the Docker Image Manually](#build-the-docker-image-manually)
- [Removing the Docker Image](#removing-the-docker-image)
- [Local/remote Installation](#localremote-installation)
- [YML File Installation (Recommended)](#yml-file-installation-recommended)
- [Manual Installation (Customizable)](#manual-installation-customizable)
- [Testing DeepRank2 Installation](#testing-deeprank2-installation)
- [Contributing](#contributing)
- [Using DeepRank2](#using-deeprank2)
- [Data Generation](#data-generation)
- [Datasets](#datasets)
- [GraphDataset](#graphdataset)
- [GridDataset](#griddataset)
- [Training](#training)
- [Run a Pre-trained Model on New Data](#run-a-pre-trained-model-on-new-data)
- [Computational Performances](#computational-performances)
- [Package Development](#package-development)
## Installation
There are two ways to install DeepRank2:
1. In a [dockerized container](#containerized-installation). This allows you to use DeepRank2, including all the notebooks within the container (a protected virtual space), without worrying about your operating system or installation of dependencies.
- We recommend this installation for inexperienced users and to learn to use or test our software, e.g. using the provided [tutorials](tutorials/TUTORIAL.md). However, resources might be limited in this installation and we would not recommend using it for large datasets or on high-performance computing facilities.
2. [Local installation](#localremote-installation) on your system. This allows you to use the full potential of DeepRank2, but requires a few additional steps during installation.
- We recommend this installation for more experienced users, for larger projects, and for (potential) [contributors](#contributing) to the codebase.
### Containerized Installation
We provide a pre-built Docker image hosted on GitHub Packages, allowing you to use DeepRank2 without worrying about installing dependencies or configuring your system. This is the recommended method for trying out the package quickly.
#### Pull and Run the Pre-build Docker Image (Recommended)
- Install [Docker](https://docs.docker.com/engine/install/) on your system, if not already installed.
- Pull the latest Docker image from GitHub Packages by running the following command:
```bash
docker pull ghcr.io/deeprank/deeprank2:latest
```
- Run the container from the pulled image:
```bash
docker run -p 8888:8888 ghcr.io/deeprank/deeprank2:latest
```
- Once the container is running, open your browser and navigate to `http://localhost:8888` to access the DeepRank2 application.
From here, you can use DeepRank2, including running the tutorial notebooks. More details about the tutorials can be found [here](https://github.com/DeepRank/deeprank2/blob/main/tutorials/TUTORIAL.md). Note that the Docker container downloads only the raw PDB files required for the tutorials. To generate processed HDF5 files, you will need to run the `data_generation_xxx.ipynb` notebooks. Since Docker containers may have limited memory resources, we reduce the number of data points processed in the tutorials. To fully utilize the package, consider [installing it locally](#localremote-installation).
#### Build the Docker Image Manually
If you prefer to build the Docker image yourself or run into issues with the pre-built image, you can manually build and run the container as follows:
- Install [Docker](https://docs.docker.com/engine/install/) on your system, if not already installed.
- Clone the DeepRank2 repository and navigate to its root directory:
```bash
git clone https://github.com/DeepRank/deeprank2
cd deeprank2
```
- Build and run the Docker image:
```bash
docker build -t deeprank2 .
docker run -p 8888:8888 deeprank2
```
- Once the container is running, open your browser and navigate to `http://localhost:8888` to access the DeepRank2 application.
#### Removing the Docker Image
If you no longer need the Docker image (which can be quite large), you can remove it after stopping the container. Follow the [container stop](https://docs.docker.com/engine/reference/commandline/stop/) and [remove the image](https://docs.docker.com/engine/reference/commandline/image_rm/) instructions. For more general information on Docker, refer to the [Docker documentation](https://docs.docker.com/get-started/) directly.
### Local/remote Installation
Local installation is formally only supported on the latest stable release of ubuntu, for which widespread automated testing through continuous integration workflows has been set up. However, it is likely that the package runs smoothly on other operating systems as well.
Before installing DeepRank2 please ensure you have [GCC](https://gcc.gnu.org/install/) installed: if running `gcc --version` gives an error, run `sudo apt-get install gcc`.
#### YML File Installation (Recommended)
You can use the provided YML file for creating a [conda environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) via [mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html), containing the latest stable release of DeepRank2 and all its dependencies.
This will install the CPU-only version of DeepRank2 on Python 3.10.
Note that this will not work for MacOS. Do the [Manual Installation](#manual-installation) instead.
```bash
# Create the environment
mamba env create -f https://raw.githubusercontent.com/DeepRank/deeprank2/main/env/deeprank2.yml
# Activate the environment
conda activate deeprank2
# Install the latest deeprank2 release
pip install deeprank2
```
We also provide a frozen environment YML file located at `env/deeprank2_frozen.yml` with all dependencies set to fixed versions. The `env/deeprank2_frozen.yml` file provides a frozen environment with all dependencies set to fixed versions. This ensures reproducibility of experiments and results by preventing changes in package versions that could occur due to updates or modifications in the default `env/deeprank2.yml`. Use this frozen environment file for a stable and consistent setup, particularly if you encounter issues with the default environment file.
#### Manual Installation (Customizable)
If you want to use the GPUs, choose a specific python version (note that at the moment we support python 3.10 only), are a MacOS user, or if the YML installation was not successful, you can install the package manually. We advise to do this inside a [conda virtual environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html).
You can first create a copy of the `deeprank2.yml` file, place it in your current directory, and remove the packages that cannot be installed properly, or the ones that you want to install differently (e.g., pytorch-related packages if you wish to install the CUDA version), and then proceed with the environment creation by using the edited YML file: `conda env create -f deeprank2.yml` or `mamba env create -f deeprank2.yml`, if you have [mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html) installed. Then activate the environment, and proceed with installing the missing packages, which might fall into the following list. If you have any issues during installation of dependencies, please refer to the official documentation for each package (linked below), as our instructions may be out of date (last tested on 19 Feb 2024):
- [MSMS](https://anaconda.org/bioconda/msms): [Here](https://ssbio.readthedocs.io/en/latest/instructions/msms.html) for MacOS with M1 chip users.
- [PyTorch](https://pytorch.org/get-started/locally/)
- Pytorch regularly publishes updates and not all newest versions will work stably with DeepRank2. Currently, the package is tested on ubuntu using [PyTorch 2.1.1](https://pytorch.org/get-started/previous-versions/#v211).
- We support torch's CPU library as well as CUDA.
- [PyG](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html) and its optional dependencies: `torch_scatter`, `torch_sparse`, `torch_cluster`, `torch_spline_conv`.
- The exact command to install pyg will depend on the version of pytorch you are using. Please refer to the source's installation instructions (we recommend using the pip installation for this as it also shows the command for the dependencies).
- [FreeSASA](https://freesasa.github.io/python/).
Finally install deeprank2 itself: `pip install deeprank2`.
Alternatively, get the latest updates by cloning the repo and installing the editable version of the package with:
```bash
git clone https://github.com/DeepRank/deeprank2
cd deeprank2
pip install -e .'[test]'
```
The `test` extra is optional, and can be used to install test-related dependencies, useful during development.
#### Testing DeepRank2 Installation
If you have cloned the repository, you can check that all components were installed correctly using `pytest`. We especially recommend doing this in case you installed DeepRank2 and its dependencies manually (the latter option above).
The quick test should be sufficient to ensure that the software works, while the full test (a few minutes) will cover a much broader range of settings to ensure everything is correct.
First run `pip install pytest`, if you did not install it above. Then run `pytest tests/test_integration.py` for the quick test or just `pytest` for the full test (expect a few minutes to run).
## Contributing
If you would like to contribute to the package in any way, please see [our guidelines](CONTRIBUTING.rst).
## Using DeepRank2
The following section serves as a first guide to start using the package, using protein-protein Interface (PPI) queries as example. For an enhanced learning experience, we provide in-depth [tutorial notebooks](https://github.com/DeepRank/deeprank2/tree/main/tutorials) for generating PPI data, generating SVR data, and for the training pipeline.
For more details, see the [extended documentation](https://deeprank2.rtfd.io/).
### Data Generation
For each protein-protein complex (or protein structure containing a missense variant), a `Query` can be created and added to the `QueryCollection` object, to be processed later on. Two subtypes of `Query` exist: `ProteinProteinInterfaceQuery` and `SingleResidueVariantQuery`.
A `Query` takes as inputs:
- A `.pdb` file, representing the molecular structure.
- The resolution (`"residue"` or `"atom"`), i.e. whether each node should represent an amino acid residue or an atom.
- `chain_ids`, the chain ID or IDs (generally single capital letter(s)).
- `SingleResidueVariantQuery` takes a single ID, which represents the chain containing the variant residue.
- `ProteinProteinInterfaceQuery` takes a pair of ids, which represent the chains between which the interface exists.
- Note that in either case this does not limit the structure to residues from this/these chain/s. The structure contained in the `.pdb` can thus have any number of chains, and residues from these chains will be included in the graphs and 3D-grids produced by DeepRank2 (if they are within the `influence_radius`).
- Optionally, the correspondent position-specific scoring matrices (PSSMs), in the form of `.pssm` files.
```python
from deeprank2.query import QueryCollection, ProteinProteinInterfaceQuery
queries = QueryCollection()
# Append data points
queries.add(ProteinProteinInterfaceQuery(
pdb_path = "tests/data/pdb/1ATN/1ATN_1w.pdb",
resolution = "residue",
chain_ids = ["A", "B"],
targets = {
"binary": 0
},
pssm_paths = {
"A": "tests/data/pssm/1ATN/1ATN.A.pdb.pssm",
"B": "tests/data/pssm/1ATN/1ATN.B.pdb.pssm"
}
))
queries.add(ProteinProteinInterfaceQuery(
pdb_path = "tests/data/pdb/1ATN/1ATN_2w.pdb",
resolution = "residue",
chain_ids = ["A", "B"],
targets = {
"binary": 1
},
pssm_paths = {
"A": "tests/data/pssm/1ATN/1ATN.A.pdb.pssm",
"B": "tests/data/pssm/1ATN/1ATN.B.pdb.pssm"
}
))
queries.add(ProteinProteinInterfaceQuery(
pdb_path = "tests/data/pdb/1ATN/1ATN_3w.pdb",
resolution = "residue",
chain_ids = ["A", "B"],
targets = {
"binary": 0
},
pssm_paths = {
"A": "tests/data/pssm/1ATN/1ATN.A.pdb.pssm",
"B": "tests/data/pssm/1ATN/1ATN.B.pdb.pssm"
}
))
```
The user is free to implement a custom query class. Each implementation requires the `build` method to be present.
The queries can then be processed into graphs only or both graphs and 3D-grids, depending on which kind of network will be used later for training.
```python
from deeprank2.features import components, conservation, contact, exposure, irc, surfacearea
from deeprank2.utils.grid import GridSettings, MapMethod
feature_modules = [components, conservation, contact, exposure, irc, surfacearea]
# Save data into 3D-graphs only
hdf5_paths = queries.process(
"<output_folder>/<prefix_for_outputs>",
feature_modules = feature_modules)
# Save data into graphs and 3D-grids
hdf5_paths = queries.process(
"<output_folder>/<prefix_for_outputs>",
feature_modules = feature_modules,
grid_settings = GridSettings(
# the number of points on the x, y, z edges of the cube
points_counts = [20, 20, 20],
# x, y, z sizes of the box in Ã…
sizes = [1.0, 1.0, 1.0]),
grid_map_method = MapMethod.GAUSSIAN)
```
### Datasets
Data can be split in sets implementing custom splits according to the specific application. Assuming that the training, validation and testing ids have been chosen (keys of the HDF5 file/s), then the `DeeprankDataset` objects can be defined.
#### GraphDataset
For training GNNs the user can create a `GraphDataset` instance:
```python
from deeprank2.dataset import GraphDataset
node_features = ["bsa", "res_depth", "hse", "info_content", "pssm"]
edge_features = ["distance"]
target = "binary"
train_ids = [<ids>]
valid_ids = [<ids>]
test_ids = [<ids>]
# Creating GraphDataset objects
dataset_train = GraphDataset(
hdf5_path = hdf5_paths,
subset = train_ids,
node_features = node_features,
edge_features = edge_features,
target = target
)
dataset_val = GraphDataset(
hdf5_path = hdf5_paths,
subset = valid_ids,
train_source = dataset_train
)
dataset_test = GraphDataset(
hdf5_path = hdf5_paths,
subset = test_ids,
train_source = dataset_train
)
```
#### GridDataset
For training CNNs the user can create a `GridDataset` instance:
```python
from deeprank2.dataset import GridDataset
features = ["bsa", "res_depth", "hse", "info_content", "pssm", "distance"]
target = "binary"
train_ids = [<ids>]
valid_ids = [<ids>]
test_ids = [<ids>]
# Creating GraphDataset objects
dataset_train = GridDataset(
hdf5_path = hdf5_paths,
subset = train_ids,
features = features,
target = target
)
dataset_val = GridDataset(
hdf5_path = hdf5_paths,
subset = valid_ids,
train_source = dataset_train,
)
dataset_test = GridDataset(
hdf5_path = hdf5_paths,
subset = test_ids,
train_source = dataset_train,
)
```
### Training
Let's define a `Trainer` instance, using for example of the already existing `GINet`. Because `GINet` is a GNN, it requires a dataset instance of type `GraphDataset`.
```python
from deeprank2.trainer import Trainer
from deeprank2.neuralnets.gnn.vanilla_gnn import VanillaNetwork
trainer = Trainer(
VanillaNetwork,
dataset_train,
dataset_val,
dataset_test
)
```
The same can be done using a CNN, for example `CnnClassification`. Here a dataset instance of type `GridDataset` is required.
```python
from deeprank2.trainer import Trainer
from deeprank2.neuralnets.cnn.model3d import CnnClassification
trainer = Trainer(
CnnClassification,
dataset_train,
dataset_val,
dataset_test
)
```
By default, the `Trainer` class creates the folder `./output` for storing predictions information collected later on during training and testing. `HDF5OutputExporter` is the exporter used by default, but the user can specify any other implemented exporter or implement a custom one.
Optimizer (`torch.optim.Adam` by default) and loss function can be defined by using dedicated functions:
```python
import torch
trainer.configure_optimizers(torch.optim.Adamax, lr = 0.001, weight_decay = 1e-04)
```
Then the `Trainer` can be trained and tested; the best model in terms of validation loss is saved by default, and the user can modify so or indicate where to save it using the `train()` method parameter `filename`.
```python
trainer.train(
nepoch = 50,
batch_size = 64,
validate = True,
filename = "<my_folder/model.pth.tar>")
trainer.test()
```
#### Run a Pre-trained Model on New Data
If you want to analyze new PDB files using a pre-trained model, the first step is to process and save them into HDF5 files [as we have done above](#data-generation).
Then, the `DeeprankDataset` instance for the newly processed data can be created. Do this by specifying the path for the pre-trained model in `train_source`, together with the path to the HDF5 files just created. Note that there is no need of setting the dataset's parameters, since they are inherited from the information saved in the pre-trained model. Let's suppose that the model has been trained with `GraphDataset` objects:
```python
from deeprank2.dataset import GraphDataset
dataset_test = GraphDataset(
hdf5_path = "<output_folder>/<prefix_for_outputs>",
train_source = "<pretrained_model_path>"
)
```
Finally, the `Trainer` instance can be defined and the new data can be tested:
```python
from deeprank2.trainer import Trainer
from deeprank2.neuralnets.gnn.vanilla_gnn import VanillaNetwork
from deeprank2.utils.exporters import HDF5OutputExporter
trainer = Trainer(
VanillaNetwork,
dataset_test = dataset_test,
pretrained_model = "<pretrained_model_path>",
output_exporters = [HDF5OutputExporter("<output_folder_path>")]
)
trainer.test()
```
For more details about how to run a pre-trained model on new data, see the [docs](https://deeprank2.readthedocs.io/en/latest/getstarted.html#run-a-pre-trained-model-on-new-data).
## Computational Performances
We measured the efficiency of data generation in DeepRank2 using the tutorials' [PDB files](https://zenodo.org/record/8187806) (~100 data points per data set), averaging the results run on Apple M1 Pro, using a single CPU.
Parameter settings were: atomic resolution, `distance_cutoff` of 5.5 Ã…, radius (for SRV only) of 10 Ã…. The [features modules](https://deeprank2.readthedocs.io/en/latest/features.html) used were `components`, `contact`, `exposure`, `irc`, `secondary_structure`, `surfacearea`, for a total of 33 features for PPIs and 26 for SRVs (the latter do not use `irc` features).
| | Data processing speed <br />[seconds/structure] | Memory <br />[megabyte/structure] |
| ---- | :--------------------------------------------------------------------: | :--------------------------------------------------------------------: |
| PPIs | graph only: **2.99** (std 0.23) <br />graph+grid: **11.35** (std 1.30) | graph only: **0.54** (std 0.07) <br />graph+grid: **16.09** (std 0.44) |
| SRVs | graph only: **2.20** (std 0.08) <br />graph+grid: **2.85** (std 0.10) | graph only: **0.05** (std 0.01) <br />graph+grid: **17.52** (std 0.59) |
## Package Development
If you're looking for developer documentation, go [here](https://github.com/DeepRank/deeprank2/blob/dev/README.dev.md).
Raw data
{
"_id": null,
"home_page": null,
"name": "deeprank2",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "graph neural network, convolutional neural network, protein-protein interfaces, missense variants, deep learning, pytorch",
"author": "Daniel Rademaker, Gayatri Ramakrishnan, Sven van der Burg, Li Xue, Daniil Lepikhov",
"author_email": "Giulia Crocioni <g.crocioni@esciencecenter.nl>, Coos Baakman <coos.baakman@radboudumc.nl>, Dani Bodor <d.bodor@esciencecenter.nl>",
"download_url": "https://files.pythonhosted.org/packages/c9/35/5c25babf78cc451424b3860196db67fc4f5e2d6dc9df975f07a14313d17a/deeprank2-3.1.0.tar.gz",
"platform": null,
"description": "# DeepRank2\n\n| Badges | |\n| :------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **fairness** | [](https://fair-software.eu) [](https://bestpractices.coreinfrastructure.org/projects/6403) |\n| **package** | [](https://badge.fury.io/py/deeprank2) [](https://app.codacy.com/gh/DeepRank/deeprank2/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade) |\n| **docs** | [](https://deeprank2.readthedocs.io/en/latest/?badge=latest) [](https://research-software-directory.org/software/deeprankcore) [](https://zenodo.org/badge/latestdoi/450496579) [](https://doi.org/10.21105/joss.05983) |\n| **tests** | [](https://github.com/DeepRank/deeprank2/actions)  [](https://coveralls.io/github/DeepRank/deeprank2?branch=main)  <!---  --> |\n| **running on** |  |\n| **license** | [](https://opensource.org/license/apache-2-0/) |\n\n## Overview\n\n<p align=\"center\">\n <img src=\"./deeprank2.png\" width=\"700\">\n</p>\n\nDeepRank2 is an open-source deep learning (DL) framework for data mining of protein-protein interfaces (PPIs) or single-residue variants (SRVs). This package is an improved and unified version of three previously developed packages: [DeepRank](https://github.com/DeepRank/deeprank), [DeepRank-GNN](https://github.com/DeepRank/Deeprank-GNN), and [DeepRank-Mut](https://github.com/DeepRank/DeepRank-Mut).\n\nAs input, DeepRank2 takes [PDB-formatted](https://www.cgl.ucsf.edu/chimera/docs/UsersGuide/tutorials/pdbintro.html) atomic structures, and map them to graphs, where nodes can represent either residues or atoms, as chosen by the user, and edges represent the interactions between them. DeepRank2 has the option to choose between two types of queries as input for the featurization phase:\n\n- PPIs, for mining interaction patterns within protein-protein complexes, implemented by the `ProteinProteinInterfaceQuery` class;\n- SRVs, for mining mutation phenotypes within protein structures, implemented by the `SingleResidueVariantQuery` class.\n\nThe physico-chemical and geometrical features are then computed and assigned to each node and edge. The user can choose which features to generate from several pre-existing options defined in the package, or define custom features modules, as explained in the documentation. The graphs can then be mapped to 3D-grids as well. The generated data can be used for training neural networks. DeepRank2 also offers a pre-implemented training pipeline, using either [CNNs](https://en.wikipedia.org/wiki/Convolutional_neural_network) (for 3D-grids) or [GNNs](https://en.wikipedia.org/wiki/Graph_neural_network) (for graphs), as well as output exporters for evaluating performances.\n\nMain features:\n\n- Predefined atom-level and residue-level feature types\n - e.g. atom/residue type, charge, size, potential energy\n - All features' documentation is available [here](https://deeprank2.readthedocs.io/en/latest/features.html)\n- Predefined target types\n - binary class, CAPRI categories, DockQ, RMSD, and FNAT\n - Detailed docking scores documentation is available [here](https://deeprank2.readthedocs.io/en/latest/docking.html)\n- Flexible definition of both new features and targets\n- Features generation for both graphs and 3D-grids\n- Efficient data storage in HDF5 format\n- Support for both classification and regression (based on [PyTorch](https://pytorch.org/) and [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/latest/))\n\n\ud83d\udcda [Documentation](https://deeprank2.rtfd.io/)\n\n\ud83d\udce3 [Discussions](https://github.com/DeepRank/deeprank2/discussions)\n\n## Table of Contents\n\n- [DeepRank2](#deeprank2)\n - [Overview](#overview)\n - [Table of Contents](#table-of-contents)\n - [Installation](#installation)\n - [Containerized Installation](#containerized-installation)\n - [Pull and Run the Pre-build Docker Image (Recommended)](#pull-and-run-the-pre-build-docker-image-recommended)\n - [Build the Docker Image Manually](#build-the-docker-image-manually)\n - [Removing the Docker Image](#removing-the-docker-image)\n - [Local/remote Installation](#localremote-installation)\n - [YML File Installation (Recommended)](#yml-file-installation-recommended)\n - [Manual Installation (Customizable)](#manual-installation-customizable)\n - [Testing DeepRank2 Installation](#testing-deeprank2-installation)\n - [Contributing](#contributing)\n - [Using DeepRank2](#using-deeprank2)\n - [Data Generation](#data-generation)\n - [Datasets](#datasets)\n - [GraphDataset](#graphdataset)\n - [GridDataset](#griddataset)\n - [Training](#training)\n - [Run a Pre-trained Model on New Data](#run-a-pre-trained-model-on-new-data)\n - [Computational Performances](#computational-performances)\n - [Package Development](#package-development)\n\n## Installation\n\nThere are two ways to install DeepRank2:\n\n1. In a [dockerized container](#containerized-installation). This allows you to use DeepRank2, including all the notebooks within the container (a protected virtual space), without worrying about your operating system or installation of dependencies.\n - We recommend this installation for inexperienced users and to learn to use or test our software, e.g. using the provided [tutorials](tutorials/TUTORIAL.md). However, resources might be limited in this installation and we would not recommend using it for large datasets or on high-performance computing facilities.\n2. [Local installation](#localremote-installation) on your system. This allows you to use the full potential of DeepRank2, but requires a few additional steps during installation.\n - We recommend this installation for more experienced users, for larger projects, and for (potential) [contributors](#contributing) to the codebase.\n\n### Containerized Installation\n\nWe provide a pre-built Docker image hosted on GitHub Packages, allowing you to use DeepRank2 without worrying about installing dependencies or configuring your system. This is the recommended method for trying out the package quickly.\n\n#### Pull and Run the Pre-build Docker Image (Recommended)\n\n- Install [Docker](https://docs.docker.com/engine/install/) on your system, if not already installed.\n- Pull the latest Docker image from GitHub Packages by running the following command:\n\n```bash\ndocker pull ghcr.io/deeprank/deeprank2:latest\n```\n\n- Run the container from the pulled image:\n\n```bash\ndocker run -p 8888:8888 ghcr.io/deeprank/deeprank2:latest\n```\n\n- Once the container is running, open your browser and navigate to `http://localhost:8888` to access the DeepRank2 application.\n\nFrom here, you can use DeepRank2, including running the tutorial notebooks. More details about the tutorials can be found [here](https://github.com/DeepRank/deeprank2/blob/main/tutorials/TUTORIAL.md). Note that the Docker container downloads only the raw PDB files required for the tutorials. To generate processed HDF5 files, you will need to run the `data_generation_xxx.ipynb` notebooks. Since Docker containers may have limited memory resources, we reduce the number of data points processed in the tutorials. To fully utilize the package, consider [installing it locally](#localremote-installation).\n\n#### Build the Docker Image Manually\n\nIf you prefer to build the Docker image yourself or run into issues with the pre-built image, you can manually build and run the container as follows:\n\n- Install [Docker](https://docs.docker.com/engine/install/) on your system, if not already installed.\n- Clone the DeepRank2 repository and navigate to its root directory:\n\n```bash\ngit clone https://github.com/DeepRank/deeprank2\ncd deeprank2\n```\n\n- Build and run the Docker image:\n\n```bash\ndocker build -t deeprank2 .\ndocker run -p 8888:8888 deeprank2\n```\n\n- Once the container is running, open your browser and navigate to `http://localhost:8888` to access the DeepRank2 application.\n\n#### Removing the Docker Image\n\nIf you no longer need the Docker image (which can be quite large), you can remove it after stopping the container. Follow the [container stop](https://docs.docker.com/engine/reference/commandline/stop/) and [remove the image](https://docs.docker.com/engine/reference/commandline/image_rm/) instructions. For more general information on Docker, refer to the [Docker documentation](https://docs.docker.com/get-started/) directly.\n\n### Local/remote Installation\n\nLocal installation is formally only supported on the latest stable release of ubuntu, for which widespread automated testing through continuous integration workflows has been set up. However, it is likely that the package runs smoothly on other operating systems as well.\n\nBefore installing DeepRank2 please ensure you have [GCC](https://gcc.gnu.org/install/) installed: if running `gcc --version` gives an error, run `sudo apt-get install gcc`.\n\n#### YML File Installation (Recommended)\n\nYou can use the provided YML file for creating a [conda environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) via [mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html), containing the latest stable release of DeepRank2 and all its dependencies.\nThis will install the CPU-only version of DeepRank2 on Python 3.10.\nNote that this will not work for MacOS. Do the [Manual Installation](#manual-installation) instead.\n\n```bash\n# Create the environment\nmamba env create -f https://raw.githubusercontent.com/DeepRank/deeprank2/main/env/deeprank2.yml\n# Activate the environment\nconda activate deeprank2\n# Install the latest deeprank2 release\npip install deeprank2\n```\n\nWe also provide a frozen environment YML file located at `env/deeprank2_frozen.yml` with all dependencies set to fixed versions. The `env/deeprank2_frozen.yml` file provides a frozen environment with all dependencies set to fixed versions. This ensures reproducibility of experiments and results by preventing changes in package versions that could occur due to updates or modifications in the default `env/deeprank2.yml`. Use this frozen environment file for a stable and consistent setup, particularly if you encounter issues with the default environment file.\n\n#### Manual Installation (Customizable)\n\nIf you want to use the GPUs, choose a specific python version (note that at the moment we support python 3.10 only), are a MacOS user, or if the YML installation was not successful, you can install the package manually. We advise to do this inside a [conda virtual environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html).\n\nYou can first create a copy of the `deeprank2.yml` file, place it in your current directory, and remove the packages that cannot be installed properly, or the ones that you want to install differently (e.g., pytorch-related packages if you wish to install the CUDA version), and then proceed with the environment creation by using the edited YML file: `conda env create -f deeprank2.yml` or `mamba env create -f deeprank2.yml`, if you have [mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html) installed. Then activate the environment, and proceed with installing the missing packages, which might fall into the following list. If you have any issues during installation of dependencies, please refer to the official documentation for each package (linked below), as our instructions may be out of date (last tested on 19 Feb 2024):\n\n- [MSMS](https://anaconda.org/bioconda/msms): [Here](https://ssbio.readthedocs.io/en/latest/instructions/msms.html) for MacOS with M1 chip users.\n- [PyTorch](https://pytorch.org/get-started/locally/)\n - Pytorch regularly publishes updates and not all newest versions will work stably with DeepRank2. Currently, the package is tested on ubuntu using [PyTorch 2.1.1](https://pytorch.org/get-started/previous-versions/#v211).\n - We support torch's CPU library as well as CUDA.\n- [PyG](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html) and its optional dependencies: `torch_scatter`, `torch_sparse`, `torch_cluster`, `torch_spline_conv`.\n - The exact command to install pyg will depend on the version of pytorch you are using. Please refer to the source's installation instructions (we recommend using the pip installation for this as it also shows the command for the dependencies).\n- [FreeSASA](https://freesasa.github.io/python/).\n\nFinally install deeprank2 itself: `pip install deeprank2`.\n\nAlternatively, get the latest updates by cloning the repo and installing the editable version of the package with:\n\n```bash\ngit clone https://github.com/DeepRank/deeprank2\ncd deeprank2\npip install -e .'[test]'\n```\n\nThe `test` extra is optional, and can be used to install test-related dependencies, useful during development.\n\n#### Testing DeepRank2 Installation\n\nIf you have cloned the repository, you can check that all components were installed correctly using `pytest`. We especially recommend doing this in case you installed DeepRank2 and its dependencies manually (the latter option above).\n\nThe quick test should be sufficient to ensure that the software works, while the full test (a few minutes) will cover a much broader range of settings to ensure everything is correct.\n\nFirst run `pip install pytest`, if you did not install it above. Then run `pytest tests/test_integration.py` for the quick test or just `pytest` for the full test (expect a few minutes to run).\n\n## Contributing\n\nIf you would like to contribute to the package in any way, please see [our guidelines](CONTRIBUTING.rst).\n\n## Using DeepRank2\n\nThe following section serves as a first guide to start using the package, using protein-protein Interface (PPI) queries as example. For an enhanced learning experience, we provide in-depth [tutorial notebooks](https://github.com/DeepRank/deeprank2/tree/main/tutorials) for generating PPI data, generating SVR data, and for the training pipeline.\nFor more details, see the [extended documentation](https://deeprank2.rtfd.io/).\n\n### Data Generation\n\nFor each protein-protein complex (or protein structure containing a missense variant), a `Query` can be created and added to the `QueryCollection` object, to be processed later on. Two subtypes of `Query` exist: `ProteinProteinInterfaceQuery` and `SingleResidueVariantQuery`.\n\nA `Query` takes as inputs:\n\n- A `.pdb` file, representing the molecular structure.\n- The resolution (`\"residue\"` or `\"atom\"`), i.e. whether each node should represent an amino acid residue or an atom.\n- `chain_ids`, the chain ID or IDs (generally single capital letter(s)).\n - `SingleResidueVariantQuery` takes a single ID, which represents the chain containing the variant residue.\n - `ProteinProteinInterfaceQuery` takes a pair of ids, which represent the chains between which the interface exists.\n - Note that in either case this does not limit the structure to residues from this/these chain/s. The structure contained in the `.pdb` can thus have any number of chains, and residues from these chains will be included in the graphs and 3D-grids produced by DeepRank2 (if they are within the `influence_radius`).\n- Optionally, the correspondent position-specific scoring matrices (PSSMs), in the form of `.pssm` files.\n\n```python\nfrom deeprank2.query import QueryCollection, ProteinProteinInterfaceQuery\n\nqueries = QueryCollection()\n\n# Append data points\nqueries.add(ProteinProteinInterfaceQuery(\n pdb_path = \"tests/data/pdb/1ATN/1ATN_1w.pdb\",\n resolution = \"residue\",\n chain_ids = [\"A\", \"B\"],\n targets = {\n \"binary\": 0\n },\n pssm_paths = {\n \"A\": \"tests/data/pssm/1ATN/1ATN.A.pdb.pssm\",\n \"B\": \"tests/data/pssm/1ATN/1ATN.B.pdb.pssm\"\n }\n))\nqueries.add(ProteinProteinInterfaceQuery(\n pdb_path = \"tests/data/pdb/1ATN/1ATN_2w.pdb\",\n resolution = \"residue\",\n chain_ids = [\"A\", \"B\"],\n targets = {\n \"binary\": 1\n },\n pssm_paths = {\n \"A\": \"tests/data/pssm/1ATN/1ATN.A.pdb.pssm\",\n \"B\": \"tests/data/pssm/1ATN/1ATN.B.pdb.pssm\"\n }\n))\nqueries.add(ProteinProteinInterfaceQuery(\n pdb_path = \"tests/data/pdb/1ATN/1ATN_3w.pdb\",\n resolution = \"residue\",\n chain_ids = [\"A\", \"B\"],\n targets = {\n \"binary\": 0\n },\n pssm_paths = {\n \"A\": \"tests/data/pssm/1ATN/1ATN.A.pdb.pssm\",\n \"B\": \"tests/data/pssm/1ATN/1ATN.B.pdb.pssm\"\n }\n))\n\n```\n\nThe user is free to implement a custom query class. Each implementation requires the `build` method to be present.\n\nThe queries can then be processed into graphs only or both graphs and 3D-grids, depending on which kind of network will be used later for training.\n\n```python\nfrom deeprank2.features import components, conservation, contact, exposure, irc, surfacearea\nfrom deeprank2.utils.grid import GridSettings, MapMethod\n\nfeature_modules = [components, conservation, contact, exposure, irc, surfacearea]\n\n# Save data into 3D-graphs only\nhdf5_paths = queries.process(\n \"<output_folder>/<prefix_for_outputs>\",\n feature_modules = feature_modules)\n\n# Save data into graphs and 3D-grids\nhdf5_paths = queries.process(\n \"<output_folder>/<prefix_for_outputs>\",\n feature_modules = feature_modules,\n grid_settings = GridSettings(\n # the number of points on the x, y, z edges of the cube\n points_counts = [20, 20, 20],\n # x, y, z sizes of the box in \u00c5\n sizes = [1.0, 1.0, 1.0]),\n grid_map_method = MapMethod.GAUSSIAN)\n```\n\n### Datasets\n\nData can be split in sets implementing custom splits according to the specific application. Assuming that the training, validation and testing ids have been chosen (keys of the HDF5 file/s), then the `DeeprankDataset` objects can be defined.\n\n#### GraphDataset\n\nFor training GNNs the user can create a `GraphDataset` instance:\n\n```python\nfrom deeprank2.dataset import GraphDataset\n\nnode_features = [\"bsa\", \"res_depth\", \"hse\", \"info_content\", \"pssm\"]\nedge_features = [\"distance\"]\ntarget = \"binary\"\ntrain_ids = [<ids>]\nvalid_ids = [<ids>]\ntest_ids = [<ids>]\n\n# Creating GraphDataset objects\ndataset_train = GraphDataset(\n hdf5_path = hdf5_paths,\n subset = train_ids,\n node_features = node_features,\n edge_features = edge_features,\n target = target\n)\ndataset_val = GraphDataset(\n hdf5_path = hdf5_paths,\n subset = valid_ids,\n train_source = dataset_train\n)\ndataset_test = GraphDataset(\n hdf5_path = hdf5_paths,\n subset = test_ids,\n train_source = dataset_train\n)\n```\n\n#### GridDataset\n\nFor training CNNs the user can create a `GridDataset` instance:\n\n```python\nfrom deeprank2.dataset import GridDataset\n\nfeatures = [\"bsa\", \"res_depth\", \"hse\", \"info_content\", \"pssm\", \"distance\"]\ntarget = \"binary\"\ntrain_ids = [<ids>]\nvalid_ids = [<ids>]\ntest_ids = [<ids>]\n\n# Creating GraphDataset objects\ndataset_train = GridDataset(\n hdf5_path = hdf5_paths,\n subset = train_ids,\n features = features,\n target = target\n)\ndataset_val = GridDataset(\n hdf5_path = hdf5_paths,\n subset = valid_ids,\n train_source = dataset_train,\n)\ndataset_test = GridDataset(\n hdf5_path = hdf5_paths,\n subset = test_ids,\n train_source = dataset_train,\n)\n```\n\n### Training\n\nLet's define a `Trainer` instance, using for example of the already existing `GINet`. Because `GINet` is a GNN, it requires a dataset instance of type `GraphDataset`.\n\n```python\nfrom deeprank2.trainer import Trainer\nfrom deeprank2.neuralnets.gnn.vanilla_gnn import VanillaNetwork\n\ntrainer = Trainer(\n VanillaNetwork,\n dataset_train,\n dataset_val,\n dataset_test\n)\n\n```\n\nThe same can be done using a CNN, for example `CnnClassification`. Here a dataset instance of type `GridDataset` is required.\n\n```python\nfrom deeprank2.trainer import Trainer\nfrom deeprank2.neuralnets.cnn.model3d import CnnClassification\n\ntrainer = Trainer(\n CnnClassification,\n dataset_train,\n dataset_val,\n dataset_test\n)\n```\n\nBy default, the `Trainer` class creates the folder `./output` for storing predictions information collected later on during training and testing. `HDF5OutputExporter` is the exporter used by default, but the user can specify any other implemented exporter or implement a custom one.\n\nOptimizer (`torch.optim.Adam` by default) and loss function can be defined by using dedicated functions:\n\n```python\nimport torch\n\ntrainer.configure_optimizers(torch.optim.Adamax, lr = 0.001, weight_decay = 1e-04)\n\n```\n\nThen the `Trainer` can be trained and tested; the best model in terms of validation loss is saved by default, and the user can modify so or indicate where to save it using the `train()` method parameter `filename`.\n\n```python\ntrainer.train(\n nepoch = 50,\n batch_size = 64,\n validate = True,\n filename = \"<my_folder/model.pth.tar>\")\ntrainer.test()\n\n```\n\n#### Run a Pre-trained Model on New Data\n\nIf you want to analyze new PDB files using a pre-trained model, the first step is to process and save them into HDF5 files [as we have done above](#data-generation).\n\nThen, the `DeeprankDataset` instance for the newly processed data can be created. Do this by specifying the path for the pre-trained model in `train_source`, together with the path to the HDF5 files just created. Note that there is no need of setting the dataset's parameters, since they are inherited from the information saved in the pre-trained model. Let's suppose that the model has been trained with `GraphDataset` objects:\n\n```python\nfrom deeprank2.dataset import GraphDataset\n\ndataset_test = GraphDataset(\n hdf5_path = \"<output_folder>/<prefix_for_outputs>\",\n train_source = \"<pretrained_model_path>\"\n)\n```\n\nFinally, the `Trainer` instance can be defined and the new data can be tested:\n\n```python\nfrom deeprank2.trainer import Trainer\nfrom deeprank2.neuralnets.gnn.vanilla_gnn import VanillaNetwork\nfrom deeprank2.utils.exporters import HDF5OutputExporter\n\ntrainer = Trainer(\n VanillaNetwork,\n dataset_test = dataset_test,\n pretrained_model = \"<pretrained_model_path>\",\n output_exporters = [HDF5OutputExporter(\"<output_folder_path>\")]\n)\n\ntrainer.test()\n```\n\nFor more details about how to run a pre-trained model on new data, see the [docs](https://deeprank2.readthedocs.io/en/latest/getstarted.html#run-a-pre-trained-model-on-new-data).\n\n## Computational Performances\n\nWe measured the efficiency of data generation in DeepRank2 using the tutorials' [PDB files](https://zenodo.org/record/8187806) (~100 data points per data set), averaging the results run on Apple M1 Pro, using a single CPU.\nParameter settings were: atomic resolution, `distance_cutoff` of 5.5 \u00c5, radius (for SRV only) of 10 \u00c5. The [features modules](https://deeprank2.readthedocs.io/en/latest/features.html) used were `components`, `contact`, `exposure`, `irc`, `secondary_structure`, `surfacearea`, for a total of 33 features for PPIs and 26 for SRVs (the latter do not use `irc` features).\n\n| | Data processing speed <br />[seconds/structure] | Memory <br />[megabyte/structure] |\n| ---- | :--------------------------------------------------------------------: | :--------------------------------------------------------------------: |\n| PPIs | graph only: **2.99** (std 0.23) <br />graph+grid: **11.35** (std 1.30) | graph only: **0.54** (std 0.07) <br />graph+grid: **16.09** (std 0.44) |\n| SRVs | graph only: **2.20** (std 0.08) <br />graph+grid: **2.85** (std 0.10) | graph only: **0.05** (std 0.01) <br />graph+grid: **17.52** (std 0.59) |\n\n## Package Development\n\nIf you're looking for developer documentation, go [here](https://github.com/DeepRank/deeprank2/blob/dev/README.dev.md).\n",
"bugtrack_url": null,
"license": "Apache-2.0 license",
"summary": "DeepRank2 is an open-source deep learning framework for data mining of protein-protein interfaces or single-residue missense variants.",
"version": "3.1.0",
"project_urls": {
"Changelog": "https://github.com/DeepRank/deeprank2/blob/main/CHANGELOG.md",
"Documentation": "https://deeprank2.readthedocs.io/en/latest/?badge=latest",
"Repository": "https://github.com/DeepRank/deeprank2"
},
"split_keywords": [
"graph neural network",
" convolutional neural network",
" protein-protein interfaces",
" missense variants",
" deep learning",
" pytorch"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dda381d8edd87f40279af1577107f76e3883fae5bbe4d209f8ab215583374518",
"md5": "552b64bd11c5724ae8b19f3b371c8d0b",
"sha256": "ef29b750d7c97ddc355ed941d5ae8348e237d1e5c9690a9f04c3904343d3aabd"
},
"downloads": -1,
"filename": "deeprank2-3.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "552b64bd11c5724ae8b19f3b371c8d0b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 118218,
"upload_time": "2024-09-16T15:20:45",
"upload_time_iso_8601": "2024-09-16T15:20:45.347549Z",
"url": "https://files.pythonhosted.org/packages/dd/a3/81d8edd87f40279af1577107f76e3883fae5bbe4d209f8ab215583374518/deeprank2-3.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c9355c25babf78cc451424b3860196db67fc4f5e2d6dc9df975f07a14313d17a",
"md5": "7248546e00e62d5f94a95866ff8fbd0a",
"sha256": "d033be23b9f85b390bf38cc29b90c007e6d89e428d6afc836f1c4cd199cfcfd1"
},
"downloads": -1,
"filename": "deeprank2-3.1.0.tar.gz",
"has_sig": false,
"md5_digest": "7248546e00e62d5f94a95866ff8fbd0a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 111287,
"upload_time": "2024-09-16T15:20:47",
"upload_time_iso_8601": "2024-09-16T15:20:47.004664Z",
"url": "https://files.pythonhosted.org/packages/c9/35/5c25babf78cc451424b3860196db67fc4f5e2d6dc9df975f07a14313d17a/deeprank2-3.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-16 15:20:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DeepRank",
"github_project": "deeprank2",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "deeprank2"
}
JSON
