# **SIMS**: Scalable, Interpretable Modeling for Single-Cell RNA-Seq Data Classification
SIMS is a pipeline for building interpretable and accurate classifiers for identifying any target on single-cell rna-seq data. The SIMS model is based on [a sequential transformer](https://arxiv.org/abs/1908.07442), a transformer model specifically built for large-scale tabular datasets.
SIMS takes in a list of arbitrarily many expression matrices along with their corresponding target variables. We assume the matrix form `cell x gene`, and NOT `gene x cell`, since our training samples are the transcriptomes of individual cells.
The code is run with `python`. To use the package, we recommend using a virtual environment such as [miniconda](https://docs.conda.io/en/latest/miniconda.html) which will allow you to install packages without harming your system `python`.
## Installation
If using conda, run
1. Create a new virtual environment with `conda create --name=<NAME> python=3.9`
2. Enter into your virtual environment with `conda activate NAME`
Otherwise, enter your virtual environment of choice and
1. Install the SIMS package with `pip install --use-pep517 git+https://github.com/braingeneers/SIMS.git`
2. Set up the model training code in a `MYFILE.py` file, and run it with `python MYFILE.py`. A tutorial on how to set up training code is shown below.
## Training and inference
To train a model, we can set up a SIMS class in the following way:
```python
from scsims import SIMS
from pytorch_lightning.loggers import WandbLogger
logger = WandbLogger(offline=True)
data = an.read_h5ad('mydata.h5ad')
sims = SIMS(data=data, class_label='class_label')
sims.setup_trainer(accelerator="gpu", devices=1, logger=logger)
sims.train()
```
This will set up the underlying dataloaders, model, model checkpointing, and everything else we need. Model checkpoints will be saved every training epoch.
To load in a model to infer new cell types on an unlabeled dataset, we load in the model checkpoint, point to the label file that we originally trained on, and run the `predict` method on new data.
```python
sims = SIMS(weights_path='myawesomemodel.ckpt')
cell_predictions = sims.predict('my/new/unlabeled.h5ad')
```
Finally, to look at the explainability of the model, we similarly run
```python
explainability_matrix = sims.explain('my/new/unlabeled.h5ad') # this can also be labeled data, of course
```
## Custom training jobs / logging
To customize the underlying `pl.Trainer` and SIMS model params, we can initialize the SIMS model like
```python
from pytorch_lightning.loggers import WandbLogger
from pytorch_lightning.callbacks import EarlyStopping, LearningRateMonitor
from scsims import SIMS
wandb_logger = WandbLogger(project=f"My Project", name=f"SIMS Model Training") # set up the logger to log data to Weights and Biases
sims = SIMS(data=adata, class_label='class_label')
sims.setup_model(n_a=64, n_d=64, weights=sims.weights) # weighting loss inversely proportional by label freq, helps learn rare cell types (recommended)
sims.setup_trainer(
logger=wandb_logger,
callbacks=[
EarlyStopping(
monitor="val_loss",
patience=50,
),
LearningRateMonitor(logging_interval="epoch"),
],
num_epochs=100,
)
sims.train()
```
This will train the SIMS model on the given expression matrices with target variable given by the `class_label` column in each label file.
## Using SIMS inside github codespaces
If you are using SIMS only for predictions using an already trained model, github codespaces is the recommended way to use this tool. You can also use this pipeline to train it in smaller datasets as the computing services offered in codespaces are modest.
To use this tool in github codespaces start by forking the repo in your github account. Then create a new codespace with the SIMS repo as the Repository of choice.
Once inside the newly created environment pull the latest SIMS image:
```docker
docker pull jmlehrer/sims:latest
```
Run the docker container mounting the file folder containing datasets and model checkpoints to the filesystem:
```docker
docker run -it -v /path/to/local/folder:/path/in/container [image_name] /bin/bash
```
Run main.py to check if the installation has been completed. You can alter this file as shown above to perform the different tasks.
```bash
python main.py
```
Raw data
{
"_id": null,
"home_page": "https://github.com/braingeneers/sims",
"name": "scsims",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "scsims",
"author": "Julian Lehrer",
"author_email": "jmlehrer@ucsc.edu",
"download_url": "https://files.pythonhosted.org/packages/09/ae/b7beee9a9fa4cf313a1344981ffe833779ffc1e23a895e5594cb0e087c50/scsims-3.0.5.tar.gz",
"platform": null,
"description": "# **SIMS**: Scalable, Interpretable Modeling for Single-Cell RNA-Seq Data Classification\n\nSIMS is a pipeline for building interpretable and accurate classifiers for identifying any target on single-cell rna-seq data. The SIMS model is based on [a sequential transformer](https://arxiv.org/abs/1908.07442), a transformer model specifically built for large-scale tabular datasets.\n\nSIMS takes in a list of arbitrarily many expression matrices along with their corresponding target variables. We assume the matrix form `cell x gene`, and NOT `gene x cell`, since our training samples are the transcriptomes of individual cells.\n\nThe code is run with `python`. To use the package, we recommend using a virtual environment such as [miniconda](https://docs.conda.io/en/latest/miniconda.html) which will allow you to install packages without harming your system `python`. \n\n## Installation\nIf using conda, run \n1. Create a new virtual environment with `conda create --name=<NAME> python=3.9`\n2. Enter into your virtual environment with `conda activate NAME`\n\nOtherwise, enter your virtual environment of choice and\n1. Install the SIMS package with `pip install --use-pep517 git+https://github.com/braingeneers/SIMS.git`\n2. Set up the model training code in a `MYFILE.py` file, and run it with `python MYFILE.py`. A tutorial on how to set up training code is shown below.\n\n## Training and inference\nTo train a model, we can set up a SIMS class in the following way:\n\n```python \nfrom scsims import SIMS\nfrom pytorch_lightning.loggers import WandbLogger\nlogger = WandbLogger(offline=True)\n\ndata = an.read_h5ad('mydata.h5ad')\nsims = SIMS(data=data, class_label='class_label')\nsims.setup_trainer(accelerator=\"gpu\", devices=1, logger=logger)\nsims.train()\n```\n\nThis will set up the underlying dataloaders, model, model checkpointing, and everything else we need. Model checkpoints will be saved every training epoch. \n\nTo load in a model to infer new cell types on an unlabeled dataset, we load in the model checkpoint, point to the label file that we originally trained on, and run the `predict` method on new data.\n\n```python\nsims = SIMS(weights_path='myawesomemodel.ckpt')\ncell_predictions = sims.predict('my/new/unlabeled.h5ad')\n```\n\nFinally, to look at the explainability of the model, we similarly run \n```python\nexplainability_matrix = sims.explain('my/new/unlabeled.h5ad') # this can also be labeled data, of course \n```\n\n## Custom training jobs / logging\nTo customize the underlying `pl.Trainer` and SIMS model params, we can initialize the SIMS model like \n```python \nfrom pytorch_lightning.loggers import WandbLogger\nfrom pytorch_lightning.callbacks import EarlyStopping, LearningRateMonitor\nfrom scsims import SIMS\n\nwandb_logger = WandbLogger(project=f\"My Project\", name=f\"SIMS Model Training\") # set up the logger to log data to Weights and Biases\n\nsims = SIMS(data=adata, class_label='class_label')\nsims.setup_model(n_a=64, n_d=64, weights=sims.weights) # weighting loss inversely proportional by label freq, helps learn rare cell types (recommended)\nsims.setup_trainer(\n logger=wandb_logger,\n callbacks=[\n EarlyStopping(\n monitor=\"val_loss\",\n patience=50,\n ),\n LearningRateMonitor(logging_interval=\"epoch\"),\n ],\n num_epochs=100,\n)\nsims.train()\n```\nThis will train the SIMS model on the given expression matrices with target variable given by the `class_label` column in each label file.\n\n## Using SIMS inside github codespaces\nIf you are using SIMS only for predictions using an already trained model, github codespaces is the recommended way to use this tool. You can also use this pipeline to train it in smaller datasets as the computing services offered in codespaces are modest.\nTo use this tool in github codespaces start by forking the repo in your github account. Then create a new codespace with the SIMS repo as the Repository of choice.\nOnce inside the newly created environment pull the latest SIMS image:\n```docker\ndocker pull jmlehrer/sims:latest\n```\nRun the docker container mounting the file folder containing datasets and model checkpoints to the filesystem:\n```docker\ndocker run -it -v /path/to/local/folder:/path/in/container [image_name] /bin/bash\n```\nRun main.py to check if the installation has been completed. You can alter this file as shown above to perform the different tasks.\n```bash\npython main.py\n```\n\n\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "Scalable, Interpretable Deep Learning for Single-Cell RNA-seq Classification",
"version": "3.0.5",
"project_urls": {
"Homepage": "https://github.com/braingeneers/sims"
},
"split_keywords": [
"scsims"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c48e3d2dc17ff011d5d20e7f36f207334406572712a579e51ed13a204683f519",
"md5": "f726d162ec54074e10eabcdb148b6a53",
"sha256": "49f217d71fab22e235b4d8f9af5ac93bb9bd97c0c461a90730c83c6a5ef3ffed"
},
"downloads": -1,
"filename": "scsims-3.0.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "f726d162ec54074e10eabcdb148b6a53",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.9",
"size": 22039,
"upload_time": "2023-11-03T21:51:34",
"upload_time_iso_8601": "2023-11-03T21:51:34.107774Z",
"url": "https://files.pythonhosted.org/packages/c4/8e/3d2dc17ff011d5d20e7f36f207334406572712a579e51ed13a204683f519/scsims-3.0.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "09aeb7beee9a9fa4cf313a1344981ffe833779ffc1e23a895e5594cb0e087c50",
"md5": "60c3b2a7aae77aa9e458f64b044f5473",
"sha256": "3b11d9559711f6086a53265abcddd69a91873659bb0ab1774dc18c9338f6ffd6"
},
"downloads": -1,
"filename": "scsims-3.0.5.tar.gz",
"has_sig": false,
"md5_digest": "60c3b2a7aae77aa9e458f64b044f5473",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 23523,
"upload_time": "2023-11-03T21:51:36",
"upload_time_iso_8601": "2023-11-03T21:51:36.049375Z",
"url": "https://files.pythonhosted.org/packages/09/ae/b7beee9a9fa4cf313a1344981ffe833779ffc1e23a895e5594cb0e087c50/scsims-3.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-03 21:51:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "braingeneers",
"github_project": "sims",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "aiohttp",
"specs": [
[
"==",
"3.8.4"
]
]
},
{
"name": "aiosignal",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "anndata",
"specs": [
[
"==",
"0.9.1"
]
]
},
{
"name": "anyio",
"specs": [
[
"==",
"3.6.2"
]
]
},
{
"name": "appdirs",
"specs": [
[
"==",
"1.4.4"
]
]
},
{
"name": "arrow",
"specs": [
[
"==",
"1.2.3"
]
]
},
{
"name": "async-timeout",
"specs": [
[
"==",
"4.0.2"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"23.1.0"
]
]
},
{
"name": "beautifulsoup4",
"specs": [
[
"==",
"4.12.2"
]
]
},
{
"name": "blessed",
"specs": [
[
"==",
"1.20.0"
]
]
},
{
"name": "boto3",
"specs": [
[
"==",
"1.26.130"
]
]
},
{
"name": "botocore",
"specs": [
[
"==",
"1.29.130"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2023.5.7"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.1.0"
]
]
},
{
"name": "click",
"specs": [
[
"==",
"8.1.3"
]
]
},
{
"name": "contourpy",
"specs": [
[
"==",
"1.0.7"
]
]
},
{
"name": "croniter",
"specs": [
[
"==",
"1.3.14"
]
]
},
{
"name": "cycler",
"specs": [
[
"==",
"0.11.0"
]
]
},
{
"name": "dateutils",
"specs": [
[
"==",
"0.6.12"
]
]
},
{
"name": "deepdiff",
"specs": [
[
"==",
"6.3.0"
]
]
},
{
"name": "docker-pycreds",
"specs": [
[
"==",
"0.4.0"
]
]
},
{
"name": "fastapi",
"specs": [
[
"==",
"0.88.0"
]
]
},
{
"name": "fonttools",
"specs": [
[
"==",
"4.39.3"
]
]
},
{
"name": "fortran-language-server",
"specs": [
[
"==",
"1.12.0"
]
]
},
{
"name": "frozenlist",
"specs": [
[
"==",
"1.3.3"
]
]
},
{
"name": "fsspec",
"specs": [
[
"==",
"2023.5.0"
]
]
},
{
"name": "gitdb",
"specs": [
[
"==",
"4.0.10"
]
]
},
{
"name": "GitPython",
"specs": [
[
"==",
"3.1.31"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.14.0"
]
]
},
{
"name": "h5py",
"specs": [
[
"==",
"3.8.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.4"
]
]
},
{
"name": "importlib-resources",
"specs": [
[
"==",
"5.12.0"
]
]
},
{
"name": "inquirer",
"specs": [
[
"==",
"3.1.3"
]
]
},
{
"name": "itsdangerous",
"specs": [
[
"==",
"2.1.2"
]
]
},
{
"name": "Jinja2",
"specs": [
[
"==",
"3.1.2"
]
]
},
{
"name": "jmespath",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "joblib",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "kiwisolver",
"specs": [
[
"==",
"1.4.4"
]
]
},
{
"name": "lightning",
"specs": [
[
"==",
"2.0.2"
]
]
},
{
"name": "lightning-cloud",
"specs": [
[
"==",
"0.5.34"
]
]
},
{
"name": "lightning-utilities",
"specs": [
[
"==",
"0.8.0"
]
]
},
{
"name": "llvmlite",
"specs": [
[
"==",
"0.40.0"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"2.2.0"
]
]
},
{
"name": "MarkupSafe",
"specs": [
[
"==",
"2.1.2"
]
]
},
{
"name": "matplotlib",
"specs": [
[
"==",
"3.7.1"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "multidict",
"specs": [
[
"==",
"6.0.4"
]
]
},
{
"name": "natsort",
"specs": [
[
"==",
"8.3.1"
]
]
},
{
"name": "networkx",
"specs": [
[
"==",
"3.1"
]
]
},
{
"name": "numba",
"specs": [
[
"==",
"0.57.0"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"1.24.3"
]
]
},
{
"name": "ordered-set",
"specs": [
[
"==",
"4.1.0"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"23.1"
]
]
},
{
"name": "pandas",
"specs": [
[
"==",
"2.0.1"
]
]
},
{
"name": "pathtools",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "patsy",
"specs": [
[
"==",
"0.5.3"
]
]
},
{
"name": "Pillow",
"specs": [
[
"==",
"9.5.0"
]
]
},
{
"name": "protobuf",
"specs": [
[
"==",
"4.23.0"
]
]
},
{
"name": "psutil",
"specs": [
[
"==",
"5.9.5"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"1.10.7"
]
]
},
{
"name": "Pygments",
"specs": [
[
"==",
"2.15.1"
]
]
},
{
"name": "PyJWT",
"specs": [
[
"==",
"2.6.0"
]
]
},
{
"name": "pynndescent",
"specs": [
[
"==",
"0.5.10"
]
]
},
{
"name": "pyparsing",
"specs": [
[
"==",
"3.0.9"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.8.2"
]
]
},
{
"name": "python-editor",
"specs": [
[
"==",
"1.0.4"
]
]
},
{
"name": "python-multipart",
"specs": [
[
"==",
"0.0.6"
]
]
},
{
"name": "pytorch-lightning",
"specs": [
[
"==",
"2.0.2"
]
]
},
{
"name": "pytorch-tabnet",
"specs": [
[
"==",
"4.0"
]
]
},
{
"name": "pytz",
"specs": [
[
"==",
"2023.3"
]
]
},
{
"name": "PyYAML",
"specs": [
[
"==",
"6.0"
]
]
},
{
"name": "readchar",
"specs": [
[
"==",
"4.0.5"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.30.0"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"13.3.5"
]
]
},
{
"name": "s3transfer",
"specs": [
[
"==",
"0.6.1"
]
]
},
{
"name": "scanpy",
"specs": [
[
"==",
"1.9.3"
]
]
},
{
"name": "scikit-learn",
"specs": [
[
"==",
"1.2.2"
]
]
},
{
"name": "scipy",
"specs": [
[
"==",
"1.10.1"
]
]
},
{
"name": "seaborn",
"specs": [
[
"==",
"0.12.2"
]
]
},
{
"name": "sentry-sdk",
"specs": [
[
"==",
"1.22.2"
]
]
},
{
"name": "session-info",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "setproctitle",
"specs": [
[
"==",
"1.3.2"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.16.0"
]
]
},
{
"name": "smmap",
"specs": [
[
"==",
"5.0.0"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.0"
]
]
},
{
"name": "soupsieve",
"specs": [
[
"==",
"2.4.1"
]
]
},
{
"name": "starlette",
"specs": [
[
"==",
"0.22.0"
]
]
},
{
"name": "starsessions",
"specs": [
[
"==",
"1.3.0"
]
]
},
{
"name": "statsmodels",
"specs": [
[
"==",
"0.14.0"
]
]
},
{
"name": "stdlib-list",
"specs": [
[
"==",
"0.8.0"
]
]
},
{
"name": "threadpoolctl",
"specs": [
[
"==",
"3.1.0"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"1.13.1"
]
]
},
{
"name": "torchmetrics",
"specs": [
[
"==",
"0.11.4"
]
]
},
{
"name": "tqdm",
"specs": [
[
"==",
"4.65.0"
]
]
},
{
"name": "traitlets",
"specs": [
[
"==",
"5.9.0"
]
]
},
{
"name": "typing_extensions",
"specs": [
[
"==",
"4.5.0"
]
]
},
{
"name": "tzdata",
"specs": [
[
"==",
"2023.3"
]
]
},
{
"name": "umap-learn",
"specs": [
[
"==",
"0.5.3"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"1.26.15"
]
]
},
{
"name": "uvicorn",
"specs": [
[
"==",
"0.22.0"
]
]
},
{
"name": "wandb",
"specs": [
[
"==",
"0.15.2"
]
]
},
{
"name": "wcwidth",
"specs": [
[
"==",
"0.2.6"
]
]
},
{
"name": "websocket-client",
"specs": [
[
"==",
"1.5.1"
]
]
},
{
"name": "websockets",
"specs": [
[
"==",
"11.0.3"
]
]
},
{
"name": "yarl",
"specs": [
[
"==",
"1.9.2"
]
]
},
{
"name": "zipp",
"specs": [
[
"==",
"3.15.0"
]
]
}
],
"lcname": "scsims"
}
