FineST


NameFineST JSON
Version 0.0.9 PyPI version JSON
download
home_pagehttps://github.com/LingyuLi-math/FineST
SummaryFineST: Fine-grained Spatial Transcriptomic
upload_time2024-12-18 15:04:21
maintainerNone
docs_urlNone
author['Lingyu Li', 'Yuanhua Huang']
requires_pythonNone
licenseApache-2.0
keywords spatial transcriptomics he stained images cell-cell communication ligand-recptor interaction
VCS
bugtrack_url
requirements anndata scanpy SpatialDE SpatialDM SparseAEH pyarrow pykdtree scikit-image numpy scipy statsmodels pandas h5py openpyxl brokenaxes pyreadr squidpy stardist tensorflow matplotlib holoviews seaborn selenium plotly tenacity nbformat kaleido numpy scikit-learn hilearn threadpoolctl tqdm
Travis-CI No Travis.
coveralls test coverage No coveralls.
            ===========================================
FineST: Fine-grained Spatial Transcriptomic
===========================================

A statistical model and toolbox to identify the super-resolved ligand-receptor interaction 
with spatial co-expression (i.e., spatial association). 
Uniquely, FineST can distinguish co-expressed ligand-receptor pairs (LR pairs) 
from spatially separating pairs at sub-spot level or single-cell level, 
and identify the super-resolved ligand-receptor interaction (LRI).

.. image:: https://github.com/StatBiomed/FineST/blob/main/docs/fig/FineST_framework_all.png?raw=true
   :width: 800px
   :align: center

It comprises three components (*Training*-*Imputation*-*Discovery*) after HE image feature is extracted: 

* Step0: HE image feature extraction
* Step1: **Training** FineST on the within spots
* Step2: Super-resolution spatial RNA-seq **imputation**
* Step3: Fine-grained LR pair and CCC pattern **discovery**

.. It comprises two main steps:

.. 1. global selection `spatialdm_global` to identify significantly interacting LR pairs;
.. 2. local selection `spatialdm_local` to identify local spots for each interaction.

Installation
============

FineST is available through `PyPI <https://pypi.org/project/FineST/>`_.
To install, type the following command line and add ``-U`` for updates:

.. code-block:: bash

   pip install -U FineST

Alternatively, install from this GitHub repository for latest (often
development) version (time: < 1 min):

.. code-block:: bash

   pip install -U git+https://github.com/StatBiomed/FineST

Installation using Conda
========================

.. code-block:: bash

   $ git clone https://github.com/StatBiomed/FineST.git
   $ conda create --name FineST python=3.8
   $ conda activate FineST
   $ cd FineST
   $ pip install -r requirements.txt

Typically installation is completed within a few minutes. 
Then install pytorch, refer to `pytorch installation <https://pytorch.org/get-started/locally/>`_.

.. code-block:: bash

   $ conda install pytorch=1.7.1 torchvision torchaudio cudatoolkit=11.0 -c pytorch

Verify the installation using the following command:

.. code-block:: text

   python
   >>> import torch
   >>> print(torch.__version__)
   >>> print(torch.cuda.is_available())


Get Started for *Visium* or *Visium HD* data
============================================

**Usage illustrations**: 

The source codes for reproducing the FineST analysis in this work are provided (see `demo` directory).
All relevant materials involved in the reproducing codes are available 
from `Google Drive <https://drive.google.com/drive/folders/10WvKW2EtQVuH3NWUnrde4JOW_Dd_H6r8>`_.

* For *Visium*, using a single slice of 10x Visium human nasopharyngeal carcinoma (NPC) data.

* For *Visium HD*, using a single slice of 10x Visium HD human colorectal cancer (CRC) data with 16-um bin.


Step0: HE image feature extraction (for *Visium*)
-------------------------------------------------

*Visium* measures about 5k spots across the entire tissue area. 
The diameter of each individual spot is roughly 55 micrometers (um), 
while the center-to-center distance between two adjacent spots is about 100 um.
In order to capture the gene expression profile across the whole tissue ASSP, 

Firstly, interpolate ``between spots`` in horizontal and vertical directions, 
using ``Spot_interpolate.py``.

.. code-block:: bash

   python ./FineST/demo/Spot_interpolate.py \
      --data_path ./Dataset/NPC/ \
      --position_list tissue_positions_list.csv \
      --dataset patient1 

.. ``Spot_interpolate.py`` also output the execution time and spot number ratio:

.. * The spots feature interpolation time is: 2.549 seconds
.. * # of interpolated between-spots are: 2.786 times vs. original within-spots
.. * # 0f final all spots are: 3.786 times vs. original within-spots
   
with **Input:**  ``tissue_positions_list.csv`` - Locations of ``within spots`` (n), 
and **Output:**  ``_position_add_tissue.csv``- Locations of ``between spots`` (m ~= 3n).


.. **Input file:**

.. * ``tissue_positions_list.csv``: Spot locations

.. **Output files:**

.. * ``_position_add_tissue.csv``: Spot locations of the ``between spots`` (m ~= 3n)
.. * ``_position_all_tissue.csv``: Spot locations of all ``between spots`` and ``within spots``

Then extracte the ``within spots`` HE image feature embeddings using ``HIPT_image_feature_extract.py``.

.. code-block:: bash

   python ./FineST/demo/HIPT_image_feature_extract.py \
      --dataset AH_Patient1 \
      --position ./Dataset/NPC/patient1/tissue_positions_list.csv \
      --image ./Dataset/NPC/patient1/20210809-C-AH4199551.tif \
      --output_path_img ./Dataset/NPC/HIPT/AH_Patient1_pth_64_16_image \
      --output_path_pth ./Dataset/NPC/HIPT/AH_Patient1_pth_64_16 \
      --patch_size 64 \
      --logging_folder ./Logging/HIPT_AH_Patient1/

.. ``HIPT_image_feature_extract.py`` also output the execution time:

.. * The image segment execution time for the loop is: 3.493 seconds
.. * The image feature extract time for the loop is: 13.374 seconds


.. **Input files:**

.. * ``20210809-C-AH4199551.tif``: Raw histology image
.. * ``tissue_positions_list.csv``: "Within spot" (Original in_tissue spots) locations

.. **Output files:**

.. * ``AH_Patient1_pth_64_16_image``: Segmeted "Within spot" histology image patches (.png)
.. * ``AH_Patient1_pth_64_16``: Extracted "Within spot" image feature embeddiings for each patche (.pth)


Similarlly, extracte the ``between spots`` HE image feature embeddings using ``HIPT_image_feature_extract.py``.

.. code-block:: bash

   python ./FineST/demo/HIPT_image_feature_extract.py \
      --dataset AH_Patient1 \
      --position ./Dataset/NPC/patient1/patient1_position_add_tissue.csv \
      --image ./Dataset/NPC/patient1/20210809-C-AH4199551.tif \
      --output_path_img ./Dataset/NPC/HIPT/NEW_AH_Patient1_pth_64_16_image \
      --output_path_pth ./Dataset/NPC/HIPT/NEW_AH_Patient1_pth_64_16 \
      --patch_size 64 \
      --logging_folder ./Logging/HIPT_AH_Patient1/

``HIPT_image_feature_extract.py`` also output the execution time:

* The image segment execution time for the loop is:  8.153 seconds
* The image feature extract time for the loop is: 35.499 seconds


**Input files:**

* ``20210809-C-AH4199551.tif``: Raw histology image 
* ``patient1_position_add_tissue.csv``: "Between spot" (Interpolated spots) locations

**Output files:**

* ``NEW_AH_Patient1_pth_64_16_image``: Segmeted "Between spot" histology image patches (.png)
* ``NEW_AH_Patient1_pth_64_16``: Extracted "Between spot" image feature embeddiings for each patche (.pth)


Step0: HE image feature extraction (for *Visium HD*)
----------------------------------------------------

*Visium HD* captures continuous squares without gaps, it measures the whole tissue area.

.. code-block:: bash

   python ./FineST/demo/HIPT_image_feature_extract.py \
      --dataset HD_CRC_16um \
      --position ./Dataset/CRC/square_016um/tissue_positions.parquet \
      --image ./Dataset/CRC/square_016um/Visium_HD_Human_Colon_Cancer_tissue_image.btf \
      --output_path_img ./Dataset/CRC/HIPT/HD_CRC_16um_pth_32_16_image \
      --output_path_pth ./Dataset/CRC/HIPT/HD_CRC_16um_pth_32_16 \
      --patch_size 32 \
      --logging_folder ./Logging/HIPT_HD_CRC_16um/

``HIPT_image_feature_extract.py`` also output the execution time:

* The image segment execution time for the loop is: 62.491 seconds
* The image feature extract time for the loop is: 1717.818 seconds

**Input files:**

* ``Visium_HD_Human_Colon_Cancer_tissue_image.btf``: Raw histology image (.btf *Visium HD* or .tif *Visium*)
* ``tissue_positions.parquet``: Spot/bin locations (.parquet *Visium HD* or .csv *Visium*)

**Output files:**

* ``HD_CRC_16um_pth_32_16_image``: Segmeted histology image patches (.png)
* ``HD_CRC_16um_pth_32_16``: Extracted image feature embeddiings for each patche (.pth)


Step1: Training FineST on the within spots
==========================================

On *Visium* dataset, if trained weights (i.e. **weight_save_path**) have been obtained, just run the following command.
Otherwise, if you want to re-train a model, just omit **weight_save_path** line.

.. code-block:: bash

   python ./FineST/FineST/demo/FineST_train_infer.py \
      --system_path '/mnt/lingyu/nfs_share2/Python/' \
      --weight_path 'FineST/FineST_local/Finetune/' \
      --parame_path 'FineST/FineST/parameter/parameters_NPC_P10125.json' \
      --dataset_class 'Visium' \
      --gene_selected 'CD70' \
      --LRgene_path 'FineST/FineST/Dataset/LRgene/LRgene_CellChatDB_baseline.csv' \
      --visium_path 'FineST/FineST/Dataset/NPC/patient1/tissue_positions_list.csv' \
      --image_embed_path 'NPC/Data/stdata/ZhuoLiang/LLYtest/AH_Patient1_pth_64_16/' \
      --spatial_pos_path 'FineST/FineST_local/Dataset/NPC/ContrastP1geneLR/position_order.csv' \
      --reduced_mtx_path 'FineST/FineST_local/Dataset/NPC/ContrastP1geneLR/harmony_matrix.npy' \
      --weight_save_path 'FineST/FineST_local/Finetune/20240125140443830148' \
      --figure_save_path 'FineST/FineST_local/Dataset/NPC/Figures/' 

``FineST_train_infer.py`` is used to train and evaluate the FineST model using Pearson Correlation, it outputs:

* Average correlation of all spots: 0.8534651812923978
* Average correlation of all genes: 0.8845136777311445

**Input files:**

* ``parameters_NPC_P10125.json``: The model parameters.
* ``LRgene_CellChatDB_baseline.csv``: The genes involved in Ligand or Receptor from CellChatDB.
* ``tissue_positions_list.csv``: It can be found in the spatial folder of 10x Visium outputs.
* ``AH_Patient1_pth_64_16``: Image feature folder from HIPT ``HIPT_image_feature_extract.py``.
* ``position_order.csv``: Ordered tissue positions list, according to image patches' coordinates.
* ``harmony_matrix.npy``: Ordered gene expression matrix, according to image patches' coordinates.
* ``20240125140443830148``: The trained weights. Just omit it if you want to newly train a model.

**Output files:**

* ``Finetune``: The logging results ``model.log`` and trained weights ``epoch_50.pt`` (.log and .pt)
* ``Figures``: The visualization plots, used to see whether the model trained well or not (.pdf)


Step2: Super-resolution spatial RNA-seq imputation
==================================================

For *sub-spot* resolution
-------------------------

This step supposes that the trained weights (i.e. **weight_save_path**) have been obtained, just run the following.

.. code-block:: bash

   python ./FineST/FineST/demo/High_resolution_imputation.py \
      --system_path '/mnt/lingyu/nfs_share2/Python/' \
      --weight_path 'FineST/FineST_local/Finetune/' \
      --parame_path 'FineST/FineST/parameter/parameters_NPC_P10125.json' \
      --dataset_class 'Visium' \
      --gene_selected 'CD70' \
      --LRgene_path 'FineST/FineST/Dataset/LRgene/LRgene_CellChatDB_baseline.csv' \
      --visium_path 'FineST/FineST/Dataset/NPC/patient1/tissue_positions_list.csv' \
      --imag_within_path 'NPC/Data/stdata/ZhuoLiang/LLYtest/AH_Patient1_pth_64_16/' \
      --imag_betwen_path 'NPC/Data/stdata/ZhuoLiang/LLYtest/NEW_AH_Patient1_pth_64_16/' \
      --spatial_pos_path 'FineST/FineST_local/Dataset/NPC/ContrastP1geneLR/position_order_all.csv' \
      --weight_save_path 'FineST/FineST_local/Finetune/20240125140443830148' \
      --figure_save_path 'FineST/FineST_local/Dataset/NPC/Figures/' \
      --adata_all_supr_path 'FineST/FineST_local/Dataset/ImputData/patient1/patient1_adata_all.h5ad' \
      --adata_all_spot_path 'FineST/FineST_local/Dataset/ImputData/patient1/patient1_adata_all_spot.h5ad' 

``High_resolution_imputation.py`` is used to predict super-resolved gene expression 
based on the image segmentation (Geometric ``sub-spot level`` or Nuclei ``single-cell level``).

**Input files:**

* ``parameters_NPC_P10125.json``: The model parameters.
* ``LRgene_CellChatDB_baseline.csv``: The genes involved in Ligand or Receptor from CellChatDB.
* ``tissue_positions_list.csv``: It can be found in the spatial folder of 10x Visium outputs.
* ``AH_Patient1_pth_64_16``: Image feature of within-spots from ``HIPT_image_feature_extract.py``.
* ``NEW_AH_Patient1_pth_64_16``: Image feature of between-spots from ``HIPT_image_feature_extract.py``.
* ``position_order_all.csv``: Ordered tissue positions list, of both within spots and between spots.
* ``20240125140443830148``: The trained weights. Just omit it if you want to newly train a model.

**Output files:**

* ``Finetune``: The logging results ``model.log`` and trained weights ``epoch_50.pt`` (.log and .pt)
* ``Figures``: The visualization plots, used to see whether the model trained well or not (.pdf)
* ``patient1_adata_all.h5ad``: High-resolution gene expression, at sub-spot level (16x3x resolution).
* ``patient1_adata_all_spot.h5ad``: High-resolution gene expression, at spot level (3x resolution).

For *single-cell* resolution
----------------------------

Replace ``AH_Patient1_pth_64_16`` and ``NEW_AH_Patient1_pth_64_16`` using ``sc Patient1 pth 16 16`` 
(saved in `Google Drive <https://drive.google.com/drive/folders/10WvKW2EtQVuH3NWUnrde4JOW_Dd_H6r8>`_), i.e.,   
the image feature of single-nuclei from ``HIPT_image_feature_extract.py``. The details will be here soon.


Step3: Fine-grained LR pair and CCC pattern discovery
=====================================================



.. Quick example
.. =============

.. Using the build-in NPC dataset as an example, the following Python script
.. will predict super-resolution ST gene expression and compute the p-value indicating whether a certain Ligand-Receptor is
.. spatially co-expressed. 

Detailed Manual
===============

The full manual is at `FineST tutorial <https://finest-rtd-tutorial.readthedocs.io>`_ for installation, tutorials and examples. 

* `Interpolate between-spots among within-spots by FineST (For Visium dataset)`_.

* `Crop region of interest (ROI) from HE image by FineST (Visium or Visium HD)`_.

* `Sub-spot level (16x resolution) prediction by FineST (For Visium dataset)`_.

* `Sub-bin level (from 16um to 8um) prediction by FineST (For Visium HD dataset)`_.

* `Super-resolved ligand-receptor interavtion discovery by FineST`_.

.. _Interpolate between-spots among within-spots by FineST (For Visium dataset): docs/source/Between_spot_demo.ipynb

.. _Crop region of interest (ROI) from HE image by FineST (Visium or Visium HD): docs/source/Crop_ROI_image.ipynb

.. _Sub-spot level (16x resolution) prediction by FineST (For Visium dataset): docs/source/NPC_Train_Impute.ipynb

.. _Sub-bin level (from 16um to 8um) prediction by FineST (For Visium HD dataset): docs/source/CRC16_Train_Impute.ipynb

.. _Super-resolved ligand-receptor interavtion discovery by FineST: docs/source/NPC_LRI_CCC.ipynb


Contact Information
===================

Please contact Lingyu Li (`lingyuli@hku.hk <mailto:lingyuli@hku.hk>`_) or Yuanhua Huang (`yuanhua@hku.hk <mailto:yuanhua@hku.hk>`_) if any enquiry.


            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/LingyuLi-math/FineST",
    "name": "FineST",
    "maintainer": null,
    "docs_url": null,
    "requires_python": null,
    "maintainer_email": null,
    "keywords": "Spatial transcriptomics, HE stained images, Cell-cell communication, Ligand-recptor interaction",
    "author": "['Lingyu Li', 'Yuanhua Huang']",
    "author_email": "lingyuli@hku.hk",
    "download_url": "https://files.pythonhosted.org/packages/08/20/ff0e6d9ff5064a43d9221709b2aa28a4ca614e71f4a83285c559564f35e6/finest-0.0.9.tar.gz",
    "platform": null,
    "description": "===========================================\nFineST: Fine-grained Spatial Transcriptomic\n===========================================\n\nA statistical model and toolbox to identify the super-resolved ligand-receptor interaction \nwith spatial co-expression (i.e., spatial association). \nUniquely, FineST can distinguish co-expressed ligand-receptor pairs (LR pairs) \nfrom spatially separating pairs at sub-spot level or single-cell level, \nand identify the super-resolved ligand-receptor interaction (LRI).\n\n.. image:: https://github.com/StatBiomed/FineST/blob/main/docs/fig/FineST_framework_all.png?raw=true\n   :width: 800px\n   :align: center\n\nIt comprises three components (*Training*-*Imputation*-*Discovery*) after HE image feature is extracted: \n\n* Step0: HE image feature extraction\n* Step1: **Training** FineST on the within spots\n* Step2: Super-resolution spatial RNA-seq **imputation**\n* Step3: Fine-grained LR pair and CCC pattern **discovery**\n\n.. It comprises two main steps:\n\n.. 1. global selection `spatialdm_global` to identify significantly interacting LR pairs;\n.. 2. local selection `spatialdm_local` to identify local spots for each interaction.\n\nInstallation\n============\n\nFineST is available through `PyPI <https://pypi.org/project/FineST/>`_.\nTo install, type the following command line and add ``-U`` for updates:\n\n.. code-block:: bash\n\n   pip install -U FineST\n\nAlternatively, install from this GitHub repository for latest (often\ndevelopment) version (time: < 1 min):\n\n.. code-block:: bash\n\n   pip install -U git+https://github.com/StatBiomed/FineST\n\nInstallation using Conda\n========================\n\n.. code-block:: bash\n\n   $ git clone https://github.com/StatBiomed/FineST.git\n   $ conda create --name FineST python=3.8\n   $ conda activate FineST\n   $ cd FineST\n   $ pip install -r requirements.txt\n\nTypically installation is completed within a few minutes. \nThen install pytorch, refer to `pytorch installation <https://pytorch.org/get-started/locally/>`_.\n\n.. code-block:: bash\n\n   $ conda install pytorch=1.7.1 torchvision torchaudio cudatoolkit=11.0 -c pytorch\n\nVerify the installation using the following command:\n\n.. code-block:: text\n\n   python\n   >>> import torch\n   >>> print(torch.__version__)\n   >>> print(torch.cuda.is_available())\n\n\nGet Started for *Visium* or *Visium HD* data\n============================================\n\n**Usage illustrations**: \n\nThe source codes for reproducing the FineST analysis in this work are provided (see `demo` directory).\nAll relevant materials involved in the reproducing codes are available \nfrom `Google Drive <https://drive.google.com/drive/folders/10WvKW2EtQVuH3NWUnrde4JOW_Dd_H6r8>`_.\n\n* For *Visium*, using a single slice of 10x Visium human nasopharyngeal carcinoma (NPC) data.\n\n* For *Visium HD*, using a single slice of 10x Visium HD human colorectal cancer (CRC) data with 16-um bin.\n\n\nStep0: HE image feature extraction (for *Visium*)\n-------------------------------------------------\n\n*Visium* measures about 5k spots across the entire tissue area. \nThe diameter of each individual spot is roughly 55 micrometers (um), \nwhile the center-to-center distance between two adjacent spots is about 100 um.\nIn order to capture the gene expression profile across the whole tissue ASSP, \n\nFirstly, interpolate ``between spots`` in horizontal and vertical directions, \nusing ``Spot_interpolate.py``.\n\n.. code-block:: bash\n\n   python ./FineST/demo/Spot_interpolate.py \\\n      --data_path ./Dataset/NPC/ \\\n      --position_list tissue_positions_list.csv \\\n      --dataset patient1 \n\n.. ``Spot_interpolate.py`` also output the execution time and spot number ratio:\n\n.. * The spots feature interpolation time is: 2.549 seconds\n.. * # of interpolated between-spots are: 2.786 times vs. original within-spots\n.. * # 0f final all spots are: 3.786 times vs. original within-spots\n   \nwith **Input:**  ``tissue_positions_list.csv`` - Locations of ``within spots`` (n), \nand **Output:**  ``_position_add_tissue.csv``- Locations of ``between spots`` (m ~= 3n).\n\n\n.. **Input file:**\n\n.. * ``tissue_positions_list.csv``: Spot locations\n\n.. **Output files:**\n\n.. * ``_position_add_tissue.csv``: Spot locations of the ``between spots`` (m ~= 3n)\n.. * ``_position_all_tissue.csv``: Spot locations of all ``between spots`` and ``within spots``\n\nThen extracte the ``within spots`` HE image feature embeddings using ``HIPT_image_feature_extract.py``.\n\n.. code-block:: bash\n\n   python ./FineST/demo/HIPT_image_feature_extract.py \\\n      --dataset AH_Patient1 \\\n      --position ./Dataset/NPC/patient1/tissue_positions_list.csv \\\n      --image ./Dataset/NPC/patient1/20210809-C-AH4199551.tif \\\n      --output_path_img ./Dataset/NPC/HIPT/AH_Patient1_pth_64_16_image \\\n      --output_path_pth ./Dataset/NPC/HIPT/AH_Patient1_pth_64_16 \\\n      --patch_size 64 \\\n      --logging_folder ./Logging/HIPT_AH_Patient1/\n\n.. ``HIPT_image_feature_extract.py`` also output the execution time:\n\n.. * The image segment execution time for the loop is: 3.493 seconds\n.. * The image feature extract time for the loop is: 13.374 seconds\n\n\n.. **Input files:**\n\n.. * ``20210809-C-AH4199551.tif``: Raw histology image\n.. * ``tissue_positions_list.csv``: \"Within spot\" (Original in_tissue spots) locations\n\n.. **Output files:**\n\n.. * ``AH_Patient1_pth_64_16_image``: Segmeted \"Within spot\" histology image patches (.png)\n.. * ``AH_Patient1_pth_64_16``: Extracted \"Within spot\" image feature embeddiings for each patche (.pth)\n\n\nSimilarlly, extracte the ``between spots`` HE image feature embeddings using ``HIPT_image_feature_extract.py``.\n\n.. code-block:: bash\n\n   python ./FineST/demo/HIPT_image_feature_extract.py \\\n      --dataset AH_Patient1 \\\n      --position ./Dataset/NPC/patient1/patient1_position_add_tissue.csv \\\n      --image ./Dataset/NPC/patient1/20210809-C-AH4199551.tif \\\n      --output_path_img ./Dataset/NPC/HIPT/NEW_AH_Patient1_pth_64_16_image \\\n      --output_path_pth ./Dataset/NPC/HIPT/NEW_AH_Patient1_pth_64_16 \\\n      --patch_size 64 \\\n      --logging_folder ./Logging/HIPT_AH_Patient1/\n\n``HIPT_image_feature_extract.py`` also output the execution time:\n\n* The image segment execution time for the loop is:  8.153 seconds\n* The image feature extract time for the loop is: 35.499 seconds\n\n\n**Input files:**\n\n* ``20210809-C-AH4199551.tif``: Raw histology image \n* ``patient1_position_add_tissue.csv``: \"Between spot\" (Interpolated spots) locations\n\n**Output files:**\n\n* ``NEW_AH_Patient1_pth_64_16_image``: Segmeted \"Between spot\" histology image patches (.png)\n* ``NEW_AH_Patient1_pth_64_16``: Extracted \"Between spot\" image feature embeddiings for each patche (.pth)\n\n\nStep0: HE image feature extraction (for *Visium HD*)\n----------------------------------------------------\n\n*Visium HD* captures continuous squares without gaps, it measures the whole tissue area.\n\n.. code-block:: bash\n\n   python ./FineST/demo/HIPT_image_feature_extract.py \\\n      --dataset HD_CRC_16um \\\n      --position ./Dataset/CRC/square_016um/tissue_positions.parquet \\\n      --image ./Dataset/CRC/square_016um/Visium_HD_Human_Colon_Cancer_tissue_image.btf \\\n      --output_path_img ./Dataset/CRC/HIPT/HD_CRC_16um_pth_32_16_image \\\n      --output_path_pth ./Dataset/CRC/HIPT/HD_CRC_16um_pth_32_16 \\\n      --patch_size 32 \\\n      --logging_folder ./Logging/HIPT_HD_CRC_16um/\n\n``HIPT_image_feature_extract.py`` also output the execution time:\n\n* The image segment execution time for the loop is: 62.491 seconds\n* The image feature extract time for the loop is: 1717.818 seconds\n\n**Input files:**\n\n* ``Visium_HD_Human_Colon_Cancer_tissue_image.btf``: Raw histology image (.btf *Visium HD* or .tif *Visium*)\n* ``tissue_positions.parquet``: Spot/bin locations (.parquet *Visium HD* or .csv *Visium*)\n\n**Output files:**\n\n* ``HD_CRC_16um_pth_32_16_image``: Segmeted histology image patches (.png)\n* ``HD_CRC_16um_pth_32_16``: Extracted image feature embeddiings for each patche (.pth)\n\n\nStep1: Training FineST on the within spots\n==========================================\n\nOn *Visium* dataset, if trained weights (i.e. **weight_save_path**) have been obtained, just run the following command.\nOtherwise, if you want to re-train a model, just omit **weight_save_path** line.\n\n.. code-block:: bash\n\n   python ./FineST/FineST/demo/FineST_train_infer.py \\\n      --system_path '/mnt/lingyu/nfs_share2/Python/' \\\n      --weight_path 'FineST/FineST_local/Finetune/' \\\n      --parame_path 'FineST/FineST/parameter/parameters_NPC_P10125.json' \\\n      --dataset_class 'Visium' \\\n      --gene_selected 'CD70' \\\n      --LRgene_path 'FineST/FineST/Dataset/LRgene/LRgene_CellChatDB_baseline.csv' \\\n      --visium_path 'FineST/FineST/Dataset/NPC/patient1/tissue_positions_list.csv' \\\n      --image_embed_path 'NPC/Data/stdata/ZhuoLiang/LLYtest/AH_Patient1_pth_64_16/' \\\n      --spatial_pos_path 'FineST/FineST_local/Dataset/NPC/ContrastP1geneLR/position_order.csv' \\\n      --reduced_mtx_path 'FineST/FineST_local/Dataset/NPC/ContrastP1geneLR/harmony_matrix.npy' \\\n      --weight_save_path 'FineST/FineST_local/Finetune/20240125140443830148' \\\n      --figure_save_path 'FineST/FineST_local/Dataset/NPC/Figures/' \n\n``FineST_train_infer.py`` is used to train and evaluate the FineST model using Pearson Correlation, it outputs:\n\n* Average correlation of all spots: 0.8534651812923978\n* Average correlation of all genes: 0.8845136777311445\n\n**Input files:**\n\n* ``parameters_NPC_P10125.json``: The model parameters.\n* ``LRgene_CellChatDB_baseline.csv``: The genes involved in Ligand or Receptor from CellChatDB.\n* ``tissue_positions_list.csv``: It can be found in the spatial folder of 10x Visium outputs.\n* ``AH_Patient1_pth_64_16``: Image feature folder from HIPT ``HIPT_image_feature_extract.py``.\n* ``position_order.csv``: Ordered tissue positions list, according to image patches' coordinates.\n* ``harmony_matrix.npy``: Ordered gene expression matrix, according to image patches' coordinates.\n* ``20240125140443830148``: The trained weights. Just omit it if you want to newly train a model.\n\n**Output files:**\n\n* ``Finetune``: The logging results ``model.log`` and trained weights ``epoch_50.pt`` (.log and .pt)\n* ``Figures``: The visualization plots, used to see whether the model trained well or not (.pdf)\n\n\nStep2: Super-resolution spatial RNA-seq imputation\n==================================================\n\nFor *sub-spot* resolution\n-------------------------\n\nThis step supposes that the trained weights (i.e. **weight_save_path**) have been obtained, just run the following.\n\n.. code-block:: bash\n\n   python ./FineST/FineST/demo/High_resolution_imputation.py \\\n      --system_path '/mnt/lingyu/nfs_share2/Python/' \\\n      --weight_path 'FineST/FineST_local/Finetune/' \\\n      --parame_path 'FineST/FineST/parameter/parameters_NPC_P10125.json' \\\n      --dataset_class 'Visium' \\\n      --gene_selected 'CD70' \\\n      --LRgene_path 'FineST/FineST/Dataset/LRgene/LRgene_CellChatDB_baseline.csv' \\\n      --visium_path 'FineST/FineST/Dataset/NPC/patient1/tissue_positions_list.csv' \\\n      --imag_within_path 'NPC/Data/stdata/ZhuoLiang/LLYtest/AH_Patient1_pth_64_16/' \\\n      --imag_betwen_path 'NPC/Data/stdata/ZhuoLiang/LLYtest/NEW_AH_Patient1_pth_64_16/' \\\n      --spatial_pos_path 'FineST/FineST_local/Dataset/NPC/ContrastP1geneLR/position_order_all.csv' \\\n      --weight_save_path 'FineST/FineST_local/Finetune/20240125140443830148' \\\n      --figure_save_path 'FineST/FineST_local/Dataset/NPC/Figures/' \\\n      --adata_all_supr_path 'FineST/FineST_local/Dataset/ImputData/patient1/patient1_adata_all.h5ad' \\\n      --adata_all_spot_path 'FineST/FineST_local/Dataset/ImputData/patient1/patient1_adata_all_spot.h5ad' \n\n``High_resolution_imputation.py`` is used to predict super-resolved gene expression \nbased on the image segmentation (Geometric ``sub-spot level`` or Nuclei ``single-cell level``).\n\n**Input files:**\n\n* ``parameters_NPC_P10125.json``: The model parameters.\n* ``LRgene_CellChatDB_baseline.csv``: The genes involved in Ligand or Receptor from CellChatDB.\n* ``tissue_positions_list.csv``: It can be found in the spatial folder of 10x Visium outputs.\n* ``AH_Patient1_pth_64_16``: Image feature of within-spots from ``HIPT_image_feature_extract.py``.\n* ``NEW_AH_Patient1_pth_64_16``: Image feature of between-spots from ``HIPT_image_feature_extract.py``.\n* ``position_order_all.csv``: Ordered tissue positions list, of both within spots and between spots.\n* ``20240125140443830148``: The trained weights. Just omit it if you want to newly train a model.\n\n**Output files:**\n\n* ``Finetune``: The logging results ``model.log`` and trained weights ``epoch_50.pt`` (.log and .pt)\n* ``Figures``: The visualization plots, used to see whether the model trained well or not (.pdf)\n* ``patient1_adata_all.h5ad``: High-resolution gene expression, at sub-spot level (16x3x resolution).\n* ``patient1_adata_all_spot.h5ad``: High-resolution gene expression, at spot level (3x resolution).\n\nFor *single-cell* resolution\n----------------------------\n\nReplace ``AH_Patient1_pth_64_16`` and ``NEW_AH_Patient1_pth_64_16`` using ``sc Patient1 pth 16 16`` \n(saved in `Google Drive <https://drive.google.com/drive/folders/10WvKW2EtQVuH3NWUnrde4JOW_Dd_H6r8>`_), i.e.,   \nthe image feature of single-nuclei from ``HIPT_image_feature_extract.py``. The details will be here soon.\n\n\nStep3: Fine-grained LR pair and CCC pattern discovery\n=====================================================\n\n\n\n.. Quick example\n.. =============\n\n.. Using the build-in NPC dataset as an example, the following Python script\n.. will predict super-resolution ST gene expression and compute the p-value indicating whether a certain Ligand-Receptor is\n.. spatially co-expressed. \n\nDetailed Manual\n===============\n\nThe full manual is at `FineST tutorial <https://finest-rtd-tutorial.readthedocs.io>`_ for installation, tutorials and examples. \n\n* `Interpolate between-spots among within-spots by FineST (For Visium dataset)`_.\n\n* `Crop region of interest (ROI) from HE image by FineST (Visium or Visium HD)`_.\n\n* `Sub-spot level (16x resolution) prediction by FineST (For Visium dataset)`_.\n\n* `Sub-bin level (from 16um to 8um) prediction by FineST (For Visium HD dataset)`_.\n\n* `Super-resolved ligand-receptor interavtion discovery by FineST`_.\n\n.. _Interpolate between-spots among within-spots by FineST (For Visium dataset): docs/source/Between_spot_demo.ipynb\n\n.. _Crop region of interest (ROI) from HE image by FineST (Visium or Visium HD): docs/source/Crop_ROI_image.ipynb\n\n.. _Sub-spot level (16x resolution) prediction by FineST (For Visium dataset): docs/source/NPC_Train_Impute.ipynb\n\n.. _Sub-bin level (from 16um to 8um) prediction by FineST (For Visium HD dataset): docs/source/CRC16_Train_Impute.ipynb\n\n.. _Super-resolved ligand-receptor interavtion discovery by FineST: docs/source/NPC_LRI_CCC.ipynb\n\n\nContact Information\n===================\n\nPlease contact Lingyu Li (`lingyuli@hku.hk <mailto:lingyuli@hku.hk>`_) or Yuanhua Huang (`yuanhua@hku.hk <mailto:yuanhua@hku.hk>`_) if any enquiry.\n\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "FineST: Fine-grained Spatial Transcriptomic",
    "version": "0.0.9",
    "project_urls": {
        "Homepage": "https://github.com/LingyuLi-math/FineST"
    },
    "split_keywords": [
        "spatial transcriptomics",
        " he stained images",
        " cell-cell communication",
        " ligand-recptor interaction"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "81f1da7a296f211ce8cd34f4aea9b1a5fef0b404cccbc55826295448e6df498f",
                "md5": "6104c5007a054a71e6ff361ddb4cd415",
                "sha256": "f782473836a7474ab00723f42e2ede19663186d16afca173fba61ac3af920c49"
            },
            "downloads": -1,
            "filename": "FineST-0.0.9-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "6104c5007a054a71e6ff361ddb4cd415",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 59096,
            "upload_time": "2024-12-18T15:04:19",
            "upload_time_iso_8601": "2024-12-18T15:04:19.674698Z",
            "url": "https://files.pythonhosted.org/packages/81/f1/da7a296f211ce8cd34f4aea9b1a5fef0b404cccbc55826295448e6df498f/FineST-0.0.9-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "0820ff0e6d9ff5064a43d9221709b2aa28a4ca614e71f4a83285c559564f35e6",
                "md5": "c7df4f70eaf2f27e8890351c0d50046e",
                "sha256": "a0fe7654203364d7af5615825030709be61cb07777d3ee654d6af1acf5ec6651"
            },
            "downloads": -1,
            "filename": "finest-0.0.9.tar.gz",
            "has_sig": false,
            "md5_digest": "c7df4f70eaf2f27e8890351c0d50046e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 59250,
            "upload_time": "2024-12-18T15:04:21",
            "upload_time_iso_8601": "2024-12-18T15:04:21.656601Z",
            "url": "https://files.pythonhosted.org/packages/08/20/ff0e6d9ff5064a43d9221709b2aa28a4ca614e71f4a83285c559564f35e6/finest-0.0.9.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-12-18 15:04:21",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "LingyuLi-math",
    "github_project": "FineST",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [
        {
            "name": "anndata",
            "specs": [
                [
                    "==",
                    "0.9.2"
                ]
            ]
        },
        {
            "name": "scanpy",
            "specs": [
                [
                    "==",
                    "1.9.8"
                ]
            ]
        },
        {
            "name": "SpatialDE",
            "specs": []
        },
        {
            "name": "SpatialDM",
            "specs": [
                [
                    "==",
                    "0.2.0"
                ]
            ]
        },
        {
            "name": "SparseAEH",
            "specs": [
                [
                    "==",
                    "0.1.0"
                ]
            ]
        },
        {
            "name": "pyarrow",
            "specs": [
                [
                    "==",
                    "17.0.0"
                ]
            ]
        },
        {
            "name": "pykdtree",
            "specs": [
                [
                    "==",
                    "1.3.7"
                ]
            ]
        },
        {
            "name": "scikit-image",
            "specs": []
        },
        {
            "name": "numpy",
            "specs": [
                [
                    ">=",
                    "1.9.0"
                ]
            ]
        },
        {
            "name": "scipy",
            "specs": [
                [
                    ">=",
                    "1.4.0"
                ]
            ]
        },
        {
            "name": "statsmodels",
            "specs": [
                [
                    ">=",
                    "0.11"
                ]
            ]
        },
        {
            "name": "pandas",
            "specs": [
                [
                    "==",
                    "2.0.3"
                ]
            ]
        },
        {
            "name": "h5py",
            "specs": [
                [
                    "==",
                    "3.10.0"
                ]
            ]
        },
        {
            "name": "openpyxl",
            "specs": [
                [
                    "==",
                    "3.1.5"
                ]
            ]
        },
        {
            "name": "brokenaxes",
            "specs": [
                [
                    "==",
                    "0.6.2"
                ]
            ]
        },
        {
            "name": "pyreadr",
            "specs": [
                [
                    "==",
                    "0.5.2"
                ]
            ]
        },
        {
            "name": "squidpy",
            "specs": [
                [
                    "==",
                    "1.2.3"
                ]
            ]
        },
        {
            "name": "stardist",
            "specs": [
                [
                    "==",
                    "0.9.1"
                ]
            ]
        },
        {
            "name": "tensorflow",
            "specs": [
                [
                    "==",
                    "2.13.1"
                ]
            ]
        },
        {
            "name": "matplotlib",
            "specs": [
                [
                    ">=",
                    "3.1.2"
                ]
            ]
        },
        {
            "name": "holoviews",
            "specs": []
        },
        {
            "name": "seaborn",
            "specs": [
                [
                    "==",
                    "0.13.2"
                ]
            ]
        },
        {
            "name": "selenium",
            "specs": []
        },
        {
            "name": "plotly",
            "specs": [
                [
                    "==",
                    "5.24.1"
                ]
            ]
        },
        {
            "name": "tenacity",
            "specs": [
                [
                    "==",
                    "9.0.0"
                ]
            ]
        },
        {
            "name": "nbformat",
            "specs": [
                [
                    "==",
                    "5.10.4"
                ]
            ]
        },
        {
            "name": "kaleido",
            "specs": [
                [
                    "==",
                    "0.2.1"
                ]
            ]
        },
        {
            "name": "numpy",
            "specs": []
        },
        {
            "name": "scikit-learn",
            "specs": [
                [
                    ">=",
                    "0.23"
                ]
            ]
        },
        {
            "name": "hilearn",
            "specs": []
        },
        {
            "name": "threadpoolctl",
            "specs": []
        },
        {
            "name": "tqdm",
            "specs": []
        }
    ],
    "lcname": "finest"
}
        
Elapsed time: 1.41665s