----
RSP is a tool for geospatial raster data processing.
RSP can preprocess Sentinel-2 and Landsat imagery, create raster mosaics, calculate vegetation indices and perform image segmentation tasks.
Read the documentation for more details: https://remote-sensing-processor.readthedocs.io
## Example
Here is an example of some features that RSP provides. We will perform a simple task - train a semantic segmentation model that will predict landcover class using Sentinel-2 and DEM data.
First, we need to preprocess data. We preprocess Sentinel-2 images and merge them into a mosaic, then we calculate NDVI of that Sentinel-2 mosaic. We also merge DEM images into mosaic, match it to the same resolution and projection as Sentinel-2 data and normalize its values. We rasterize landcover shape file and match it to the same resolution and projection as Sentinel-2 data.
Then we prepare data to semantic segmentation model training. We use Sentinel-2 and DEM data as training data and landcover data as a target variable. We cut our data into small tiles and split it into train, validation and test subsets. Then we train and test UperNet model that predicts landcover based on Sentinel-2 and DEM data. Finally, we use this model to create a landcover map.
```python
from glob import glob
import remote_sensing_processor as rsp
# Getting a list of Sentinel-2 images
sentinel2_imgs = glob('/home/rsp_test/sentinels/*.zip')
# Preprocessing sentinel-2 images
# It includes converting L1 products to L2, superresolution for 20 and 60m bands and cloud masking
# We also normalize data values to range from 0 to 1
output_sentinels = rsp.sentinel2(sentinel2_imgs, normalize=True)
# Merging sentinel-2 images into one mosaic
# in order from images with less nodata pixels on top to images with most nodata on bottom
# clipping it to the region of interest and reprojecting to the crs we need
border = '/home/rsp_test/border.gpkg'
mosaic_sentinel = rsp.mosaic(
output_sentinels,
'/home/rsp_test/mosaics/sentinel/',
clip=border,
crs='EPSG:4326',
nodata_order=True,
)
# Calculating NDVI for Sentinel-2 mosaic
ndvi = rsp.calculate_index('NDVI', '/home/rsp_test/mosaics/sentinel/')
# Merging DEM files into mosaic
# and matching it to resolution and projection of a reference file (one of Sentinel mosaic bands)
dems = glob('/home/rsp_test/dem/*.tif')
mosaic_dem = rsp.mosaic(
dems,
'/home/rsp_test/mosaics/dem/',
clip=border,
reference_raster='/home/rsp_test/mosaics/sentinel/B1.tif',
nodata=0,
)
# Applying min/max normalization to DEM mosaic (heights in our region of interest are in range from 100 to 1000)
rsp.normalize(mosaic_dem[0], mosaic_dem[0], 100, 1000)
# Rasterizing landcover type classification shapefile
# and matching it to resolution and projection of a reference file (one of Sentinel mosaic bands)
landcover_shp = '/home/rsp_test/landcover/types.shp'
landcover = '/home/rsp_test/landcover/types.tif'
rsp.rasterize(
landcover_shp,
reference_raster='/home/rsp_test/mosaics/sentinel/B1.tif',
value="type",
output_file=landcover
)
# Preparing data for semantic segmentation
# Cut Sentinel and DEM (training data) and landcover (target variable) data to 256x256 px tiles,
# random shuffle samples, split data into train, validation and test subsets in proportion 3 to 1 to 1
x = mosaic_sentinel + mosaic_dem
y = landcover
x_tiles, y_tiles = rsp.segmentation.generate_tiles(
x,
y,
tile_size=256,
shuffle=True,
split=[3, 1, 1],
split_names=['train', 'val', 'test'],
)
# Training UperNet that predicts landcover class based on Sentinel and DEM
train_ds = [x_tiles, y_tiles[0], 'train']
val_ds = [x_tiles, y_tiles[0], 'val']
model = rsp.segmentation.train(
train_ds,
val_ds,
model='UperNet',
backbone='ConvNeXTV2',
model_file='/home/rsp_test/model/upernet.ckpt',
epochs=100,
)
# Testing model
test_ds = [x_tiles, y_tiles[0], 'test']
rsp.segmentation.test(test_ds, model = model)
# Mapping landcover using predictions of our UperNet
reference = landcover
output_map = '/home/rsp_test/prediction.tif'
rsp.segmentation.generate_map(x_tiles, y_tiles[0], reference, model, output_map)
```
## Requirements
To run RSP you need these packages to be installed:
- Python >= 3.8
- Numpy
- Xarray
- Dask
- Zarr
- GDAL
- Rasterio
- Rioxarray
- Pyproj
- Geopandas
- Geocube
- Scikit-learn
- Scikit-image
- XGBoost
- Pytorch >= 2.0
- Torchvision >= 0.10
- Lightning
- Transformers
If you have a GPU that supports CUDA we strongly recommend you to install Pytorch version that is built with CUDA support before installing RSP. You can find out how to do it on [Pytorch official site](https://pytorch.org/get-started/locally/).
Also you need a Sen2Cor to be installed to process Sentinel-2 data. Required version is 2.11 for Windows or Linux and 2.9 for Mac OS. You should install it via SNAP plugin installer. [Here](http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Installation_of_SNAP) is the instruction how you can do it.
## Installation
From PyPI:
```
pip install remote-sensing-processor
```
From Conda:
```
conda install -c moskovchenkomike remote-sensing-processor
```
From source:
```
git clone https://github.com/simonreise/remote-sensing-processor
cd remote-sensing-processor
pip install .
```
> :warning: This package is still in early development stage, so its API can change significantly, sometimes without backward compatibility. Consider this before updating the package.
## License
See [LICENSE](LICENSE).
## Credits
RSP uses code from some other projects.
Sentinel-2 superresolution is based on [s2-superresolution](https://github.com/up42/s2-superresolution) by UP42, which is based on [DSen-2](https://github.com/lanha/DSen2) by lanha.
Sentinel-2 atmospheric correction is performed with [Sen2Cor](https://step.esa.int/main/snap-supported-plugins/sen2cor/).
Landsat processing module uses code from [Semi-Automatic Classification Plugin](https://fromgistors.blogspot.com/p/semi-automatic-classification-plugin.html).
Raw data
{
"_id": null,
"home_page": "https://github.com/simonreise/remote-sensing-processor",
"name": "remote-sensing-processor",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "remote sensing, landsat, sentinel, gdal, rasterio",
"author": "Mikhail Moskovchenko",
"author_email": "moskovchenkomike@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/70/4b/840d174b973bce8bdb2d71d318b2f400a99f8b990396c629b37b3840e60e/remote_sensing_processor-0.2.2.tar.gz",
"platform": null,
"description": "\r\n\r\n\r\n----\r\n\r\n\r\n\r\n\r\n\r\nRSP is a tool for geospatial raster data processing.\r\n\r\nRSP can preprocess Sentinel-2 and Landsat imagery, create raster mosaics, calculate vegetation indices and perform image segmentation tasks.\r\n\r\nRead the documentation for more details: https://remote-sensing-processor.readthedocs.io\r\n\r\n## Example\r\n\r\nHere is an example of some features that RSP provides. We will perform a simple task - train a semantic segmentation model that will predict landcover class using Sentinel-2 and DEM data.\r\n\r\nFirst, we need to preprocess data. We preprocess Sentinel-2 images and merge them into a mosaic, then we calculate NDVI of that Sentinel-2 mosaic. We also merge DEM images into mosaic, match it to the same resolution and projection as Sentinel-2 data and normalize its values. We rasterize landcover shape file and match it to the same resolution and projection as Sentinel-2 data. \r\n\r\nThen we prepare data to semantic segmentation model training. We use Sentinel-2 and DEM data as training data and landcover data as a target variable. We cut our data into small tiles and split it into train, validation and test subsets. Then we train and test UperNet model that predicts landcover based on Sentinel-2 and DEM data. Finally, we use this model to create a landcover map. \r\n```python\r\nfrom glob import glob\r\nimport remote_sensing_processor as rsp\r\n\r\n\r\n# Getting a list of Sentinel-2 images\r\nsentinel2_imgs = glob('/home/rsp_test/sentinels/*.zip')\r\n\r\n# Preprocessing sentinel-2 images\r\n# It includes converting L1 products to L2, superresolution for 20 and 60m bands and cloud masking\r\n# We also normalize data values to range from 0 to 1\r\noutput_sentinels = rsp.sentinel2(sentinel2_imgs, normalize=True)\r\n\r\n# Merging sentinel-2 images into one mosaic \r\n# in order from images with less nodata pixels on top to images with most nodata on bottom\r\n# clipping it to the region of interest and reprojecting to the crs we need\r\nborder = '/home/rsp_test/border.gpkg'\r\nmosaic_sentinel = rsp.mosaic(\r\n\toutput_sentinels, \r\n\t'/home/rsp_test/mosaics/sentinel/', \r\n\tclip=border, \r\n\tcrs='EPSG:4326', \r\n\tnodata_order=True,\r\n)\r\n\r\n# Calculating NDVI for Sentinel-2 mosaic\r\nndvi = rsp.calculate_index('NDVI', '/home/rsp_test/mosaics/sentinel/')\r\n\r\n# Merging DEM files into mosaic \r\n# and matching it to resolution and projection of a reference file (one of Sentinel mosaic bands)\r\ndems = glob('/home/rsp_test/dem/*.tif')\r\nmosaic_dem = rsp.mosaic(\r\n\tdems, \r\n\t'/home/rsp_test/mosaics/dem/', \r\n\tclip=border, \r\n\treference_raster='/home/rsp_test/mosaics/sentinel/B1.tif', \r\n\tnodata=0,\r\n)\r\n\r\n# Applying min/max normalization to DEM mosaic (heights in our region of interest are in range from 100 to 1000)\r\nrsp.normalize(mosaic_dem[0], mosaic_dem[0], 100, 1000)\r\n\r\n# Rasterizing landcover type classification shapefile\r\n# and matching it to resolution and projection of a reference file (one of Sentinel mosaic bands)\r\nlandcover_shp = '/home/rsp_test/landcover/types.shp'\r\nlandcover = '/home/rsp_test/landcover/types.tif'\r\nrsp.rasterize(\r\n\tlandcover_shp, \r\n\treference_raster='/home/rsp_test/mosaics/sentinel/B1.tif', \r\n\tvalue=\"type\", \r\n\toutput_file=landcover\r\n)\r\n\r\n# Preparing data for semantic segmentation\r\n# Cut Sentinel and DEM (training data) and landcover (target variable) data to 256x256 px tiles, \r\n# random shuffle samples, split data into train, validation and test subsets in proportion 3 to 1 to 1\r\nx = mosaic_sentinel + mosaic_dem\r\ny = landcover\r\nx_tiles, y_tiles = rsp.segmentation.generate_tiles(\r\n\tx, \r\n\ty, \r\n\ttile_size=256, \r\n\tshuffle=True, \r\n\tsplit=[3, 1, 1], \r\n\tsplit_names=['train', 'val', 'test'],\r\n)\r\n\r\n# Training UperNet that predicts landcover class based on Sentinel and DEM\r\ntrain_ds = [x_tiles, y_tiles[0], 'train']\r\nval_ds = [x_tiles, y_tiles[0], 'val']\r\nmodel = rsp.segmentation.train(\r\n\ttrain_ds, \r\n\tval_ds, \r\n\tmodel='UperNet', \r\n\tbackbone='ConvNeXTV2',\r\n\tmodel_file='/home/rsp_test/model/upernet.ckpt', \r\n\tepochs=100,\r\n)\r\n\r\n# Testing model\r\ntest_ds = [x_tiles, y_tiles[0], 'test']\r\nrsp.segmentation.test(test_ds, model = model)\r\n\r\n# Mapping landcover using predictions of our UperNet\r\nreference = landcover\r\noutput_map = '/home/rsp_test/prediction.tif'\r\nrsp.segmentation.generate_map(x_tiles, y_tiles[0], reference, model, output_map)\r\n```\r\n\r\n## Requirements\r\nTo run RSP you need these packages to be installed:\r\n- Python >= 3.8\r\n- Numpy\r\n- Xarray\r\n- Dask\r\n- Zarr\r\n- GDAL\r\n- Rasterio\r\n- Rioxarray\r\n- Pyproj\r\n- Geopandas\r\n- Geocube\r\n- Scikit-learn\r\n- Scikit-image\r\n- XGBoost\r\n- Pytorch >= 2.0\r\n- Torchvision >= 0.10\r\n- Lightning\r\n- Transformers\r\n\r\nIf you have a GPU that supports CUDA we strongly recommend you to install Pytorch version that is built with CUDA support before installing RSP. You can find out how to do it on [Pytorch official site](https://pytorch.org/get-started/locally/).\r\n\r\nAlso you need a Sen2Cor to be installed to process Sentinel-2 data. Required version is 2.11 for Windows or Linux and 2.9 for Mac OS. You should install it via SNAP plugin installer. [Here](http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Installation_of_SNAP) is the instruction how you can do it.\r\n\r\n## Installation\r\n\r\nFrom PyPI:\r\n```\r\npip install remote-sensing-processor\r\n```\r\nFrom Conda:\r\n```\r\nconda install -c moskovchenkomike remote-sensing-processor\r\n```\r\nFrom source:\r\n```\r\ngit clone https://github.com/simonreise/remote-sensing-processor\r\ncd remote-sensing-processor\r\npip install .\r\n```\r\n\r\n> :warning: This package is still in early development stage, so its API can change significantly, sometimes without backward compatibility. Consider this before updating the package.\r\n\r\n\r\n## License\r\nSee [LICENSE](LICENSE).\r\n\r\n## Credits\r\nRSP uses code from some other projects.\r\n\r\nSentinel-2 superresolution is based on [s2-superresolution](https://github.com/up42/s2-superresolution) by UP42, which is based on [DSen-2](https://github.com/lanha/DSen2) by lanha.\r\n\r\nSentinel-2 atmospheric correction is performed with [Sen2Cor](https://step.esa.int/main/snap-supported-plugins/sen2cor/).\r\n\r\nLandsat processing module uses code from [Semi-Automatic Classification Plugin](https://fromgistors.blogspot.com/p/semi-automatic-classification-plugin.html).\r\n \r\n \r\n\r\n",
"bugtrack_url": null,
"license": null,
"summary": "RSP is a tool for geospatial raster data processing",
"version": "0.2.2",
"project_urls": {
"Bug Tracker": "https://github.com/simonreise/remote-sensing-processor/issues",
"Documentation": "https://remote-sensing-processor.readthedocs.io",
"Homepage": "https://github.com/simonreise/remote-sensing-processor",
"Source": "https://github.com/simonreise/remote-sensing-processor"
},
"split_keywords": [
"remote sensing",
" landsat",
" sentinel",
" gdal",
" rasterio"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2d8f2a35f1ce92fa2e94c7869079b9666b2a914dd514e13e4e9381941b43c86a",
"md5": "ad3eafbdbf12f2e6ca83fd39bd2f792b",
"sha256": "670943931ea31f1898f30b578978f8b3b63f8a7ab044a434b5af05b82f30f950"
},
"downloads": -1,
"filename": "remote_sensing_processor-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ad3eafbdbf12f2e6ca83fd39bd2f792b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 26649086,
"upload_time": "2024-04-29T17:31:27",
"upload_time_iso_8601": "2024-04-29T17:31:27.879574Z",
"url": "https://files.pythonhosted.org/packages/2d/8f/2a35f1ce92fa2e94c7869079b9666b2a914dd514e13e4e9381941b43c86a/remote_sensing_processor-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "704b840d174b973bce8bdb2d71d318b2f400a99f8b990396c629b37b3840e60e",
"md5": "5ebbaa6d64c7f5f83e29805bcdda0b36",
"sha256": "2dcbe7247e60778be513524944188f49d7f60c7e452b7b6ade9571f7d1c95233"
},
"downloads": -1,
"filename": "remote_sensing_processor-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "5ebbaa6d64c7f5f83e29805bcdda0b36",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 26764192,
"upload_time": "2024-04-29T17:38:11",
"upload_time_iso_8601": "2024-04-29T17:38:11.623824Z",
"url": "https://files.pythonhosted.org/packages/70/4b/840d174b973bce8bdb2d71d318b2f400a99f8b990396c629b37b3840e60e/remote_sensing_processor-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-29 17:38:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "simonreise",
"github_project": "remote-sensing-processor",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"appveyor": true,
"tox": true,
"lcname": "remote-sensing-processor"
}
JSON
