| Name | py-qgis-wps JSON |
| Version |
1.8.6
JSON |
| download |
| home_page | https://github.com/3liz/py-qgis-wps |
| Summary | Py-Qgis-WPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium. qgis-wps is written in Python and is a fork of PyWPS 4.0. |
| upload_time | 2023-04-13 13:33:06 |
| maintainer | David Marteau |
| docs_url | None |
| author | David Marteau |
| requires_python | >=3.6 |
| license | |
| keywords |
qgis
wps
ogc
processing
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Py-QGIS-WPS
[](https://pypi.org/project/py-qgis-wps/)
[](https://pypi.org/project/py-qgis-wps/)
[](https://pypi.org/project/py-qgis-wps/)
**New in 1.8: OGC api `processes` support**
Py-QGIS-WPS is an implementation of the [Web Processing Service](https://www.ogc.org/standards/wps)
standard from the Open Geospatial Consortium based on the QGIS Processing API.
Since 1.8 Py-QGIS-WPS supports [OGC API REST processes api](https://ogcapi.ogc.org/processes/)
This implementation allows you to expose and run on a server:
* QGIS Processing algorithms available on Desktop
* QGIS Processing models and scripts
* QGIS plugins having a Processing provider according to their `metadata.txt`file
It is written in Python and was originally a fork of [PyWPS](https://pywps.org/).
Requirements and limitations :
- Python 3.7 minimum
- Windows not officially supported
- Redis server
# Documentation
Latest documentation is available on [docs.3liz.org](https://docs.3liz.org/py-qgis-wps/)
# Why Py-QGIS-WPS ?
Py-QGIS-WPS differs from [PyWPS](https://pywps.org/) in the following:
* QGIS centric
* Handle all request in asynchronous way: all jobs run in a non-blocking way, even
when synchronous operation is requested.
* Use multiprocessing Pool to handle task queue instead instantiating a new process each time.
* Uniform Logging with the 'logging' module
* Implements OGC `processes` api.
* Use Redis for asynchronous status storage.
* Support streamed/chunked requests for stored data
* Add extensions to WPS: TIMEOUT and EXPIRE
* No Windows support
All these changes where not easy to implement without some drastic changes of the original code and we think
that it deviates too much from the PyWPS original intentions.
That is, we have decided to fork the original project and go along with it.
So, we are really grateful to the original authors of PyWPS for the nice piece of software that helped us very much
to start quickly this project.
## Why moving to Tornado instead WSGI
* We need to support asyncio: asyncio requires a blocking running loop. This cannot be achieved simply in a WSGI architecture.
* Tornado is fully integrated with native python `asyncio` library and provide a great framework for developing a http server.
## Extensions to WPS
### TIMEOUT extension
Specify the timeout for a process: if the process takes more than TIMEOUT seconds to run, the worker is then killed and an
error status is returned.
Set the `TIMEOUT=<seconds>` in GET requests.
In POST requests, set the `timeout=<seconds>` attribut in the `<ResponseDocument>` tag.
The server may configure maximum timeout value.
### EXPIRE extension
Specify the expiration time for stored results: after EXPIRE seconds after the end of the wps process, all results will be
flushed from disks and local cache. Trying to request the results again will return a 404 HTTP error.
Set the `EXPIRE=<seconds>` in GET requests.
In POST requests, set the `expire=<seconds>` attribut int the `<ResponseDocument>` tag.
The server may configure maximum expiration value.
### Status API
Now implemented with the processes api:
The status REST api will return the list of the stored status for all running and terminated wps processes.
Example for returning all stored status:
```
http://localhost:8080/jobs
```
Example for returning status for one given job from its id:
```
http://localhost:8080/jobs/<job_id>
```
## Extensions to `processes` api:
### Files
```
http://localhost:8080/jobs/<job_id>/files
```
# Running QGIS processing
## WPS input/output layer mapping
With QGIS desktop, QGIS processing algorithms usually apply on a QGIS source project and computed layers are displayed in the same context as the source project.
Py-qgis-wps works the same way: a qgis project will be used as a source of input layers.
The difference is that, when an algorithm runs, it creates a qgis project file associated to the current task and register computed layers to it.
The created project may be used as OWS source with Qgis Server. Output layers are returned as complex objects
holding a reference to a WMS/WFS uri that can be used directly with Qgis server. The uri template is configurable
using the `server/wms_response_uri` configuration setting.
## Contextualized input parameters
Tasks parameters are contextualized using the `MAP` query param. If a `MAP` parameters is given when
doing a `DescripProcess` requests, allowed values for input layers will be taken from the qgis source project
according the type of the input layers.
QGIS project (.qgs) files and project stored in Postgres databases are both supported.
The best practice is to always provide a `MAP` parameters and include the possible input layer in a qgs project. This way you
may connect whatever data source supported by qgis and use them as input data in a safe way.
If you need to pass data to your algorithm from client-side, prefer inputs file parameter and small payloads.
# Dependencies
See [requirements.txt](requirements.txt) file.
# Installation from python package
*ADVICE*: You should always install in a python virtualenv. If you want to use system packages, setup your environment
with the `--system-site-packages` option.
See the official documentation for how to setup a python virtualenv: https://virtualenv.pypa.io/en/stable/.
## From source
Install in development mode
```bash
make build
pip install -e .
```
## From python package server
```bash
pip install py-qgis-wps
```
# Running the server
The server from a command line interface:
The server does not run as a daemon by itself, there are several ways to run a command as a daemon.
For example:
* Use Supervisor http://supervisord.org/ will give you full control over logs and server status notifications.
* Use the `daemon` command.
* Use Docker
# Running the server
## Usage
```
usage: wpsserver [-h] [-d] [-c [PATH]]
[--version] [-p PORT] [-b IP] [-u SETUID]
WPS server
optional arguments:
-h, --help show this help message and exit
-d, --debug Set debug mode
-c [PATH], --config [PATH]
Configuration file
--version Return version number and exit
-p PORT, --port PORT http port
-b IP, --bind IP Interface to bind to
-u SETUID, --setuid SETUID
uid to switch to
```
### Requests to OWS services
The OWS requests use the following format: `/ows/?<ows_query_params>`
Example:
```
http://myserver:8080/ows/?SERVICE=WPS&VERSION=1.0.0&REQUEST=GetCapabilities
```
## Configuration
### From config ini file
By default, the wps server is not using any config file, but one can be used with the `--config` option.
A config file is a simple ini file, a sample config file is given with the sources.
### From environment variables
The server can be configured with environnement variables:
Configuration is done with environment variables:
- QGSWPS\_SERVER\_WORKDIR: set the current dir processes, all processes will be running in that directory.
- QGSWPS\_SERVER\_HOST\_PROXY: When the service is behind a reverse proxy, set this to the proxy entrypoint.
- QGSWPS\_SERVER\_PARALLELPROCESSES: Number of parallel process workers
- QGSWPS\_SERVER\_RESPONSE\_TIMEOUT: The max response time before killing a process.
- QGSWPS\_SERVER\_RESPONSE\_EXPIRATION: The max time (in seconds) the response from a WPS process will be available.
- QGSWPS\_SERVER\_WMS\_SERVICE\_URL: The base url for WMS service. Default to <hosturl>/wms. Responses from processing will
be retourned as WMS urls. This configuration variable set the base url for accessing results.
- QGSWPS\_SERVER\_RESULTS\_MAP\_URI
### Logging
- QGSWPS\_LOGLEVEL: the log level, should be `INFO` in production mode, `DEBUG` for debug output.
### REDIS storage configuration
- QGSWPS\_REDIS\_HOST: The redis host
- QGSWPS\_REDIS\_PORT: The redis port. Default to 6379
- QGSWPS\_REDIS\_DBNUM: The redis database number used. Default to 0
### Qgis project Cache configuration
- QGSWPS\_CACHE\_ROOTDIR: Absolute path to the qgis projects root directory, projects referenced with the MAP parameter will be searched at this location
### Processing configuration
- QGSWPS\_PROCESSSING\_PROVIDERS\_MODULE\_PATH: Path to look for processing algorithms provider to publish, algorithms from providers specified here will be runnable as WPS processes.
# Exposing algorithms as WPS services
Note that since 1.1 , the `__algorithms__.py` method for declaring providers is no longer supported.
Processing providers following the same rules as Qgis regular plugin with a special factory entrypoint: `WPSClassFactory(iface)` in the `__init__.py` file.
### The `metadata.txt` file
As regular QGIS plugin, a metadata.txt file must be present with a special entry `wps=True` indicating that
the plugin is available as a WPS service provider.
### Registering providers
The `iface` parameter is a instance of `WPSServerInterface` which provide a
`registerProvider( provider: QgsAlgorithmProvider, expose: bool = True) -> Any` method.
Exposed providers as WPS services must be registered using the `registerProvider` method
Example:
```python
def WPSClassFactory(iface: WPSServerInterface) -> Any:
from TestAlgorithmProvider1 import AlgorithmProvider1
from TestAlgorithmProvider2 import AlgorithmProvider2
iface.registerProvider( AlgorithmProvider1() )
iface.registerProvider( AlgorithmProvider2() )
```
## Controlling what is exposed:
Processing algorithm with the flag [FlagHideFromToolbox](https://qgis.org/pyqgis/latest/core/QgsProcessingAlgorithm.html#qgis.core.QgsProcessingAlgorithm.FlagHideFromToolbox) set will not be exposed as WPS process.
Parameters with the flag [FlagHidden](https://qgis.org/pyqgis/latest/core/QgsProcessingParameterDefinition.html#qgis.core.QgsProcessingParameterDefinition.FlagHidden) set won't be exposed in a `DescribeProcess` request
# References
* [OGC standards](https://www.ogc.org/standards)
* [OGC Api processes](https://ogcapi.ogc.org/processes/)
* [Introduction to WPS](http://opengeospatial.github.io/e-learning/wps/text/basic-index.html)
* [Py-qgis-server at FOSS4G 2019](https://www.youtube.com/watch?v=YL1tdcJwimA).
Raw data
{
"_id": null,
"home_page": "https://github.com/3liz/py-qgis-wps",
"name": "py-qgis-wps",
"maintainer": "David Marteau",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "david.marteau@3liz.com",
"keywords": "QGIS WPS OGC processing",
"author": "David Marteau",
"author_email": "david.marteau@3liz.com",
"download_url": "https://files.pythonhosted.org/packages/81/fb/af32236e488a75a6070e30197d41e090a10a048e38773d89b6ff7dbd3522/py-qgis-wps-1.8.6.tar.gz",
"platform": null,
"description": "# Py-QGIS-WPS \n\n[](https://pypi.org/project/py-qgis-wps/)\n[](https://pypi.org/project/py-qgis-wps/)\n[](https://pypi.org/project/py-qgis-wps/)\n\n**New in 1.8: OGC api `processes` support** \n\nPy-QGIS-WPS is an implementation of the [Web Processing Service](https://www.ogc.org/standards/wps)\nstandard from the Open Geospatial Consortium based on the QGIS Processing API.\n\nSince 1.8 Py-QGIS-WPS supports [OGC API REST processes api](https://ogcapi.ogc.org/processes/)\n\nThis implementation allows you to expose and run on a server:\n* QGIS Processing algorithms available on Desktop\n* QGIS Processing models and scripts\n* QGIS plugins having a Processing provider according to their `metadata.txt`file\n\nIt is written in Python and was originally a fork of [PyWPS](https://pywps.org/).\n\nRequirements and limitations :\n\n- Python 3.7 minimum\n- Windows not officially supported\n- Redis server \n\n# Documentation\n\nLatest documentation is available on [docs.3liz.org](https://docs.3liz.org/py-qgis-wps/)\n\n# Why Py-QGIS-WPS ?\n\nPy-QGIS-WPS differs from [PyWPS](https://pywps.org/) in the following: \n\n* QGIS centric\n* Handle all request in asynchronous way: all jobs run in a non-blocking way, even\n when synchronous operation is requested.\n* Use multiprocessing Pool to handle task queue instead instantiating a new process each time.\n* Uniform Logging with the 'logging' module\n* Implements OGC `processes` api.\n* Use Redis for asynchronous status storage.\n* Support streamed/chunked requests for stored data\n* Add extensions to WPS: TIMEOUT and EXPIRE\n* No Windows support\n\nAll these changes where not easy to implement without some drastic changes of the original code and we think\nthat it deviates too much from the PyWPS original intentions.\n\nThat is, we have decided to fork the original project and go along with it. \n\nSo, we are really grateful to the original authors of PyWPS for the nice piece of software that helped us very much\nto start quickly this project. \n\n## Why moving to Tornado instead WSGI\n\n* We need to support asyncio: asyncio requires a blocking running loop. This cannot be achieved simply in a WSGI architecture.\n* Tornado is fully integrated with native python `asyncio` library and provide a great framework for developing a http server.\n\n## Extensions to WPS\n\n### TIMEOUT extension\n\nSpecify the timeout for a process: if the process takes more than TIMEOUT seconds to run, the worker is then killed and an \nerror status is returned.\n\nSet the `TIMEOUT=<seconds>` in GET requests.\n\nIn POST requests, set the `timeout=<seconds>` attribut in the `<ResponseDocument>` tag.\n\nThe server may configure maximum timeout value.\n\n\n### EXPIRE extension\n\nSpecify the expiration time for stored results: after EXPIRE seconds after the end of the wps process, all results will be\nflushed from disks and local cache. Trying to request the results again will return a 404 HTTP error.\n\nSet the `EXPIRE=<seconds>` in GET requests.\n\nIn POST requests, set the `expire=<seconds>` attribut int the `<ResponseDocument>` tag.\n\nThe server may configure maximum expiration value.\n\n\n### Status API\n\nNow implemented with the processes api:\nThe status REST api will return the list of the stored status for all running and terminated wps processes.\n\nExample for returning all stored status:\n```\nhttp://localhost:8080/jobs\n```\n\nExample for returning status for one given job from its id:\n```\nhttp://localhost:8080/jobs/<job_id>\n```\n\n## Extensions to `processes` api:\n\n### Files\n\n```\nhttp://localhost:8080/jobs/<job_id>/files\n```\n\n# Running QGIS processing\n\n## WPS input/output layer mapping\n\nWith QGIS desktop, QGIS processing algorithms usually apply on a QGIS source project and computed layers are displayed in the same context as the source project. \n\nPy-qgis-wps works the same way: a qgis project will be used as a source of input layers. \nThe difference is that, when an algorithm runs, it creates a qgis project file associated to the current task and register computed layers to it. \n\nThe created project may be used as OWS source with Qgis Server. Output layers are returned as complex objects\nholding a reference to a WMS/WFS uri that can be used directly with Qgis server. The uri template is configurable \nusing the `server/wms_response_uri` configuration setting.\n\n## Contextualized input parameters\n\nTasks parameters are contextualized using the `MAP` query param. If a `MAP` parameters is given when\ndoing a `DescripProcess` requests, allowed values for input layers will be taken from the qgis source project\naccording the type of the input layers. \n\nQGIS project (.qgs) files and project stored in Postgres databases are both supported.\n\nThe best practice is to always provide a `MAP` parameters and include the possible input layer in a qgs project. This way you\nmay connect whatever data source supported by qgis and use them as input data in a safe way.\n\nIf you need to pass data to your algorithm from client-side, prefer inputs file parameter and small payloads.\n\n\n# Dependencies\n\nSee [requirements.txt](requirements.txt) file.\n\n\n# Installation from python package\n\n*ADVICE*: You should always install in a python virtualenv. If you want to use system packages, setup your environment\nwith the `--system-site-packages` option.\n\nSee the official documentation for how to setup a python virtualenv: https://virtualenv.pypa.io/en/stable/.\n\n## From source\n\nInstall in development mode\n```bash\nmake build\npip install -e .\n```\n\n## From python package server\n\n```bash\npip install py-qgis-wps\n```\n\n# Running the server\n\nThe server from a command line interface:\n\nThe server does not run as a daemon by itself, there are several ways to run a command as a daemon.\n\nFor example:\n\n* Use Supervisor http://supervisord.org/ will give you full control over logs and server status notifications.\n* Use the `daemon` command.\n* Use Docker\n\n# Running the server\n\n## Usage\n\n```\nusage: wpsserver [-h] [-d] [-c [PATH]]\n [--version] [-p PORT] [-b IP] [-u SETUID]\n\nWPS server\n\noptional arguments:\n -h, --help show this help message and exit\n -d, --debug Set debug mode \n -c [PATH], --config [PATH]\n Configuration file\n --version Return version number and exit\n -p PORT, --port PORT http port\n -b IP, --bind IP Interface to bind to\n -u SETUID, --setuid SETUID\n uid to switch to\n```\n\n### Requests to OWS services \n\nThe OWS requests use the following format: `/ows/?<ows_query_params>`\n\nExample:\n\n```\nhttp://myserver:8080/ows/?SERVICE=WPS&VERSION=1.0.0&REQUEST=GetCapabilities\n```\n\n## Configuration\n\n### From config ini file\n\nBy default, the wps server is not using any config file, but one can be used with the `--config` option.\nA config file is a simple ini file, a sample config file is given with the sources.\n\n### From environment variables\n\nThe server can be configured with environnement variables:\n\nConfiguration is done with environment variables:\n\n- QGSWPS\\_SERVER\\_WORKDIR: set the current dir processes, all processes will be running in that directory.\n- QGSWPS\\_SERVER\\_HOST\\_PROXY: When the service is behind a reverse proxy, set this to the proxy entrypoint.\n- QGSWPS\\_SERVER\\_PARALLELPROCESSES: Number of parallel process workers\n- QGSWPS\\_SERVER\\_RESPONSE\\_TIMEOUT: The max response time before killing a process.\n- QGSWPS\\_SERVER\\_RESPONSE\\_EXPIRATION: The max time (in seconds) the response from a WPS process will be available.\n- QGSWPS\\_SERVER\\_WMS\\_SERVICE\\_URL: The base url for WMS service. Default to <hosturl>/wms. Responses from processing will\nbe retourned as WMS urls. This configuration variable set the base url for accessing results.\n- QGSWPS\\_SERVER\\_RESULTS\\_MAP\\_URI\n\n### Logging\n\n- QGSWPS\\_LOGLEVEL: the log level, should be `INFO` in production mode, `DEBUG` for debug output. \n\n### REDIS storage configuration\n\n- QGSWPS\\_REDIS\\_HOST: The redis host\n- QGSWPS\\_REDIS\\_PORT: The redis port. Default to 6379\n- QGSWPS\\_REDIS\\_DBNUM: The redis database number used. Default to 0\n\n\n### Qgis project Cache configuration\n\n- QGSWPS\\_CACHE\\_ROOTDIR: Absolute path to the qgis projects root directory, projects referenced with the MAP parameter will be searched at this location\n\n### Processing configuration\n\n- QGSWPS\\_PROCESSSING\\_PROVIDERS\\_MODULE\\_PATH: Path to look for processing algorithms provider to publish, algorithms from providers specified here will be runnable as WPS processes.\n\n# Exposing algorithms as WPS services\n\nNote that since 1.1 , the `__algorithms__.py` method for declaring providers is no longer supported.\n\nProcessing providers following the same rules as Qgis regular plugin with a special factory entrypoint: `WPSClassFactory(iface)` in the `__init__.py` file.\n\n### The `metadata.txt` file\n\nAs regular QGIS plugin, a metadata.txt file must be present with a special entry `wps=True` indicating that\nthe plugin is available as a WPS service provider.\n\n### Registering providers\n\nThe `iface` parameter is a instance of `WPSServerInterface` which provide a \n`registerProvider( provider: QgsAlgorithmProvider, expose: bool = True) -> Any` method.\n\nExposed providers as WPS services must be registered using the `registerProvider` method\n\nExample:\n\n```python\ndef WPSClassFactory(iface: WPSServerInterface) -> Any:\n\n from TestAlgorithmProvider1 import AlgorithmProvider1\n from TestAlgorithmProvider2 import AlgorithmProvider2\n\n iface.registerProvider( AlgorithmProvider1() )\n iface.registerProvider( AlgorithmProvider2() )\n\n``` \n\n## Controlling what is exposed:\n\nProcessing algorithm with the flag [FlagHideFromToolbox](https://qgis.org/pyqgis/latest/core/QgsProcessingAlgorithm.html#qgis.core.QgsProcessingAlgorithm.FlagHideFromToolbox) set will not be exposed as WPS process.\n\nParameters with the flag [FlagHidden](https://qgis.org/pyqgis/latest/core/QgsProcessingParameterDefinition.html#qgis.core.QgsProcessingParameterDefinition.FlagHidden) set won't be exposed in a `DescribeProcess` request\n\n# References\n\n* [OGC standards](https://www.ogc.org/standards)\n* [OGC Api processes](https://ogcapi.ogc.org/processes/)\n* [Introduction to WPS](http://opengeospatial.github.io/e-learning/wps/text/basic-index.html)\n* [Py-qgis-server at FOSS4G 2019](https://www.youtube.com/watch?v=YL1tdcJwimA).\n",
"bugtrack_url": null,
"license": "",
"summary": "Py-Qgis-WPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium. qgis-wps is written in Python and is a fork of PyWPS 4.0.",
"version": "1.8.6",
"split_keywords": [
"qgis",
"wps",
"ogc",
"processing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "81fbaf32236e488a75a6070e30197d41e090a10a048e38773d89b6ff7dbd3522",
"md5": "fe6df873ccc189d93725f12048fd653f",
"sha256": "2450ce391b6046be8df7664fa906c3008d9becd2b6412ed95a051ad22a8def2c"
},
"downloads": -1,
"filename": "py-qgis-wps-1.8.6.tar.gz",
"has_sig": false,
"md5_digest": "fe6df873ccc189d93725f12048fd653f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 786691,
"upload_time": "2023-04-13T13:33:06",
"upload_time_iso_8601": "2023-04-13T13:33:06.989343Z",
"url": "https://files.pythonhosted.org/packages/81/fb/af32236e488a75a6070e30197d41e090a10a048e38773d89b6ff7dbd3522/py-qgis-wps-1.8.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-13 13:33:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "3liz",
"github_project": "py-qgis-wps",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "py-qgis-wps"
}
