| Name | ngr-services-spider JSON |
| Version |
0.7.0
JSON |
| download |
| home_page | None |
| Summary | test |
| upload_time | 2024-06-12 11:42:30 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10.6 |
| license | MIT |
| keywords |
ngr
csw
pdok
ows
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# NGR SPIDER
[](https://github.com/PDOK/ngr-services-spider/blob/master/LICENSE)
[](https://github.com/PDOK/ngr-services-spider/releases)
[](https://hub.docker.com/r/pdok/ngr-services-spider/tags)
[](https://pypi.org/project/ngr-services-spider/)
CLI (command line interface) application to retrieve list of services and datasets in a simple JSON format from [nationaalgeoregister.nl](https://nationaalgeoregister.nl/) (NGR), leveraging the NGR [CSW service](https://nationaalgeoregister.nl/geonetwork/srv/dut/csw?service=CSW&request=GetCapabilities).
To install from source run (from the root this repo):
```sh
python3 -m pip install -e .
```
This should install the cli tool `ngr-spider`:
```sh
$ ngr-spider
usage: ngr-spider [-h] subcommand ...
Generate list of PDOK services and/or service layers
positional arguments:
subcommand
services Generate list of all PDOK services
layers Generate list of all PDOK layers
options:
-h, --help show this help message and exit
```
## Examples
1. To generate a list of layers per service run the following:
```sh
ngr-spider layers -p 'OGC:WMS,OGC:WMTS,OGC:API features' pdok-services.json
ngr-spider services -p 'OGC:WMS,OGC:WMTS,OGC:API features' pdok-services.json
```
Note: you may need to install pyproj manually (`pip install pyproj`) on some systems.
## Development
Install dev dependencies and package from source:
```sh
python3 -m pip install -e ".[dev]"
```
To format code install [`black`](https://pypi.org/project/black/) en run from the root of this repo:
```sh
black .
```
Delete unused imports with [`autoflake`](https://pypi.org/project/autoflake/):
```sh
autoflake --remove-all-unused-imports -i -r .
```
Organise and order imports with [`isort`](https://pypi.org/project/isort/):
```sh
isort -m 3 .
```
Run mypy for type checking (from root):
```sh
mypy ngr_spider
```
## Sorting Rules Example
When running the `layers` command in flat mode (`--mode flat`), it is possible to sort the layers in the output by passing a path to a JSON file containing, sorting rules. See below for an example a sorting rules JSON file (use for example with: `ngr-spider layers --sort sorting-rules.json -n 20 -m flat --pretty -p "OGC:WMS,OGC:WMTS" output.json`):
```json
[
{ "index": 0, "names": ["opentopo+"], "types": ["OGC:WMTS"] },
{ "index": 10, "names": ["^actueel_orthohr$"], "types": ["OGC:WMTS"] },
{ "index": 11, "names": ["^actueel_ortho25$"], "types": ["OGC:WMTS"] },
{ "index": 12, "names": ["^actueel_ortho25ir$"], "types": ["OGC:WMTS"] },
{ "index": 12, "names": ["lufolabels"], "types": ["OGC:WMTS"] },
{
"index": 20,
"names": ["landgebied", "provinciegebied", "gemeentegebied"],
"types": ["OGC:WFS"]
},
{ "index": 30, "names": ["top+"], "types": ["OGC:WMTS"] },
{
"index": 32,
"names": ["^standaard$", "^grijs$", "^pastel$", "^water$"],
"types": ["OGC:WMTS"]
},
{
"index": 34,
"names": ["bgtstandaardv2", "bgtachtergrond"],
"types": ["OGC:WMTS"]
},
{ "index": 60, "names": ["ahn3+"], "types": ["OGC:WMTS"] }
]
```
Raw data
{
"_id": null,
"home_page": null,
"name": "ngr-services-spider",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10.6",
"maintainer_email": null,
"keywords": "ngr, csw, pdok, ows",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/cd/d2/acc801b302c9599ca8dc5351371923be7706c1e6e4a04a7456f28a1eb7ac/ngr_services_spider-0.7.0.tar.gz",
"platform": null,
"description": "# NGR SPIDER\n\n[](https://github.com/PDOK/ngr-services-spider/blob/master/LICENSE)\n[](https://github.com/PDOK/ngr-services-spider/releases)\n[](https://hub.docker.com/r/pdok/ngr-services-spider/tags)\n[](https://pypi.org/project/ngr-services-spider/)\n\n\nCLI (command line interface) application to retrieve list of services and datasets in a simple JSON format from [nationaalgeoregister.nl](https://nationaalgeoregister.nl/) (NGR), leveraging the NGR [CSW service](https://nationaalgeoregister.nl/geonetwork/srv/dut/csw?service=CSW&request=GetCapabilities).\n\nTo install from source run (from the root this repo):\n\n```sh\npython3 -m pip install -e .\n```\n\nThis should install the cli tool `ngr-spider`:\n\n```sh\n$ ngr-spider\nusage: ngr-spider [-h] subcommand ...\n\nGenerate list of PDOK services and/or service layers\n\npositional arguments:\n subcommand\n services Generate list of all PDOK services\n layers Generate list of all PDOK layers\n\noptions:\n -h, --help show this help message and exit\n```\n\n## Examples\n\n1. To generate a list of layers per service run the following:\n\n```sh\nngr-spider layers -p 'OGC:WMS,OGC:WMTS,OGC:API features' pdok-services.json\nngr-spider services -p 'OGC:WMS,OGC:WMTS,OGC:API features' pdok-services.json\n\n```\n\nNote: you may need to install pyproj manually (`pip install pyproj`) on some systems.\n\n## Development\n\nInstall dev dependencies and package from source:\n\n```sh\npython3 -m pip install -e \".[dev]\" \n```\n\nTo format code install [`black`](https://pypi.org/project/black/) en run from the root of this repo:\n\n```sh\nblack .\n```\n\nDelete unused imports with [`autoflake`](https://pypi.org/project/autoflake/):\n\n```sh\nautoflake --remove-all-unused-imports -i -r .\n```\n\nOrganise and order imports with [`isort`](https://pypi.org/project/isort/):\n\n```sh\nisort -m 3 .\n```\n\nRun mypy for type checking (from root):\n\n```sh\nmypy ngr_spider \n```\n\n\n## Sorting Rules Example\n\nWhen running the `layers` command in flat mode (`--mode flat`), it is possible to sort the layers in the output by passing a path to a JSON file containing, sorting rules. See below for an example a sorting rules JSON file (use for example with: `ngr-spider layers --sort sorting-rules.json -n 20 -m flat --pretty -p \"OGC:WMS,OGC:WMTS\" output.json`):\n\n\n```json\n[\n { \"index\": 0, \"names\": [\"opentopo+\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 10, \"names\": [\"^actueel_orthohr$\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 11, \"names\": [\"^actueel_ortho25$\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 12, \"names\": [\"^actueel_ortho25ir$\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 12, \"names\": [\"lufolabels\"], \"types\": [\"OGC:WMTS\"] },\n {\n \"index\": 20,\n \"names\": [\"landgebied\", \"provinciegebied\", \"gemeentegebied\"],\n \"types\": [\"OGC:WFS\"]\n },\n { \"index\": 30, \"names\": [\"top+\"], \"types\": [\"OGC:WMTS\"] },\n {\n \"index\": 32,\n \"names\": [\"^standaard$\", \"^grijs$\", \"^pastel$\", \"^water$\"],\n \"types\": [\"OGC:WMTS\"]\n },\n {\n \"index\": 34,\n \"names\": [\"bgtstandaardv2\", \"bgtachtergrond\"],\n \"types\": [\"OGC:WMTS\"]\n },\n { \"index\": 60, \"names\": [\"ahn3+\"], \"types\": [\"OGC:WMTS\"] }\n]\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "test",
"version": "0.7.0",
"project_urls": null,
"split_keywords": [
"ngr",
" csw",
" pdok",
" ows"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2de49f6ed2e925ec91c17a302593924ed1a3859517b8dce9c356b2392a04dbc0",
"md5": "9bd997b1cbe6a10f5d306b6771b80c35",
"sha256": "55bb4e390750a32a4867737990166485cbf64d2e85ef218ee2c578aad590e243"
},
"downloads": -1,
"filename": "ngr_services_spider-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9bd997b1cbe6a10f5d306b6771b80c35",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10.6",
"size": 21878,
"upload_time": "2024-06-12T11:42:29",
"upload_time_iso_8601": "2024-06-12T11:42:29.339725Z",
"url": "https://files.pythonhosted.org/packages/2d/e4/9f6ed2e925ec91c17a302593924ed1a3859517b8dce9c356b2392a04dbc0/ngr_services_spider-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cdd2acc801b302c9599ca8dc5351371923be7706c1e6e4a04a7456f28a1eb7ac",
"md5": "f9609d04b8999befc9debfabfbfde2e5",
"sha256": "ce18764577a8de4537406c6bdd1ab49674711bddcde9b98eeb444448f222b810"
},
"downloads": -1,
"filename": "ngr_services_spider-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "f9609d04b8999befc9debfabfbfde2e5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10.6",
"size": 18840,
"upload_time": "2024-06-12T11:42:30",
"upload_time_iso_8601": "2024-06-12T11:42:30.661744Z",
"url": "https://files.pythonhosted.org/packages/cd/d2/acc801b302c9599ca8dc5351371923be7706c1e6e4a04a7456f28a1eb7ac/ngr_services_spider-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-12 11:42:30",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ngr-services-spider"
}
