| Name | ngr-services-spider JSON |
| Version |
0.6.1
JSON |
| download |
| home_page | |
| Summary | test |
| upload_time | 2023-04-05 13:12:54 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.10.6 |
| license | MIT |
| keywords |
ngr
csw
pdok
ows
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# NGR SPIDER
[](https://github.com/PDOK/ngr-services-spider/blob/master/LICENSE)
[](https://github.com/PDOK/ngr-services-spider/releases)
[](https://hub.docker.com/r/pdok/ngr-services-spider/tags)
[](https://pypi.org/project/ngr-services-spider/)
CLI (command line interface) application to retrieve list of services and datasets in a simple JSON format from [nationaalgeoregister.nl](https://nationaalgeoregister.nl/) (NGR), leveraging the NGR [CSW service](https://nationaalgeoregister.nl/geonetwork/srv/dut/csw?service=CSW&request=GetCapabilities).
To install from source run (from the root this repo):
```sh
python3 -m build
python3 -m pip install dist/ngr_spider-0.0.1-py3-none-any.whl
```
This should install the cli tool `ngr-spider`:
```sh
$ ngr-spider
usage: ngr-spider [-h] subcommand ...
Generate list of PDOK services and/or service layers
positional arguments:
subcommand
services Generate list of all PDOK services
layers Generate list of all PDOK layers
options:
-h, --help show this help message and exit
```
## Examples
1. To generate a list of layers per service per dataset run the following:
```sh
ngr-spider layers -p OGC:WMS,OGC:WMTS -m datasets pdok-services.json
```
## Development
Install dev dependencies and package from source:
```sh
python3 -m pip install -e ".[dev]"
```
To format code install [`black`](https://pypi.org/project/black/) en run from the root of this repo:
```sh
black .
```
Delete unused imports with [`autoflake`](https://pypi.org/project/autoflake/):
```sh
autoflake --remove-all-unused-imports -i -r .
```
Organise and order imports with [`isort`](https://pypi.org/project/isort/):
```sh
isort -m 3 .
```
Run mypy for type checking (from root):
```sh
mypy ngr_spider
```
## Sorting Rules Example
When running the `layers` command in flat mode (`--mode flat`), it is possible to sort the layers in the output by passing a path to a JSON file containing, sorting rules. See below for an example a sorting rules JSON file (use for example with: `ngr-spider layers --sort sorting-rules.json -n 20 -m flat --pretty -p "OGC:WMS,OGC:WMTS" output.json`):
```json
[
{ "index": 0, "names": ["opentopo+"], "types": ["OGC:WMTS"] },
{ "index": 10, "names": ["^actueel_orthohr$"], "types": ["OGC:WMTS"] },
{ "index": 11, "names": ["^actueel_ortho25$"], "types": ["OGC:WMTS"] },
{ "index": 12, "names": ["^actueel_ortho25ir$"], "types": ["OGC:WMTS"] },
{ "index": 12, "names": ["lufolabels"], "types": ["OGC:WMTS"] },
{
"index": 20,
"names": ["landgebied", "provinciegebied", "gemeentegebied"],
"types": ["OGC:WFS"]
},
{ "index": 30, "names": ["top+"], "types": ["OGC:WMTS"] },
{
"index": 32,
"names": ["^standaard$", "^grijs$", "^pastel$", "^water$"],
"types": ["OGC:WMTS"]
},
{
"index": 34,
"names": ["bgtstandaardv2", "bgtachtergrond"],
"types": ["OGC:WMTS"]
},
{ "index": 60, "names": ["ahn3+"], "types": ["OGC:WMTS"] }
]
```
Raw data
{
"_id": null,
"home_page": "",
"name": "ngr-services-spider",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10.6",
"maintainer_email": "",
"keywords": "ngr,csw,pdok,ows",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/81/ca/c34473a1c9c30a608cb4676921b13fc0a0add80d1656a23a80625a2b48c6/ngr-services-spider-0.6.1.tar.gz",
"platform": null,
"description": "# NGR SPIDER\n\n[](https://github.com/PDOK/ngr-services-spider/blob/master/LICENSE)\n[](https://github.com/PDOK/ngr-services-spider/releases)\n[](https://hub.docker.com/r/pdok/ngr-services-spider/tags)\n[](https://pypi.org/project/ngr-services-spider/)\n\n\nCLI (command line interface) application to retrieve list of services and datasets in a simple JSON format from [nationaalgeoregister.nl](https://nationaalgeoregister.nl/) (NGR), leveraging the NGR [CSW service](https://nationaalgeoregister.nl/geonetwork/srv/dut/csw?service=CSW&request=GetCapabilities).\n\nTo install from source run (from the root this repo):\n\n```sh\npython3 -m build\npython3 -m pip install dist/ngr_spider-0.0.1-py3-none-any.whl \n```\n\nThis should install the cli tool `ngr-spider`:\n\n```sh\n$ ngr-spider\nusage: ngr-spider [-h] subcommand ...\n\nGenerate list of PDOK services and/or service layers\n\npositional arguments:\n subcommand\n services Generate list of all PDOK services\n layers Generate list of all PDOK layers\n\noptions:\n -h, --help show this help message and exit\n```\n\n## Examples\n\n1. To generate a list of layers per service per dataset run the following:\n\n```sh\nngr-spider layers -p OGC:WMS,OGC:WMTS -m datasets pdok-services.json\n```\n\n## Development\n\nInstall dev dependencies and package from source:\n\n```sh\npython3 -m pip install -e \".[dev]\" \n```\n\nTo format code install [`black`](https://pypi.org/project/black/) en run from the root of this repo:\n\n```sh\nblack .\n```\n\nDelete unused imports with [`autoflake`](https://pypi.org/project/autoflake/):\n\n```sh\nautoflake --remove-all-unused-imports -i -r .\n```\n\nOrganise and order imports with [`isort`](https://pypi.org/project/isort/):\n\n```sh\nisort -m 3 .\n```\n\nRun mypy for type checking (from root):\n\n```sh\nmypy ngr_spider \n```\n\n\n## Sorting Rules Example\n\nWhen running the `layers` command in flat mode (`--mode flat`), it is possible to sort the layers in the output by passing a path to a JSON file containing, sorting rules. See below for an example a sorting rules JSON file (use for example with: `ngr-spider layers --sort sorting-rules.json -n 20 -m flat --pretty -p \"OGC:WMS,OGC:WMTS\" output.json`):\n\n\n```json\n[\n { \"index\": 0, \"names\": [\"opentopo+\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 10, \"names\": [\"^actueel_orthohr$\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 11, \"names\": [\"^actueel_ortho25$\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 12, \"names\": [\"^actueel_ortho25ir$\"], \"types\": [\"OGC:WMTS\"] },\n { \"index\": 12, \"names\": [\"lufolabels\"], \"types\": [\"OGC:WMTS\"] },\n {\n \"index\": 20,\n \"names\": [\"landgebied\", \"provinciegebied\", \"gemeentegebied\"],\n \"types\": [\"OGC:WFS\"]\n },\n { \"index\": 30, \"names\": [\"top+\"], \"types\": [\"OGC:WMTS\"] },\n {\n \"index\": 32,\n \"names\": [\"^standaard$\", \"^grijs$\", \"^pastel$\", \"^water$\"],\n \"types\": [\"OGC:WMTS\"]\n },\n {\n \"index\": 34,\n \"names\": [\"bgtstandaardv2\", \"bgtachtergrond\"],\n \"types\": [\"OGC:WMTS\"]\n },\n { \"index\": 60, \"names\": [\"ahn3+\"], \"types\": [\"OGC:WMTS\"] }\n]\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "test",
"version": "0.6.1",
"split_keywords": [
"ngr",
"csw",
"pdok",
"ows"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0b173ac0a986beb974b00abeffd39ad5fb713757e3aa2b7aa1ffe631e030782e",
"md5": "0f3d93e54cf7bb830c24766315648c6c",
"sha256": "eda6c28e0367ce481b2fc8ecff98a718470faa802a9319b45b4347914a6088e7"
},
"downloads": -1,
"filename": "ngr_services_spider-0.6.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0f3d93e54cf7bb830c24766315648c6c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10.6",
"size": 20479,
"upload_time": "2023-04-05T13:12:53",
"upload_time_iso_8601": "2023-04-05T13:12:53.076179Z",
"url": "https://files.pythonhosted.org/packages/0b/17/3ac0a986beb974b00abeffd39ad5fb713757e3aa2b7aa1ffe631e030782e/ngr_services_spider-0.6.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "81cac34473a1c9c30a608cb4676921b13fc0a0add80d1656a23a80625a2b48c6",
"md5": "ac66ecaf73ffb4cea70cf43c84c7a63a",
"sha256": "9769a384bc4bdfcd5271caeb7fdf8b95e03b0bec350e353518b64e9f6c9f121e"
},
"downloads": -1,
"filename": "ngr-services-spider-0.6.1.tar.gz",
"has_sig": false,
"md5_digest": "ac66ecaf73ffb4cea70cf43c84c7a63a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10.6",
"size": 17935,
"upload_time": "2023-04-05T13:12:54",
"upload_time_iso_8601": "2023-04-05T13:12:54.707223Z",
"url": "https://files.pythonhosted.org/packages/81/ca/c34473a1c9c30a608cb4676921b13fc0a0add80d1656a23a80625a2b48c6/ngr-services-spider-0.6.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-05 13:12:54",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "ngr-services-spider"
}
