# fiboa CLI
A command-line interface (CLI) for working with fiboa.
- [Getting Started](#getting-started)
- [Documentation / Commands](#commands)
- [Development](#development)
## Getting Started
In order to make working with fiboa easier we have developed command-line interface (CLI) tools such as
inspection, validation and file format conversions.
### Installation
You will need to have **Python 3.9** or any later version installed.
Run `pip install fiboa-cli` in the CLI to install the validator.
**Optional:** To install additional dependencies for specific [converters](#converter-for-existing-datasets),
you can for example run: `pip install fiboa-cli[xyz]` with xyz being the converter name.
**Note on versions:**
- fiboa CLI >= 0.3.0 works with fiboa version > 0.2.0
- fiboa CLI < 0.3.0 works with fiboa version = 0.1.0
### Execute a command
After the installation you should be able to run the following command: `fiboa`
You should see usage instructions and [available commands](#commands) for the CLI.
fiboa CLI supports various commands to work with the files:
- [fiboa CLI](#fiboa-cli)
- [Getting Started](#getting-started)
- [Installation](#installation)
- [Execute a command](#execute-a-command)
- [Commands](#commands)
- [Validation](#validation)
- [Create fiboa GeoParquet from GeoJSON](#create-fiboa-geoparquet-from-geojson)
- [Create fiboa GeoJSON from GeoParquet](#create-fiboa-geojson-from-geoparquet)
- [Inspect fiboa GeoParquet file](#inspect-fiboa-geoparquet-file)
- [Merge fiboa GeoParquet files](#merge-fiboa-geoparquet-files)
- [Create JSON Schema from fiboa Schema](#create-json-schema-from-fiboa-schema)
- [Validate a fiboa Schema](#validate-a-fiboa-schema)
- [Improve a fiboa Parquet file](#improve-a-fiboa-parquet-file)
- [Update an extension template with new names](#update-an-extension-template-with-new-names)
- [Converter for existing datasets](#converter-for-existing-datasets)
- [Development](#development)
- [Implement a converter](#implement-a-converter)
## Commands
### Validation
To validate a fiboa GeoParquet or GeoJSON file, you can for example run:
- GeoJSON: `fiboa validate example.json --collection collection.json`
- GeoParquet: `fiboa validate example.parquet --data`
Check `fiboa validate --help` for more details.
The validator also supports remote files.
- `http://` or `https://`: no further configuration is needed.
- `s3://`: `s3fs` needs to be installed (run `pip install .[s3]`) and you may need to set environment variables.
Refer [here](https://s3fs.readthedocs.io/en/latest/#credentials) for how to define credentials.
- `gs://`: `gcsfs` needs to be installed (run `pip install .[gcs]`).
By default, `gcsfs` will attempt to use your default gcloud credentials or, attempt to get credentials from the google metadata service, or fall back to anonymous access.
### Create fiboa GeoParquet from GeoJSON
To create a fiboa-compliant GeoParquet for a fiboa-compliant set of GeoJSON files containing Features or FeatureCollections,
you can for example run:
- `fiboa create-geoparquet geojson/example.json -o example.parquet -c geojson/collection.json`
Check `fiboa create-geoparquet --help` for more details.
### Create fiboa GeoJSON from GeoParquet
To create one or multiple fiboa-compliant GeoJSON file(s) for a fiboa-compliant GeoParquet file,
you can for example run:
- GeoJSON FeatureCollection:
`fiboa create-geojson example.parquet -o dest-folder`
- GeoJSON Features (with indentation and max. 100 features):
`fiboa create-geojson example.parquet -o dest-folder -n 100 -i 2 -f`
Check `fiboa create-geoparquet --help` for more details.
### Inspect fiboa GeoParquet file
To look into a fiboa GeoParquet file to get a rough understanding of the content, the following can be executed:
- `fiboa describe example.parquet`
Check `fiboa describe --help` for more details.
### Merge fiboa GeoParquet files
Merges multiple fiboa datasets to a combined fiboa dataset:
- `fiboa merge ec_ee.parquet ec_lv.parquet -o merged.parquet -e https://fiboa.github.io/hcat-extension/v0.1.0/schema.yaml -i ec:hcat_name -i ec:hcat_code -i ec:translated_name`
Check `fiboa merge --help` for more details.
### Create JSON Schema from fiboa Schema
To create a JSON Schema for a fiboa Schema YAML file, you can for example run:
- `fiboa jsonschema example.json --id=https://fiboa.github.io/specification/v0.1.0/geojson/schema.json -o schema.json`
Check `fiboa jsonschema --help` for more details.
### Validate a fiboa Schema
To validate a fiboa Schema YAML file, you can for example run:
- `fiboa validate-schema schema/schema.yaml`
Check `fiboa validate-schema --help` for more details.
### Improve a fiboa Parquet file
Various "improvements" can be applied to a fiboa GeoParquet file.
The commands allows to
- change the CRS (`--crs`)
- change the GeoParquet version (`-gp1`) and compression (`-pc`)
- add/fill missing perimeter/area values (`-sz`)
- fix invalid geometries (`-g`)
- rename columns (`-r`)
Example:
- `fiboa improve file.parquet -o file2.parquet -g -sz -r old=new -pc zstd`
Check `fiboa improve --help` for more details.
### Update an extension template with new names
Once you've created and git cloned a new extension, you can use the CLI
to update all template placeholders with proper names.
For example, if your extension is meant to have
- the title "Timestamps Extension",
- the prefix `ts` (e.g. field `ts:created` or `ts:updated`),
- is hosted at `https://github.io/fiboa/timestamps-extension`
(organization: `fiboa`, repository `timestamps-extension`),
- and you run fiboa in the folder of the extension.
Then the following command could be used:
- `fiboa rename-extension . -t Timestamps -p ts -s timestamps-extension -o fiboa`
Check `fiboa rename-extension --help` for more details.
### Converter for existing datasets
The CLI ships various converters for existing datasets.
To get a list of available converters/datasets with title, license, etc. run:
- `fiboa converters`
Use any of the IDs from the list to convert an existing dataset to fiboa:
- `fiboa convert de_nrw`
See [Implement a converter](#implement-a-converter) for details about how to
## Development
To install in development mode run `pip install -e .` in this folder.
For the tests first run `pip install -r requirements-dev.txt` to install pytest.
Then you can run `pytest` to execute the tests.
### Implement a converter
The following high-level description gives an idea how to implement a converter in fiboa CLI:
1. Create a new file in `fiboa_cli/datasets` based on the `template.py`
2. Implement the `convert()` function / test it / run it
3. Add missing dependencies into a separate dependency group in `setup.py`
4. Add the converter to the list above
5. Create a PR to submit your converter for review
An in-depth guide how to create a cloud-native fiboa dataset using fiboa CLI is available at:
<https://github.com/fiboa/data/blob/main/HOWTO.md>
Raw data
{
"_id": null,
"home_page": "https://github.com/fiboa/cli",
"name": "fiboa-cli",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Matthias Mohr",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/b6/92/9123a42b344981dd64b2ba78f548239e1e671489448fb67c540c8788840f/fiboa_cli-0.9.0.tar.gz",
"platform": null,
"description": "# fiboa CLI\n\nA command-line interface (CLI) for working with fiboa.\n\n- [Getting Started](#getting-started)\n- [Documentation / Commands](#commands)\n- [Development](#development)\n\n## Getting Started\n\nIn order to make working with fiboa easier we have developed command-line interface (CLI) tools such as\ninspection, validation and file format conversions.\n\n### Installation\n\nYou will need to have **Python 3.9** or any later version installed.\n\nRun `pip install fiboa-cli` in the CLI to install the validator.\n\n**Optional:** To install additional dependencies for specific [converters](#converter-for-existing-datasets),\nyou can for example run: `pip install fiboa-cli[xyz]` with xyz being the converter name.\n\n**Note on versions:**\n\n- fiboa CLI >= 0.3.0 works with fiboa version > 0.2.0\n- fiboa CLI < 0.3.0 works with fiboa version = 0.1.0\n\n### Execute a command\n\nAfter the installation you should be able to run the following command: `fiboa`\n\nYou should see usage instructions and [available commands](#commands) for the CLI.\n\nfiboa CLI supports various commands to work with the files:\n\n- [fiboa CLI](#fiboa-cli)\n - [Getting Started](#getting-started)\n - [Installation](#installation)\n - [Execute a command](#execute-a-command)\n - [Commands](#commands)\n - [Validation](#validation)\n - [Create fiboa GeoParquet from GeoJSON](#create-fiboa-geoparquet-from-geojson)\n - [Create fiboa GeoJSON from GeoParquet](#create-fiboa-geojson-from-geoparquet)\n - [Inspect fiboa GeoParquet file](#inspect-fiboa-geoparquet-file)\n - [Merge fiboa GeoParquet files](#merge-fiboa-geoparquet-files)\n - [Create JSON Schema from fiboa Schema](#create-json-schema-from-fiboa-schema)\n - [Validate a fiboa Schema](#validate-a-fiboa-schema)\n - [Improve a fiboa Parquet file](#improve-a-fiboa-parquet-file)\n - [Update an extension template with new names](#update-an-extension-template-with-new-names)\n - [Converter for existing datasets](#converter-for-existing-datasets)\n - [Development](#development)\n - [Implement a converter](#implement-a-converter)\n\n## Commands\n\n### Validation\n\nTo validate a fiboa GeoParquet or GeoJSON file, you can for example run:\n\n- GeoJSON: `fiboa validate example.json --collection collection.json`\n- GeoParquet: `fiboa validate example.parquet --data`\n\nCheck `fiboa validate --help` for more details.\n\nThe validator also supports remote files.\n\n- `http://` or `https://`: no further configuration is needed.\n- `s3://`: `s3fs` needs to be installed (run `pip install .[s3]`) and you may need to set environment variables.\n Refer [here](https://s3fs.readthedocs.io/en/latest/#credentials) for how to define credentials.\n- `gs://`: `gcsfs` needs to be installed (run `pip install .[gcs]`).\n By default, `gcsfs` will attempt to use your default gcloud credentials or, attempt to get credentials from the google metadata service, or fall back to anonymous access.\n\n### Create fiboa GeoParquet from GeoJSON\n\nTo create a fiboa-compliant GeoParquet for a fiboa-compliant set of GeoJSON files containing Features or FeatureCollections,\nyou can for example run:\n\n- `fiboa create-geoparquet geojson/example.json -o example.parquet -c geojson/collection.json`\n\nCheck `fiboa create-geoparquet --help` for more details.\n\n### Create fiboa GeoJSON from GeoParquet\n\nTo create one or multiple fiboa-compliant GeoJSON file(s) for a fiboa-compliant GeoParquet file,\nyou can for example run:\n\n- GeoJSON FeatureCollection:\n `fiboa create-geojson example.parquet -o dest-folder`\n- GeoJSON Features (with indentation and max. 100 features):\n `fiboa create-geojson example.parquet -o dest-folder -n 100 -i 2 -f`\n\nCheck `fiboa create-geoparquet --help` for more details.\n\n### Inspect fiboa GeoParquet file\n\nTo look into a fiboa GeoParquet file to get a rough understanding of the content, the following can be executed:\n\n- `fiboa describe example.parquet`\n\nCheck `fiboa describe --help` for more details.\n\n### Merge fiboa GeoParquet files\n\nMerges multiple fiboa datasets to a combined fiboa dataset:\n\n- `fiboa merge ec_ee.parquet ec_lv.parquet -o merged.parquet -e https://fiboa.github.io/hcat-extension/v0.1.0/schema.yaml -i ec:hcat_name -i ec:hcat_code -i ec:translated_name`\n\nCheck `fiboa merge --help` for more details.\n\n### Create JSON Schema from fiboa Schema\n\nTo create a JSON Schema for a fiboa Schema YAML file, you can for example run:\n\n- `fiboa jsonschema example.json --id=https://fiboa.github.io/specification/v0.1.0/geojson/schema.json -o schema.json`\n\nCheck `fiboa jsonschema --help` for more details.\n\n### Validate a fiboa Schema\n\nTo validate a fiboa Schema YAML file, you can for example run:\n\n- `fiboa validate-schema schema/schema.yaml`\n\nCheck `fiboa validate-schema --help` for more details.\n\n### Improve a fiboa Parquet file\n\nVarious \"improvements\" can be applied to a fiboa GeoParquet file.\nThe commands allows to\n\n- change the CRS (`--crs`)\n- change the GeoParquet version (`-gp1`) and compression (`-pc`)\n- add/fill missing perimeter/area values (`-sz`)\n- fix invalid geometries (`-g`)\n- rename columns (`-r`)\n\nExample:\n\n- `fiboa improve file.parquet -o file2.parquet -g -sz -r old=new -pc zstd`\n\nCheck `fiboa improve --help` for more details.\n\n### Update an extension template with new names\n\nOnce you've created and git cloned a new extension, you can use the CLI\nto update all template placeholders with proper names.\n\nFor example, if your extension is meant to have\n\n- the title \"Timestamps Extension\",\n- the prefix `ts` (e.g. field `ts:created` or `ts:updated`),\n- is hosted at `https://github.io/fiboa/timestamps-extension`\n (organization: `fiboa`, repository `timestamps-extension`),\n- and you run fiboa in the folder of the extension.\n\nThen the following command could be used:\n\n- `fiboa rename-extension . -t Timestamps -p ts -s timestamps-extension -o fiboa`\n\nCheck `fiboa rename-extension --help` for more details.\n\n### Converter for existing datasets\n\nThe CLI ships various converters for existing datasets.\n\nTo get a list of available converters/datasets with title, license, etc. run:\n\n- `fiboa converters`\n\nUse any of the IDs from the list to convert an existing dataset to fiboa:\n\n- `fiboa convert de_nrw`\n\nSee [Implement a converter](#implement-a-converter) for details about how to\n\n## Development\n\nTo install in development mode run `pip install -e .` in this folder.\n\nFor the tests first run `pip install -r requirements-dev.txt` to install pytest.\nThen you can run `pytest` to execute the tests.\n\n### Implement a converter\n\nThe following high-level description gives an idea how to implement a converter in fiboa CLI:\n\n1. Create a new file in `fiboa_cli/datasets` based on the `template.py`\n2. Implement the `convert()` function / test it / run it\n3. Add missing dependencies into a separate dependency group in `setup.py`\n4. Add the converter to the list above\n5. Create a PR to submit your converter for review\n\nAn in-depth guide how to create a cloud-native fiboa dataset using fiboa CLI is available at:\n<https://github.com/fiboa/data/blob/main/HOWTO.md>\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "CLI tools such as validation and file format conversion for fiboa.",
"version": "0.9.0",
"project_urls": {
"Homepage": "https://github.com/fiboa/cli"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ea24369507d0d8f6217e22c1056c0945b9d38e073bcc8c825e5b535647bb0ac8",
"md5": "81ec164904b9e0c0a32b8632002de53f",
"sha256": "9280cdd1762f4bbdf2071e3bdbef999688283e275f0d5c71b1e4ea19b4f75ffb"
},
"downloads": -1,
"filename": "fiboa_cli-0.9.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "81ec164904b9e0c0a32b8632002de53f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 106122,
"upload_time": "2025-01-07T12:48:42",
"upload_time_iso_8601": "2025-01-07T12:48:42.544628Z",
"url": "https://files.pythonhosted.org/packages/ea/24/369507d0d8f6217e22c1056c0945b9d38e073bcc8c825e5b535647bb0ac8/fiboa_cli-0.9.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b6929123a42b344981dd64b2ba78f548239e1e671489448fb67c540c8788840f",
"md5": "843f3802ac18fc80799b341862e2b099",
"sha256": "319e6ea52e05c5f3beb07cf389e51804955e29134ac42a66531259d9d0cc15da"
},
"downloads": -1,
"filename": "fiboa_cli-0.9.0.tar.gz",
"has_sig": false,
"md5_digest": "843f3802ac18fc80799b341862e2b099",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 73644,
"upload_time": "2025-01-07T12:48:44",
"upload_time_iso_8601": "2025-01-07T12:48:44.914820Z",
"url": "https://files.pythonhosted.org/packages/b6/92/9123a42b344981dd64b2ba78f548239e1e671489448fb67c540c8788840f/fiboa_cli-0.9.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-07 12:48:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fiboa",
"github_project": "cli",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "fiboa-cli"
}
JSON
