# ogr2vrt *simple*
**Generate a simple VRT file from an OGR-compatible dataset**
## Python package
This is mostly a python package, destined to be used by other applications. By itself, it is quite limited. However, we provide a small CLI tool that acts as a commandline VRT generator.
## Command line interface
Generate a VRT file from an OGR-compatible source.
The result is to be considered as a "kickoff" VRT file, to refine according to your desires
but it will save you some time.
It mostly targets and was tested with spreadsheet data (xls, xslx, ods, csv) but actually should work with most data sources supported by the OGR drivers
Are currently detected:
- the datasource path
- for each layer (depending on the source, there might be one or more layers):
- the layer name
- for each field in this layer
- name
- type if available (defaults to string)
- width (precision) if available
By default, the VRT file will be generated alongside the source file, extension .vrt
## Features
- Support file as datasource
- Support plain URL as datasource:
- E.g. https://www.data.gouv.fr/fr/datasets/r/c53cd4d4-4623-4772-9b8c-bc72a9cdf4c2
- Autodetects the file format if provided in the headers
- Supports streaming service (e.g. https://www.data.gouv.fr/fr/datasets/r/d22ba593-90a4-4725-977c-095d1f654d28)
- find path to dataset inside an archive e.g. https://open-data.s3.fr-par.scw.cloud/bdnb_millesime_2022-10-d/millesime_2022-10-d_dep59/open_data_millesime_2022-10-d_dep59_gpkg.zip)
- determine if needs to download the file as a first step or can use vsicurl functions
**_Does not support (yet)_**:
- non-UTF-8 csv files
---
## Install
- _**You will need the GDAL/OGR library**_. It is not listed in the requirements.txt file, because the installation depends on having the proper libraries already installed on your computer.
The simplest way:
- install gdal, libgdal-dev, python3-dev packages on your computer
- install [GDAL package](https://pypi.org/project/GDAL/) using
```
pip install GDAL==$(gdal-config --version)
```
- Create a virtual env and install the app
```
python3 -m venv .venv
source .venv/bin/activate
pip install ogr2vrt-simple
```
## Use the CLI
Once installed, you will have the `ogr2vrt_cli` command available. For now, it is limited to only one sub-command, `generate-vrt`:
```
# Get help
ogr2vrt_cli generate-vrt --help
# Extract VRT from a remote resource
ogr2vrt_cli generate-vrt https://raw.githubusercontent.com/OSGeo/gdal/master/autotest/ogr/data/shp/poly.zip
# Works also on a API remote source:
ogr2vrt_cli generate-vrt -d 'https://data.statistiques.developpement-durable.gouv.fr/dido/api/v1/datafiles/37dd7056-6c4d-44e0-a720-32d4064f9a26/csv?millesime=2023-05&withColumnName=true&withColumnDescription=true&withColumnUnit=true&orderBy=-COMMUNE_CODE&columns=COMMUNE_CODE,COMMUNE_LIBELLE,CLASSE_VEHICULE,CATEGORIE_VEHICULE,CARBURANT,CRITAIR,PARC_2011,PARC_2012,PARC_2013,PARC_2014,PARC_2015,PARC_2016,PARC_2017,PARC_2018,PARC_2019,PARC_2020,PARC_2021,PARC_2022&COMMUNE_CODE=contains%3A09241'
```
_**Note**: as in the example above, if you are tapping into a remote URL that has special characters in it (e.g. parenthesis), you will have to surround the URL with quotes or escape the characters (this is a shell issue, not a python issue, but an issue that needs to be taken care of anyway)_
---
## Develop
### Compatibility
**python >=3.8**
Some effort was done to support python 3.8+, but if possible, we would advise to use a more recent one (3.10 or above)
### Using poetry
This is now the recommended way
- Install [poetry](https://python-poetry.org/docs/#installation)
- Install the dependencies: `poetry install`
- Run the script:
```
# Install the dependencies:
poetry install
#activate the environment
poetry shell
# You have to install the GDAL library using pip, it doesn't seem to work with poetry directly
pip install GDAL==$(gdal-config --version)
cd ogr2vrt_simple/
python3 cli.py generate-vrt --help
```
### Simply using pip
_This should still work._
Create a virtual env and install the requirements
```
python3 -m venv .venv
source .venv/bin/activate
pip install GDAL==$(gdal-config --version)
pip install -r requirements.txt
```
_**You will need the GDAL/OGR library**_. See above in the [install](#install) section
### Run it
```bash
cd ogr2vrt_simple/
# Generate VRT file for a local file (zipped shapefile)
python3 cli.py generate-vrt https://raw.githubusercontent.com/OSGeo/gdal/master/autotest/ogr/data/shp/poly.zip
# Generate VRT file for a remote CSV resource
python3 cli.py generate-vrt --with_vsicurl https://www.data.gouv.fr/fr/datasets/r/c53cd4d4-4623-4772-9b8c-bc72a9cdf4c2
```
List options:
```bash
python3 cli.py generate-vrt --help
````
### Build
Use Poetry to build this script:
```
# Build
poetry build
# install
poetry install
# test it
ogr2vrt_cli --help
# Publish it to test-pypi
poetry publish -r test-pypi
# ... or to pypi
poetry publish
```
Raw data
{
"_id": null,
"home_page": "https://github.com/jeanpommier/ogr2vrt_simple/",
"name": "ogr2vrt-simple",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "gdal,ogr,VRT,geospatial",
"author": "Jean Pommier",
"author_email": "jean.pommier@pi-geosolutions.fr",
"download_url": "https://files.pythonhosted.org/packages/cc/97/082e89b8b02c70c1de0beeb5c81f6fc750b08ba62bd1e91c4571fd3a902c/ogr2vrt_simple-0.2.4.tar.gz",
"platform": null,
"description": "# ogr2vrt *simple*\n**Generate a simple VRT file from an OGR-compatible dataset**\n\n## Python package\nThis is mostly a python package, destined to be used by other applications. By itself, it is quite limited. However, we provide a small CLI tool that acts as a commandline VRT generator.\n\n## Command line interface\nGenerate a VRT file from an OGR-compatible source.\nThe result is to be considered as a \"kickoff\" VRT file, to refine according to your desires\nbut it will save you some time.\n\nIt mostly targets and was tested with spreadsheet data (xls, xslx, ods, csv) but actually should work with most data sources supported by the OGR drivers\n\nAre currently detected:\n- the datasource path\n- for each layer (depending on the source, there might be one or more layers):\n - the layer name\n - for each field in this layer\n - name\n - type if available (defaults to string)\n - width (precision) if available\n\nBy default, the VRT file will be generated alongside the source file, extension .vrt\n\n## Features\n- Support file as datasource\n- Support plain URL as datasource:\n - E.g. https://www.data.gouv.fr/fr/datasets/r/c53cd4d4-4623-4772-9b8c-bc72a9cdf4c2\n - Autodetects the file format if provided in the headers\n - Supports streaming service (e.g. https://www.data.gouv.fr/fr/datasets/r/d22ba593-90a4-4725-977c-095d1f654d28)\n - find path to dataset inside an archive e.g. https://open-data.s3.fr-par.scw.cloud/bdnb_millesime_2022-10-d/millesime_2022-10-d_dep59/open_data_millesime_2022-10-d_dep59_gpkg.zip)\n - determine if needs to download the file as a first step or can use vsicurl functions\n\n**_Does not support (yet)_**:\n- non-UTF-8 csv files\n\n---\n\n## Install\n- _**You will need the GDAL/OGR library**_. It is not listed in the requirements.txt file, because the installation depends on having the proper libraries already installed on your computer.\n\n The simplest way:\n - install gdal, libgdal-dev, python3-dev packages on your computer\n - install [GDAL package](https://pypi.org/project/GDAL/) using \n ```\n pip install GDAL==$(gdal-config --version)\n ```\n\n- Create a virtual env and install the app\n```\npython3 -m venv .venv\nsource .venv/bin/activate\npip install ogr2vrt-simple\n```\n\n## Use the CLI\nOnce installed, you will have the `ogr2vrt_cli` command available. For now, it is limited to only one sub-command, `generate-vrt`:\n```\n# Get help\nogr2vrt_cli generate-vrt --help\n\n# Extract VRT from a remote resource\nogr2vrt_cli generate-vrt https://raw.githubusercontent.com/OSGeo/gdal/master/autotest/ogr/data/shp/poly.zip\n\n# Works also on a API remote source:\nogr2vrt_cli generate-vrt -d 'https://data.statistiques.developpement-durable.gouv.fr/dido/api/v1/datafiles/37dd7056-6c4d-44e0-a720-32d4064f9a26/csv?millesime=2023-05&withColumnName=true&withColumnDescription=true&withColumnUnit=true&orderBy=-COMMUNE_CODE&columns=COMMUNE_CODE,COMMUNE_LIBELLE,CLASSE_VEHICULE,CATEGORIE_VEHICULE,CARBURANT,CRITAIR,PARC_2011,PARC_2012,PARC_2013,PARC_2014,PARC_2015,PARC_2016,PARC_2017,PARC_2018,PARC_2019,PARC_2020,PARC_2021,PARC_2022&COMMUNE_CODE=contains%3A09241'\n```\n\n_**Note**: as in the example above, if you are tapping into a remote URL that has special characters in it (e.g. parenthesis), you will have to surround the URL with quotes or escape the characters (this is a shell issue, not a python issue, but an issue that needs to be taken care of anyway)_\n\n---\n## Develop\n \n### Compatibility\n**python >=3.8**\n\nSome effort was done to support python 3.8+, but if possible, we would advise to use a more recent one (3.10 or above)\n\n### Using poetry\nThis is now the recommended way\n\n- Install [poetry](https://python-poetry.org/docs/#installation)\n- Install the dependencies: `poetry install`\n- Run the script:\n```\n# Install the dependencies: \npoetry install\n#activate the environment\npoetry shell \n# You have to install the GDAL library using pip, it doesn't seem to work with poetry directly\npip install GDAL==$(gdal-config --version)\n\ncd ogr2vrt_simple/\npython3 cli.py generate-vrt --help\n```\n\n\n### Simply using pip\n_This should still work._ \n\nCreate a virtual env and install the requirements\n```\npython3 -m venv .venv\nsource .venv/bin/activate\npip install GDAL==$(gdal-config --version)\npip install -r requirements.txt\n```\n\n_**You will need the GDAL/OGR library**_. See above in the [install](#install) section\n \n### Run it\n```bash\ncd ogr2vrt_simple/\n# Generate VRT file for a local file (zipped shapefile)\npython3 cli.py generate-vrt https://raw.githubusercontent.com/OSGeo/gdal/master/autotest/ogr/data/shp/poly.zip\n\n# Generate VRT file for a remote CSV resource\npython3 cli.py generate-vrt --with_vsicurl https://www.data.gouv.fr/fr/datasets/r/c53cd4d4-4623-4772-9b8c-bc72a9cdf4c2\n```\n\nList options: \n```bash\npython3 cli.py generate-vrt --help\n````\n\n### Build\n\nUse Poetry to build this script:\n```\n# Build\npoetry build\n\n# install \npoetry install\n\n# test it\nogr2vrt_cli --help\n\n# Publish it to test-pypi\npoetry publish -r test-pypi\n# ... or to pypi\npoetry publish\n```",
"bugtrack_url": null,
"license": "",
"summary": "Generate a simple VRT file for data transformations",
"version": "0.2.4",
"project_urls": {
"Homepage": "https://github.com/jeanpommier/ogr2vrt_simple/",
"Repository": "https://github.com/jeanpommier/ogr2vrt_simple/"
},
"split_keywords": [
"gdal",
"ogr",
"vrt",
"geospatial"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "671192be1394d74c3a5de7f3a59197b268611db13e409577a29204ec157391d8",
"md5": "9f03dc35c88c93af5b3eab671058f439",
"sha256": "dc974643c924a004676bdafc9db26c5a7e6cd9ced2d6d55b42d780e0114a39bf"
},
"downloads": -1,
"filename": "ogr2vrt_simple-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9f03dc35c88c93af5b3eab671058f439",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 18670,
"upload_time": "2023-11-10T13:41:58",
"upload_time_iso_8601": "2023-11-10T13:41:58.392325Z",
"url": "https://files.pythonhosted.org/packages/67/11/92be1394d74c3a5de7f3a59197b268611db13e409577a29204ec157391d8/ogr2vrt_simple-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cc97082e89b8b02c70c1de0beeb5c81f6fc750b08ba62bd1e91c4571fd3a902c",
"md5": "074cfdf31a7f12edc63abba7ed0bec3a",
"sha256": "7bfa2fa73a9cefe2f42bc5513f936a6c1e2ded05a44d86da91fd477a00341f63"
},
"downloads": -1,
"filename": "ogr2vrt_simple-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "074cfdf31a7f12edc63abba7ed0bec3a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 15661,
"upload_time": "2023-11-10T13:42:03",
"upload_time_iso_8601": "2023-11-10T13:42:03.207181Z",
"url": "https://files.pythonhosted.org/packages/cc/97/082e89b8b02c70c1de0beeb5c81f6fc750b08ba62bd1e91c4571fd3a902c/ogr2vrt_simple-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-10 13:42:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jeanpommier",
"github_project": "ogr2vrt_simple",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "ogr2vrt-simple"
}
JSON
